One advantage of the Java is that the interfaces are named, type-checked documentation. Yes it's more verbose, but a maintainer of that code can understand exactly the intention just from the interface names, without having to decipher it line-by-line.
For the Clojure code, after you've added enough comments to explain what you're doing, so that the code can be reasonably maintained, then it's the same size as the Java code, but without the type safety.
Granted for some of the simpler patterns like strategy, Java's verbosity just gets in the way, though much of that has been simplified in Java 8. FWIW I prefer Scala over both in all cases. But if Scala didn't exist, I'd prefer Java for the enterprise use case, and Clojure for the small applet use case.
Judging by his comment I have the feeling that entreprise level means huge projects with thousands of lines of bloat code and comments which need vast amount of underpaid programmers to analyse and debug just to add minor features.
Big, not-very-interesting CRUD app. Every time the data model changes, you've got to go through manually and find all the places the code needs changed, and hope you didn't miss anything. It probably didn't help that it was running on MongoDB, so two levels of dynamic typing and impossible to remember what went where, and a DB that takes whatever you give without complaining. So no compile errors and often no runtime errors either, just incorrect behavior.
Switched to Scala and SQL, and code is almost as compact but typed relations are far easier to reason about; if a relation changes then you change the core type and follow the compile errors. Once it compiles, it usually works. We've not missed Clojure in the least.
Scala (and even Java) simply feels more structured, with relations between types, the resulting app more "designed", which seems necessary at the enterprise level. In my 2 years, Clojure constantly felt like just hacking.
Would love to see some examples of Clojure being used well at an enterprise level so I could understand better how to do so, but CircleCI's recent flopping between core.typed and prismatic just makes me think it's a square peg in a round hole.
> Every time the data model changes, you've got to go through manually and find all the places the code needs changed, and hope you didn't miss anything.
No, that's what types are for. Types are exactly for this. They're far less verbose than tests, they prevent compilation completely on error, they're integrated with your code, they're 100% comprehensive, and they don't contain bugs. Contrarily, tests don't work at all for model changes because the test still retains the old model anyway.
I agree with your sentiment - why bother testing something that we can get the compiler to prove, once and for all? However, in Java, a typeful model of a program's specification is bound to be:
(0) Imprecise, because of the holes in Java's static safety. You can establish a convention that statically unsafe features are verboten, but it will remain just that - a convention.
(1) Verbose, because Java can't express even the simplest ideas without using at least 5 classes: Finite case analysis? Visitors. Higher-order functions? Strategies. Persistent data structures? Memento. Things don't get better if we move to a higher architectural level: Embedded DSLs? You need a separate class for each production rule in your abstract syntax. Type-safe database queries? Java has no structurally typed records, so you need to make a custom POJO class for every query that returns different columns.
These things are incredibly unpleasant to deal with, and while it might be true that "unpleasant work builds character", I don't think it should come as a surprise that some people say "enough is enough", and seek alternatives that minimize or at least reduce their pain (like dynamically checked conformance to a schema), even at the cost of sacrificing some type safety (dynamic checks can't enforce higher-order invariants).
I agree with you that Java is painful. But if "enough is enough", abandoning all types is throwing out the baby with bathwater. You make a compelling argument for languages with type inference and richer typing systems. There are many such languages. For the JVM, there is Scala.
I'm a static typing lover. My two favorite languages are Standard ML and Haskell, which I have found very well suited to the way I think: denotationally (expressions are given meaning by a mapping to a semantic domain), algebraically (substituting variables for expressions), abstractly (selectively ignoring implementation specifics) and structurally (the only operations allowed on an expression are those sanctioned by its type).
However, I don't think Scala supports your argument very well. Its type system is:
(0) Redundant: When should I use an abstract class or a trait? When should I provide extended functionality using a subclass or an implicit? What are the relative benefits of poor man's ML-style type abstraction (with path-dependent types) vs. poor man's type classes (with implicits)?
(1) Inconsistent: Why are closed sums [(generalized) algebraic data types] implemented in terms of an open-ended inheritance mechanism? This makes `match` exhaustiveness checks unnecessarily more complicated, and hiliarity ensues when the implementors get it wrong.
(2) Unhelpful: Scala's type inference is local and unidirectional, from arguments to results, which limits the usefulness of using types for exploratory programming. GHC Haskell, which has all sorts of crazy type system extensions (some even more ambitious than Scala's), does a much better job of globally inferring types.
So, in spite of my general preference for statically typed languages, I find Clojure easier to live with than Scala, simply because Scala has so many warts.
Well, against the standard of ML, Scala does come up short.
Almost all of that is due to the intentional choice of running on the JVM with high interop compatibility. Choose ML-family or Haskell, and you don't have Java baggage.
But I'd take types with class-vs-trait design choices, less-than-ideal match exhaustiveness checks, and type inference incompleteness any day over a language lacking all static typing features.
> Almost all of that is due to the intentional choice of running on the JVM with high interop compatibility.
Here are some counterexamples:
(0) Clojure is compatible with Java, but has less warts than Scala. It is not statically typed, but it is internally more unified and consistent. With some exceptions, you only get to see ugly compatibility compromises when you need to interact with Java code.
(1) Rust is compatible with C - in fact, it interoperates with other languages that have C FFIs by simply pretending to be C. Nevertheless, outside of specifically designated `unsafe` blocks, which idiomatic Rust only uses sparingly, most code is written in a "modern C++ meets ML" style.
> But I'd take types with class-vs-trait design choices, less-than-ideal match exhaustiveness checks, and type inference incompleteness any day over a language lacking all static typing features.
Static typing is a useful feature, but I think it's narrow-minded to use it as the only criterion for judging a language's merits. For instance, Clojure's macros are objectively nicer than Scala's, even if both pale in comparison to Racket's. Writing multimethods in Clojure is straightforward once you know the syntax, whereas emulating multiparameter type classes in Scala can be confusing unless you know the specifics of how implicit parameters are resolved. (This is, ironically, a direct consequence of Scala's richer type system.)
When I said Java interop, I meant two-way. With few exceptions (traits being the only I can think of) all Scala code can be used by Java. You can even use implicit parameters, though of course the Java compiler won't resolve them for you. And almost all Java constructs can be created with Scala (Java enumerations being an exception). For example, using Scala you can define an abstract class that Java code can use.
You can't do that in Clojure.
Real world examples: Play and Akka are two large projects that include Java development in their primary use cases.
Two-way interop is the cause for a huge number of less than ideal decisions: different Array equality than Seq, @specialized, universal equality, hash codes, and toString, abstract class vs trait, generic type erasure, etc.
You can call the compromise misguided (and you wouldn't be the first to do so) but that's why.
Why is there a need for a one-to-one correspondence between Java and Scala features? In any case, you can call Clojure code just fine from Java. Of course, the API won't look like one a Java programmer would've designed, but that's okay - Clojure isn't trying to be Java. It's trying to leverage the JVM environment.
Take twitter for example. They do a ton of stuff in Scala but make everything straightforward enough to use from Java that you wouldn't know it was Scala. This is fairly trivial in Scala but difficult in Clojure.
Types are a "proof" to the compiler that your code will run in a certain way. No missed "edge cases", no fuzziness, just pure comprehensive logic.
Of course, the success of this varies by language and a lot of people have worked to find better and more powerful approaches.
But you should regard any type system as a proof system for runtime behavior.
(I'm not saying that running tests with arbitrary examples is bad; I'm just saying that HUGE classes of potential bugs reliably go away with good static typing.)
I'm using postgres and clojure for quite a few enterprises and my typical approach is to keep the raw sql files somewhere (resources/sql/) so that I can execute it via CLI (say psql) as well as from within clojure (by parsing the files and putting them in a JDBC transaction)
From this I derive both the database schema itself as well as implementation details (e.g., check what columns are in a table or see if the current DB schema agrees with the current SQL schema in resources/sql)
For program internal representation I have my own "classes" (hash maps with a few pre/post assertions on key points) and I just use a single db.clj file that needs to be adapted when the SQL representation changes.
I also worked a lot with Java and Hibernate and it just feels verbose to change a simple datatype definition in both the application and the db. In a way, with clojure I get DB / App consistency that I wouldn't get with Java and/or types. I haven't had the chance to use scala, however.
I also worked with MongoDB (when I started with clojure) and I agree, there it is very easy to make mistakes but maybe I would do things now differently if I used it again.
Yeah I'm using raw JDBC for the same reason; I like knowing the exact SQL that's going to be used and it's nice to be able to freely use non-SQL-standard things that your particular database supports. It's trivial to write a custom wrapper to make things easier, and simple case classes in Scala to represent the DB records make for a single line code change when the DB model changes.
JDBC. I tried a few libraries and found they all provide either too much or too little. It's not so hard to write your own wrapper around it to make JDBC more friendly. Plus then you get to use features specific to your DB rather than just the lowest common denominator.
Another thing that helped was to use underscore_naming for any case-class in Scala that maps directly to a query, so it's easier to identify them.
Just out of curiosity, have you run into the "how did (our / users') (password / private key) get into the ABC data record that gets returned on XYZ API call?" regret (even though it's not displayed by the GUI, still it's transmitted)?
I'm thinking it's a total potential for exploit on web apps backed by mongodb and any dynamic language.
Well, in lisp languages you just write an abstraction layer to account for such problem areas.
It's really no different than the OO argument. If one wants OO functionality in lisp it can be implemented with the introduction of a few functions. But by using an OO only language you have just forced that abstraction to the surface leaving you with only one way to write your code even when it's not suitable for the problem space.
So the same could be said for Types. Without Type Inference you're locking yourself in to model/framework that has a list of disadvantages too.
I think the real problem is when people don't recognize they need to account for those problem areas, and they don't take the time to write the abstraction layer when needed and naturally they end up with bad results and feel like it's just hacking, because, well, they are.
Implementing statically checked abstractions in Lisp or Clojure using `defmacro` is a royal pain in the rear hole. Error messages from macros being used wrong range from "confusing" at best, to "WTF is this %#$&%#$&?" at worst.
The problem outlined was in dealing with data model changes. So the problem as I see is in managing the alignment of parameter types used in code with database field types. For that I've written a schema-template abstraction that manages that relationship. This template engine is about 100 lines of code and for every point the code intersect to/from the data it goes through the engine, looks up the a template and uses that for access/processing (a hook if you will). This allows automation for finding all the code entry spots.
Bottom line is I don't hit these issues because I identified the problem and wrote an abstraction layer to handle it and for the rest of my code I can happily rely on type inference.
What exactly are you calling "type inference"? It cannot be ML-like type reconstruction, because doing that requires either whole program analysis or a prohibition from interacting with untyped code.
I worked in java for 6 years before moving to clojure. Your experiences don't remotely reflect mine, I can't think of a single reason I'd ever choose java over clojure anymore (as tooling matures) but I appreciate you taking the time to comment!
> For the Clojure code, after you've added enough comments to explain what you're doing, so that the code can be reasonably maintained, then it's the same size as the Java code, but without the type safety.
Funny, I feel the same about design patterns. So you bothered to find an abstraction in the code, you described it, gave it a name, and yet - you didn't tell the computer - here I am using this abstraction! And you rely on other humans to correctly infer what abstraction you used from reading your code, instead of just telling them.
So what should really be done, design patterns, once identified, should be replaced with proper abstractions in the programming language (mostly functions?) and die. Frankly, most of them are just really fancy names for trivial things.
Totally agree. A fair amount of that is done in C#, with proper events, observables, enumerables, and functional constructs. Really when I say Java previously I mean C#; I haven't worked in Java proper in over a decade, so it's entirely possible I forget how bad it was.
While it’s definitely the case that there are more concise domain-specific examples, as you’ve pointed out, the article’s point stands: in Clojure, a Strategy is simply passing a function as an argument to another function.
Every "language comparison" between dominant ones and less popular ones (even implicitly) ends up being a little contrived, which is unfortunate because in general Clojure is the real deal.
I love the idea! A few examples are a bit strawmanish but really when you're comparing languages like that who can resist the strawman.
The Clojure solutions do sacrifice a lot of safety though, due to dynamic typing (e.g. the visitor pattern, which relies on basically strings to do its dispatching).
Brilliant-- this is among the best design pattern writeups I've ever seen, and will greatly help developers who are coming to Clojure.
This would be excellent for expanding to book-length guide to help people learn functional programming along with the patterns. Thank you Misha for writing this!
Really nice writeup of patterns. What turned me off Clojure is that the dynamic typing makes it very hard to refactor. e.g. even renaming a function is really quite difficult, let alone adding function parameters. Forget extracting a function. The dynamic typing also means a typo can yield a nil that travels far from the mistake site making debugging difficult. There are hints towards better debugging in clojure, but last time I checked it was still very much a work in progress. Also lazy lists are easy to screw up, e.g. if you put a concat inside a reduce then stack go boom.
Still this is a great post because it teaches you both good Clojure and the design patterns. Since java has closures now you can take a lot of what you learn from Clojure and put it in your well typed and refactorable Java programs.
Refactoring is a breeze when your editor is connected to the repl, you should try clj-refactor for instance.
I'm not sure what you mean with the typo story since most of the functions in your program should be free of side effects you should be able to test them as you write them.
I'm always curious about the latter sentiment. Maybe I just haven't been on enough interesting projects, but I've never had a program that didn't have/need side-effecting code nearly every other line. Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc. Pure data manipulation has always been a fairly minor component limited to dashboard statistics for me. Is purity that really your experience? What industry / types of projects?
Thinking in data & functions just takes some practice. Traditionally imperative code can often be reimagined in terms of reducing over a command sequence, or interpreting some more sophisticated representation. The benefits for implementation, testing, debugging, refactoring, etc can be massive.
Consider your chain of examples for "Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc." You could represent this as a sequence of steps which could be interpreted by a central side-effecting loop. Let's call it an "action plan", in this case for your "upload" operation:
While this is initially a tad more verbose, there's great reasons to do this sort of thing.
For example, now have a central place to mock all side effects! You could substitute a different execute method during testing, or automatically log the "action" object that failed.
More importantly, you can develop each of your "plan" functions independently. You can reuse them, re-order them, etc.
You can also do hypotheticals, for example: "Which plan would cause less network round trips?" can be answered before committing to a plan.
You can _record_ plans. Instrument your execute method to record and save plans to a file, then you can just check them right in as test cases.
You can optimize plans. If five different functions all notify the same target of the same thing, your central execution method can filter subsequent calls. Or you could separate your "optimization" from your "interpretation" and you can ask "After accounting for idempotence, which operations will be performed?"
As an aside, you just (sort of) implented the IO monad.
The simplest explanation of the IO monad is that your Haskell application outputs a recipe or "execution plan". This output is side effect free and pure. The runtime then executes your plan, producing those side effects.
One key difference between this and the IO monad or a free monad: There's no continuation.
That is, you can't communicate between the code that produces the command sequence and the code that interprets it. This is either a shortcoming, or a dramatic simplification, depending on your needs. I find that it's often the latter, since it enforces a one-way information flow.
I looked at this, and isn't it inside-out though? Given that code/data equivalence being the fundamental thing that makes a lisp a lisp, wouldn't the lisp way to go about it be, to write a whole-program-transformation macro that can replace certain function calls with specified equivalents? That way you can write code just like normal, but have it do completely different things based on the context. The idea of being able to do this is what made me consider a lisp in the first place. Yet I don't see much evidence of people working in that direction.
There's a lot of stuff packed in to your comment here.
The code/data equivalence isn't necessary to make this data-first approach work. Consider JavaScript, where it's quite popular to power complex processes with simple JSON (maybe + extensions, like functions). The approach also works well in typed functional languages, albeit with greater declaration burden. Moreover, it would even work well in languages like Java, if you're willing to buck the XML trend.
That said, data being a subset of code (which is also implied by equivalence) is what makes this technique much more popular in Lisps (and JavaScript). Being able to easily move forms between evaluation, quoting, interpretation, etc is critical to evolving the right approach.
Regarding whole program transforms: They don't compose, at least not without a substantially more sophisticated macro system (like Racket's) or evaluator (see work on "extensible interpreters"). Check out Racket's rich literature on macros if you want to see the state of the art here.
Lastly, changing behavior based on context can be accomplished by many techniques including interpretation, dynamically scoped binding, etc. Read about Common Lisp's condition system, or Eff's "Algebraic Effects and Handlers" to see the state of the art of the dynamic substitution of effects. However, each new context needs to carefully reason about the interleaving of effects, since time is linear.
If you want more flexibility with time, you have to turn it in to space. That's what happens when you take a step-by-step procedure and turn it in to data you can interpret. Consider code that prints a lot of stuff. Instead, you can return a lazy sequence of strings and then at the very end do something like `(apply str parts)` to concatenate it all together. Now that each sub-printer doesn't have side effects, you can re-order and refactor the calls without worrying about messing up the printed output. You've decoupled the evaluation timeline from the linear order of the output.
Not that you should necessarily use it as a production language, but I'd suggest checking Racket if you want to see dynamic typing done right. `#lang racket` doesn't have static types, but it has dynamically checked contracts with a blame feature that pinpoints at the part of the program that is responsible for a contract violation. `#lang typed/racket` has a more traditional type system (programs are checked before they even run), but the type checker is implemented entirely as a series of Racket macros, and the resulting program integrates seamlessly with untyped Racket code. Regarding refactoring, DrRacket does a pretty good job of identifying dependencies between defined symbols, even across modules. It's true that more popular dynamic languages (JavaScript, Python, etc.) are bad at tracking useful static information, but you shouldn't generalize from this to all dynamic languages.
Excellent, definitely going to read this. I would like to see this for Haskell, though. And some Lisp idioms (from On Lisp, for example) could be thrown in for a good measure, too.
Whilst I agree in general with the sentiment, it really doesn't help if you start with a misunderstanding of the command pattern. The essential property of command is that it's data that can be executed. Implicitly, it's possible to do other things with them: log them, undo them and so on. A better model would be a record that implements IFn.
I think the Visitor example is a mistake. Couldn't you replicate the Clojure version in Java by having a "Formatter" singleton that has a function for each of the different item type/format type combinations?
I don't think so, the whole point is that Clojure will select the correct function dynamically (that's what multiple dispatch is about). In all cases you need 4 different functions (for each type/format combinations).
63 comments
[ 3.0 ms ] story [ 122 ms ] threadOne advantage of the Java is that the interfaces are named, type-checked documentation. Yes it's more verbose, but a maintainer of that code can understand exactly the intention just from the interface names, without having to decipher it line-by-line.
For the Clojure code, after you've added enough comments to explain what you're doing, so that the code can be reasonably maintained, then it's the same size as the Java code, but without the type safety.
Granted for some of the simpler patterns like strategy, Java's verbosity just gets in the way, though much of that has been simplified in Java 8. FWIW I prefer Scala over both in all cases. But if Scala didn't exist, I'd prefer Java for the enterprise use case, and Clojure for the small applet use case.
Switched to Scala and SQL, and code is almost as compact but typed relations are far easier to reason about; if a relation changes then you change the core type and follow the compile errors. Once it compiles, it usually works. We've not missed Clojure in the least.
Scala (and even Java) simply feels more structured, with relations between types, the resulting app more "designed", which seems necessary at the enterprise level. In my 2 years, Clojure constantly felt like just hacking.
Would love to see some examples of Clojure being used well at an enterprise level so I could understand better how to do so, but CircleCI's recent flopping between core.typed and prismatic just makes me think it's a square peg in a round hole.
Isn't that what tests are for?
(0) Imprecise, because of the holes in Java's static safety. You can establish a convention that statically unsafe features are verboten, but it will remain just that - a convention.
(1) Verbose, because Java can't express even the simplest ideas without using at least 5 classes: Finite case analysis? Visitors. Higher-order functions? Strategies. Persistent data structures? Memento. Things don't get better if we move to a higher architectural level: Embedded DSLs? You need a separate class for each production rule in your abstract syntax. Type-safe database queries? Java has no structurally typed records, so you need to make a custom POJO class for every query that returns different columns.
These things are incredibly unpleasant to deal with, and while it might be true that "unpleasant work builds character", I don't think it should come as a surprise that some people say "enough is enough", and seek alternatives that minimize or at least reduce their pain (like dynamically checked conformance to a schema), even at the cost of sacrificing some type safety (dynamic checks can't enforce higher-order invariants).
However, I don't think Scala supports your argument very well. Its type system is:
(0) Redundant: When should I use an abstract class or a trait? When should I provide extended functionality using a subclass or an implicit? What are the relative benefits of poor man's ML-style type abstraction (with path-dependent types) vs. poor man's type classes (with implicits)?
(1) Inconsistent: Why are closed sums [(generalized) algebraic data types] implemented in terms of an open-ended inheritance mechanism? This makes `match` exhaustiveness checks unnecessarily more complicated, and hiliarity ensues when the implementors get it wrong.
(2) Unhelpful: Scala's type inference is local and unidirectional, from arguments to results, which limits the usefulness of using types for exploratory programming. GHC Haskell, which has all sorts of crazy type system extensions (some even more ambitious than Scala's), does a much better job of globally inferring types.
So, in spite of my general preference for statically typed languages, I find Clojure easier to live with than Scala, simply because Scala has so many warts.
Almost all of that is due to the intentional choice of running on the JVM with high interop compatibility. Choose ML-family or Haskell, and you don't have Java baggage.
But I'd take types with class-vs-trait design choices, less-than-ideal match exhaustiveness checks, and type inference incompleteness any day over a language lacking all static typing features.
Here are some counterexamples:
(0) Clojure is compatible with Java, but has less warts than Scala. It is not statically typed, but it is internally more unified and consistent. With some exceptions, you only get to see ugly compatibility compromises when you need to interact with Java code.
(1) Rust is compatible with C - in fact, it interoperates with other languages that have C FFIs by simply pretending to be C. Nevertheless, outside of specifically designated `unsafe` blocks, which idiomatic Rust only uses sparingly, most code is written in a "modern C++ meets ML" style.
> But I'd take types with class-vs-trait design choices, less-than-ideal match exhaustiveness checks, and type inference incompleteness any day over a language lacking all static typing features.
Static typing is a useful feature, but I think it's narrow-minded to use it as the only criterion for judging a language's merits. For instance, Clojure's macros are objectively nicer than Scala's, even if both pale in comparison to Racket's. Writing multimethods in Clojure is straightforward once you know the syntax, whereas emulating multiparameter type classes in Scala can be confusing unless you know the specifics of how implicit parameters are resolved. (This is, ironically, a direct consequence of Scala's richer type system.)
You can't do that in Clojure.
Real world examples: Play and Akka are two large projects that include Java development in their primary use cases.
Two-way interop is the cause for a huge number of less than ideal decisions: different Array equality than Seq, @specialized, universal equality, hash codes, and toString, abstract class vs trait, generic type erasure, etc.
You can call the compromise misguided (and you wouldn't be the first to do so) but that's why.
Types are a "proof" to the compiler that your code will run in a certain way. No missed "edge cases", no fuzziness, just pure comprehensive logic.
Of course, the success of this varies by language and a lot of people have worked to find better and more powerful approaches.
But you should regard any type system as a proof system for runtime behavior.
(I'm not saying that running tests with arbitrary examples is bad; I'm just saying that HUGE classes of potential bugs reliably go away with good static typing.)
From this I derive both the database schema itself as well as implementation details (e.g., check what columns are in a table or see if the current DB schema agrees with the current SQL schema in resources/sql)
For program internal representation I have my own "classes" (hash maps with a few pre/post assertions on key points) and I just use a single db.clj file that needs to be adapted when the SQL representation changes.
I also worked a lot with Java and Hibernate and it just feels verbose to change a simple datatype definition in both the application and the db. In a way, with clojure I get DB / App consistency that I wouldn't get with Java and/or types. I haven't had the chance to use scala, however.
I also worked with MongoDB (when I started with clojure) and I agree, there it is very easy to make mistakes but maybe I would do things now differently if I used it again.
What do you use to interface with SQL on the Scala side? Squeryl?
Another thing that helped was to use underscore_naming for any case-class in Scala that maps directly to a query, so it's easier to identify them.
I'm thinking it's a total potential for exploit on web apps backed by mongodb and any dynamic language.
It's really no different than the OO argument. If one wants OO functionality in lisp it can be implemented with the introduction of a few functions. But by using an OO only language you have just forced that abstraction to the surface leaving you with only one way to write your code even when it's not suitable for the problem space.
So the same could be said for Types. Without Type Inference you're locking yourself in to model/framework that has a list of disadvantages too.
I think the real problem is when people don't recognize they need to account for those problem areas, and they don't take the time to write the abstraction layer when needed and naturally they end up with bad results and feel like it's just hacking, because, well, they are.
Racket supports your argument much better. And Racket is not Lisp: https://news.ycombinator.com/item?id=10763584
The problem outlined was in dealing with data model changes. So the problem as I see is in managing the alignment of parameter types used in code with database field types. For that I've written a schema-template abstraction that manages that relationship. This template engine is about 100 lines of code and for every point the code intersect to/from the data it goes through the engine, looks up the a template and uses that for access/processing (a hook if you will). This allows automation for finding all the code entry spots.
Bottom line is I don't hit these issues because I identified the problem and wrote an abstraction layer to handle it and for the rest of my code I can happily rely on type inference.
Funny, I feel the same about design patterns. So you bothered to find an abstraction in the code, you described it, gave it a name, and yet - you didn't tell the computer - here I am using this abstraction! And you rely on other humans to correctly infer what abstraction you used from reading your code, instead of just telling them.
So what should really be done, design patterns, once identified, should be replaced with proper abstractions in the programming language (mostly functions?) and die. Frankly, most of them are just really fancy names for trivial things.
For instance, the sorting example can be simplified:
A lot less verbose than it's made out to be.I'd love to see patterns struck down as obsolete with Java language updates.
Will userOrdering.reversed() account for this?
The Clojure solutions do sacrifice a lot of safety though, due to dynamic typing (e.g. the visitor pattern, which relies on basically strings to do its dispatching).
This would be excellent for expanding to book-length guide to help people learn functional programming along with the patterns. Thank you Misha for writing this!
Still this is a great post because it teaches you both good Clojure and the design patterns. Since java has closures now you can take a lot of what you learn from Clojure and put it in your well typed and refactorable Java programs.
I'm not sure what you mean with the typo story since most of the functions in your program should be free of side effects you should be able to test them as you write them.
Consider your chain of examples for "Upload something to S3, if X then call some web service, otherwise log and call something else, etc etc etc." You could represent this as a sequence of steps which could be interpreted by a central side-effecting loop. Let's call it an "action plan", in this case for your "upload" operation:
Now you can process this plan centrally: While this is initially a tad more verbose, there's great reasons to do this sort of thing.For example, now have a central place to mock all side effects! You could substitute a different execute method during testing, or automatically log the "action" object that failed.
More importantly, you can develop each of your "plan" functions independently. You can reuse them, re-order them, etc.
You can also do hypotheticals, for example: "Which plan would cause less network round trips?" can be answered before committing to a plan.
You can _record_ plans. Instrument your execute method to record and save plans to a file, then you can just check them right in as test cases.
You can optimize plans. If five different functions all notify the same target of the same thing, your central execution method can filter subsequent calls. Or you could separate your "optimization" from your "interpretation" and you can ask "After accounting for idempotence, which operations will be performed?"
The list goes on and on...
The simplest explanation of the IO monad is that your Haskell application outputs a recipe or "execution plan". This output is side effect free and pure. The runtime then executes your plan, producing those side effects.
That is, you can't communicate between the code that produces the command sequence and the code that interprets it. This is either a shortcoming, or a dramatic simplification, depending on your needs. I find that it's often the latter, since it enforces a one-way information flow.
The code/data equivalence isn't necessary to make this data-first approach work. Consider JavaScript, where it's quite popular to power complex processes with simple JSON (maybe + extensions, like functions). The approach also works well in typed functional languages, albeit with greater declaration burden. Moreover, it would even work well in languages like Java, if you're willing to buck the XML trend.
That said, data being a subset of code (which is also implied by equivalence) is what makes this technique much more popular in Lisps (and JavaScript). Being able to easily move forms between evaluation, quoting, interpretation, etc is critical to evolving the right approach.
Regarding whole program transforms: They don't compose, at least not without a substantially more sophisticated macro system (like Racket's) or evaluator (see work on "extensible interpreters"). Check out Racket's rich literature on macros if you want to see the state of the art here.
Lastly, changing behavior based on context can be accomplished by many techniques including interpretation, dynamically scoped binding, etc. Read about Common Lisp's condition system, or Eff's "Algebraic Effects and Handlers" to see the state of the art of the dynamic substitution of effects. However, each new context needs to carefully reason about the interleaving of effects, since time is linear.
If you want more flexibility with time, you have to turn it in to space. That's what happens when you take a step-by-step procedure and turn it in to data you can interpret. Consider code that prints a lot of stuff. Instead, you can return a lazy sequence of strings and then at the very end do something like `(apply str parts)` to concatenate it all together. Now that each sub-printer doesn't have side effects, you can re-order and refactor the calls without worrying about messing up the printed output. You've decoupled the evaluation timeline from the linear order of the output.
Cursive IDE (IntelliJ based) has no problem renaming functions.
These are all tooling issues, not specific to Clojure nor Dynamic Typing. Give it time & give it a try!
I like Eve.
Subtle.
Paul Graham mentioned Norvig's work in 2002: http://www.paulgraham.com/icad.html