26 comments

[ 2.3 ms ] story [ 61.9 ms ] thread
Maybe I'm just missing something, but the "domain expert" that is described here is just... a function? The big win in Clojure is apparently using code instead of types?

  (defn apply-shipping-rules [order]
    (cond-> order
      (and (= :premium (:customer-type order))
           (> (:order-total order) 100))
      (assoc :shipping-cost 0)))
Yes, the point of the article is that people should do this (as is common in Clojure) rather than try and encode the rules in the type system (be it as a class hierarchy or a sum type).
The "domain expert" is the business-person who is, it is suggested, more capable of reading and comprehending the Clojure code than the Haskell code.

Since there is an equivalence between types and propositions, the Clojure program also models a "type", in the sense that the (valid) inputs to the program are obviously constrained by what the program can (successfully) process. One ought, in principle, to be able to transform between the two, and generate (parts of) one from the other.

We do a limited form of this when we do type inference. There are also (more limited) cases where we can generate code from type signatures.

I think op's point is that the Clojure code, which lays the system out as a process with a series of decision points, is closer to the mental model of the domain expert than the Haskell code which models it as a set of types. This seems plausible to me, although it's obviously subjective (not all domain experts are alike!).

The secondary point is that the Clojure system may be more malleable - if you want to add a new state, you just directly add some code to handle that state at the appropriate points in the process. The friction here is indeed lower. But this does give up some safety in cases where you have failed to grasp how the system works; a type system is more likely to complain if your change introduces an inconsistency. The cost of that safety is that you have two representations of how the system works: the types and the logic, and you can't experiment with different logic in a REPL-like environment until you have fully satisfied the type-checker. Obviously a smarter system might allow the type-checker to be overridden in such cases (on a per-REPL-session basis, rather than by further editing the code) but I'm not aware of any systems that actually do this.

I agree with the article, but I will note that we have a great tool for this problem that is (or at least can be) "statically typed": the relational model. Databases are precisely what you want for this sort of problem (even better if it's Datalog and you can encode rules that derive relevant information).

Most mainstream languages are very poorly equipped to do relational modeling. ORMs are a disaster (object-relational mismatch) and you don't necessarily need an actual database running in the background.

Clojure's approach is superior to the class hierarchy or sum type solution for this sort of very loose business domain modelling, for the reasons stated in the article, but it's also a local optima, and so is the "fat struct" solution (which is the statically typed equivalent). Even entity component systems are but a shadow of the relational model.

I mostly agree. I have quipped once that I write "spaghetti and meatballs" code. The meatballs are the core domain objects, explicitly typed. The spaghetti is the business rules, untyped. With experience you get a good intuition where to draw the line. But the untyped code needs extensive testing.

Where I disagree with the article is on refactoring. It's identically hard both ways. Migrating to new business rules while simultaneously running the old and new system is the hard part. I don't find static typing helps or hurts me in particular. Compiler warnings are useful, but my unit tests catch the dynamic parts as well. Either way a lot breaks and often needs temporary scaffolding between the versions.

I feel like this article is missing the point by a country mile. FP proponents very much know that requirements can change and wreak havoc with their type systems forcing them to change large numbers of likes. What the author is missing is that this is be welcomed and vastly preferable to the situation we find ourselves in with Python codebases where those lines still need updating but the code will happily run incorrectly if you fail to find them all. Switching off the alarm doesn’t stop the fire spreading.
A mereological nihilist will never have anything but primitive types.

Everything more complex than those building block aren't in reality a Type.

Reality doesn't consiste of: X type made up of these primitives and other defined sub-types and let's hide the primitives as far down as we can.

It's instead primitives arranged X wise.

Or mapped a little better to programming terminology: A Schema.

It's about having the mental model that complex types can be useful as an abstraction but they aren't real and aren't worth fighting for or defending.

Types are for devs, devs aren't for types.

It's supposed to break when your assumptions break. That's the whole point.
I’ve watched a few good talks by people that have coded this kind of thing at the maximum level of scale and complexity, such as Amazon’s checkout system.

The endgame of this problem always turns into some sort of “log of events” with loosely coupled subscribers.

A single state machine suffers from a combinatorial explosion of states as it has to handle every corner case, combinations of every scenario, etc…

What if a single shopping basket contains both a digital good and a physically shipped one? What if some items are shipped separately and/or delayed? Etc…

Instead the business rules are encoded into smaller state machines that listen to events on the log and pay attention only to relevant events. This avoids much of the complexity and allows the OOP types to remain relatively clean and stable over time.

Now the “digital goods” shipping handler can simply listen to events where “delivery=authorized” and “type=digital”, allowing it to ignore how the payment was authorised (or just store credit!) and ignore anything with physical shipping constraints.

It then writes an event that marks that line item in the shopping cart as “delivered”, allowing partial cancellations later, etc…

while this does bring up a valid point,

"classic" sql databases are still safer for many things then mongodb.

it is easier to do away with types and constraints, but in many cases they do end up being important safeguards

I think that many of the problems of this are solved by abstract types and good module interfaces to interacting with those types.

If the business logic changes, the internal type representation can be modified or a new module that fills the same signature but uses a different internal type can be written.

Modularity matters most. If you have to do massive refactoring of code because of a change in type representation, there’s an issue with the modularity of your design. Good modularity prevents refactoring from impacting the rest of the system unless you need to change your module — that’s something that’s true even if you have no static typing.

Every year someone figures out that encoding complex logic in type systems leads to complex type systems. YES. Types will not simplify your logic, they mean to represent it, in all its glorious complexity, in a way that is checkable automatically, and will break and require refectoring when you violate the previous invariants.

This is types working.

> The trap is that both OOP hierarchies and FP "make illegal states unrepresentable" create premature crystallization of domain understanding into rigid technical models. When domains evolve (and they always do), this coupling demands expensive refactoring.

At least when you refactor your types, the compiler is going to pinpoint every line of code where you now have missing pattern checks, unhandled nulls, not enough parameters, type mismatches etc.

I find refactoring in languages like Python/JavaScript/PHP terrifying because of the lack of this and it makes me much less likely to refactor.

Even with a test suite (which you should have even when using types), it's not going to exhaustively catch problems the type system could catch (maybe you can trudge through several null errors your tests triggered but there could be many more lurking), working backwards to figure out what caused each runtime test error is ad-hoc and draining (like tracing back where a variable value came and why it was unexpectedly null), and having to write + refactor extra tests to make up for the lack of types is a maintenance burden.

Also, most test suites I see do not contain type related tests like sending the wrong types to function parameters because it's so tedious and verbose to do this for every function and parameter, which is a massive test coverage hole. This is especially true for nested data structures that contain a mixture of types, arrays, and optional fields.

I feel like I'm never going to understand how some people are happy with a test suite and figuring out runtime errors over a magic tool that says "without even running any parameters through this function, this line can have an unhandled null error you should fix". How could you not want that and the peace of mind that comes with it?

Godel's incompleteness theorem indicates illegal states as unrepresentable is impossible at a fundamental level.
This is a misinterpretation equivalent to saying "The halting problem indicates it is impossible to say if this loop terminates: while true: break "

It is true that it is impossible in general, but that says nothing about whether or not it is possible in almost every useful case

I sometimes wonder how big the category of "programs that break (or rather, cause) the halting problem" really is.

If we carve out "programs that run themselves on themselves and then do the opposite", what remains?

You both rather miss the point that the op ref talks of making illegal states unrepresentable. An objective that either depends on only handling simplistic data or ... most likely is impossible.
Basically, this article is saying "modelling your domain logic in a type system is hard" and suggests wimping out.

Just get good!

Your job as a programmer is to think through the domain logic, find the right abstractions to represent that logic, and then tell the computer to enforce those abstractions for you. Punting business logic to "it's all messy, type systems can't be used to model it" is just saying "I cant take the time to find the right abstractions, sorry"

I'm not sure, but a general feeling I've been having on this topic is that advanced type systems are like a separate higher-order language atop the grammar that actually does things. And by encoding more and more responsibility in the type system, you're just pushing your business logic to a different language. The same propensity for human error exists there as well.

Which is why I'm partial to the data-oriented approach that clojure seems to promote. If at its core, programming is just Data And Its Various Transformations, you can encode your data in simple structs or lists, and then your transformations encode the rules and logic. In this model you're still making illegal states unrepresentable, it's just being done in the base programming language instead of the type system. Having a type system that can verify that things aren't null, a string isn't a number etc is a benefit of course. But I don't see much difference in putting that logic in the base language or the higher order type system language, except the base language is more expressive and flexible.

I'm not sure if this is correct of course, I don't have enough experience to really be certain. But it does sound reasonable and some much smarter and more experienced developers seem to think so as well. But I'm open to having my mind changed.

I work on a Haskell codebase that is close to or over a million lines of code. It’s a bear to work with for a number of reasons but inflexibility to changes in business logic isn’t one of them. People need to stop waving their hands and running around claiming that the sky is falling.

Try writing some Haskell for a while.

Explore the compiler output. If you can ignore the language runtime it’s pretty reasonable. Good, even. There are a lot of cases where GHC can optimize away a lot of things you might think are unreasonable by reading the surface language.

There’s states that are technologically incorrect, like dangling data because of a broken foreign reference - and states that are incorrect for the business.

Only the former should be represented by and constrained by the type system.

I have the feeling that people who don't like compilers are people don't like it when they're wrong, as compilers point that out more often. The problem is they don't have enough patience to learn the language, not just the syntax, but how to express solutions in that language. At that point, the compiler has your back and you can do fearless refactoring.
> When domains evolve (and they always do), this coupling demands expensive refactoring.

What is the evidence for that?

Admitting it's true, how expensive is it really?

regardless of the style, when business domain changes a lot the most expensive part in general are...tests. not apis, not types. But tests.

I agree with the article. I was trying to statically type event calculus in OCaml and didn't found any sustainable way to implement it.

In order to even get a piece of system tracking the amount of boilerplate was enormous.

It won't help in delivering complex logic and might even be a costly mistake. E.g. if exploratory attempt costs 3 days of refactoring to implement and then it uncovers unexpected behavior those 3 days on type adjustment is just time lost.

Today seeing many complicated systems in both non-static typed and strongly typed languages I have opinion that's only matter of preference. Dragons live everywhere.