Yeah, Evans' book really changed the way I do a bunch of design. That software is infinitely flexible is generally great, but it can allow complexity to metastasize. Anchoring design to the domain, which tends to be pretty stable, can provide a lot of clarity. It does require taking an anthropological approach to one's users, but I think that pays off in all sorts of ways.
Important to mention, but when I checked out the specification pattern. It's useful to know that it can only be applied if your programming language of choice supports expression trees ( eg. C# )
What is an expression tree? I've done this with arrays of structs in C as well as nested dictionaries in python, and sexps in both lisp and python. I could do it trivially in x86 if needed. Maybe sexps are "expression trees", but it seems almost any language can implement this pattern. It was much nicer with data literals than without, but data literals aren't a prerequisite.
Expression trees are a .NET capability to compile a series of expressions during runtime. It's not required for this at all; it's just an optimization. It allows the complex tree of Specification objects to be compiled down into a simple delegate function, rather than evaluating by walking the tree.
That's just an optimization layer, though. I actually have a hard time telling this apart from `java.util.function.Predicate` with its default `and`, `or`, and `negate` methods...
I can certainly understand using something like this when your users are building rules, because you don't know what they want to build so you just provide the base implementations and the operators. But if engineers are implementing all the logic, it seems like the criticisms called out in Wikipedia regarding obfuscation and re-implementation of basic operators would apply.
A predicate is really just a test that accepts an input and produces true or false. The expression only becomes necessary when you want to translate the predicate somehow (e.g. to SQL).
That's not true. You don't need expression trees (or equivalent), you can walk and interpret the specification directly. Expression trees are just a trick to compile this as code. But you can do that in any language that can compile code at runtime (of which there are, unfortunately, few).
From Common Lisp's perspective, the example given is just:
You can walk that tree recursively as-is, calling appropriate operators and substituting correct values.
Or, you can wrap it in a (lambda (divident payout-ratio pe-ratio price) ...) and pass to (compile ...), and get a piece of executable code. With perhaps a line or two of book-keeping (that can be added by your code, and not necessarily typed in by the user), you can make the rule declare which variables it cares about. So a full example:
There's a lot of interesting things you can build on top of these sorts of rules engines, like optimizations. I had to deal with a system that allowed users to enter arbitrary boolean expressions and we had to use them to filter out records from a very large stream of data, and optimizing for speed was very important. Thee's the obvious changes, like flattening out deeply-nested expressions as much as possible. You can sort sub-clauses in an AndSpecification so that the ones most likely to be false are first, so that you are more likely to short-circuit and cut down on evaluation time (and you sort in the reverse order for the OrSpecification). But you can actually do some more advanced optimizations. In my case, we knew that for a given expression provided by users the overwhelming majority of records would not match. So for a given expression A we would derive a simpler formula B such that A => B (and thus not B => not A) where B was cheaper to evaluate, then we would evaluate the expression as B and A. If B was false it means A would be false anyway, so we could short-circuit and skip the more expensive test.
Production rules systems in the 1970s (when they were in vogue for "expert systems") did not use indexes and generally had poor execution performance compared to modern "business rules" engine.
Production rules are related to other logic-based methods of programming such as "logic programming" (which tends to progress backwards from a desired consequence to prove it is satisfied) or "theorem provers" that could demonstrate that a section of code satisfies certain invariants.
Tangent re: similar idea applicable to speedup old school minimax game search
> You can sort sub-clauses in an AndSpecification so that the ones most likely to be false are first, so that you are more likely to short-circuit and cut down on evaluation time (and you sort in the reverse order for the OrSpecification).
Similar trick also works in context of Alpha beta pruning when minimax searching through a game tree: at each node various moves are available, getting tighter Alpha beta bounds for pruning earlier in the search gives a massive speedup, getting tighter bounds corresponds to selecting a relatively strong move to investigate first. So if you have some heuristic to guess what a good move is for the current player and use that heuristic to define the order that moves are attempted, that can give a large speedup.
Unlike some other situations with heuristics (e.g. a*, where heuristic estimating distance to goal must always be a lower bound to true difference to goal in order to guarantee that the solution obtained by the search is optimal) , there's no requirement that the heuristic used to order moves during search have any guaranteed relationship to the true order -- so that frees you to use wacky empirical heuristics -- e.g. fitting a statistical machine learning model to suggest the search order based on features that can be defined and quickly evaluated at each search node.
When I played around with this idea to accelerate minimax search it generated a fun little project to instrument the search with metrics to collect training data, fit a simple model to predict a strong move (i used random forest) and then make the model evaluation run really fast at predict time: the model was fitted offline before search so I could encode the fitted model -- a bunch of decision trees -- into C code (lots of gotos!) that was then included into the game search code
As soon as I saw "public class PriceSpecification" I got a creepy feeling.
Why shouldn't the specifications be more purely data, like
{field:"price", operator:">", level: 150 }
Then you DON'T need to write new classes for DividendYieldSpecification and PeRatioSpecification - you just need to make sure those fields are available in the data, or write support to derive them when they are requested.
Things would work both ways, of course. For me the difference is a) the extent to which I want to make the domain model explicit, and b) how much I value build-time elimination of bugs vs run-time flexibility.
Data-driven approaches leave a lot implicit, meaning that it can be hard to answer the question, "What's really possible here?" And they can only be verified to the extent you have all the data, which usually means a lot of checks are deferred to run time.
So happy I did not have to work with Java for a long time. The amount of boilerplate and cognitive overhead is insane!
The factory was really the cherry on the cake...
24 comments
[ 4.4 ms ] story [ 63.4 ms ] threadStill useful to know the "pattern" regardless of OOP cruft, Javascript prototypes, or Haskell data types.
Orm's support expression trees ( eg, nhibernate,...)
So the specifications pattern for my use-case require expression trees.
There are workarounds ofc, but translating specifications to Sql is not the desired use-case of the pattern.
Specifications make it easy to have a 100% equal business filter ( that translate it to Sql through the ORM) and making it testable.
- Specification.AssertAll(resultset, "not all elements are valid according to this specification");
( Using pseudocode )
I can certainly understand using something like this when your users are building rules, because you don't know what they want to build so you just provide the base implementations and the operators. But if engineers are implementing all the logic, it seems like the criticisms called out in Wikipedia regarding obfuscation and re-implementation of basic operators would apply.
https://en.wikipedia.org/wiki/Specification_pattern#Criticis...
From Common Lisp's perspective, the example given is just:
You can walk that tree recursively as-is, calling appropriate operators and substituting correct values.Or, you can wrap it in a (lambda (divident payout-ratio pe-ratio price) ...) and pass to (compile ...), and get a piece of executable code. With perhaps a line or two of book-keeping (that can be added by your code, and not necessarily typed in by the user), you can make the rule declare which variables it cares about. So a full example:
Which, for me, is 90% of the use-cases
Real business rules engines use something like
https://en.wikipedia.org/wiki/Rete_algorithm
Production rules systems in the 1970s (when they were in vogue for "expert systems") did not use indexes and generally had poor execution performance compared to modern "business rules" engine.
Production rules are related to other logic-based methods of programming such as "logic programming" (which tends to progress backwards from a desired consequence to prove it is satisfied) or "theorem provers" that could demonstrate that a section of code satisfies certain invariants.
> You can sort sub-clauses in an AndSpecification so that the ones most likely to be false are first, so that you are more likely to short-circuit and cut down on evaluation time (and you sort in the reverse order for the OrSpecification).
Similar trick also works in context of Alpha beta pruning when minimax searching through a game tree: at each node various moves are available, getting tighter Alpha beta bounds for pruning earlier in the search gives a massive speedup, getting tighter bounds corresponds to selecting a relatively strong move to investigate first. So if you have some heuristic to guess what a good move is for the current player and use that heuristic to define the order that moves are attempted, that can give a large speedup.
Unlike some other situations with heuristics (e.g. a*, where heuristic estimating distance to goal must always be a lower bound to true difference to goal in order to guarantee that the solution obtained by the search is optimal) , there's no requirement that the heuristic used to order moves during search have any guaranteed relationship to the true order -- so that frees you to use wacky empirical heuristics -- e.g. fitting a statistical machine learning model to suggest the search order based on features that can be defined and quickly evaluated at each search node.
When I played around with this idea to accelerate minimax search it generated a fun little project to instrument the search with metrics to collect training data, fit a simple model to predict a strong move (i used random forest) and then make the model evaluation run really fast at predict time: the model was fitted offline before search so I could encode the fitted model -- a bunch of decision trees -- into C code (lots of gotos!) that was then included into the game search code
Why shouldn't the specifications be more purely data, like {field:"price", operator:">", level: 150 }
Then you DON'T need to write new classes for DividendYieldSpecification and PeRatioSpecification - you just need to make sure those fields are available in the data, or write support to derive them when they are requested.
Data-driven approaches leave a lot implicit, meaning that it can be hard to answer the question, "What's really possible here?" And they can only be verified to the extent you have all the data, which usually means a lot of checks are deferred to run time.
If you see this pattern at work, just run!