This is an awesome pattern. Seems like it could easily be added as a language feature to Java and C#. Add an alternative to the "virtual" keyword called "extensible", and an alternative to "override" called "extend".
Define an "inner" interface, set up the "outer" object with a default implementation (available to caller so they can wrap it rather than replace it entirely) and the "inner" calls simply delegate to it. Make all classes involved sealed/final.
That’s basically the template method pattern and I remember some Windows API using it (maybe GDI) where the protected overridable method was called DrawInternal and the public Draw method was non-virtual.
In fact, depending on whether the delegated methods depend on (protected) super-class state, this is probably better served with composition than inheritance.
Whenever I’ve worked, I was always advocating to use subclassing as a last resort solution. It’s the most intimate and brittle relationship you can have between your classes.
Unfortunately the majority of OOP books don’t emphasize this, so beginners try to do everything by subclassing, as if that was what OOP is about...
I've hardly ever found a good reason to inherit functionality - plain old data objects yes, but I always favor aggregation over inheritance.
When it comes to inheriting a base object to reuse cross cutting concerns, not only do you run into the fragile base class problem - there is usually a better way.
I personally prefer aspect oriented programming when possible.
I cannot recall an instance where virtual methods were definitely my best option. I have used them sparingly, but I suspect further review would reveal a perfectly good alternative. I have always considered them one of the least readable constructs.
I've read about this (public non virtual interfaces calling virtual functions that are meant to be overridden) for C++ a long time ago: http://www.gotw.ca/publications/mill18.htm
Well, it's the Template Method pattern from the GoF book and, like most design patterns, probably independently discovered by most people writing OO code.
It's interesting, but I can see it leading to much frustration when, as an overrider, you don't want it to run that before/after code.
The thing is, the developer doing the overriding generally has more information about the final system than the original - after all, the code is more specific. So taking away their power is generally problematic.
I could see it being useful if it was optional. Something like:
Another perspective: the developer doing the overriding doesn't in fact have more information; they don't know the future plans of the developer of the class they're overriding, and they don't know about the usage models of other descendants of the class and how they'll affect evolution into the future. The more "power" they use, the more likely they'll be left behind on a version bump.
Implementation inheritance means accepting an extremely tight coupling to the ancestor, and overriding is the main mechanism of coupling. Coupling means correlated changes, and when the developers of the ancestor and descendant are in different organizations, it means versioning risk. This is an argument for very careful selection of methods available for overriding, and strictly limiting the choices of the descendant, funneling them into designed-for usage modes.
Versioning is single biggest reason why C# elected to make methods non-virtual by default, where Java makes them virtual by default. Breaking all the clients of your code because of an implementation design change isn't very desireable, thus limiting the surface area for coupling is usually a good idea.
If ancestors and descendants are in the same repository so upgrades of definition site and use site can be coordinated, or the probability of design change in the ancestor is low, then these concerns are more relaxed. But they're valid in many situations.
The library developers can provide that information without enforcing it. For example, in Python any method starting with an underscore is considered private and liable to be changed or removed, but the language doesn't treat them in a special way.
Enforcement mechanisms don't have anything to do with passing information, but with removing liability (not necessarily legal - might only be social) from the original developers. They're effectively an EULA.
>It's interesting, but I can see it leading to much frustration when, as an overrider, you don't want it to run that before/after code.
It's about ensuring the invariants the parent class needs -- not giving subclasses freedom to do whatever they want is meant as a feature.
This allows library designers to design more future proof APIs (since they control more invariants of how they're gonna be extended), and to prevent users of their libs from shooting themselves in the foot.
And it requires users that want to override it (a common issue with final stuff on Android) to use native extensions to patch the memory of the parent class at runtime (or use other forms of reflection).
It is because often you need to do stuff that the system doesn’t allow on older versions.
As updates are rarely ever applied, you have to deal with 10 year old versions frequently.
So you use these tricks to get the same functionality on old versions.
Sometimes you also need to do the same on modern versions, because Google releases UI libraries where everything is final and internal, and so you either have to fork a library with hundredthousands of LOC, or you gotta use reflection.
As on android every library is bundled statically, there’s no risk of anything breaking either.
>(It is because often you need to do stuff that the system doesn’t allow on older versions.
As updates are rarely ever applied, you have to deal with 10 year old versions frequently.
That's not a programming problem that this capability allows us to solve though.
That's a management/market/etc problem, that necessitates that we use this method as a kludge to bypass it.
The same is true for the features we're talking about, though. The relationship between the library developers and its users is a social problem, not a programming problem. There's nothing technical that prevents you from changing a non-sealed/virtual/etc API, it's just that you don't want to have people who didn't read or ignored the documentation complaining that their code broke.
Common Lisp’s object system (CLOS) allows you to specify on a method by method basis whether or not the most-specific method has priority or the least-specific one does. Generally people just use the default behavior—least specific first—but this post makes the alternative sound interesting.
It also has daemon methods, which solve the problem of making sure the superclass gets to do something before, or after, the subclass method runs, without the subclass method having to request this explicitly.
Attempting to google variants on the them of "CLOS daemon methods" isn't bringing up anything recognisable as what you're describing - happen to know how to find a link?
(I love it when my attempt to RTFM is defeated by my being too stupid to find the FM in the first place ;)
The FM for Common Lisp is called Hyperspec, or CLHS. Just append "CLHS" to your search query.
The parent is probably talking about method combinations - in particular, the method combination used by default, which supports :before, :after and :around methods.
It's the first time I see the term "daemon method".
I guess the term isn't as common as I thought, but it appears to have been used explicitly in the Flavor system, which was the object system used on the MIT Lisp Machines and which predated CLOS. That must be where I picked it up.
A daemon / demon was a procedure which runs when a slot is accessed / added / deleted / .... My bet is that this name was used in frame languages in the end 70s where there were if-needed, if-added, ... demons for a frame slot. Flavors probably got it from there - though I'm not sure.
But the concept/term of a demon goes back to the end 60s to Hewitt's planner language and probably even earlier.
Later with generic functions in CLOS, I'd guess the term wasn't used, since something like an :after method is no longer conceptually a specific demon procedure attached to a slot via a facet, but a method in a method combination.
Right, sorry, "daemon methods" is the collective term but may not actually appear in the documentation. There are three types, "before", "after", and "around"; there's a brief description at the bottom of this page: http://www.aiai.ed.ac.uk/~jeff/clos-guide.html
> Great systems also have advice. There's no universally accepted name for this feature. Sometimes it's called hooks, or filters, or aspect-oriented programming. As far as I know, Lisp had it first, and it's called advice in Lisp. Advice is a mini-framework that provides before, around, and after hooks by which you can programmatically modify the behavior of some action or function call in the system. Not all advice systems are created equal. The more scope that is given to an advice system — that is, the more reach it has in the system it's advising — the more powerful the parent system will be.
It doesn't sound like such a great alternative to me. The derived class should have maximum flexibility. If I have to struggle with some OOP system where the chain of control is backwards and my derivation gets the control last, I'm just gonna throw my hands up and write my own thing from scratch and not use the inflexible stuff. This end-result is counterproductive, resulting in more work and code duplication.
Or else I will wrap their objects with mine such that mine have new, unrelated methods that do not automatically yield control to those underlying object. My methods will call theirs at any point in the processing that I wish.
Basically, I will get the last word no matter what, so either work with me by giving me a normal OOP system, or I will build detours around you anyway.
Covered in Common Lisp by custom method combinations.
A "method combination" determines, when a generic function call denotes multiple methods, in which order they are called.
There is a default method combination with standard features: :pre/:post/:around auxiliary methods in relation to a primary method. This has the conventional ordering: derived controls dispatch. E.g. an :around method can decide that the primary method won't be executed at all by not calling (call-next-method).
The documentation for the define-method-combination macro is a bit of a challenging read. One of the examples of its "long forms" shows how the standard method combination would be defined with this macro, if it didn't exist. A simple example shows how to arrange for methods to be dispatched one by one until one returns something other than nil.
It seems like the debate boils down to "should subclasses be able to override behaviour in ways the base class doesn't anticipate?".
Pro: sometimes it lets you extend classes in useful ways that the original author didn't anticipate.
Con: it makes it difficult or impossible to reason about invariants in the base class.
Seems like it's a trade-off between hack-ability and robustness.
> Seems like it's a trade-off between hack-ability and robustness.
Precisely. This is where Python's "We're all adults, here" philosophy comes into play: just trust me with the footgun. I'll take responsibility for the risk but dammit, if I'm not going to be pissed when I can't do something I know I need to do.
Maybe it's the distinction between "languages for large teams of mediocre programmers" vs "languages for hackers or small experienced teams"
Yes, I think in C++ at-least you are supposed to protect from this by never having the child directly implement the virtual method.
This means inheritors only need to ensure they implement according to the specified constraints.
I also suspect some of the new C++14 and beyond meta tagging can even enforce a `final`.
class Abstract {
public:
// Even though this function might do nothing but
// redirect, it is a useful abstraction.
void
SomeMethod() {
// Do things which must be done first, if any.
ImplSomeMethod();
// Do things which must be done after, if any.
}
protected:
// Document all expected invariant behavior here.
virtual void
ImplSomeMethod() = 0;
};
Or: OOP theory essentially says that classes should never be "subclassed." As in, the code isn't supposed to evolve that way; you're Doing It Wrong.
In OOP, the only reason for something to be a "subclass" of something else, is that you first had several subclasses (FooA, FooB, FooC) as plain classes, and then noticed that they all obeyed the exact same interface (a hypothetical IFoo) and had common logic, and so you then took the opportunity to factor out the interface + common logic into a common superclass (Foo).
The interface, if separated into an abstract contract (a real IFoo) is reusable by downstream code; but the common logic, 99% of the time, isn't. (It's potentially reusable by people contributing to your library, but more-than-likely they'll have to modify the "shape" of your common logic when they add a new concrete subclass with its own new concerns, even if the contract IFoo would stay the same.)
Given a full prover-level type system, you would be essentially unable to usefully do anything with the superclass Foo once it's been created, other than what the "causally prior" subclasses FooA, FooB, and FooC are already doing with it. Foo is the reificiation of an equivalence class whose members are exactly FooA, FooB, and FooC. A "FooD" shouldn't be able to be defined without changing the definition of Foo.
Assuming you accept this, how does code reuse work?
• Specializing behavior: this is what the delegate pattern (composition!) is for. Rather than SomeLib exposing a base class SomeLib.Foo for subclassing, SomeLib instead gives you a (final, sealed) SomeLib.Foo class which takes a SomeLib.IBar as a delegate, and calls methods on it. You create YourLib.Bar that implements SomeLib.IBar, and pass it to the constructor of a new SomeLib.Foo.
• Extending behavior: this is what plain-old component-wise composition is for. Call it an Adapter, or a Decorator, or a Facade; it's just a new object of a class you've defined, that is implemented by holding onto an instance of the (final, sealed) SomeLib.Foo class. (If you want nice interoperation, the upstream library author should provide a SomeLib.IFoo interface and define their methods to take/return it; and you should have your wrapper class implement SomeLib.IFoo. In less statically-typed languages, this translates to adding a contract method to your object that can be probed for, e.g. Ruby's #to_str.)
I would say you `never never subclass` most of the time its just proves your design is turning into a taxonomy. You really don't need classes for a OOP language, its just simply a convention that developers use that is convenient.
>The cleanliness and "refactorability" of my code improved significantly when I drastically reduced the amount of subclassing I was doing.
>I wouldn't go so far to say "never subclass", but I'd say most of the time, you won't need to subclass.
Yeah, I would agree with that. A lot of OOP mechanisms like inheritance, or design patterns, get overused a lot. I'm not saying it's what you were doing, but I've seen it mostly in code written by newish programmers. It seems that once the concepts of OOP click in their mind, they get excited and eager to put them to work, so they use them where they're not really appropriate. I was definitely guilty of that, way back in the day. But once the novelty wore off, and I realized that I was just creating over-architected messes, I quit doing it.
They're not saying never create subclasses, so much as never start with a base class and then create subclasses from it. The idea presented is that you should instead be starting with the specific classes you need, and if they have common operations, possibly creating a base class out of those.
Wow ... I usually lurk hackernews and don’t comment but I feel offended by this post.
If he started with; say “in javascript” I’d agree completely.
And I know that unless you‘re writing code for others to use (SDK, os ..) you usually don’t need inheretance.
But for generations people overridden methods and no one complained about it... infact it’s a good idea in so many cases and easily any proper OOP class can protect it’s behaviour (see encapsulation ..)
The only impolitness I see here is someone refusing a very valid pattern in a very solid space just because it’s invalid in js (where a lot of things are invalid)
52 comments
[ 2.9 ms ] story [ 89.4 ms ] threadDefine an "inner" interface, set up the "outer" object with a default implementation (available to caller so they can wrap it rather than replace it entirely) and the "inner" calls simply delegate to it. Make all classes involved sealed/final.
The degenerate case is the humble callback.
Unfortunately the majority of OOP books don’t emphasize this, so beginners try to do everything by subclassing, as if that was what OOP is about...
When it comes to inheriting a base object to reuse cross cutting concerns, not only do you run into the fragile base class problem - there is usually a better way.
I personally prefer aspect oriented programming when possible.
The thing is, the developer doing the overriding generally has more information about the final system than the original - after all, the code is more specific. So taking away their power is generally problematic.
I could see it being useful if it was optional. Something like:
Implementation inheritance means accepting an extremely tight coupling to the ancestor, and overriding is the main mechanism of coupling. Coupling means correlated changes, and when the developers of the ancestor and descendant are in different organizations, it means versioning risk. This is an argument for very careful selection of methods available for overriding, and strictly limiting the choices of the descendant, funneling them into designed-for usage modes.
Versioning is single biggest reason why C# elected to make methods non-virtual by default, where Java makes them virtual by default. Breaking all the clients of your code because of an implementation design change isn't very desireable, thus limiting the surface area for coupling is usually a good idea.
If ancestors and descendants are in the same repository so upgrades of definition site and use site can be coordinated, or the probability of design change in the ancestor is low, then these concerns are more relaxed. But they're valid in many situations.
Enforcement mechanisms don't have anything to do with passing information, but with removing liability (not necessarily legal - might only be social) from the original developers. They're effectively an EULA.
It's about ensuring the invariants the parent class needs -- not giving subclasses freedom to do whatever they want is meant as a feature.
This allows library designers to design more future proof APIs (since they control more invariants of how they're gonna be extended), and to prevent users of their libs from shooting themselves in the foot.
Is it because of a design issue in the Android libraries or to try and circumvent some limitations that might have some reasons to exist?
As updates are rarely ever applied, you have to deal with 10 year old versions frequently.
So you use these tricks to get the same functionality on old versions.
Sometimes you also need to do the same on modern versions, because Google releases UI libraries where everything is final and internal, and so you either have to fork a library with hundredthousands of LOC, or you gotta use reflection.
As on android every library is bundled statically, there’s no risk of anything breaking either.
That's not a programming problem that this capability allows us to solve though.
That's a management/market/etc problem, that necessitates that we use this method as a kludge to bypass it.
(I love it when my attempt to RTFM is defeated by my being too stupid to find the FM in the first place ;)
The parent is probably talking about method combinations - in particular, the method combination used by default, which supports :before, :after and :around methods.
It's the first time I see the term "daemon method".
But the concept/term of a demon goes back to the end 60s to Hewitt's planner language and probably even earlier.
Later with generic functions in CLOS, I'd guess the term wasn't used, since something like an :after method is no longer conceptually a specific demon procedure attached to a slot via a facet, but a method in a method combination.
https://franz.com/support/documentation/8.1/doc/flavors.htm#...
> Great systems also have advice. There's no universally accepted name for this feature. Sometimes it's called hooks, or filters, or aspect-oriented programming. As far as I know, Lisp had it first, and it's called advice in Lisp. Advice is a mini-framework that provides before, around, and after hooks by which you can programmatically modify the behavior of some action or function call in the system. Not all advice systems are created equal. The more scope that is given to an advice system — that is, the more reach it has in the system it's advising — the more powerful the parent system will be.
https://en.wikipedia.org/wiki/Advice_(programming)
Or else I will wrap their objects with mine such that mine have new, unrelated methods that do not automatically yield control to those underlying object. My methods will call theirs at any point in the processing that I wish.
Basically, I will get the last word no matter what, so either work with me by giving me a normal OOP system, or I will build detours around you anyway.
E.g. Eric Lippert's comments: https://blogs.msdn.microsoft.com/ericlippert/2004/01/22/why-...
A StackOverflow q&a: https://stackoverflow.com/questions/268251/why-seal-a-class/...
A "method combination" determines, when a generic function call denotes multiple methods, in which order they are called.
There is a default method combination with standard features: :pre/:post/:around auxiliary methods in relation to a primary method. This has the conventional ordering: derived controls dispatch. E.g. an :around method can decide that the primary method won't be executed at all by not calling (call-next-method).
The documentation for the define-method-combination macro is a bit of a challenging read. One of the examples of its "long forms" shows how the standard method combination would be defined with this macro, if it didn't exist. A simple example shows how to arrange for methods to be dispatched one by one until one returns something other than nil.
Pro: sometimes it lets you extend classes in useful ways that the original author didn't anticipate. Con: it makes it difficult or impossible to reason about invariants in the base class.
Seems like it's a trade-off between hack-ability and robustness.
Precisely. This is where Python's "We're all adults, here" philosophy comes into play: just trust me with the footgun. I'll take responsibility for the risk but dammit, if I'm not going to be pissed when I can't do something I know I need to do.
Maybe it's the distinction between "languages for large teams of mediocre programmers" vs "languages for hackers or small experienced teams"
This means inheritors only need to ensure they implement according to the specified constraints.
I also suspect some of the new C++14 and beyond meta tagging can even enforce a `final`.
In OOP, the only reason for something to be a "subclass" of something else, is that you first had several subclasses (FooA, FooB, FooC) as plain classes, and then noticed that they all obeyed the exact same interface (a hypothetical IFoo) and had common logic, and so you then took the opportunity to factor out the interface + common logic into a common superclass (Foo).
The interface, if separated into an abstract contract (a real IFoo) is reusable by downstream code; but the common logic, 99% of the time, isn't. (It's potentially reusable by people contributing to your library, but more-than-likely they'll have to modify the "shape" of your common logic when they add a new concrete subclass with its own new concerns, even if the contract IFoo would stay the same.)
Given a full prover-level type system, you would be essentially unable to usefully do anything with the superclass Foo once it's been created, other than what the "causally prior" subclasses FooA, FooB, and FooC are already doing with it. Foo is the reificiation of an equivalence class whose members are exactly FooA, FooB, and FooC. A "FooD" shouldn't be able to be defined without changing the definition of Foo.
Assuming you accept this, how does code reuse work?
• Specializing behavior: this is what the delegate pattern (composition!) is for. Rather than SomeLib exposing a base class SomeLib.Foo for subclassing, SomeLib instead gives you a (final, sealed) SomeLib.Foo class which takes a SomeLib.IBar as a delegate, and calls methods on it. You create YourLib.Bar that implements SomeLib.IBar, and pass it to the constructor of a new SomeLib.Foo.
• Extending behavior: this is what plain-old component-wise composition is for. Call it an Adapter, or a Decorator, or a Facade; it's just a new object of a class you've defined, that is implemented by holding onto an instance of the (final, sealed) SomeLib.Foo class. (If you want nice interoperation, the upstream library author should provide a SomeLib.IFoo interface and define their methods to take/return it; and you should have your wrapper class implement SomeLib.IFoo. In less statically-typed languages, this translates to adding a contract method to your object that can be probed for, e.g. Ruby's #to_str.)
Got a reference for that? I've heard "prefer composition over inheritance" (GoF) but never have I heard that classes should never be subclassed.
I wouldn't go so far to say "never subclass", but I'd say most of the time, you won't need to subclass.
>I wouldn't go so far to say "never subclass", but I'd say most of the time, you won't need to subclass.
Yeah, I would agree with that. A lot of OOP mechanisms like inheritance, or design patterns, get overused a lot. I'm not saying it's what you were doing, but I've seen it mostly in code written by newish programmers. It seems that once the concepts of OOP click in their mind, they get excited and eager to put them to work, so they use them where they're not really appropriate. I was definitely guilty of that, way back in the day. But once the novelty wore off, and I realized that I was just creating over-architected messes, I quit doing it.
It helps you realize that programming is simply about data, and how you manipulate it.
Design patterns, OOP, conventions, etc are all just secondary tools that sometimes help you do that.
Well, whatever they said, they started off with "OOP theory essentially says that..."
So, if it's OO theory, I'd appreciate if someone could provide me a reference that shows that it's OO theory, and not just someone's personal opinion.
If he started with; say “in javascript” I’d agree completely.
And I know that unless you‘re writing code for others to use (SDK, os ..) you usually don’t need inheretance.
But for generations people overridden methods and no one complained about it... infact it’s a good idea in so many cases and easily any proper OOP class can protect it’s behaviour (see encapsulation ..)
The only impolitness I see here is someone refusing a very valid pattern in a very solid space just because it’s invalid in js (where a lot of things are invalid)