Yo dawg, I heard you like dependencies, so we created a dependency injector dependency for you to inject so you can inject your dependencies.
Sorry couldn't resist. The additional dependency is the biggest problem I have with OP's approach though. Now none of my classes are easily portable across projects, and this DI framework is going to spread around everywhere.
All you've done is push the burden of constructing the dependency to whomever is calling an passing that dependency to __init__
Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root".
Now that all construction is taking place at once, the order matters as you can't pass a dependency to its dependent until the dependency has been constructed. But it may have its own dependencies. So now there is a topological sorting problem.
Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
I know, but just thinking though what you're doing and building a program for right now, rather than building for a future that may never happen is probably going to deliver more success.
I like constructor based dependency injection personally. It's simple and I don't really see a stack of value obsessing about the possibility of code reuse.
Curiously i've actually seen people use @Lazy in Spring projects to allow for circular dependencies, to deal with situations where the services or their dependencies aren't structured like a neat tree with leafs, but rather some interdependent graph with cycles.
Honestly, in practice it worked and wasn't too bad to work with, which was interesting to behold - in practice everyone talks about how circular dependencies are bad for a variety or reasons (trying to print or process data and ending up with endless loops comes to mind), but then there just was that system that was chugging along without a care in the world.
If you have so many top-level components or change your dependency graph that often that sorting becomes an issue you want to automate, you might want to reconsider your architecture.
Of course the order matters - but you know what? It's good that this is explicit in the sourcecode and even checked by the compiler.
Since now, I can go to the project's root and read step by step how the application is architected and build up and which are core dependencies and which are not.
With dependency injection containers, everything is shuffled around and I'm now at the mercy of a library to tell me what I want to know.
>you can't pass a dependency to its dependent until the dependency has been constructed.
> But it may have its own dependencies. So now there is a topological sorting problem.
I guess I just don't see the problem.
That solves itself naturally through normal programming.
Some function takes A as argument? Well I obviously make that A first.
A requires B to setup? Well obviously I make that first.
I don't need to "sort" anything, it just follows naturally from the types and constraints of the API.
> Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root".
I never knew this has an explicit name.
> Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
What exactly is a "dependency injection container"? I've searched the internet for the definition, but I've only gotten more confused by all the PHP and C#. You mention topological sorting of dependencies - does that assume a hierarchical object model or can it be used with basically anything?
EDIT:
From what I've read, it appears that a Dependency Injection Container is just an object that:
1) for each dependency type, has a `GetDependency: Type -> Object` method
2) lazily creates dependencies as required by the dependents and the dependency dependencies.
it is at heart quite simple. But this characteristic doesn't always mean that it has few benefits. Off the top of my head, things that DI enables include:
* swapping out a class for an equivalent becomes an configuration concern.
* whether a type is a singleton or transient becomes a separate concern to that class's implementation, and it can be changed in the app startup config code.
There can be drawbacks too, if done badly, but I don't think that the argument that "this doesn't do very much" is entirely relevant to the pros and cons.
While some configurability is desireable, being explicit is even more so.
I wonder why Zope died out? It has everything, it's a completely sound design, and makes all components interchangeable, and it's very pretty when done properly.
It died out because it's verbose and gets hairy quickly in the real world where not everyone is on the same page, so it never ends up being used "properly".
Go for simpler and you have a higher chance of everybody getting the intent on their first reading of code.
It's going to be hard to get people to not do DI "properly" if you simply ask them to pass dependencies in (though I am sure they still could by eg. passing classes vs instances in).
I really like that. Not just for testing - it makes the libraries much more flexible. But to make it a bit less burdensome for the most common usage patterns I like to relax it a bit with the possibility of building default object params in __init__.
class MyDep:
def do_foo(self):
print("fooo!")
class Fooser:
def __init__(self, foo_dooer: MyDep):
self.foo_dooer = foo_dooer
def act(self):
self.foo_dooer.do_foo()
class FakeDep(MyDep):
def do_foo(self):
print("booo!")
# With the grand unveiling:
fooser = Fooser(MyDep())
Seems a lot simpler than 100 lines they brag about, yet this includes the business logic implementation for the code as well (dependency injection takes a couple of lines total; if you add interface handling or defaults, you'll add a few lines).
When discussing these in Python, you need to be careful to differentiate between "mocking" and "monkey patching", especially so because both of those are provided by Python's standard library "mock" module.
Mocking, and specifically mocking with Python mock module, is a quick approach to construct faked objects that simply do not fail and which log any method execution on them. Dependency injection is usually done with more complete fakes, ideally with tested fakes (fakes which pass the same sets of tests as the original, "production" implementation does).
Monkey-patching is what allows Python developers to avoid dependency injection altogether. It has its own set of risks (basically, it might affect things you are unaware of), though you can reduce those by patching exactly at the import place of your dependency (eg. patch("mymodule.utc_now") vs patch("datetime.datetime.utc_now") — if you are not doing that today, start now! :).
Still, I consider readability the main concern when programming, with testability being second (no surprise they are closely related, as tests will document behaviour precisely). With Python's monkey-patching and mocking, you are at a large risk of producing illegible gibberish, both in tests and code.
If you can at all avoid it, you should! Simple dependency injection by passing dependencies directly in (functions or classes) makes everything explicit and trivial to comprehend (and test).
Unfortunately, the biggest problem is that the world of Python libraries is completely filled with magic side effects, so you sometimes can't effectively avoid doing either of those (it would be too costly).
What about when there's some environment-specific (or otherwise problematic) dependency that you need to mock because it can't be imported where you want to run the unit tests? Patching it afterwards doesn't work when the import of the default implementation fails.
If you patch it ahead of time by inserting a dummy module, that dummy module might be used to initialize other modules that you weren't targeting, so undoing the patch after the test might be very tricky.
You can sort of make it work, but it seems really hacky, so for that reason I'm currently refactoring towards DI (without frameworks). But I'm open to better ideas.
100% this. Having worked on projects using and not using DI in several languages, I'd prefer a world in which we just say no to DI. What it offers, relative to the indirection and cognitive overhead has almost always been net negative.
What do you suggest instead? new operators everywhere? Service Locator? Global Singletons everywhere?
Honestly, I don't know how you came from "my object needs objects A, B, and C to work" to the "it's cognitive overload". Your object still needs A, B and C to work, just with any other approach it's implicit which is spooky action at a distance in its finest and way more cognitive load.
I agree with you completely! As the matter of fact, I prefer this approach. But that is dependency injection at its finest, and something that NewMountain above said is "cognitive overhead" and "indirection".
I can't speak for NewMountain but I assume he was talking about using a DI framework or container (e.g. Spring) where dependencies are defined declaratively and the framework handles the instantiation. This is what brings the indirection and the cognitive overhead.
If a, b and c are MyObject's "dependencies", that's dependency injection. If MyObject has no other dependencies (produces no side effects other than on a/b/c), it is fully dependency injected.
Magic, monkey-patching mess of dependency injection libraries is terrible.
Dependency injection as a concept is simple functional approach to imperative and OO programming.
> If a, b and c are MyObject's "dependencies", that's dependency injection.
In practice, though, at least coming from a Java/.NET background, DI would be understood as using something like a decorator/annotation/some registration mechanism to let the framework handle giving you a valid instance of the class that you're after, rather than straight up using the constructor whilst filling out the parameters manually.
While your example is technically true, it wouldn't be the answer that anyone in a programming interview (at least for those languages) might expect you to provide as an example.
That said, adopting the composition root approach in a Java project (or using a god-object for all of the services) was oddly liberating and felt way more simple than mucking about with appeasing Spring's @Autowired annotation and its finnicky nature.
Like most fields of endeavour, programming has gotten infested with "smart" speak and too much terminology (let's talk the rest of Design Patterns next :D), and this same terminology changes even between different teams and not just different companies or technologies. Basically, people have a tendency to want to name pretty simple things names just so they are something special, even if they need to re-explain it every single time (there is no value in such a name, imho).
The fact that most of that terminology is misunderstood simply supports that claim above. The fact that interviews are also generally silly is besides the point too. :)
Still, none of that should be a benchmark for what something is. Dependency injection is simply about injecting a dependency instead of using it as a side-effect. You can achieve that in many ways, and the simplest is just to pass it in.
I'm catching up on this thread now, sorry for the delay. You're absolutely correct. My function needs A, B and C to work is valid. That's perfectly fine.
if __name__ == "__main__":
a = os.getenv("foo")
b = get_from_somewhere()
c = "some_hardcoded_variable"
my_app(a, b, c)
Is what I would expect in any reasonable codebase. The indirection and cognitive overhead was taken from Java where seeing @Autowired or @Inject, hitting reverse search and not being able to keyword search for the thing being injected is nightmare fuel. That is not something I would wish on an enemy, let alone on any user of a programming language I care for.
My original reply was in response to "please don't make Python Java", where that flavor of spooky action DI seems to be common.
To your original question: `what do I suggest`. I point to the authors example:
di = Di()
@di.register('A')
class A:
def __init__(self, di: Di):
pass
def action(self):
print("Hi from A")
@di.register('B')
class B:
def __init__(self, di: Di):
self._di = di
self._a = None
def run(self):
self.get_a().action()
def get_a(self):
if self._a is None:
self._a = self._di.get('A')
return self._a
@di.override('A') # replace class A definition with a new one
class C:
def __init__(self, di: Di):
pass
def action(self):
print("Hi from C")
di.get('B').run() # prints "Hi from C"
di.remove('B') # removes the B dependency from the registry
My suggestion: please not that.
How about:
class A:
def __init__(self):
pass
def action(self):
print("Hi from A")
class B:
# A is default for some reason
def __init__(self, a=A()):
self._a = a
def run(self):
self._a.action()
class C:
def __init__(self):
pass
def action(self):
print("Hi from C")
if __name__ == "__main__":
if os.getenv('ENV') == "PROD":
a = A()
b = B()
# test/non-prod/whatever version
else:
a = A()
b = B(C())
As someone who's been writing python for a while, I could walk into this having never read it before and say...ok sure, when prod, B takes A behavior, but when not prod, B takes C behavior.
In the proposed example, I would go in, read it (assuming I know to even look for this non-idiomatic use of the language) and then read through this code to figure out what should have just been an argument passed to B.
Which leads back to my original question: what is this buying me over dunder main or handing some arguments? In terms of cost, I have to understand something and walk through the code to understand something that could have been done more simply and more explicitly without this DI pattern.
Thank you for very detailed reply! As the matter of fact, I mostly agree with you and prefer such a direct (or manually wired) approach to DI, over Spring's declarative one. Somehow I was under impression that you are against DI as a concept. I am sorry for that.
Dependency Injection does not require a DI Container.
You know, like have an app_init.py and instantiate all services/dependencies manually; providing them as constructor args to other instances as needed.
Use other implementations/mocks in tests setup or dev environments.
It bothers me very much that people who don't even understand the basics of Python want to teach complex concepts. This is why there are lots of misconceptions and terrible code in Python.
__init__ is NOT the constructor in Python, the object is already created at the time when __init__ is started. Maybe this detail doesn't matter in this article, but still teaching the wrong thing.
At the risk of showing my ignorance... what makes Python's __init__ fundamentally different from Java's constructors? The latter also has access to the "this" reference with some of its state initialized and some not.
I am not a Python expert, but I think strictly speaking __init__ is the object instance initialiser.
The "constructor" is the __new__ method (which is a static method) which will create an instance of the class. Afterwards a call is made to the __init__ method on the new instance, passing any parameters specified in the __new__ method and the newly created instance (as self).
As kissgyorgy rightly points out, it's the interpreter that performs these calls.
Python is a bit more revealing about how the sausages are made.
Hmm, perhaps there is no such thing as a constructor in Python then? As the documentation says, "__new__() and __init__() work together in constructing objects".
In Java's constructors you already have access to the instance (and could leak it outside the object if you wanted to) whereas in __new__ you don't, so it doesn't seem like __new__ by itself could be called the constructor either.
Again, this is what I'm talking about. If you don't know the exact details, why do you want to teach how it works? Learn it first, try it, THEN teach about it. Silence or questions arebetter when you don't exactly know how things work.
__new__ doesn't call __init__, the interpreter will call the __init__ when an instance is returned from __new__, otherwise it won't even be called.
This does not seem to implement dependency inversion (DIP).
(IIRC, Robert Martin states that DIP is one of the principles[1] on which dependency injection (DI) can be built as a pattern.)
Since python does not have interfaces (unless you write classes that to nothing, except perhaps raise errors if methods are not overwritten) this does not seem to actually do DI/IoC completely.
[1]: Together with DIP as the second principle.
<Edit>Flipped IoC (e.g. factories) and DIP (interfaces for abstraction) which I seem to had memorized with swapped meanings</edit>
I don't understand the point of this implementation. If you have to call di.get("A") in __init__, why is it better than just instantiating A() directly? It's not even a dependency injection I guess, as the framework should take care of the automatic instantiation of objects without the object having to know even that a Di is in place.
Yeah it doesn't make any sense, the whole point of DI is to work around Java's static type system to enable mocking for testing, which is a problem you don't have outside of static typing.
But the "why" around these design patterns have been long lost, and people will argue to death that the patterns should always be used in every language/runtime and come up with ever more vague and circular reasons until you can't argue against it.
69 comments
[ 2.7 ms ] story [ 130 ms ] threadWhen an object requires other objects for its work, pass references to those other objects into its __init__ method. That’s it.
Sorry couldn't resist. The additional dependency is the biggest problem I have with OP's approach though. Now none of my classes are easily portable across projects, and this DI framework is going to spread around everywhere.
Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root".
Now that all construction is taking place at once, the order matters as you can't pass a dependency to its dependent until the dependency has been constructed. But it may have its own dependencies. So now there is a topological sorting problem.
Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
The compiler generally has better attention to detail and the ability to deal with larger object graphs than the typical human.
Honestly, in practice it worked and wasn't too bad to work with, which was interesting to behold - in practice everyone talks about how circular dependencies are bad for a variety or reasons (trying to print or process data and ending up with endless loops comes to mind), but then there just was that system that was chugging along without a care in the world.
Since now, I can go to the project's root and read step by step how the application is architected and build up and which are core dependencies and which are not.
With dependency injection containers, everything is shuffled around and I'm now at the mercy of a library to tell me what I want to know.
I guess I just don't see the problem.
That solves itself naturally through normal programming. Some function takes A as argument? Well I obviously make that A first. A requires B to setup? Well obviously I make that first.
I don't need to "sort" anything, it just follows naturally from the types and constraints of the API.
I never knew this has an explicit name.
> Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
What exactly is a "dependency injection container"? I've searched the internet for the definition, but I've only gotten more confused by all the PHP and C#. You mention topological sorting of dependencies - does that assume a hierarchical object model or can it be used with basically anything?
EDIT:
From what I've read, it appears that a Dependency Injection Container is just an object that:
1) for each dependency type, has a `GetDependency: Type -> Object` method
2) lazily creates dependencies as required by the dependents and the dependency dependencies.
Is this accurate?
[1] https://blog.ploeh.dk/2011/07/28/CompositionRoot/
[2] https://blog.ploeh.dk/2012/11/06/WhentouseaDIContainer/
* swapping out a class for an equivalent becomes an configuration concern.
* whether a type is a singleton or transient becomes a separate concern to that class's implementation, and it can be changed in the app startup config code.
There can be drawbacks too, if done badly, but I don't think that the argument that "this doesn't do very much" is entirely relevant to the pros and cons.
I wonder why Zope died out? It has everything, it's a completely sound design, and makes all components interchangeable, and it's very pretty when done properly.
It died out because it's verbose and gets hairy quickly in the real world where not everyone is on the same page, so it never ends up being used "properly".
Go for simpler and you have a higher chance of everybody getting the intent on their first reading of code.
It's going to be hard to get people to not do DI "properly" if you simply ask them to pass dependencies in (though I am sure they still could by eg. passing classes vs instances in).
1. An add_foo instance method.
2. Arguments to the `__init__` method, ideally as *args, using add_foo internally.
3. Optionally, an @instance.foo decorator which inserts the dependency as it's being defined (again, using add_foo internally).
Because you don't need dependency injection in python. You can mock out any object anywhere during your tests.
Exactly! I recommend anyone who comes in contact with dependency injection to really think about this.
Mocking, and specifically mocking with Python mock module, is a quick approach to construct faked objects that simply do not fail and which log any method execution on them. Dependency injection is usually done with more complete fakes, ideally with tested fakes (fakes which pass the same sets of tests as the original, "production" implementation does).
Monkey-patching is what allows Python developers to avoid dependency injection altogether. It has its own set of risks (basically, it might affect things you are unaware of), though you can reduce those by patching exactly at the import place of your dependency (eg. patch("mymodule.utc_now") vs patch("datetime.datetime.utc_now") — if you are not doing that today, start now! :).
Still, I consider readability the main concern when programming, with testability being second (no surprise they are closely related, as tests will document behaviour precisely). With Python's monkey-patching and mocking, you are at a large risk of producing illegible gibberish, both in tests and code.
If you can at all avoid it, you should! Simple dependency injection by passing dependencies directly in (functions or classes) makes everything explicit and trivial to comprehend (and test).
Unfortunately, the biggest problem is that the world of Python libraries is completely filled with magic side effects, so you sometimes can't effectively avoid doing either of those (it would be too costly).
If you patch it ahead of time by inserting a dummy module, that dummy module might be used to initialize other modules that you weren't targeting, so undoing the patch after the test might be very tricky.
You can sort of make it work, but it seems really hacky, so for that reason I'm currently refactoring towards DI (without frameworks). But I'm open to better ideas.
If you are using something like Java or C++ then it's easy to see the benefits (some C++ developers would disagree though, read https://accu.org/journals/overload/25/140/pamudurthy_2403/).
More code = more cognitive overhead
Honestly, I don't know how you came from "my object needs objects A, B, and C to work" to the "it's cognitive overload". Your object still needs A, B and C to work, just with any other approach it's implicit which is spooky action at a distance in its finest and way more cognitive load.
Magic, monkey-patching mess of dependency injection libraries is terrible.
Dependency injection as a concept is simple functional approach to imperative and OO programming.
> If a, b and c are MyObject's "dependencies", that's dependency injection.
In practice, though, at least coming from a Java/.NET background, DI would be understood as using something like a decorator/annotation/some registration mechanism to let the framework handle giving you a valid instance of the class that you're after, rather than straight up using the constructor whilst filling out the parameters manually.
While your example is technically true, it wouldn't be the answer that anyone in a programming interview (at least for those languages) might expect you to provide as an example.
That said, adopting the composition root approach in a Java project (or using a god-object for all of the services) was oddly liberating and felt way more simple than mucking about with appeasing Spring's @Autowired annotation and its finnicky nature.
The fact that most of that terminology is misunderstood simply supports that claim above. The fact that interviews are also generally silly is besides the point too. :)
Still, none of that should be a benchmark for what something is. Dependency injection is simply about injecting a dependency instead of using it as a side-effect. You can achieve that in many ways, and the simplest is just to pass it in.
My original reply was in response to "please don't make Python Java", where that flavor of spooky action DI seems to be common.
To your original question: `what do I suggest`. I point to the authors example:
My suggestion: please not that.How about:
As someone who's been writing python for a while, I could walk into this having never read it before and say...ok sure, when prod, B takes A behavior, but when not prod, B takes C behavior.In the proposed example, I would go in, read it (assuming I know to even look for this non-idiomatic use of the language) and then read through this code to figure out what should have just been an argument passed to B.
Which leads back to my original question: what is this buying me over dunder main or handing some arguments? In terms of cost, I have to understand something and walk through the code to understand something that could have been done more simply and more explicitly without this DI pattern.
Edit: formatting
You know, like have an app_init.py and instantiate all services/dependencies manually; providing them as constructor args to other instances as needed.
Use other implementations/mocks in tests setup or dev environments.
__init__ is NOT the constructor in Python, the object is already created at the time when __init__ is started. Maybe this detail doesn't matter in this article, but still teaching the wrong thing.
The "constructor" is the __new__ method (which is a static method) which will create an instance of the class. Afterwards a call is made to the __init__ method on the new instance, passing any parameters specified in the __new__ method and the newly created instance (as self).
As kissgyorgy rightly points out, it's the interpreter that performs these calls.
Python is a bit more revealing about how the sausages are made.
https://docs.python.org/3.10/reference/datamodel.html#object...
https://docs.python.org/3.10/reference/datamodel.html#object...
In Java's constructors you already have access to the instance (and could leak it outside the object if you wanted to) whereas in __new__ you don't, so it doesn't seem like __new__ by itself could be called the constructor either.
__new__ doesn't call __init__, the interpreter will call the __init__ when an instance is returned from __new__, otherwise it won't even be called.
In the __new__ method docs they mention an "object constructor expression" which I take to mean:
And in the __init__ method docs there's mention of a "class constructor expression" which I also take to mean the same as above.Yet oddly we have no mention of constructors or constructor expressions in:
https://docs.python.org/3/tutorial/classes.html
And the act of instantiating a new instance of a class is referred to as "“calling” a class object)"
https://docs.python.org/3/tutorial/classes.html#class-object...
Would you not agree that the docs could be a bit more consistent with regards to terminology?
Since python does not have interfaces (unless you write classes that to nothing, except perhaps raise errors if methods are not overwritten) this does not seem to actually do DI/IoC completely.
[1]: Together with DIP as the second principle.
<Edit>Flipped IoC (e.g. factories) and DIP (interfaces for abstraction) which I seem to had memorized with swapped meanings</edit>
You can have them in earlier versions with the typing-extensions library. [0]
> For static type checking
Protocols can be used for runtime checks as well. [1]
[0] https://pypi.org/project/typing-extensions/
[1] https://docs.python.org/3.9/library/typing.html#typing.runti...
But the "why" around these design patterns have been long lost, and people will argue to death that the patterns should always be used in every language/runtime and come up with ever more vague and circular reasons until you can't argue against it.