As someone has used Django extensively over the last 10 years for countless projects, I think that these are exaggerated claims of “harmful” defaults, particularly examples 2 to 6.
Furthermore, this:
> As you can see, Django doesn’t have your best interest at heart. Instead of providing sane defaults, it punishes you for not being a master at the framework.
Is just nonsense.
Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. It’s also heavily documented by Django and one of the first things mentioned. Lazy querysets are actually very useful when you need to build queries over a number of steps. Explicit query sets can have their own subtle issues with performance.
2. The default user model is fine for most users. It’s not a bad default and there is clear documentation on how to extend it. It’s designed to be extended!
3. I rarely rename migrations. Maybe this is a good idea, but it’s hardly a harmful default.
4. I almost never rename my tables. Again, maybe a sensible suggestion, not a harmful default.
5. Strangely this seems to be the most un-Django suggestion that would have the largest effect on a project but hardly any detail is given here. I never do this and wouldn’t recommend anyone else to when staring off.
6. I think project structure and CLI is one place where Django is weak but again this is hardly a harmful default. Use django-cookiecutter.
Anyway I feel like the author is trying to manufacturer controversy to help define their expertise (not to say they aren’t a good Django developer, I have no idea). Some of these are good suggestions and “top tips” but they’re certainly not instances of Django trying to harm you.
The default user model is fine, but I *always* start a project by extending it, for future-proofing. If you need to add to it, this is *so* much better than having to add a one-to-one, and at *some* point down the line you find you do want to add something. The problem is that doing it later when you need it is a pain, because of poor support by the migrations framework.
It's like 4 lines of code that saves you so much grief later on:
from django.contrib.auth.models import AbstractUser
class User(AbstractUser):
pass
AUTH_USER_MODEL = "myapp.User"
If you never need it, those 3 lines are not hurting you. In terms of a default, I think it probably would be nicer if Django pushed you to do this up front.
I agree 100%. Even if you don't need it, these lines don't hurt you, but most of the time, the additional model is put to good use. I usually add metadata, such as additional timestamps, flags, tags, custom user settings, consent or preferences.
> Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere.
No, there are frameworks out there that doesn't allow queries during template rendering. See Lucky for an example, https://luckyframework.org/.
The problem is not whether the queries are ran in templates or in code, it's which queries are ran. Doing the same he did in the template but in code (by constructing a dictionary of items to be passed to the template) will have exactly the same problem.
The solution of using eager loading has nothing to do with whether you run your query in the template or in code.
It's easier to notice expensive SQL queries in the view vs in the template.
Anyone who claims that it's fine to run queries in templates has smoked something really good. Personally, I'll continue to aggregate all my data using JOINs before presenting them.
When it comes to "queries" keep in mind that I'm not saying you should call "Model.objects.filter(...)" in the template (in fact that would be very difficult to do unless you pass a "Model" reference to the template). You still do that in code.
However due to how Django handles queries by default, calling the ORM won't actually execute the query until the queryset is evaluated in most cases (unless requesting a single object or doing an aggregation), so your queries are still technically "ran" from the template - even if you are doing prefetch_related.
I personally don't have a problem with that; explicitly casting the queryset to a list before rendering a template seems like unnecessary boilerplate and still doesn't actually isolate you from the "N+1" queries problem if your template then accesses an object's relation that hasn't been prefetched.
That's the point. Instead of worrying about whether you're accessing a related attribute, join everything and evaluate the QuerySet before passing it to context.
You'll never have to worry about whether {{ pizza.topping.name }} is hitting the database ever again, especially, if you install django-zen-queries because it will raise an exception if you try to.
Got it - so the magic here isn't merely to evaluate querysets ahead of time (which can still return objects with unfetched relations that'll cause a query at runtime if accessed) but to also rely on the third-party package to explicitly disallow that.
It's highly unlikely that will you execute unnecessary queries from view code because usually you'll have a context like {"pizzas": page_obj.object_list}. You'll almost never execute something like:
for pizza in pizzas:
print(pizza.topping.name)
But you'll certainly do the following in templates:
{% for pizza in pizzas %}
<p>{{ pizza.topping.name }}</p>
{% endfor %}
I also don't understand why you want to avoid 3rd party packages, especially if it helps you write better code.
My argument was that the third-party package is what's key here. Merely converting your querysets to a list in code before handing it off to the template won't fully protect you against extra queries.
I'm not necessarily trying to avoid third-party packages, in fact I've installed the package you suggested in one of my projects and will be trying it out.
Nothing I wrote is controversial. I explained my thought process at length but you flat out rejected everything without explaining why. I wonder who's creating controversy here.
All of these "issues" are well documented if you bother to read just the main Django documentation.
Some solutions are questionable. Executing querysets as soon as you define them means you can't compose them. Lazy quersets are not dangerous and you don't execute that many because they are lazy, but because under the hood you haven't asked for a JOIN statement.
Automatic database table names are definitely not a problematic feature. 99% of the time it gives you very intuitive mapping between model name and database table name. If you actually want to move model to a different app, then you can use the same feature (db_table) to point to its old location.
Just to note: django-zen-queries doesn’t encourage you to execute querysets as soon as you define them, but rather just before you pass them to a template or serializer. This is a pattern we’ve found really useful.
(I’m the author of django-zen-queries)
I’m not a fan of the hyperbole in the article either. Some of the points are genuine problems but none of them should put off beginners.
As an experienced Django developer, I found django-zen-queries very good for hygiene, and I'm going to start using it. I should also pay attention to django-debug-toolbar's query view more often.
I generally don't need to optimize my queries because my apps are small, but I should do that by default, because otherwise it's wasteful and it's not hard to do. It's just that Django's default doesn't encourage you too, but hopefully with Zen Queries it will.
> I generally don't need to optimize my queries because my apps are small
Same situation here. I find that being mindful of the ORM is enough to save me from obviously bad queries (fetching 100 objects and then running additional queries for each one), for the rest I get away with it as it's not a problem in practice. Sure, I will probably save an extra milisecond here and there and maybe in the end be able to run my entire DB off a very small computer, but (bare-metal) hardware is so cheap that a few stray queries here and there won't break the bank.
Nevertheless I've installed the package and will be trying it out, but I have yet to be convinced that eliminating additional queries all the time is worth the extra developer effort.
I tried it, and came to the same conclusion as you. Fortunately, the package doesn't require you to do it all the time, you can pick a few at-risk functions and only disallow template queries there.
I just found out this package and absolutely love the idea of "sanitizing" the queries. However I'm just wondering: if I'm a solo dev and I already monitor all my endpoints using Django Debug toolbar, how is it still useful to use this package?
In my entire career I've got away with just relying on a "gut feeling" about whether a query needs a select/prefetch_related and it has yet to fail me. Yes, you're going to have a handful of stray queries here and there that could technically be avoided, but is the extra developer effort worth it? Hardware is very cheap (well if you know where to look, definitely don't look in the clouds), take advantage of that.
Unless you're programming purely for the sake of programming, what matters is the end result and the problem your application is solving. Your code doesn't have to be 100% perfect (otherwise why are you using a dynamically-typed, interpreted language?), it just has to work well enough to solve the business problem - other tools such as monitoring will quickly tell you if you've made a mistake and you can go back to the drawing board and add that extra select/prefetch_related you missed the first time.
I wrote this post for beginners and most of them never read the docs before hacking on production code.
Lazy querysets aren't dangerous but it's always best to evaluate them before rendering anything in order to avoid secret SQL queries. Many developers never run EXPLAIN, so these extra queries go unnoticed until your server crashes during Black Friday.
I don't see why your database table needs to encapsulate the app's name. Most apps get rewritten at some point but databases can live for centuries. Why couple code with your data?
Maybe I should be more humble to please sensitive people like you but no thanks. If a simple blog post hurts your feelings then you need to professional help.
> I don't see why your database table needs to encapsulate the app's name.
Because Django assumes that in most cases your DB will be primarily (only?) be used by the Django app. In my experience that is true of most startups and early-stage projects, and the default names have never been a blocker even when you start giving others access to that database (hopefully with a read-only user account).
> Most apps get rewritten at some point but databases can live for centuries.
You can rename tables when this becomes needed. I don't think it's worth wasting time doing so at the beginning (you could equally argue that whatever name you pick manually isn't going to be suitable in the future either, so a rename will be needed either way).
> Why couple code with your data?
Renaming tables doesn't realistically address that coupling by any means.
As others here have pointed out, the article is much more clickbaity than its contents justify. None of the issues are dealbreakers and some expectations of the author are simply unreasonable. He complains that Django doesn't name generated files with a good human description of the intentions inside his head. I think we'll have to wait for Neuralink for that.
My biggest objection to that naming scheme would be the numbering. If you have multiple people working on an app, it is likely that you at some point get multiples of a number as people add models simultaneously. What happens if there are multiples using a single number?
I much prefer the timestamp based system that is used by Rails, it make the risk of that kind of conflict a lot lower.
The migration engine will order the dependencies at runtime, and it will bail (and suggest creating a "merge" migration) if you have diverging trees of migrations. I find it pretty robust in practice.
> My biggest objection to that naming scheme would be the numbering. If you have multiple people working on an app, it is likely that you at some point get multiples of a number as people add models simultaneously. What happens if there are multiples using a single number?
We have this happen at work, when merging branches; this is what `./manage.py makemigrations --merge` exists for.
The only thing that annoys me to this day is that they have the numbers first. I get the logic why: it means they're ordered in the directory properly.
That said, it's very annoying in that they're not tab-completable... any time I have to edit one, I have to stop and hunt in the entire list for what I'm looking for.
I have been a user of Django for 15 years, I think I first used version 0.95 just after the “magic removal”.
I wouldn’t say any of these are “harmful”, there are certainly some things here that have valid arguments for doing differently, but most is rubbish. But it needs to be taken into account that Django is an incredibly mature framework with literally millions of projects built on it. Large changes are hard, and Django has had to make many of them before. All of these have been debated and haven’t changed either due to immovable technical debt, a decision that there isn’t at this time a need, or there is valid arguments on both sides.
A couple of examples: the user model, providing a default is important but it is also well documented that if you want to extend it you should subclass it from the beginning to make your life easier. There are ways to migrate later, they aren’t exactly easy, but it is doable.
Multiple small apps, this architecture is due to the history of Django and its development at LWJ. They wanted a way of being able to quickly build new websites out of pre-defined “apps”, and this system works for that. At the time the Python packaging ecosystem was very different, hence then slightly odd design. I’m sure in hindsight the founding team would have done things differently, but this is 17 years ago.
The general rule of thumb for “app” architecture now with Django is if you are a consultancy/agency building multiple similar sites for multiple clients (this is a huge use case for Django, closest to it original vision) it makes sense to package up your functionality into reusable apps. If you are a “startup” or building an internal app for a company with no need for reusability it’s best to just have one “core” app to contain all your functionality. It comes down to your personal needs, but it’s easy to make both architectures work.
Django "apps" are sometimes reusable, but more often they aren't. For me the benefit in splitting a project into apps is that it provides a more manageable structure.
You also avoid merge conflicts in teams, especially with migrations. I'm now working with Dotnet, because I have to (nobody wants to hire me as a Django developer even though I have 10+ years experience in Django and <0.5y in .NET), and this is an ongoing problem with EF core...
Lovely website - just a note that your contact pages / widgets all list support@example.com as your contact email...Ignore me if you're just building but thought it was worth pointing out.
Believe me, I've tried. People commended my technical skills and still rejected me. Between a screwed up CV und unwillingness to move outside Germany during the pandemic, it just never worked out.
Out of curiosity, why is your CV screwed up? Feel free to reach out (email in my profile) if you need advice and/or are looking for remote Django-focused work.
I wouldn't call those harmful. Most are non-issues, while others are "unfortunate" at best.
1. I found that the ORM works quite well and it's easy to optimize later. Using `prefetch_related` or `select_related` in the ORM still feels weird (they're just joins), but I wouldn't call them harmful.
2. I found it unfortunate that the info on how to change the User model is hidden so deep in the Documentation. I don't like to RTFM, but with Django it sure is worth it; same goes for 1.
3. I occasionally do it, but most of the time, I don't and never had an issue.
4. It do rely on that and found django's naming scheme to be a non-issue. I guess this might be more important if you need to change your database out of band (out of django band, that is)
5. This point is unsubstantiated. Apps aren't mere packages but also define a scope for the admin tool (which I use a lot and think is an integral part of Django) so that it will group models and allow individual access (i.e. apps also provide scope for your permissions). I use tons of apps but I wouldn't call it a harmful default.
6. Yeah, dynamic languages might produce naming collisions. Still beats imports like `org.example.publib.boulder.yourmoms.maidenname.models.impl.Account`
I consider them harmful for beginners who need to start working on non trivial production code.
Multiple apps are definitely good for the admin but what I was advocating isn't 100% avoidance of apps but to avoid creating apps for every tiny feature.
Personally, I like my databases to be designed for longevity and avoid couple code to my data. Code can die at any point in time but data is an asset that needs to live for as long as the company is in operation.
> beginners who need to start working on non trivial production code.
If you're a beginner dropped into a non-trivial production codebase your best bet is to follow existing conventions until you gain enough experience to be able to evaluate such advice and consider it against your projects' and business' requirements.
I've been using Django for over a decade. I don't think most of these are a big deal and outright disagree with others. Happy to offer my own criticisms of django to add value to this comment thread.
My real criticism of Django is that, at long last, I think it's starting to look dated. Templating hasn't been the preferred way of making web app UIs for some time. Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Setting up a project to do this is cumbersome. While there are third party solves, Django's greatness comes in part from how opinionated it is.
I understand DSF may see adding more javascript as out of scope, but in keeping with the "batteries included" philosophy I hope they one day embrace something in contrib.
I'd love to hear your criticism of django, issues you encountered and problems you have solved :)
> My real criticism of Django is that, at long last, I think it's starting to look dated.
Actually, that is something I found really great about django. I found that django is reliable over a long period of time. The js/node/react/express/angular apps I had developed in the past (~2012+) can't be built these days without redoing major components. I never had these problems with django.
As for the responsiveness of the results, I found that a good understanding of django's performance helps. Using a good caching strategy (on models, views + nginx) yields results have almost instant reactions, even though there is the whole HTTP to HTML roundtrip. While they do have the noticable page-reload semantics, I found that these are way easier to build and sometimes yield very enjoyable experiences for my users. After all, I'm not building a YouTube so PictureInPicture and SPA-semantics aren't that important to my users.
> Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets.
Julia Elman explains how to do this with Django + DRF and Backbone.js in the book "Lightweight Django" (O'Reilly). Is this a viable compromise?
I've implemented Django+DRF+React in several projects. The thrust of my arguement is that, were Django started today and not 17 years ago, it would account for these concepts more than templates do.
Django contrib exists to be a home for first party solutions to specific problems [1]. It need not be an entire frontend framework that follows trends like react, but something to make syncing state easier sits squarely in that namespace.
One advantage of REST is that it's easier to use from a programmatic client perspective; this might become useful if you want to offer an API to third-parties.
Not adding JS (other than what's needed for the admin) is in retrospect a good move - when you see the pain Rails developers went through in multiple iterations of RJS, rails-ujs, Turbolinks, CoffeeScript and now Hotwire/Stimulus depending on DHH's vision of what was hot in frontend. Django's JS agnoticism has made it easier to use different frontends without having to tear stuff out, whether you want a full Vue/React SPA or something more minimal like htmx or Alpine (or even plain old server-side templates + jQuery).
Wow this article is so poor it is almost malicious.
> This means that you don’t own your database — the Django foundation does!
This is a completely baseless accusation. At no point are you unable to change the schema or replace the user model (which we have done with admittedly some kerfuffle).
If the author is reading this comment, and this is the reaction you were looking for from the community. Congratulations. You have written a completely asinine article about a technology you obviously do not understand yourself. Telling people to rename tables because "a future data analyst is not going to know what a table is for". On what planet is a data analyst given raw access to the production instance's tables of a database and not a view on a read replica.
Having one migration per model is dumb as all hell, as you would need to manually separate these, achieving nothing of value in the process.
Know-it-all anuses like the author of the article is why software engineering has become to be known as "a black hole of cash" because these are the type of people who pretend to know what they are talking about but make decisions that cost people exponential money and time.
Please stop writing sensationalist, clickbait bullshit like this and please, please read at least a small part of the documentation before writing something misleading like this.
I was a data analyst working at a company where all the data was in a Django-mamaged database.
Not once was I confused by the table or column naming.
I was kind of annoyed by all the long table names and all the joins I had to write out, but it was all fairly tidy and organized, and I almost prefer it to the human-designed databases I've worked with.
Theoretically yes if they one day renamed the app then there would be some legacy names in the table. Who cares, shit happens.
Regarding read-only access: we were a small enough organization that I actually did have full access to the prod database. I hated having it and tried to work off of our "QA" clone whenever possible, or made my own local timestamped copy of the prod data.
I’ve been developing Django for two weeks and I agree.
1) ORMs have the concept of lazy vs eager, no matter which framework. It’ll bite if you don’t know what you’re doing in any language. They’re still less bad than raw sql.
2) a default user model is better than any attempt an average developer would implement themselves. Go on, I dare you to try and hash passwords correctly.
3) if I wanna know wtf a data migration is for, I’ll look at the commit message, and code changes.
4-6 ) these are opinions of the Django team. I don’t really agree with them, but I chose to use their software. You know what doesn’t have opinions? Raw php, and I know which language/framework I’d prefer..
Point 2) is not about implementing authentication or even the user model from scratch. It is about using a custom user model, something the django documentation explicitly recommends doing, for the same reason the author recommends it: Changing it later is a pain.
"If you’re starting a new project, it’s highly recommended to set up a custom user model, even if the default User model is sufficient for you."
Adding an empty, but custom, user model derived from the base model could be the default. But it isn't, instead there is this recommendation, which you will only read when it is too late (on your first project).
I upvoted the article just because of this point as I myself had to go through the pain of moving to a custom user model on an already-deployed app with real data (so nuking the migrations and starting fresh wasn't an option).
I agree that most other points are garbage though.
No need to hash password if you subclass AbstractUser.
Tell me, if I have {{ customer.auth_user.email }}. Where in this line of code does it say that a SQL query is going to be performed? So much love for lazy queries eh.
You're so new to Django and yet you pretend to understand everything.
I don't understand how I'm supposed to help someone like you. I hope your tech lead is able to guide you through this.
> Where in this line of code does it say that a SQL query is going to be performed?
It is implied when you're using a framework with an ORM - after all, how do you end up with a "customer" object to begin with instead of a list representing a DB row?
> So much love for lazy queries eh.
When it comes to performance there's always a trade-off. The same reasoning could be applied to using Python as opposed to languages that compile to machine code, or for that matter using Linux instead of running your code on bare-metal.
Lazy queries trade-off performance in favour of developer productivity. In some cases it's a major problem (fetching 100 objects and looking up related objects for each one), in other cases it might be imperceptible and the time spent fixing them can be probably spent better elsewhere.
> You're so new to Django and yet you pretend to understand everything.
Knowledge of lazy queries and how they work is definitely valuable, but my impression is that most people here do know how they work and what are their pitfalls - they simply disagree with your opinion that "all lazy queries are bad".
> I don't understand how I'm supposed to help someone like you. I hope your tech lead is able to guide you through this.
Some people here (including me) don't agree with your black & white thinking about how lazy queries are always bad. They know their application, requirements and performance targets better than you do and have decided that for their use-case lazy queries aren't a problem in most cases (and can address the problematic cases individually without disabling lazy queries globally and having to waste hours trawling through their codebase for every single place to put a select/prefetch_related with no worthwhile performance gain in practice).
1. How are you supposed to change the concrete User model that Django provides without forking Django itself? It's 1000x better to subclass AbstractUser and provide your own User model that you own.
2. One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file.
I wish you'd think straight instead of being angry. It's not like I'm claiming that the earth is flat.
> One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file.
Squashing is something Django will handle for you through a management command, at which point it doesn't matter whether you're doing one model per migration file or not. Stuff that cannot be squashed (RunSQL/Python operations) will be pointed out by Django and I'm not sure how having one model per migration will help - the difficulty here isn't locating the offending operation, it's to actually figure out whether that operation is still needed and if so whether it can be reworked.
I looked for a simple solution to this recently and I couldn't find one. It would be great if implicit SQL queries that occur as a result of missing out either select_related() or prefetch_related() would generate a console warning.
I think this should be the default behaviour given that most beginning Django developers fall into this trap.
The Django logging configuration is IMO unnecessarily complex and while it can be tuned to do anything with enough effort, I wish there was a one-click setting that says "tell me shit I should probably be aware of". I always find myself searching as to how to log SQL queries and copy/pasting the entire "LOGGING" setting temporarily despite working with the framework for several years.
I think these are simply "gotchas" rather than reasons not to use django. We've probably all committed an n+1 query at some point, but that doesn't mean the language/framework we were using was bad.
Meta point: I really enjoy articles that list the places noobies go wrong (for any language/framework, not exclusively django). E.g. an experienced dev I taught R to was mind-blown that vec[1] accesses the first element of a vector (not vec[0]).
Long lists of these unexpected (albeit simple) things can quickly be compiled. It might only take 30 minutes for a noobie to read but probably ends up saving days of pain down the track.
I never said that people need to stop using Django. It's still my favorite framework and will continue to be.
I just wanted to point out some mistakes that many beginners make and that I made when I first started.
Personally, I think Django could use better opinions from the core team because beginners should be set for success by default. In the current state, that's not the case.
> I never said that people need to stop using Django.
I know :) I was alluding to that exact point. I think other commenters thought that's what you meant, and jumped to the defence of the framework. But I agree that criticism is healthy, and it doesn't mean you don't enjoy or would stop using it. As someone who hasn't done much django, I appreciate your tips for beginners (especially in list format).
I worked on Django projects for around 7–8 years professionally. Most of these weren't really issues, though I think we did run into name conflicts at some point with an app vs package.
But the first pain point I very much agree with. Implicitly sending queries to the backend is one of those things that is an attempt to be very helpful to programmers, but that ends up as an obstacle that you always need to be wary of and avoid. Over time I have grown to appreciate explicitness in queries more and more, though I probably wouldn't go so far to remove lazy queries completely since they can be handy in query building, just remove the implicit loading feature if possible.
Django's query builder is also quite far removed from SQL. Especially once you get into more complex queries, I sometimes just went raw SQL as I knew I wouldn't be able to remember how to translate the Django expression into SQL in a month's time. Simple queries are easy to write and look very sleek, but complex ones can turn into a mess that is much more understandable in SQL form. The sleekness of the queries may also betray how complex they are to execute as it's not "in your face".
I still like Django's ORM quite a bit for many things, but after discovering Elixir's Ecto, I've enjoyed how explicit it is most of the time. I haven't needed implicit loading of relations once. It's also much closer to actual SQL. But there are things where Ecto also tries to be too helpful in my taste, like the humanised types in migrations (i.e. :utc_datetime instead of :timestamptz or :string instead of :"character varying"/:text) -- I strongly believe it would be easier in the long run for the programmer to specify the database types they want directly rather than through aliases which may result in surprising choices. Of course, that would cause issues in supporting multiple different databases, but that's not a goal I would go for in the first place, but I digress…
What I guess I'm saying is that sometimes projects build in helpful features to make the code nicer to look at, more approachable to beginners, or just shorter to write, but they often end up being a burden on those that go past the beginner stage. People invariably trip on them because who can remember everything mentioned in all the documentation? And then it's just a game of "who fails to avoid the trap every time" and that's a game that everyone loses eventually.
I mostly agree with other commenters, but I really must hard disagree with point 5.
Proper abstraction is the one thing you need to get right, and django makes it so easy to make clean separate modules work together using a shared base and way of working (user model, groups, permissions, urls, signals, etc.).
One day you'll need to use your payments logic to sell something else, embrace it.
I wish the tutorial really explored this more. I’ve got some apps which have clear boundaries, but could be split into distinctive modules but I don’t see much value in doing so. I’d love to see an official example of this.
If my payment logic is getting too coupled, I can easily extract it because I use one migration file per model and separate code using regular Python packages.
I can easily extract the logic into it's own app if I need to publish it to PyPi for others to enjoy as well.
Of course, if you code is getting coupled, then you need to revise your design process.
I relate with the author about what a Django's database could become if not handled with care. In my experience if one creates models using a lot of inheritance then the database becomes a dumpster and it's basically impossible to get any data out of it with something different than Python itself.
The only sane way is to think about tables first, create models that mimic those tables, use makemigrations + migrate and absolutely no inheritance in the models.
An extra bonus is only one app per project. I've got a customer with two projects: one with many apps and a lot of inheritance, the other one with one big app and a few very small ones and no inheritance. We can use any SQL tool to extract data from the database of the second project. The database of the first one is "spaghetti data" and "python3 manage.py shell" is the only way to make sense of it.
On the other side we could say that at last a djangified database is a truly object oriented database.
You'd be surprised how many developers are working on Django projects right now who have never written a line of SQL in their entire lives.
Instead of thinking about data and business logic, they're preoccupied by code and separation of concerns. That's why we end up with database that are unqueryable from outside of Django's ORM itself.
Good luck if you end up working in a project that makes heavy use of GenericForeignKeys.
Model inheritance is indeed hell. Doesn't necessarily mean it should be removed (it might be a good shortcut and might be fine depending on your performance requirements) but it should be handled with care for sure. Same with generic foreign keys.
I don't necessarily agree with your objections around how Django apps structure their database though. The need to access said database outside of Django should be a business requirement that's considered and if needed taken into account when designing the models. For projects that don't have said requirement it's not a problem because nothing meaningful should be accessing the DB (and if it does, then the application should be changed). It's kind of complaining that some third-party application's internal data file format doesn't suit your needs - well yes obviously it as it was never designed with you in mind in the first place.
I'm sorry you didn't find it useful. No need to dissuade others from reading it, especially beginners, because it will help them write better Django code.
I found it useful. I've been using Django casually in hobby projects for a few years now and your article just taught me about the N+1 query problem, so thanks!
An article about harmful defaults in Django with no mention of how the default settings.py encourages developers to commit secrets to their git history? Sad!
I like Django: it’s admin interface is worth the price of admission, but encouraging people to put their database credentials directly into settings.py is setting developers up for failure. Anyone who claims “just be more diligent, bro” is someone who does not take security seriously.
I wish Django was more "12-factor" compliant by default. Reading secrets from environment variables should be a first-class citizen and it's not. I typically use python-decouple (https://github.com/henriquebastos/python-decouple/) for that but it would be great to have it as default.
79 comments
[ 4.1 ms ] story [ 162 ms ] threadFurthermore, this:
> As you can see, Django doesn’t have your best interest at heart. Instead of providing sane defaults, it punishes you for not being a master at the framework.
Is just nonsense.
Regarding 1. While this is certainly an issue, it’s an issue for anyone using any framework and a challenge of database-backed web applications everywhere. It’s also heavily documented by Django and one of the first things mentioned. Lazy querysets are actually very useful when you need to build queries over a number of steps. Explicit query sets can have their own subtle issues with performance.
2. The default user model is fine for most users. It’s not a bad default and there is clear documentation on how to extend it. It’s designed to be extended!
3. I rarely rename migrations. Maybe this is a good idea, but it’s hardly a harmful default.
4. I almost never rename my tables. Again, maybe a sensible suggestion, not a harmful default.
5. Strangely this seems to be the most un-Django suggestion that would have the largest effect on a project but hardly any detail is given here. I never do this and wouldn’t recommend anyone else to when staring off.
6. I think project structure and CLI is one place where Django is weak but again this is hardly a harmful default. Use django-cookiecutter.
Anyway I feel like the author is trying to manufacturer controversy to help define their expertise (not to say they aren’t a good Django developer, I have no idea). Some of these are good suggestions and “top tips” but they’re certainly not instances of Django trying to harm you.
It's like 4 lines of code that saves you so much grief later on:
If you never need it, those 3 lines are not hurting you. In terms of a default, I think it probably would be nicer if Django pushed you to do this up front.No, there are frameworks out there that doesn't allow queries during template rendering. See Lucky for an example, https://luckyframework.org/.
The solution of using eager loading has nothing to do with whether you run your query in the template or in code.
Anyone who claims that it's fine to run queries in templates has smoked something really good. Personally, I'll continue to aggregate all my data using JOINs before presenting them.
However due to how Django handles queries by default, calling the ORM won't actually execute the query until the queryset is evaluated in most cases (unless requesting a single object or doing an aggregation), so your queries are still technically "ran" from the template - even if you are doing prefetch_related.
I personally don't have a problem with that; explicitly casting the queryset to a list before rendering a template seems like unnecessary boilerplate and still doesn't actually isolate you from the "N+1" queries problem if your template then accesses an object's relation that hasn't been prefetched.
You'll never have to worry about whether {{ pizza.topping.name }} is hitting the database ever again, especially, if you install django-zen-queries because it will raise an exception if you try to.
I'm not necessarily trying to avoid third-party packages, in fact I've installed the package you suggested in one of my projects and will be trying it out.
Some solutions are questionable. Executing querysets as soon as you define them means you can't compose them. Lazy quersets are not dangerous and you don't execute that many because they are lazy, but because under the hood you haven't asked for a JOIN statement.
Automatic database table names are definitely not a problematic feature. 99% of the time it gives you very intuitive mapping between model name and database table name. If you actually want to move model to a different app, then you can use the same feature (db_table) to point to its old location.
This article would benefit from more humility.
(I’m the author of django-zen-queries)
I’m not a fan of the hyperbole in the article either. Some of the points are genuine problems but none of them should put off beginners.
I generally don't need to optimize my queries because my apps are small, but I should do that by default, because otherwise it's wasteful and it's not hard to do. It's just that Django's default doesn't encourage you too, but hopefully with Zen Queries it will.
It's just a way to make less mistakes.
Same situation here. I find that being mindful of the ORM is enough to save me from obviously bad queries (fetching 100 objects and then running additional queries for each one), for the rest I get away with it as it's not a problem in practice. Sure, I will probably save an extra milisecond here and there and maybe in the end be able to run my entire DB off a very small computer, but (bare-metal) hardware is so cheap that a few stray queries here and there won't break the bank.
Nevertheless I've installed the package and will be trying it out, but I have yet to be convinced that eliminating additional queries all the time is worth the extra developer effort.
In my entire career I've got away with just relying on a "gut feeling" about whether a query needs a select/prefetch_related and it has yet to fail me. Yes, you're going to have a handful of stray queries here and there that could technically be avoided, but is the extra developer effort worth it? Hardware is very cheap (well if you know where to look, definitely don't look in the clouds), take advantage of that.
Unless you're programming purely for the sake of programming, what matters is the end result and the problem your application is solving. Your code doesn't have to be 100% perfect (otherwise why are you using a dynamically-typed, interpreted language?), it just has to work well enough to solve the business problem - other tools such as monitoring will quickly tell you if you've made a mistake and you can go back to the drawing board and add that extra select/prefetch_related you missed the first time.
Lazy querysets aren't dangerous but it's always best to evaluate them before rendering anything in order to avoid secret SQL queries. Many developers never run EXPLAIN, so these extra queries go unnoticed until your server crashes during Black Friday.
I don't see why your database table needs to encapsulate the app's name. Most apps get rewritten at some point but databases can live for centuries. Why couple code with your data?
Maybe I should be more humble to please sensitive people like you but no thanks. If a simple blog post hurts your feelings then you need to professional help.
Because Django assumes that in most cases your DB will be primarily (only?) be used by the Django app. In my experience that is true of most startups and early-stage projects, and the default names have never been a blocker even when you start giving others access to that database (hopefully with a read-only user account).
> Most apps get rewritten at some point but databases can live for centuries.
You can rename tables when this becomes needed. I don't think it's worth wasting time doing so at the beginning (you could equally argue that whatever name you pick manually isn't going to be suitable in the future either, so a rename will be needed either way).
> Why couple code with your data?
Renaming tables doesn't realistically address that coupling by any means.
I found that django's naming scheme for migrations is good enough; here are some real ones that I pulled out of my current project:
While they aren't semantically rich ('Why did you change absolute url on menu item?') I found that they are easy to distinguish and navigate.I much prefer the timestamp based system that is used by Rails, it make the risk of that kind of conflict a lot lower.
The migration file will contain explicit dependency information, something like:
The migration engine will order the dependencies at runtime, and it will bail (and suggest creating a "merge" migration) if you have diverging trees of migrations. I find it pretty robust in practice.We have this happen at work, when merging branches; this is what `./manage.py makemigrations --merge` exists for.
That said, it's very annoying in that they're not tab-completable... any time I have to edit one, I have to stop and hunt in the entire list for what I'm looking for.
I wouldn’t say any of these are “harmful”, there are certainly some things here that have valid arguments for doing differently, but most is rubbish. But it needs to be taken into account that Django is an incredibly mature framework with literally millions of projects built on it. Large changes are hard, and Django has had to make many of them before. All of these have been debated and haven’t changed either due to immovable technical debt, a decision that there isn’t at this time a need, or there is valid arguments on both sides.
A couple of examples: the user model, providing a default is important but it is also well documented that if you want to extend it you should subclass it from the beginning to make your life easier. There are ways to migrate later, they aren’t exactly easy, but it is doable.
Multiple small apps, this architecture is due to the history of Django and its development at LWJ. They wanted a way of being able to quickly build new websites out of pre-defined “apps”, and this system works for that. At the time the Python packaging ecosystem was very different, hence then slightly odd design. I’m sure in hindsight the founding team would have done things differently, but this is 17 years ago.
The general rule of thumb for “app” architecture now with Django is if you are a consultancy/agency building multiple similar sites for multiple clients (this is a huge use case for Django, closest to it original vision) it makes sense to package up your functionality into reusable apps. If you are a “startup” or building an internal app for a company with no need for reusability it’s best to just have one “core” app to contain all your functionality. It comes down to your personal needs, but it’s easy to make both architectures work.
You also avoid merge conflicts in teams, especially with migrations. I'm now working with Dotnet, because I have to (nobody wants to hire me as a Django developer even though I have 10+ years experience in Django and <0.5y in .NET), and this is an ongoing problem with EF core...
Shoot me an email ;) hn@initialise.io
1. Relying on implicit SQL queries
2. Using the default User model
3. Using automatic migration names
4. Relying on automatic database table names
5. Using multiple tiny apps
6. Dumping apps in the root directory
I wouldn't call those harmful. Most are non-issues, while others are "unfortunate" at best.
1. I found that the ORM works quite well and it's easy to optimize later. Using `prefetch_related` or `select_related` in the ORM still feels weird (they're just joins), but I wouldn't call them harmful.
2. I found it unfortunate that the info on how to change the User model is hidden so deep in the Documentation. I don't like to RTFM, but with Django it sure is worth it; same goes for 1.
3. I occasionally do it, but most of the time, I don't and never had an issue.
4. It do rely on that and found django's naming scheme to be a non-issue. I guess this might be more important if you need to change your database out of band (out of django band, that is)
5. This point is unsubstantiated. Apps aren't mere packages but also define a scope for the admin tool (which I use a lot and think is an integral part of Django) so that it will group models and allow individual access (i.e. apps also provide scope for your permissions). I use tons of apps but I wouldn't call it a harmful default.
6. Yeah, dynamic languages might produce naming collisions. Still beats imports like `org.example.publib.boulder.yourmoms.maidenname.models.impl.Account`
Multiple apps are definitely good for the admin but what I was advocating isn't 100% avoidance of apps but to avoid creating apps for every tiny feature.
Personally, I like my databases to be designed for longevity and avoid couple code to my data. Code can die at any point in time but data is an asset that needs to live for as long as the company is in operation.
If you're a beginner dropped into a non-trivial production codebase your best bet is to follow existing conventions until you gain enough experience to be able to evaluate such advice and consider it against your projects' and business' requirements.
My real criticism of Django is that, at long last, I think it's starting to look dated. Templating hasn't been the preferred way of making web app UIs for some time. Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets. Setting up a project to do this is cumbersome. While there are third party solves, Django's greatness comes in part from how opinionated it is.
I understand DSF may see adding more javascript as out of scope, but in keeping with the "batteries included" philosophy I hope they one day embrace something in contrib.
> My real criticism of Django is that, at long last, I think it's starting to look dated.
Actually, that is something I found really great about django. I found that django is reliable over a long period of time. The js/node/react/express/angular apps I had developed in the past (~2012+) can't be built these days without redoing major components. I never had these problems with django.
As for the responsiveness of the results, I found that a good understanding of django's performance helps. Using a good caching strategy (on models, views + nginx) yields results have almost instant reactions, even though there is the whole HTTP to HTML roundtrip. While they do have the noticable page-reload semantics, I found that these are way easier to build and sometimes yield very enjoyable experiences for my users. After all, I'm not building a YouTube so PictureInPicture and SPA-semantics aren't that important to my users.
> Modern web apps are maintaining models in the client and staying current though a combination of ajax queries and websockets.
Julia Elman explains how to do this with Django + DRF and Backbone.js in the book "Lightweight Django" (O'Reilly). Is this a viable compromise?
Django contrib exists to be a home for first party solutions to specific problems [1]. It need not be an entire frontend framework that follows trends like react, but something to make syncing state easier sits squarely in that namespace.
1 https://docs.djangoproject.com/en/4.0/ref/contrib/
> This means that you don’t own your database — the Django foundation does!
This is a completely baseless accusation. At no point are you unable to change the schema or replace the user model (which we have done with admittedly some kerfuffle).
If the author is reading this comment, and this is the reaction you were looking for from the community. Congratulations. You have written a completely asinine article about a technology you obviously do not understand yourself. Telling people to rename tables because "a future data analyst is not going to know what a table is for". On what planet is a data analyst given raw access to the production instance's tables of a database and not a view on a read replica.
Having one migration per model is dumb as all hell, as you would need to manually separate these, achieving nothing of value in the process.
Know-it-all anuses like the author of the article is why software engineering has become to be known as "a black hole of cash" because these are the type of people who pretend to know what they are talking about but make decisions that cost people exponential money and time.
Please stop writing sensationalist, clickbait bullshit like this and please, please read at least a small part of the documentation before writing something misleading like this.
Not once was I confused by the table or column naming.
I was kind of annoyed by all the long table names and all the joins I had to write out, but it was all fairly tidy and organized, and I almost prefer it to the human-designed databases I've worked with.
Theoretically yes if they one day renamed the app then there would be some legacy names in the table. Who cares, shit happens.
Regarding read-only access: we were a small enough organization that I actually did have full access to the prod database. I hated having it and tried to work off of our "QA" clone whenever possible, or made my own local timestamped copy of the prod data.
1) ORMs have the concept of lazy vs eager, no matter which framework. It’ll bite if you don’t know what you’re doing in any language. They’re still less bad than raw sql.
2) a default user model is better than any attempt an average developer would implement themselves. Go on, I dare you to try and hash passwords correctly.
3) if I wanna know wtf a data migration is for, I’ll look at the commit message, and code changes.
4-6 ) these are opinions of the Django team. I don’t really agree with them, but I chose to use their software. You know what doesn’t have opinions? Raw php, and I know which language/framework I’d prefer..
A custom user model doesn't even have to contain anything, it can be empty. See: https://docs.djangoproject.com/en/4.0/topics/auth/customizin...
"If you’re starting a new project, it’s highly recommended to set up a custom user model, even if the default User model is sufficient for you."
Adding an empty, but custom, user model derived from the base model could be the default. But it isn't, instead there is this recommendation, which you will only read when it is too late (on your first project).
I agree that most other points are garbage though.
Tell me, if I have {{ customer.auth_user.email }}. Where in this line of code does it say that a SQL query is going to be performed? So much love for lazy queries eh.
You're so new to Django and yet you pretend to understand everything.
I don't understand how I'm supposed to help someone like you. I hope your tech lead is able to guide you through this.
It is implied when you're using a framework with an ORM - after all, how do you end up with a "customer" object to begin with instead of a list representing a DB row?
> So much love for lazy queries eh.
When it comes to performance there's always a trade-off. The same reasoning could be applied to using Python as opposed to languages that compile to machine code, or for that matter using Linux instead of running your code on bare-metal.
Lazy queries trade-off performance in favour of developer productivity. In some cases it's a major problem (fetching 100 objects and looking up related objects for each one), in other cases it might be imperceptible and the time spent fixing them can be probably spent better elsewhere.
> You're so new to Django and yet you pretend to understand everything.
Knowledge of lazy queries and how they work is definitely valuable, but my impression is that most people here do know how they work and what are their pitfalls - they simply disagree with your opinion that "all lazy queries are bad".
> I don't understand how I'm supposed to help someone like you. I hope your tech lead is able to guide you through this.
Some people here (including me) don't agree with your black & white thinking about how lazy queries are always bad. They know their application, requirements and performance targets better than you do and have decided that for their use-case lazy queries aren't a problem in most cases (and can address the problematic cases individually without disabling lazy queries globally and having to waste hours trawling through their codebase for every single place to put a select/prefetch_related with no worthwhile performance gain in practice).
2. One migration file per model is easier to manage. It's my opinion of course, but think about how you'd squash migrations for old models if you're in the habit of bundling multiple RunPython or RunSQL methods in the same migration file.
I wish you'd think straight instead of being angry. It's not like I'm claiming that the earth is flat.
Squashing is something Django will handle for you through a management command, at which point it doesn't matter whether you're doing one model per migration file or not. Stuff that cannot be squashed (RunSQL/Python operations) will be pointed out by Django and I'm not sure how having one model per migration will help - the difficulty here isn't locating the offending operation, it's to actually figure out whether that operation is still needed and if so whether it can be reworked.
I think this should be the default behaviour given that most beginning Django developers fall into this trap.
Meta point: I really enjoy articles that list the places noobies go wrong (for any language/framework, not exclusively django). E.g. an experienced dev I taught R to was mind-blown that vec[1] accesses the first element of a vector (not vec[0]).
Long lists of these unexpected (albeit simple) things can quickly be compiled. It might only take 30 minutes for a noobie to read but probably ends up saving days of pain down the track.
I just wanted to point out some mistakes that many beginners make and that I made when I first started.
Personally, I think Django could use better opinions from the core team because beginners should be set for success by default. In the current state, that's not the case.
I know :) I was alluding to that exact point. I think other commenters thought that's what you meant, and jumped to the defence of the framework. But I agree that criticism is healthy, and it doesn't mean you don't enjoy or would stop using it. As someone who hasn't done much django, I appreciate your tips for beginners (especially in list format).
But the first pain point I very much agree with. Implicitly sending queries to the backend is one of those things that is an attempt to be very helpful to programmers, but that ends up as an obstacle that you always need to be wary of and avoid. Over time I have grown to appreciate explicitness in queries more and more, though I probably wouldn't go so far to remove lazy queries completely since they can be handy in query building, just remove the implicit loading feature if possible.
Django's query builder is also quite far removed from SQL. Especially once you get into more complex queries, I sometimes just went raw SQL as I knew I wouldn't be able to remember how to translate the Django expression into SQL in a month's time. Simple queries are easy to write and look very sleek, but complex ones can turn into a mess that is much more understandable in SQL form. The sleekness of the queries may also betray how complex they are to execute as it's not "in your face".
I still like Django's ORM quite a bit for many things, but after discovering Elixir's Ecto, I've enjoyed how explicit it is most of the time. I haven't needed implicit loading of relations once. It's also much closer to actual SQL. But there are things where Ecto also tries to be too helpful in my taste, like the humanised types in migrations (i.e. :utc_datetime instead of :timestamptz or :string instead of :"character varying"/:text) -- I strongly believe it would be easier in the long run for the programmer to specify the database types they want directly rather than through aliases which may result in surprising choices. Of course, that would cause issues in supporting multiple different databases, but that's not a goal I would go for in the first place, but I digress…
What I guess I'm saying is that sometimes projects build in helpful features to make the code nicer to look at, more approachable to beginners, or just shorter to write, but they often end up being a burden on those that go past the beginner stage. People invariably trip on them because who can remember everything mentioned in all the documentation? And then it's just a game of "who fails to avoid the trap every time" and that's a game that everyone loses eventually.
There are whole DjangoCon conference talks that go in length into these features even though I find them obvious.
Makes you realize how many beginners the Django community actually has.
Proper abstraction is the one thing you need to get right, and django makes it so easy to make clean separate modules work together using a shared base and way of working (user model, groups, permissions, urls, signals, etc.).
One day you'll need to use your payments logic to sell something else, embrace it.
I can easily extract the logic into it's own app if I need to publish it to PyPi for others to enjoy as well.
Of course, if you code is getting coupled, then you need to revise your design process.
The only sane way is to think about tables first, create models that mimic those tables, use makemigrations + migrate and absolutely no inheritance in the models.
An extra bonus is only one app per project. I've got a customer with two projects: one with many apps and a lot of inheritance, the other one with one big app and a few very small ones and no inheritance. We can use any SQL tool to extract data from the database of the second project. The database of the first one is "spaghetti data" and "python3 manage.py shell" is the only way to make sense of it.
On the other side we could say that at last a djangified database is a truly object oriented database.
Instead of thinking about data and business logic, they're preoccupied by code and separation of concerns. That's why we end up with database that are unqueryable from outside of Django's ORM itself.
Good luck if you end up working in a project that makes heavy use of GenericForeignKeys.
I don't necessarily agree with your objections around how Django apps structure their database though. The need to access said database outside of Django should be a business requirement that's considered and if needed taken into account when designing the models. For projects that don't have said requirement it's not a problem because nothing meaningful should be accessing the DB (and if it does, then the application should be changed). It's kind of complaining that some third-party application's internal data file format doesn't suit your needs - well yes obviously it as it was never designed with you in mind in the first place.
inb4 some grumpy guy shoots me down for claiming that his favorite framework isn't perfect.