I am quite amazed this has to be done manually in Django. I work a lot with Symfony2 in PHP and that kind of Object serialization thing is pretty much built in. I just return my model from the controller and it gets serialized to JSON including child objects with pretty fine grained controls about what to serialize.
Besides, after working a lot with REST webservices and AngularJS, i don't know if serializing child objects is really a good idea vs just getting the id and then using the api endpoint for the specific children separately. Having child objects serialized creates problems when doing PUT/UPDATE requests and you can also end up with fairly large sets of JSON. Of course doing dozens of requests instead (especially for lists) is also not the best of ideas, so i am not really sure how others handle this ?
Correct. It's very easy to create a simple model serializer and this is what the general model serializer does. It's no big deal just saves a few line of code.
In Django Rest Framework (DRF) you need serializer class. Maybe this is because DRF is not tightly coupled with the models from Django framework.
As far as I know DRF does not support PUT/UPDATE operations for nested serializers. But there are cases when you just need to read. Like getting an activity feed which can't be modified directly by the users. I think if you have a good caching system it's better to return all the data for one request instead of creating new requests to get details about each entity.
We tend to load things (GET) in one large request that pulls a big hunks of JSON for everything certain parts of the app need. These have lots of nested serialization going on.
We then have separate, traditional REST endpoints for updates/creates and incremental updates of the larger object.
This is an edge case for something called Django Content Types. It's basically a way to handle some sort of polymorphism in a table.
Let's say you have a table called activity feed like in this example. Whether the items in the feed are images, blog posts, or status updates they're all going to have something in common and you'll have endpoints that should link any relevant tables and combine them. So you'd have an activity table and it would have a column for content_type and then content_object_id. Content type is basically just a reference to another DB table (blog or status update) with more info on this activity and content_object_id is the actual id of the row in that table.
I've used Symfony, Laravel, and pretty much every other trendy PHP framework and nothing comes close to the ease of building APIs that is inherent in Django Rest Framework.
This is just a very specific use case that you normally have to handle using it's own custom class. This just shows a way you can have DRF serialize it like a normal object without issue.
There's not a general solution to serializing child objects, it totally depends on the use case and the number of children. So for example if you have a lot of child objects it might not even make sense to include theirs ids, but have a seperate (maybe nested) resource to retrieve them. On the other hand if there are only few children you might want to include them in their parent resource.
I really have a problem with the concept of the Serializer in DRF. It handles so much that has nothing to do with serialization. It performs something like de-serialization, where it takes an incoming dictionary and turns it into an ORM object. The docs even recommend using its ".save()" method. Save? on a Serializer?
I think it's a really cool concept if you think of it as more of a Lens. The so-called serializer lets you declare properties of "fields", and these properties will influence both the native-to-ORM conversion, as well as the ORM-to-native conversion.
"The best documentation is the source code" :-) . I use to read it but sometimes is really hard to understand what's going on. As you said the serializer does a lot of thing.
The model / model manager layer is the right place at which to design your state changing API.
REST framework absolutely you to work with the grain there. Some good practice I'd recommend...
* Write the `create()` and/or `update()` methods explicitly on the serializer class.
* Push logic into the model and model manager where possible and only have the serializer `.save()` as a thin layer on top of that.
That way a serializer class still has all the behavior it needs to map both ways between persisted objects and their corresponding native python representations, but you still have a well separated model API.
Interesting. In practice, I have started following a pattern where I have two Serializer classes - a "serializer" and a "deserializer".
The "deserializer"'s job is to perform validation and type coercion of incoming requests. It returns a "native" dictionary, which the application code then saves to ORM.
The "serializer" is basically a presentation layer. It calls out to other nested serializers [this is awesome!], and it throws in convenience-fields that make the API response easier to consume.
This pattern makes it a little bit less "magical", and it's easier to distinguish between the API's "incoming" behavior and its "outgoing" behavior.
Their API/role is very similar to the `forms` in standard Django. I agree with you that they handle too much stuff and that saving to the database has nothing to do in a serializer, just like it has nothing to do in a form. But I also think it's a very good choice to follow common Django practices, so that codebases stay somewhat consistent, and new users are able to understand the concepts very quickly.
I was thinking the exact same thing when I first read about DRF's Serializers. Also I don't quite get why serialization is always expected to work automatically, at least not in the context of mapping a business layer to an API layer (e.g. for JSON consumption), as those domains don't have to map directly onto each other.
Therefore, for an ongoing project I've implemented something along the lines of your idea (if I got that correctly), by describing API-Layer models in terms of properties, which are just named sets of pure lambdas. The most general property only requires a single one to compute the property value from an input business-model.
This allows for some very nice things, especially with regards to API-level functionality like support for nested ?expand=, ?only=, and such niceties.
The big problem for me is that serializers are bidirectional. They should only work in one direction: server to client. Client to server should be either handled by a wrapper around Django form or by a separate deserializer type. Why? Take a simple example: you have a game character model which consists of things like name, color, and a set of randomly generated attributes such as strength, agility, etc. Now, you wan the server to generate the attributes, but not the name and the color. You write your code, do a POST to /characters/ and BAM: {"strength": ["This field is required."], ...}. Now you have to tell the serializer that this field is not required...
Oh, and try customizing error messages or providing customer error conditions... It's just not fun. I think this is the weakest part of DRF.
I think it's somewhat rare to have generic relations (content type + model id) popping up this way in serializers. I tend to use generic FKs for things like "tags" or or "comments" that can apply to multiple objects. As a result, I'm almost always requesting the parent object ("post") and want the model with the generic FK serialized in that response ("post.comments"), not the other way around.
Just looking through largish DRF codebase, I'm not sure there are many places I could use this, thought it seems like a good idea if your code has a lot of these scenarios.
You can use Spyne[1]'s HttpRpc and JsonDocument protocols via Flask-Spyne[2].
You can actually use Spyne with Django as well[3] (and Spyne has a "general model serializer" that can be used with a lot less boilerplate than what's shown here).
Spyne supports a lot of protocols as well as persisting to a relational database via SQLAlchemy.
Hi languagehacker,
this is not science fiction. It just shows you a way of implementing a serializer that works for all kind of models in Django Rest Framework.
26 comments
[ 3.6 ms ] story [ 53.1 ms ] threadBesides, after working a lot with REST webservices and AngularJS, i don't know if serializing child objects is really a good idea vs just getting the id and then using the api endpoint for the specific children separately. Having child objects serialized creates problems when doing PUT/UPDATE requests and you can also end up with fairly large sets of JSON. Of course doing dozens of requests instead (especially for lists) is also not the best of ideas, so i am not really sure how others handle this ?
But DRF adds lots of extra functionality. Even with DRF it's as simple as adding:
to get full automatic serialization of a model.> I don't know if serializing child objects is really a good idea
Depends on the specific case. For internal APIs I tend to create endpoints that give a view what it needs in a single request.
As far as I know DRF does not support PUT/UPDATE operations for nested serializers. But there are cases when you just need to read. Like getting an activity feed which can't be modified directly by the users. I think if you have a good caching system it's better to return all the data for one request instead of creating new requests to get details about each entity.
We then have separate, traditional REST endpoints for updates/creates and incremental updates of the larger object.
This is an edge case for something called Django Content Types. It's basically a way to handle some sort of polymorphism in a table.
Let's say you have a table called activity feed like in this example. Whether the items in the feed are images, blog posts, or status updates they're all going to have something in common and you'll have endpoints that should link any relevant tables and combine them. So you'd have an activity table and it would have a column for content_type and then content_object_id. Content type is basically just a reference to another DB table (blog or status update) with more info on this activity and content_object_id is the actual id of the row in that table.
I've used Symfony, Laravel, and pretty much every other trendy PHP framework and nothing comes close to the ease of building APIs that is inherent in Django Rest Framework.
This is just a very specific use case that you normally have to handle using it's own custom class. This just shows a way you can have DRF serialize it like a normal object without issue.
I think it's a really cool concept if you think of it as more of a Lens. The so-called serializer lets you declare properties of "fields", and these properties will influence both the native-to-ORM conversion, as well as the ORM-to-native conversion.
* Write the `create()` and/or `update()` methods explicitly on the serializer class.
* Push logic into the model and model manager where possible and only have the serializer `.save()` as a thin layer on top of that.
That way a serializer class still has all the behavior it needs to map both ways between persisted objects and their corresponding native python representations, but you still have a well separated model API.
The "deserializer"'s job is to perform validation and type coercion of incoming requests. It returns a "native" dictionary, which the application code then saves to ORM.
The "serializer" is basically a presentation layer. It calls out to other nested serializers [this is awesome!], and it throws in convenience-fields that make the API response easier to consume.
This pattern makes it a little bit less "magical", and it's easier to distinguish between the API's "incoming" behavior and its "outgoing" behavior.
Therefore, for an ongoing project I've implemented something along the lines of your idea (if I got that correctly), by describing API-Layer models in terms of properties, which are just named sets of pure lambdas. The most general property only requires a single one to compute the property value from an input business-model.
This allows for some very nice things, especially with regards to API-level functionality like support for nested ?expand=, ?only=, and such niceties.
Oh, and try customizing error messages or providing customer error conditions... It's just not fun. I think this is the weakest part of DRF.
Just looking through largish DRF codebase, I'm not sure there are many places I could use this, thought it seems like a good idea if your code has a lot of these scenarios.
You can actually use Spyne with Django as well[3] (and Spyne has a "general model serializer" that can be used with a lot less boilerplate than what's shown here).
Spyne supports a lot of protocols as well as persisting to a relational database via SQLAlchemy.
Disclaimer: I'm the author of spyne.
[1]: http://spyne.io
[2]: https://pypi.python.org/pypi/Flask-Spyne
[3]: https://github.com/arskom/spyne/tree/master/examples/django/...