Ask HN: Nested Resources in REST/HTTP API URLs?
If you're building a REST/HTTP API, how do you think about when to express the hierarchy of resources in the URLs?
For example, let's say you had a blog site. Organizations have blogs. Blogs are made up of sections and comment threads. Comment threads have individual comments.
Would you opt for:
- /organizations/:organizationId/blogs/:blogId OR /blogs/:blogId (and get the organization from somewhere else like an auth token)
- /organizations/:organizationId/blogs/:blogId/sections/:sectionId OR /blogs/:blogId/sections/:sectionId OR /sections/:sectionId
- /organizations/:organizationId/blogs/:blogId/threads/:threadId/comments/:commentId... you get the idea
71 comments
[ 3.8 ms ] story [ 126 ms ] threadQuick 3 min draft example: https://gist.github.com/djames-bloom/2131d021a8f6d3d09c5a808...
I think it doesn’t quite answer what I’m after though.
I’m thinking less about resource discoverability and more about authorization and fetching resources from the database. For example, if the URL doesn’t include the post ID and I need that to check if the user is authorized to post a comment, I’d first need to look up the comment thread to get the post.
In this case, the URLs never included anything in the route path that would have let you know that a user could execute an action. Instead, resource actions were limited to/owned by the shared key. A request to a route that had a different secret key owner would simply fail authorization unless the signature matched. The authorization headers included the API_ID that let the server side know which secret key to use to check the signature.
This felt pretty good to me at the time - I basically looked at what AWS was doing with REST signatures and implemented a version of that for my purposes.
- /organizations/:id
- /blogs/:id
- /sections/:id
- /threads/:id
- /comments/:id
Why?
What determines how resources are related are links, not patterns in the URL. It's a graph, the URLs are just nodes, the links are what connect them.
If you _want_ to have some sort of hierarchy in the URL, you can redirect:
- /organizations/123/blogs/1234 -> /blogs/1234
Then each resource expresses how it is related to others using links (hrefs, or other mechanism for orther media types).
URIs can be anything like /ajndkandkjnasd, totally unreadable for humans. If it is a resource that contains links that can be followed, then it is a system that answers to a uniform interface, and that is the part of the REST dissertation that really matters (the other stuff are just implications of having such uniform interface).
For machine APIs, payloads could use "comment_id", "thread_id" and so on referring to a single resource. If any API users need to build URLs, they would do so by using a single property of the data (which is good enough as a link for me nowadays, for private APIs at least).
Nesting entities seems like the right approach at first but can turn into a nightmare as you continue to build things out.
> - /blogs/:id
The pragmatic, large-company-only counterpoint is the narrow edge case where:
- :id must be human-readable for "SEO reasons"
- there are many competing organizations and blogs to the point where there may be a name collision.
Although in that case, I'd still suggest:
/:organization-name/:blog-name
Whether it's a good idea to use this kind of composite key is a separate question, though.
Complicated URLs are hard to remember and annoying to share. The one benefit of the nested URL is that you get is of being able to "inspect" what organization or section or comment a piece of content belongs to. This is probably a rare edge case and something you should only implement if it makes your system much more usable, for some reason.
e.g. I'd probably rather have podcastsite.com/:podcastname/:podcastepisode/:comment_id
The thing after the '#', which has been called many things in the past (hash fragment, I guess), never goes into the server, so you can play with it using JavaScript to make it meaningful to users, while keeping it simple and flat on the real HTTP requests.
> - /organizations/123/blogs/1234 -> /blogs/1234
And:
/organizations/123/blogs -> /blogs?organization_id=123
Because what you're doing is just applying a filter to an index of blog resources.
I.e. the "/organizations/123/blogs" route is simply an alias to "/blogs?organization_id=123" which you can just redirect to in your gateway.
If it's /org/orgId/blog/blogId and a user has access to only a single orgId and needs to be logged in then doing it in /blog/blogId makes sense.
But if a user has acess to multiple orgs via different orgIds, then it makes more sense to do the former with /org/orgId/blog/blogId because then the page is sharable via URL. If you do just /blog/blogId that prevents a user from easily accessing a new organization (unless you use query parameters)
Eg. List all comments of a blog?
It has to be GET /blog/:blogid/comments
Then, how about creating a new comment?
It'll be POST /blog/:blogid/comments
When both of these have the blog's hierarchy in the URL, should we take it away when fetching a single item? Or the stay consistent, we should simply do:
/blog/:blogid/comments/:commentid
It's hard.
You're just asking for one of the most basic features in resource-oriented architectures: query a collection resource with a filter predicate and possibly pagination.
GET /comments?blog=<blogId>
> Then, how about creating a new comment?
Same thing: just POST it to /comments with a relevant request document, such as
{"blog":<blogId>,"comment":<blah blah blah>}
Your POST request then receives a response with a link to the newly-created comment resource, and that's it.
> When both of these have the blog's hierarchy in the URL, should we take it away when fetching a single item?
Resources do not have a hierarchy per se. You've just been passing request parameters through the path. That's it.
> It's hard.
Not really. It's a matter of understanding that resource-based architectures just deal with resources, and not resource hierarchies which don't really exist per se.
What you mean by hierarchy is actually just passing parameters through the path, but those parameters can be passed elsewhere.
GET /comments?blogId=<id>
Creating a new comment: POST /comments (blog_id, author_id, category_id all in the post body)
Single comment: GET /comments/1234
You can pretty much follow all the same normalization rules you would for a SQL database. /blog_comments is a kind of query in a virtual N-to-N resource that maps blogs to comments, get it?
Consider for example, "all posts that this particular author commented on". That would be insane to put into a hierarchy URL. For a flat one, you can just:
GET /commented_by/:authorId
Boom, done. Sometimes, you'll need more than one id, of course, but that does not imply "nesting". Consider "all posts where these two authors interact on comments":
GET /discussions_involving/:authorId1/:authorId2
There is a cap in what you can do with a hierachy, and the web is not a filesystem with folders, it's a graph with unlimited possibilities.
GET /discussions?author=:authorId1,:authorId2
If both authors had participated.
Or if only one or the other had participated:
GET /discussions?author=:authorId1&author=:authorId2
I guess the choice on the URL format depends on how you're going to route that on the server side, so there are multiple ways of doing it.
Depending on the nature of the resource, you might have to be careful with URI canonicalization.
Consider for example the "diff between account balances" endpoint:
GET /account_balance_diff?accid=1&accid=2
Should the diff be presented as from 1 to 2, or from 2 to 1? Query string parameters don't have a particular order to them, and when canonicalizing, some user agent might decide to reorder these parameters.
If you do:
GET /account_balance_diff/1/2
Then, there are two distinct URIs (one for diff 1->2 and other 2->1) and no ambiguity on meaning.
You could also use some kind of index on the parameters to preserve order:
GET /account_balance_diff?acc[1]=123&acc[2]=456
Your other example using a comma should also be fine:
GET /account_balance_diff?accs=1,2
Let's go back to the "discussions" example. What should happen if I GET /discussions without any author id? Our inner guts tell us that there should be something there (after all, I'm filtering _something_), but REST implies absolutely nothing about this relation. To REST and HTTP, you could have a /discussions URL that is completely unrelated to /discussions?filter.
Having a concise, clear URL forming pattern is great. It's not REST though, it's a separate thing, incredibly relevant to us humans, but irrelevant to the architectural style.
A similar confusion happens with error codes. I've seen a lot of people answer 406 Not Acceptable as a status for lock errors and invalid requests. It sounds nice, but it's not designed for that. 406 means the server can't deliver that media type (you asked for PNG but I only have JPG, for example). That 406 is part of the content negotiation mechanism of HTTP, not a lego block to reuse as application logic. The URI is the same, it's role is to be a primary key for the web, not to express hierarchy.
(btw sorry for the long post, I ended up venting a lot and got into several tangents completely unrelated to your comment)
You could try a HATEOAS approach if you want to get a bit more fancy.
https://restfulapi.net/hateoas/
If you are following REST strictly there should be only objects, actions and verbs.
Not quite. That would be a resource-based architectures much like REST, but REST dials requirements a notch higher, with stuff like adding capabiliy-discovering semantics to resources such as navigation links to related resources and operations to express it's state and state-changed, in the form of HATEOAS.
APIs are consumed by humans first. Machines don't care about your structure.
HATEOS is the part that actually doesn't matter in practice.
The part for humans is the HTML form I'm typing on. Humans _can_ type in HTTP/1.1 non-chunked requests by hand, but I don't see it that often.
For developers, you might think that the URIs are "the UI" of an API, but they were not designed to be used in that way.
We were meant to use the HTML rel= attribute to inform our machine clients on how to navigate links. We still do, sometimes. rel=stylesheet is a vestigial trace of that. rel=nofollow still informs clients of irrelevant URLs to this day.
We use HATEOAS all the time, we're just not using the standards designed to express them. A complete API schema (what seems to be the cool thing these days), in the eyes of REST, is nothing but a giant complex hypermedia form written in an unspecified media type. The "blog_id" is nothing but a lofi link, and so on.
Thanks!
I guess the one case where it might still make sense to have the organization ID in the path is if users can be in multiple organizations and some APIs involve listing all of a given resource for a certain organization.
I'd really recommend putting the organization name in the content of the page in that example. Users don't really look at URLs, browsers de-prioritize them and often only display part of the URL.
Also, depending on front-end implementation the REST API URL may not be displayed in the URL bar. A single page app may use example.com/#create-article for posting an article. A mobile app wouldn't even have a URL bar.
One solution is to always pass the intended org along in the POST, but requires something disciplined and nesting POST /org/A/articles is one way to do that.
example.org/2/1/4/12/326
If you want jsonapi, you probably really want graphql...
> ou cannot represent an API backed by jsonapi with swagger/openapi
This doesn't seem right to me. What issues were you having trying to specify a JSON:API service using OpenAPI?
I personally often use jsonapi.net[1], a C# implementation of JSONAPI. This supports OpenAPI/Swagger with swashbuckle, has a very good filtering implementation and together with Orbit.js[2] it is pretty much without having to decide too many things...
[1]: https://www.jsonapi.net/
[2]: https://orbitjs.com/
You can do the same even if some entry points are known, and then I'd follow the approach of /top-level/:id, because it’s at least a predictable way to start. If further API access requires discovery, go wild and format your URLs how it works best for your service. Just keep them unique and stable, because stable URLs are cool!
This is REST. As a backend dev I've attempted this several times, and the consumers of my API never cared about the hyperlinks.
I've tried explaining that the 'delete' hyperlink is only going to be present when the user is authorized to delete the resource, and isn't that wonderful and making your life easier? It turns out, (my) API consumers want to hardcode URLs on their side and have the payload include a can-delete flag. Same across several companies.
I've since mostly given up on making interfaces restful. Anyone has experience with this going the same or differently?
You're not going to fix that unless you (initially) make the URLs unpredictable (that is, make the links the only way to know a URL).
But that will (initially) be costly to do. And is probably annoying to your API consumers as well.
So only consider this if you need reduced coupling and there is clear value in avoiding hardcoded URLs.
I’ve seen it work exceedingly well with a browsable interface, like the one provided by Django REST Framework (not an endorsement of the framework per se; it does a lot of stuff very nicely, but other stuff can be a pain). If the API is human-discoverable, that makes the appeal of hyperlinks (actually linked in the browsable interface) a lot more obvious.
> I've tried explaining that the 'delete' hyperlink is only going to be present when the user is authorized to delete the resource, and isn't that wonderful and making your life easier? It turns out, (my) API consumers want to hardcode URLs on their side and have the payload include a can-delete flag. Same across several companies.
Not to nitpick the RESTfulness of this design, but I think it’s interesting that a more purist HATEOAS approach would be closer to what they wanted. Example response:
Note `Allow`, that’s a standard HTTP header many clients can parse and understand automatically, and might even give them their can-delete flag for free. They can hard code the URL if they like, but they already have the URL because it’s the same one, for the same resource they already requested. Or, if they requested a listing… sure, again, they can hard-code the listed resources but I think the convenience of including URLs is a probably a lot more obviously appealing there.Coming back to DRF’s browsable interface, it similarly uses standard HTTP semantics, in this case content negotiation. If it receives a request with `Accept: text/html` it responds with a web representation of whichever resource you requested. If the request is authorized to perform other actions on that resource—determined by the same `Allow` header—the web representation also includes a form to perform those actions.
This isn’t to say everyone will see the appeal! But in my experience it’s a lot easier to grasp when an API really does embrace HATEOAS.
Exact same experience. Advocating for HATEOS actually confused developers in my org. In the end, I had precisely zero consumers not hardcode URLs client-side.
I was pushing on my team for us to adopt more generic client-side code so that we could iterate on web UIs quicker but just couldn't get buy-in. I had a basic collection+json web client that I demo'd for browsing various services that returned that hypermedia type but people just weren't interested.
It was (and still is) the case that most of our internal apps fit the "simple collection" data and manipulation model. I probably blew it by not doing any nice css styling of the demo web client.
I stay from the web app space internally now. If I'm being honest, my org's web projects mostly fail these days. The common pattern goes something like legacy desktop app -> web app generates excitement (modern UI! node.js! CSS improvements!), dev starts, dev continues for indeterminate time, users lose interest and buy external tools, team inherits other legacy desktop apps, dev team adopts new legacy app to repeat on.
I'm embarrassed about my teams' results in this general space.
They do things like
/orgs/ORG/repos /repos/OWNER/REPO/issues
In other words, GH scopes their endpoints usually one maybe two layers. I think conceptually that makes a lot of sense
[0] https://docs.github.com/en/rest/repos/repos#list-organizatio...
You make it sound like the blog posts are keyed by (organizationId, blogId). If this is so, then /organizations/:organizationId/blogs/:blogId is clearly superior. But if blogId alone is sufficient to identify the blog post, then /blogs/:blogId is very likely to be wiser.
But you also make it sound like sessions might be scoped to an organization, that organization is, for API purposes, a singleton. In that case, /organizations/:organizationId (the route, not the prefix) would not make a great deal of sense: it should either be just /organization or not exist, and be removed as a prefix from other things.
This reasoning can be applied to the rest as well.
The other post on this thread are accurate, in that normally keeping it simpler is better, but in this case I disagree.
With the nested approach you have the ability to do multiple levels of validation and mitigate a lot of key guessing attacks.
Imagine a different scenario:
facility/:facility_id/provider/:provider_id/patient/:patient_id
Vs
patient/:patient_id
In the first case there are three tokens I'd need to possess or guess in order to access a patient record. In the second, there's just one.
The implementation will still only grab the patient id for the query, but it'll validate that the two other keys are correct as well.
It also makes role scoping easier. I.e. my role grants admin access to all patients in a facility. Middleware to validate the access is a lot easier to implement if you have all those keys in the query string. Declarative permissions on the routes are a lot easier.
So if you can find blogs just by their blogId, then `/blogs/:blogId` makes sense since you only provide the necessary parameters. `/organizations/:organizationId/blogs/:blogId` would imply that `blogId` is not unique across organizations. This may also make sense if that's how it is.
What I recommend is follow the resource standards from something like jsonapi, and organize your routes like this:
/organizations/:organization_id
/organizations/:organization_id/blogs -> returns a list of links to blogs (maybe including the very most basic data, like blog title and byline)
/blogs/:blog_id -> one blog, has a link to organization, for going "up" the hirearchy, but only returned for authenticated users (if that's necessary in your use case)
/blogs/:blog_id/sections -> links to list of sections
/blogs/:blog_id/threads -> links to list of threads
/threads/:thred_id -> one thread, but links to parents as above
/sections/:section_id -> same as threads
you get the idea.
I've done it this way and I feel like it ends up being very easy to wrap my head around where things are, and how they relate to one another.
One of my favorite example APIs is stripe. the layout makes sense, routes make sense, and it's a very complex system which becomes easy to understand and the routes aren't insanity, if you are looking for a very public example
It's also possible you will need at some point to re-use sub objects in a new context, so you'll have different kinds of paths to query those sub objects.
I'd go with Python's guideline "Flat is better than nested" here, and in doubt, follow it.
I think nested is only appropriate if you want to ensure the requester knows the oragnisationId, the blogId and the threadId of a particular comment to access it. Otherwise, it will lead to more complexity everywhere.
Or you don't check and just discard all the superfluous parameters, I've seen that too unfortunately.
All your points are valid of course :)
Quite like I would not use a function argument if I don't need it. So I would avoid putting these parameters in the function signature in the first place. It just leads to confusion at best, and there's a good reason why many compilers emit warnings for this.
I believe a function should only take the minimum it needs, and I believe APIs should not be different in this matter.
Redundancy is good when you actually need it, and not quite when you don't.
If you want your URLs to continue to work then you either need to (a) keep using the previous URL, meaning it's no longer meaningful to the content, (b) change the URL to match the new content, meaning existing links break or (c) support the old and the new versions with a redirection mechanism which is a bunch of effort.
Not saying you shouldn't use meaningful URLs but make sure you're going into the decision with your eyes open and preparing to accept the downsides.
Consistency across an API surface is more valuable than local optimizations.
However as soon as you need a M:M or many:1 relationship this breaks down. If a blog can be shared between multiple orgs, how do you represent that?
Therefore as others have recommended, a single ID per URL with a sub-endpoint for the nested list like /organizations/:organizationId/blogs/ (but no two-ID endpoints) is the generalizable and future-proof option. I strongly prefer to keep all the state in the URL rather than hitting /blogs/ and getting the org implicitly from the auth state. Way harder to debug if the auth context can change what results your API produces.
I would imagine something like graphql type of URL schema:
That said, I wouldn't recommend going the allow everything flexible resolver way like GraphQL: it's terrible for performance (eg. most APIs use N+1 queries unless you have something like https://github.com/join-monster/join-monster), the complexity of the codebase skyrockets and having to specify all the fields you want is not exactly ergonomic in most situations.
If you plan querying blogs per organization, then make it a su resource under the organization.
If you are planning only to get a full list, go for root level blogs.
If you are planning to do both then do both :-)
https://dev.tasubo.com/2021/08/quick-practical-introduction-...
A common pattern I follow with URLs is that I sometimes allow the user to omit the final ID in the URL; in this case I do a front-end redirect to the URL with the ID of the main/default resource appended at the end - This is useful if the user account has (for example) a 'main blog' associated with it.
- /organizations/:organizationId/blogs/:blogId
- /organizations/:organizationId/blogs/:blogId/sections/:sectionId
- /organizations/:organizationId/blogs/:blogId/threads/:threadId/comments/:commentId
I do this because it expresses the structure and the constraints of the data. A blog cannot exist without being linked to an organisation. And then you typically have the following CRUD endpoints:
- /organizations/:organizationId/blogs, with GET and POST to retrieve a list of blogs belonging to an organization and to create a blog for an organisation
- /organizations/:organizationId/blogs/:blogId, with GET, PUT and DELETE to retrieve, update and remove a blog
The fact that you always have the :organizationId in the url likely means that you can easily verify authorization, because I assume you would have a link between the user and an organization and maybe that link would already be available in the access token.
I don't think it is a problem that the urls are heavily nested. The urls are not supposed to be manually entered, but are using from within an application. I like the use of HATEOAS links, because I don't really like the frontend trying to assemble urls by itself.
Also note that next to the hierarchical links, you might need some extra for search that are not hierarchical. For example, say that there is a need to search blogs across organizations. This could be provided using the following:
- /blogs?description=test&language=en
You can provide a search endpoint in the root for blogs. This endpoint could then return some kind of blog object with a reference to the actual blog location (like /organizations/34/blogs/11). When you use this approach, the frontend has no need to assemble urls and the structure or complexity of the url does not matter. Do note that the frontend still needs some generic "start" urls like the blogs search endpoint.
Additional advantage (like others have mentioned), is that with the hierarchical urls it becomes more difficult to "guess" an url. Altough, if you want to properly protect against that, you should transform the ids before putting them in the url and transform them back when reading them out of an incoming url, for example using symmetric encryption with a secret only known in the backend.
Be careful with that! If there's a network error sending the response, the blog will have been created and you won't have received the ID on the client.
In a distributed environment (such as REST) it's better to make all calls idempotent. Choose an ID on the client e.g. generate a random UUID. Then do a PUT to ..../:id. That will update the blog if it exists or create it if not. If the create call fails, you can just retry it (with the same ID), if the request previously failed and it never got created it'll create it now, and if the response previously failed and it did get created but you never got the successful response it'll get "edited" to the same values as it already has.
* may be harder to route if you break your app into services, you are now relying on the server implementation to have fast regex routing
* harder to version the data models, if I need to add path versioning I’m stuck versioning and breaking the entire tree