32 comments

[ 4.7 ms ] story [ 98.4 ms ] thread
We do I18N by providing a (cachable) name/value based dictionary using the same REST API and return on error/validation the error source element and an I18N error key. Makes rendering error messages a breeze as we name the input elements by convention the same as the JSON property and therefore can easily use that property in JQuery selectors.
Sorry, but it lacks the most misunderstood parts of REST:

- Hypermedia as the Engine of Application State (HATEOAS)

- Returning concrete mediatypes (standard is great, custom is OK) instead of generic 'application/json' or 'application/xml'

It also contains some misunderstandings:

- Sending a sha1 of the password to the server is bad, for two reasons:

- - First, and more importantly, if you send the hash of the password over the wire, the hash becomes a plain-text password! Seems strange? Think about it: if the attacker somehow gets the hash, he can just use it as-is to authenticate to your API - he doesn't need to crack it.

If you're using HTTPS, just use Basic Auth. Otherwise, you should implement (read: use a library!) proper request signing.

- - Second, sha1 by itself is pitiful - it can be cracked in mere seconds. Use bcrypt (this, on the server).

    Once you know the top level objects and actions in your product,
    designing the endpoints becomes easier and clearer. For example,
    to “add” a new venue, you would probably have to call a method
    similar to /venues/add
Sorry, but no, no.

In REST, there should be a single endpoint. Everything else is done by navigating.

And in REST, there are no method calls! As Roy Fielding would say, this screams RPC! To add a venue you should just POST to /venues/ (URL which you discovered by navigating the API).

Read his blog post with some of the rules that make a REST API: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...

And that's a clear case of not being able to downvote submissions being regrettable.
not to mention version numbers (a part of content negotiation) in user agents

this isn't REST and only serves to confuse things further

I realise it's only part 1, but this post doesn't even mention hypermedia. Its advice on documentation says that you should explain how clients construct URIs. Groan.

I don't fully understand what it is about HATEOAS that people find difficult or unconvincing. They must use web sites every day. When you want to buy a book at Amazon, do you read their documentation and then construct a URL? No, you go to amazon.com and click on shit.

I must admit that the author's description of REST sounds awfully like what I thought REST was before I sat down and tried to design a RESTful API. There was a very distinct moment when I "got" the concept of only requiring a single URL and then navigating the resulting documents - as you say exactly like a web browser (which is, of course, the whole point).

It's clearly non-obvious to work with an API primarily as a set of documents rather than as a set of "methods" to be called - I have no idea why it's non-obvious at the start as it is exactly how the Web works and it's pretty obvious and elegant in retrospect.

(comment deleted)
I feel the same way. My first API on my current project was a horrible mess, and did I only realize why it sucked and what I needed to change when I decided to play with Backbone.js over a weekend using my API as a data source. The only way, I think, one can build a RESTful API is to do it from the client's perspective, much like BDD builds software using a list of actions the user wants to make.

If you're not using your own API, then it's probably broken.

I found that manually using my API from the command line using cURL helped a lot (with a bit of cutting and pasting to extract the URLs I want).
HATEOAS is absolutely not a simple matter of being able to 'click on shit' in a web service.

As a browser user, I can generally look at a web page and figure out what's a clickable link. The HATEOAS constraint is rather trickier than that: it requires that a client understand what is a "clickable" link in an HTTP response based on nothing other than the content type and the response body.

For the past couple of years I've been building RESTish web services using JSON. They use HTTP methods and status codes appropriately, each resource has a URL, my response objects include URLs to subsidiary resources, and so on. However, the services I've built are not RESTful because JSON is not a hypermedia content type.

To be truly RESTful, I would need to adopt and/or define a hypermedia content-type using JSON and make sure my response objects conform to that content-type.

I don't get it: what's so difficult about defining a format - encoded as JSON - in which certain strings are defined as being URLs you can navigate to?

You just say "This is the format application/vnd.myservice.userprofile+json". In it there's an object, which contains the key "avatar_url", which has the URL of the user's avatar image.

Frankly, I don't see what's tricky about this.

It's yet another implementation detail that drags the reality of RESTful web service development farther away from the principle that it's easy because it's based on HTTP and we already know HTTP.

I'm not writing this to dump on REST. I still think it's the right approach to build a web service over HTTP, but I'm also conscious of the diminishing returns on each incremental step toward pure RESTfulness. I'm just trying to make a web service, and suddenly I'm defining new content-types?

For a client, is it worth the trouble to learn a new content-type to determine what's a hyperlink in my JSON response, when they can just look at the response and see something like the following?

    GET /articles HTTP/1.1

    {
        "articles": [
            {
                "url": "/articles/1",
                "title": "This is my first article",
                "pubDate": "2012-01-05T08:39:47.625000
            },
            {
                "url": "/articles/2",
                "title": "This is my second article",
                "pubDate": "2012-01-13T11:07:35.219000
            }
        ]
    }
After a while it starts to feel like the HTML v. XHTML debate.
Maybe I'm missing something, but why the resistance to returning a "meaningful" content type in a response? It's just a name for the structure you have outlined there.
Two reasons that I can see...

If the server generates custom mimetypes, clients have to ensure they "Accept:" the correct mimetype for every request. And if you have multiple data types (and you will have multiple data types), then this "Accept:" header will vary for every request. Getting this right is an irritant to client developers with no actual benefit to them, and a great potential source of bugs.

Much easier is if the client can just send "Accept: application/json" or "Accept: application/xml" for every request.

If the server generates custom mimetypes, and you update your API to v2 which produces an article listing in an incompatible format, you either have to introduce a version number into your mimetype (which seems messy and pointless busywork), or stick with the same mimetype and accept that your custom mimetypes aren't actually particularly meaningful as their meaning can change substantially over time (so why did you have them in the first place?)

Much easier is if the server had just used "Content-Type: application/json" or "Content-Type: application/xml" which will always be correct.

If the server generates custom mimetypes, clients have to ensure they "Accept:" the correct mimetype for every request.

No, he doesn't. First, the server can send whatever it wants - nothing breaks if you reply with your custom media type to a generic "application/json" request.

Second, even if you want to be correct, that's what wildcards are for: the client can simply send always the same header:

     Accept: application/*+json
If the server generates custom mimetypes, and you update your API to v2 which produces an article listing in an incompatible format, you either have to introduce a version number into your mimetype (which seems messy and pointless busywork),

I still don't get what's the problem with changing a string. I think you're using a bad framework.

I doubt that sending a mimetype which the client has said it can't accept would be on anyone's list of best practices.

This is a good point, though, not sure why it escaped me:

    Accept: application/*+json
I think you're using a bad framework.

Probably. This isn't why, though.

(comment deleted)
So if you don't communicate that the version of the structure of your response has changed through the Content-Type, where would you do it?
I'm happy to change a server's content-type from application/json to application/vnd.my-adhoc-service+json, but that won't change the client's need to read my documentation and play with the service before they can figure out how to use it. That's true even if I move the documentation out to an RFC.

One real advantage I can see is to use the media type for versioning, but even that doesn't remove the need for a client to intervene if they want to upgrade their code to work with the newer API version.

So it's not really clear to me that a custom media type is significantly more useful for the client, even if it is more 'proper'. When a constraint feels ceremonial, I become suspicious that it exists to drive purity-for-its-own-sake rather than a real benefit.

What trouble? Are you not documenting what you return anyway? What's the cost of just changing the mediatype to some different string?

For a client, is it worth the trouble to learn a new content-type to determine what's a hyperlink in my JSON response, when they can just look at the response and see something like the following?

Personally, I think that mindset will hold us back. People still think of RESTful services as closed silos instead of open, truly uniform interfaces, and that will prevent us from growing, in my opinion.

Your view is of a developer painstakingly coding against each API (s)he wants to use, manually prodding and poking it and writing a custom library that doesn't work with any other service.

My view is of generic, easily adaptable clients, using machine-readable specification of formats¹ and providing graphical UIs where even non-programmers can input the single endpoints of various services and easily combine the data and functionality to create applications that cater to their specific needs.

I see a service like IFTTT[1], but more flexible and where the services are not hardcoded but imported dynamically by the user and then strung together to fit their needs.

Using mediatypes, even if custom, is a step towards that integration and standardization that makes this possible.

¹ RDF seems like the best bet to me - possibly encoded as JSON

[1]: http://ifttt.com/

What I'm trying to suggest is that the point at which a schema is powerful and generic enough to work as a general-purpose web service API for an arbitrary use case, it has become more difficult and complicated to learn, configure and use than a much simpler, discoverable ad-hoc schema that closely matches the functionality of your web service.

In general, REST principles give more than they take, which is why I try to follow them; but you're always going to have to find the sweet spot of abstraction that provides the right amount of power and flexibility combined with the right amount of discoverability and predictability.

> What's the cost of just changing the mediatype to some different string?

I'm happy to change a server's content-type from application/json to application/vnd.my-adhoc-service+json, but that won't change the client's need to read my documentation and play with the service before they can figure out how to use it. That's true even if I move the documentation out to an RFC.

One real advantage I can see is to use the media type for versioning, but even that doesn't remove the need for a client to intervene if they want to upgrade their code to work with the newer API version.

So it's not really clear to me that a custom media type is significantly more useful for the client, even if it is more 'proper'. When a constraint feels ceremonial, I become suspicious that it exists to drive purity-for-its-own-sake rather than a real benefit.

What I'm trying to suggest is that the point at which a schema is powerful and generic enough to work as a general-purpose web service API for an arbitrary use case, it has become more difficult and complicated to learn, configure and use than a much simpler, discoverable ad-hoc schema that closely matches the functionality of your web service.

Is RSS difficult and complicated to learn and use? Now think what a pain it would be to develop a feed reader if each blog had its own "simpler" ad-hoc schema.

Standards don't have to be complicated, especially if they use an extensible encoding like JSON or XML so that instead of having the kitchen sink they can remain simple and be extended for specific purposes if really needed, without affecting generic clients.

I'm happy to change a server's content-type from application/json to application/vnd.my-adhoc-service+json, but that won't change the client's need to read my documentation and play with the service before they can figure out how to use it. That's true even if I move the documentation out to an RFC.

The end goal is for different services to agree on standard formats for common resources. Using custom mimetypes is a step in the right direction because it decouples the format recognition and parsing from the service-specific code - it allows clients to use more modular code that works across different services with plenty of reusable code.

Is it terribly important? No. But since the costs are essentially none - changing a couple of strings here and there - not doing it seems unreasonable to me. You asked why, I ask why not?

First of all, I want to thank you for your thoughtful, clearly-articulated comments. Frankly, you've done a better job of explaining and defending dedicated media types than anything else I've seen and I appreciate you taking the time to engage my comments with seriousness rather than condescension.

> Is RSS difficult and complicated to learn and use?

No, but RSS is a schema with a specific, narrowly defined purpose: to syndicate article-based web pages (and even so, there are 5 or 6 different versions, not including Atom - though your point stands and I don't want to get sidetracked over a quibble).

A more apt comparator may be WSDL, which theoretically allows a SOAP web service's remote procedure calls to be discovered programmatically but in practice is such a messy, complicated, leaky abstraction that it's often easier just to build XML templates manually and POST them to the endpoint.

I'd be happy to use and support some kind of lightweight schema for JSON that puts URLs in predictable places in the response object and still allows flexibility to provide the functionality a given service needs. I'm not aware of anything that fits this description, but I'm certainly open to it if you have any recommendations.

> You asked why, I ask why not?

The model of HTTP resources and methods is useful: it unambiguously delivers real, tangible benefits to the developer and the client over ad-hoc protocols riding on top of HTTP like `GET /create_user.php`.

Similarly, it makes sense to use the HTTP status codes we already have instead of reinventing your own arbitrary set of status codes and, e.g. sticking them somewhere in the response body. If someone e.g. does a POST request on /users/1337 to create a user, it makes sense to return a 405 status. (However, I tend to put status codes in the response headers and the response body, just to make it easier for clients to notice what's going on.)

It also makes sense to put URLs inside the response body in an obvious way so that the client can "click through" to subsidiary resources in a manner analogous to how web pages do it. Again, it makes the web service more or less self-documenting to the client, especially if it uses URLs and HTTP Methods in a sane, predictable way.

I'm struggling to understand the benefit of a custom media type beyond merely satisfying one of Roy Fielding's constraints. If the only reason to use a custom media type is to satisfy a constraint, the constraint might be more ceremonial than useful.

If two web services are identical except one serves `Content-Type: application/vnd.some-arbitrary-format+json` and the other serves `Content-Type: application/json`, I get suspicious about the benefits of REST when people say the latter is doing it wrong and doesn't deserve to be called RESTful.

No, but RSS is a schema with a specific, narrowly defined purpose: to syndicate article-based web pages (and even so, there are 5 or 6 different versions, not including Atom - though your point stands and I don't want to get sidetracked over a quibble).

A more apt comparator may be WSDL, which theoretically allows a SOAP web service's remote procedure calls to be discovered programmatically but in practice is such a messy, complicated, leaky abstraction that it's often easier just to build XML templates manually and POST them to the endpoint.

Well, two points:

Firstly, I think WSDL (and WADL) are working at the wrong abstraction level. I think one of the reasons why REST is cleaner is because unlike SOAP it's data oriented, not RPC. And I believe it's possible, as long as your API is data oriented too, to map that to generic formats that don't feel messy.

I had already mentioned RDF (Resource Description Framework), but I'll mention it again; RDF is a layer between the serialization and the concrete schemas, giving very specific but open ended semantics to its documents.

Unlike WSDL, it's very simple: it consists only of four concepts: triples, which are composed of objects (which is always a resource, and is represented as an URL), subjects (which can be other resource or a literal value) and a predicate (which relates that object with the subject in some way, and which are specified by a schema). Then, a document is just a bunch of statements about one or more resources.

This model is hypermedia (since it clearly points to resources), and it's at the same time constrained enough to not end up being messy, and open enough to define mostly anything. I think it really captures the sweet spot.

There's no JSON serialization, unfortunately, but I like Turtle[1].

---------

The second point is that even if this day dream of mine never comes to pass, I think we'd still benefit a lot from using more of those standard "schemas with specific, narrowly defined purposes" and keep the custom formats to when they're really needed. Instead of defining your own user profile schema - which 99% of the APIs out there need - why not just share a single format, which can be easily:

1) Parsed by generic clients, libraries and plugins

2) Shared between services and applications

But for that, mediatypes make perfect sense, since they make it easy for a single HTTP client to dispatch the responses to the right readers, without involving a bunch of hairy rules that say "when response is JSON and the service being called is X, call parser Y".

This post is against WADL, but I think it expresses that power of using proper mediatypes: http://bitworking.org/news/193/Do-we-need-WADL

I'm struggling to understand the benefit of a custom media type beyond merely satisfying one of Roy Fielding's constraints. If the only reason to use a custom media type is to satisfy a constraint, the constraint might be more ceremonial than useful.

I might be wrong, but I'm convinced some great things could come up from it. Unfortunately, I doubt that anyone will seriously explore the possibilities unless there are already many APIs with support for it, and in typical Prisoner's dilemma style, very few API developers will consider supporting it unless they see a short term benefit.

If two web services are identical except one serves `Content-Type: application/vnd.some-arbitrary-format+json` and the other serves `Content-Type: application/json`, I get suspicious about the benefits of REST when people say the latter is doing it wrong and doesn't deserve to be called RESTful.

Well, I think as long as it supports HATEOAS it's RESTful enough, but on the other hand, I can see the their point. There's nothing wrong with not fully implementing REST, but still calling it REST gives false expectations to the clients.

Of course, at this stage nobody expects a RESTful API to be actually RESTful, but then again that's the problem, isn't it?

Well, so...

(comment deleted)
I haven't written anything about HATEOAS, caching and Internationalization intentionally as I'm reserving it for another post.
That's a bit like writing an article on how to design boats and reserving mentioning water.
I did mention that it's incomplete. It's impossible to write the complete thing in one post.
About RESTful API, if you have many applications working together, do you build an API and use it in every application or is the API only for applications you don't control and you use the same code that you use for the API in every application you control?
what's with the PHP hate?
Because if you build your startup's site using PHP, you'll never be able to scale like a Flickr, Digg, Wikipedia, or Facebook.

// Ahem.