Since their core product is an API, they've done a really good job building in all the levels of REST discussed in the article, including discoverability.
Any website that uses HTML forms (properly) is at the last level. Also, using the <a href ... > tag is also using HTML as the engine of application state transitions.
This article makes things out to be more complicated than they actually are. Just make decent HTML. REST was extracted from the web, it doesn't need to be imposed on it.
I'm with him until he gets to "Hypermedia Controls", which strikes me as needless bureaucracy.
The 'link rel = "royalhope.nhs.uk/linkrels/slot/book"' element isn't really helping anything. Before you can consume that service, you need to know what "royalhope.nhs.uk/linkrels/slot/book" actually means, and if you already know that, then having it listed here in the response justs wastes space.
He's disguising a custom verb under a cloak of some sort of 'standardization' which isn't actually part of HTTP or any common web practice. The consumer of this service will need to know what all the verb URIs mean and precisely how to use them anyway, so listing them here doesn't provide any benefit and significantly complicates the communication.
In this way, the "Hypermedia Protocol" doesn't demonstrate any edge over old-school XML RPC.
Up until that point, I agreed with the article. Simple objects exposed using standard HTTP verbs is the way to go -- but I stop short of supporting random lists of verbs labeled with URIs as just more needless bureaucracy.
I understood it to be decoupling the what from the how. In this case, it decoupled knowing that you needed to book an appointment from the URI that lets you do that.
If I already what what I want to do, but I allow a service to tell me how to do it, me and the service have a looser coupling. If the service decides to change how I accomplish the task, then I should seamlessly adapt.
But, in practice, how often does a service change an URI only, without changing the input or output parameters (causing you to have to re-factor your client anyway)?
Often merely moving from the dev environment to the release environment changes the URIs. Sure, you could try to minimize the part that changes between environments and teach all your clients to form correct URIs between environments, but it's still coupling. And dev/prod is hardly the only thing people find themselves adjusting in URIs; it was just an example.
One example, if you move domains, all you'd need to do is redirect the single entry point URL to the new one without having to force clients to change anything and with having to redirect every single API request.
You can also allow other sites to implement the same API and clients would only have to select a different entry point. Look at AtomPub for instance.
He states very explicitly that his examples weren't any kind of standard, as one doesn't truly exist at that level.
I think the point he was trying to get across, and really what a lot of people who are advocates for REST are trying to accomplish, is that in a fully realized model computers would be able to understand how to talk to each other to get information and update resource states without the need for human intervention.
Clearly we are a long way from that goal, but I think it's still something worth talking about.
Whilst the notion of "computers talking to each other without human intervention" sounds like a utopian ideal, I think his method is a recipe for disaster.
The domain of possible consequences of changing protocols and semantics on the fly is hugely complex. What about testing? Are we advocating that we change live systems without tests?
The author writes:
"Level 3 introduces discoverability, providing a way of making a protocol more self-documenting."
I reply:
The same way good design separates presentation, logic and data, it follows that good protocols separate data from documentation.
> possible consequences of changing protocols and semantics on the fly is hugely complex
Except when you assume that you're reading the semantics from each and every response. It introduces some complexity to how clients need to accept responses. However, once that part is done the changing protocol and semantics should not disrupt the system.
And testing? That's not a very difficult component to test. Throw a few different 'schemas' at the HTTP client response module and then feel safe to drop that into your application. I believe you're imagining a system that is more highly coupled to the response schema. This approach is meant to remove that coupling entirely (well, except for a base uri :))
I agree that embedding <link />'s into the response is a violation of a separation of concerns. They should probably be separated in some way (HTTP headers?). However, there are precious few examples of this in the wild and we may see some creative ways to separate these (I just hope we don't go the way of XML to have a different resource to specify schema).
I don't know if it is true that that is what REST advocates are trying to accomplish. If you want to remove the human than SOAP and WSDL have gone a long way towards that goal. I always viewed REST as the opposite of that, make things simple so the developer is able to easily manipulate the data as he sees fit.
I agree. If he truly wants to create a system that can be updated only from the server side he needs some sort of human readable description for the control. displayAs = "Book Appointment" for example in this case.
The idea is that the client code will search for the link element whose rel attribute is equal to the known value the client is looking for, and then get the uri attribute for the request. In other words, the client knows the name of the uri for the functionality it wants, but it doesn't have to know the uri. This allows the uris to change if needed.
It's like using constants instead of literals in your code. If the literal value has to change you only have to change the constant definition, not every place in the code where the value is used.
Look at it this way. If I'm at a REPL, and I have this:
o = SomeObjectOfUnknownCapabilities.instance
then I can do this:
p o.methods
...and see what I can call. The method names give me no guarantee as to what the methods will actually do ("slice" could return a subset of data, or cut a pie), but once I have the verbs, in either case, I can experiment. That's the important bit here—discoverability isn't about knowing what new things do right off the bat, because that's impossible unless there's a finite set of things you can do; discoverability is simply about being told the affordances, so you can manipulate them and observe the results.
... all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations.The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]
He's explicate in that a REST system must return options for further action. Links. This is the default behavior of the web, request a page get a page full of links. Martin Fowler is just describing a system that uses a hypermedia implementation other than html.
The benefit to this is that you don't need out-of-band information to interact with a new server you run into.
curious: why is the initial resource used to get the list of slots /doctors/mjones/slots?date=20100104&status=open and not /slots?doctor=mjones&date=20100104&status=open?
if i am setting up future communications at /slots/1234, to me /slots?doctor=mjones&date=20100104&status=open makes more sense. would you typically just have both resources available at those locations?
Something like depends on the context of the whole application and how you want to organize it. The first way puts the emphasis on the /doctors/mjones/...
If you have other actions related to doctors like /doctors/mjones/contact, this way can provide a more coherent organization. Plus you can extend it to something like /labs/questdiagnostic/mri/slots?
So you can decouple the concept of 'slots' from doctor, allowing you to extend it to other types of appointments without having to alter slots. The parameters are the items that are also the most likely to vary. When making an appointment you will check multiple days, but you are much less likely to check multiple doctors. Plus the days are more temporal in nature while the list of doctors is going to stay the same for longer.
It describes an interface (Atom XML based) that further self-describes how it was called and what you can do to it (how to search within results, for example).
Simply applying that thinking to JSON based RESTful services gives you that question "What can I do with this?", and that makes building the API easier... for everything you serve describe what can be done with it and how you were called.
This then totally de-couples the request from the client, the client can send many queries and not care about them, and get back something that encapsulates the state of the request (search terms), the state of the result (search pagination) and the verbs for each entity (what you can do with it).
I never would've called this "Hypermedia Controls", I think that's confusing.
Perhaps we all just need a new catchphrase or buzzword though, just to get the wider industry onboard? I'd rather just say that your services should be discoverable, and that means within the results telling us what we can do, in addition to a top level thing to describe which services exist.
I find it hard to believe that he's built real systems that work.
Yes, "decomposing services into multiple components" works well when everything is living in the same address space, but if you want to build fast and reliable systems, minimizing round trips is important.
If I didn't have anything better to do and if I liked dealing with idiots, I'd hang my shingle as a high-priced consultant who helps "enterprises" dig their way out of the REST morass. Just last year I managed to speed up an application 100x by switching from REST to POX. In this case, the app was doing thousands of round trips for no good reason other than architectural purity.
In the real world, composite operations should be done, more or less, in transactions. For instance, in one user authentication system I've worked on, there are about ten database updates to create a new user. There really ought to be one "createUser" call that creates the user. You could probably get away with not having database transactions if you're talking to the database locally, but in a distributed app on the wider internet, you just can't expect to do ten communications in a row and have it work "reliably enough".
You find it hard to believe that _Martin Fowler_ has built real systems that work? He is a rather well known programmer, author, speaker, and Chief Scientist at ThoughtWorks.
It's a fact about the industry that most such consultants-authors-speakers don't actually build systems themselves and haven't for a long time. If you want to get prominent at it, telling other people how to build things is itself a full-time job. The contradiction here is unfortunately very real.
I agree in principle, but not when applied to Martin Fowler. I doubt the guy who literally wrote the book on refactoring did so without any real-world experience.
But I don't think the book, in itself, can cited as evidence that he's built large, real-world systems.
There's a big step between "no experience" and "major experience with large scale, distributed systems".
When I had six months of programming experience, I could do a fine rendition of the standard OO doctrines. With far more experience now, I can mostly describe how they fail to operate in the real world.
A follow-up question regarding REST, transactions, and doing multiple things at the same time ...
I see the REST services in the article expose some very fine-grained concepts, e.g., existence of patients, doctors, and "slots", i.e., appointments. It appears the REST services can readily let one patient book one slot to see one doctor, but most real-world systems aren't like this. So ...
What's the REST-y way to address this situation then, that's a bit more complex: (a) there are a number of conference rooms; (b) each conference room can be reserved for a contiguous set of 15-minute slots; (c) a client using the REST-y interface wants to fully reserve N 15-minute slots in a particular room, and either completely succeed or completely fail. Is there a REST-y way to do this or would one need to rely on some client-driven pseudo-transaction mechanism? (This is kind of similar to the parent's complaint about needing the "createUser" call.)
I saw a nice example that covered the classic "bank account transfer" case. The answer was that you POST something that's like a deposit slip.
Some systems, like the Kuali accounting system, use a document-centric approach to business software. You can have a workflow that's much like a traditional paper-based system, so this can make a lot of sense. You can overlay this on top of REST to solve many of the problems that REST has, but then you end up with much of REST being just a fifth wheel.
As for ranges, lately I've been sketching up a hypertext system that has some inspiration from Ted Nelson's Xanadu system. In particular, hyperlinks can be made to and from arbitrary ranges in documents, without touching the documents directly.
You can certainly encode ranges in a URL, so something like
As terra_t outlines, introduce a "Reservation" resource which contains 1 or more slots, allowing someone to (attempt to) create a Reservation containing several 15-minute slots in a single request.
If "N 15 minute slots" are something that you want to either all suceed or all fail at the same time, then what you really have in this concept is another entity/resource to add to your model.
And if you do that you're really writing POX, with any true-REST features being fifth, sixth, and seventh wheels.
Other than systems that post documents that describe state transitions, REST is all about having an inappropriate coupling between your API and your underlying data models.
I'm curious, how do you suggest designing a system where the API and domain model are decoupled?
Unless we're using the same words to mean different things, I don't see how you could interact with an API method with understanding what you are passing to or receiving from it at a certain level.
On the other hand, one of the large selling points of REST-style design is the ability to know how to interact with resource URLs using common http verbs without necessarily knowing what that URL represents - for example, GET or POST have the same meaning throughout the API.
I don't think the question of transaction boundaries and REST design are mutually exclusive. One can build a REST system that uses meaningful transactions around it's operations - the obvious solution is to make the operations that are exposed the same as what you want to wrap transactions about, and only expose these operations. In fact I find it strange that some might think these are two exclusive concepts.
The other problem you describe sounds like it would be easily solved with client-side caching. REST and caching remote operations are not exclusive of one another either. HTTP provides a very natural way to control caching. Having to make 100 network round trips for the same resource should be unnecessary in any system, REST or not.
It's not 100 trips for the same resource, it's 100 round trips for 100 ~different~ resources that are required to construct a page. Cacheing won't help, and in fact, http caching is one of the #1 obstacles in AJAX programming. People who write AJAX APIs are always using cache-busting methods to make sure that browser caches don't screw up their apps with stale data.
The GET model is acceptable for data access (unless you want to radically reduce round trips -- one answer to this is to "virtualize" the REST model and let an app ask for N chunks of data in one request) however, it's bad for data updates, where you're better off exposing operations that exposing data directly.
Security, for instance, is easier to implement if you expose operations because security rules most frequently govern state transitions: to apply security rules in purist REST, you're always having to write comparisons between the old state and the new state and you've got to test for every single possible thing that could break security, even combinations of things that would be harmless by themselves.
If a security-sensitive operation is exposed as an operation, the implementation of security rules is much more clear.
Cacheing won't help, and in fact, http caching is one of the #1 obstacles in AJAX programming. People who write AJAX APIs are always using cache-busting methods to make sure that browser caches don't screw up their apps with stale data.
Ah, I was referring to caching requests from the webapp producing the web page to the remote services it's consuming, assuming this was the setup - not browser caching.
>And if you do that you're really writing POX, with any true-REST features being fifth, sixth, and seventh wheels.
Not at all. In REST, the resource represents an object. When you GET the resource, you get a document encoding the state of the object. So, if you POST such a document to the creation resource, you're creating the object.
CREATE would be a better verb, but WebDAV defines CREATE as creating a resource at the specified URI; in an API, when you want to create a new object, you probably need the API to tell you the URI.
It sounds like you've worked only with deeply flawed REST APIs. You don't need a 1:1 relation between database tables and REST resources. In fact, you don't actually deal with resources directly, but rather resource representations which can abstract many actual data resources Ina single document. If you're using REST like a naive low-level CRUD ORM, you're probably doing something wrong.
You read the article? The first thing he does is call POX "level 0" and "the swamp".
Generally, real http APIs from companies like Amazon, Flickr, Twitter, etc. are POX-RPC APIs, the idea is that you're exposing operations, not data structures.
REST is a castles-in-the-air exercise in exploring the dark corners of the http specification and exposing data directly instead of exposing operations on the data.
It's always seemed to me that the difficulty with hypermedia controls is not how to have your server expose them, which is straightforward enough, but how to have your client do something useful with them.
It basically means writing your client as a state-graph navigating agent, which makes it closer to AI-ish programming (not surprising, since the argument for REST has been that web services should be structured more like the human-browsable web) than the input -> output or interactive event-driven paradigms that most programmers are more conversant with.
Yet the books and online evangelism etc. for REST that I've seen don't really address this issue.
I don't know if he had intended to put the REST zealotry to rest, but Martin Fowler and Roy field seem to be on the same page when it comes to level 2 REST. I submitted this to HN a few weeks ago, I'm posting it here again:
50 comments
[ 4408 ms ] story [ 239 ms ] threadisn't bad
edit: Btw, any well designed website is at level 3.
Since their core product is an API, they've done a really good job building in all the levels of REST discussed in the article, including discoverability.
This article makes things out to be more complicated than they actually are. Just make decent HTML. REST was extracted from the web, it doesn't need to be imposed on it.
The 'link rel = "royalhope.nhs.uk/linkrels/slot/book"' element isn't really helping anything. Before you can consume that service, you need to know what "royalhope.nhs.uk/linkrels/slot/book" actually means, and if you already know that, then having it listed here in the response justs wastes space.
He's disguising a custom verb under a cloak of some sort of 'standardization' which isn't actually part of HTTP or any common web practice. The consumer of this service will need to know what all the verb URIs mean and precisely how to use them anyway, so listing them here doesn't provide any benefit and significantly complicates the communication.
In this way, the "Hypermedia Protocol" doesn't demonstrate any edge over old-school XML RPC.
Up until that point, I agreed with the article. Simple objects exposed using standard HTTP verbs is the way to go -- but I stop short of supporting random lists of verbs labeled with URIs as just more needless bureaucracy.
If I already what what I want to do, but I allow a service to tell me how to do it, me and the service have a looser coupling. If the service decides to change how I accomplish the task, then I should seamlessly adapt.
You can also allow other sites to implement the same API and clients would only have to select a different entry point. Look at AtomPub for instance.
I think the point he was trying to get across, and really what a lot of people who are advocates for REST are trying to accomplish, is that in a fully realized model computers would be able to understand how to talk to each other to get information and update resource states without the need for human intervention.
Clearly we are a long way from that goal, but I think it's still something worth talking about.
The domain of possible consequences of changing protocols and semantics on the fly is hugely complex. What about testing? Are we advocating that we change live systems without tests?
The author writes: "Level 3 introduces discoverability, providing a way of making a protocol more self-documenting."
I reply: The same way good design separates presentation, logic and data, it follows that good protocols separate data from documentation.
Except when you assume that you're reading the semantics from each and every response. It introduces some complexity to how clients need to accept responses. However, once that part is done the changing protocol and semantics should not disrupt the system.
And testing? That's not a very difficult component to test. Throw a few different 'schemas' at the HTTP client response module and then feel safe to drop that into your application. I believe you're imagining a system that is more highly coupled to the response schema. This approach is meant to remove that coupling entirely (well, except for a base uri :))
I agree that embedding <link />'s into the response is a violation of a separation of concerns. They should probably be separated in some way (HTTP headers?). However, there are precious few examples of this in the wild and we may see some creative ways to separate these (I just hope we don't go the way of XML to have a different resource to specify schema).
It's like using constants instead of literals in your code. If the literal value has to change you only have to change the constant definition, not every place in the code where the value is used.
This is from Roy Fielding's blog: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...
... all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations.The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]
He's explicate in that a REST system must return options for further action. Links. This is the default behavior of the web, request a page get a page full of links. Martin Fowler is just describing a system that uses a hypermedia implementation other than html.
The benefit to this is that you don't need out-of-band information to interact with a new server you run into.
if i am setting up future communications at /slots/1234, to me /slots?doctor=mjones&date=20100104&status=open makes more sense. would you typically just have both resources available at those locations?
OpenSearch does stuff like this: http://www.opensearch.org/Specifications/OpenSearch/1.1/Draf...
It describes an interface (Atom XML based) that further self-describes how it was called and what you can do to it (how to search within results, for example).
Simply applying that thinking to JSON based RESTful services gives you that question "What can I do with this?", and that makes building the API easier... for everything you serve describe what can be done with it and how you were called.
This then totally de-couples the request from the client, the client can send many queries and not care about them, and get back something that encapsulates the state of the request (search terms), the state of the result (search pagination) and the verbs for each entity (what you can do with it).
I never would've called this "Hypermedia Controls", I think that's confusing.
It's just having your interfaces describe themselves and be discoverable. Coupled with a WADL-style thing (but probably not WADL itself... perhaps JSON such as on the Freebase API: http://code.google.com/p/google-refine/wiki/ReconciliationSe... ).
Perhaps we all just need a new catchphrase or buzzword though, just to get the wider industry onboard? I'd rather just say that your services should be discoverable, and that means within the results telling us what we can do, in addition to a top level thing to describe which services exist.
Yes, "decomposing services into multiple components" works well when everything is living in the same address space, but if you want to build fast and reliable systems, minimizing round trips is important.
If I didn't have anything better to do and if I liked dealing with idiots, I'd hang my shingle as a high-priced consultant who helps "enterprises" dig their way out of the REST morass. Just last year I managed to speed up an application 100x by switching from REST to POX. In this case, the app was doing thousands of round trips for no good reason other than architectural purity.
In the real world, composite operations should be done, more or less, in transactions. For instance, in one user authentication system I've worked on, there are about ten database updates to create a new user. There really ought to be one "createUser" call that creates the user. You could probably get away with not having database transactions if you're talking to the database locally, but in a distributed app on the wider internet, you just can't expect to do ten communications in a row and have it work "reliably enough".
http://refactoring.com/catalog/index.html
But I don't think the book, in itself, can cited as evidence that he's built large, real-world systems.
There's a big step between "no experience" and "major experience with large scale, distributed systems".
When I had six months of programming experience, I could do a fine rendition of the standard OO doctrines. With far more experience now, I can mostly describe how they fail to operate in the real world.
On the other hand, once you get famous, you can go around and get big $ telling people what they want to hear and you can lose your edge.
If this is representative of his work these days, he should preface it with something like "I'm not a doctor but I play one on TV"
Really? That bad? Care to explain?
I see the REST services in the article expose some very fine-grained concepts, e.g., existence of patients, doctors, and "slots", i.e., appointments. It appears the REST services can readily let one patient book one slot to see one doctor, but most real-world systems aren't like this. So ...
What's the REST-y way to address this situation then, that's a bit more complex: (a) there are a number of conference rooms; (b) each conference room can be reserved for a contiguous set of 15-minute slots; (c) a client using the REST-y interface wants to fully reserve N 15-minute slots in a particular room, and either completely succeed or completely fail. Is there a REST-y way to do this or would one need to rely on some client-driven pseudo-transaction mechanism? (This is kind of similar to the parent's complaint about needing the "createUser" call.)
Some systems, like the Kuali accounting system, use a document-centric approach to business software. You can have a workflow that's much like a traditional paper-based system, so this can make a lot of sense. You can overlay this on top of REST to solve many of the problems that REST has, but then you end up with much of REST being just a fifth wheel.
As for ranges, lately I've been sketching up a hypertext system that has some inspiration from Ted Nelson's Xanadu system. In particular, hyperlinks can be made to and from arbitrary ranges in documents, without touching the documents directly.
You can certainly encode ranges in a URL, so something like
/conference_room_3/time_slot_7,time_slot_12
might be POSTable to
If "N 15 minute slots" are something that you want to either all suceed or all fail at the same time, then what you really have in this concept is another entity/resource to add to your model.
That strikes me as a strawman, because the obvious RESTful way to do that would be to POST a document fully describing the new user.
Mind you, I don't doubt that you've seen this in a RESTful API. You can write Java Beans in any language.
Other than systems that post documents that describe state transitions, REST is all about having an inappropriate coupling between your API and your underlying data models.
Unless we're using the same words to mean different things, I don't see how you could interact with an API method with understanding what you are passing to or receiving from it at a certain level.
On the other hand, one of the large selling points of REST-style design is the ability to know how to interact with resource URLs using common http verbs without necessarily knowing what that URL represents - for example, GET or POST have the same meaning throughout the API.
I don't think the question of transaction boundaries and REST design are mutually exclusive. One can build a REST system that uses meaningful transactions around it's operations - the obvious solution is to make the operations that are exposed the same as what you want to wrap transactions about, and only expose these operations. In fact I find it strange that some might think these are two exclusive concepts.
The other problem you describe sounds like it would be easily solved with client-side caching. REST and caching remote operations are not exclusive of one another either. HTTP provides a very natural way to control caching. Having to make 100 network round trips for the same resource should be unnecessary in any system, REST or not.
The GET model is acceptable for data access (unless you want to radically reduce round trips -- one answer to this is to "virtualize" the REST model and let an app ask for N chunks of data in one request) however, it's bad for data updates, where you're better off exposing operations that exposing data directly.
Security, for instance, is easier to implement if you expose operations because security rules most frequently govern state transitions: to apply security rules in purist REST, you're always having to write comparisons between the old state and the new state and you've got to test for every single possible thing that could break security, even combinations of things that would be harmless by themselves.
If a security-sensitive operation is exposed as an operation, the implementation of security rules is much more clear.
Ah, I was referring to caching requests from the webapp producing the web page to the remote services it's consuming, assuming this was the setup - not browser caching.
Not at all. In REST, the resource represents an object. When you GET the resource, you get a document encoding the state of the object. So, if you POST such a document to the creation resource, you're creating the object.
CREATE would be a better verb, but WebDAV defines CREATE as creating a resource at the specified URI; in an API, when you want to create a new object, you probably need the API to tell you the URI.
Generally, real http APIs from companies like Amazon, Flickr, Twitter, etc. are POX-RPC APIs, the idea is that you're exposing operations, not data structures.
REST is a castles-in-the-air exercise in exploring the dark corners of the http specification and exposing data directly instead of exposing operations on the data.
POX is complementary to REST: REST refers to a communication pattern, while POX refers to an information format style
It basically means writing your client as a state-graph navigating agent, which makes it closer to AI-ish programming (not surprising, since the argument for REST has been that web services should be structured more like the human-browsable web) than the input -> output or interactive event-driven paradigms that most programmers are more conversant with.
Yet the books and online evangelism etc. for REST that I've seen don't really address this issue.
http://roy.gbiv.com/untangled/2009/it-is-okay-to-use-post
I hope these 2 blog posts will just stop people from equating REST to full HTTP.