This is essentially trying to solve the same problem as Swagger. Swagger is "a specification and complete framework implementation for describing, producing, consuming, and visualizing RESTful web services" [1]. Check out the spec on GitHub here [2].
When I see this post, the first thing to pop up is SOAP as well. Just that SOAP is not human-readable. Then I suddenly remembered that it wasn't SOAP itself that include the schema but the SOAP providers would generate WSDL alongside a SOAP endpoint.
SOAP is an abstraction that is fundamentally unhelpful (at least in my opinion). Without SOAP, to call a webservice, as a developer, I need to read the documentation for the webservice to understand what data needs to go into what places. With SOAP, as a developer, I need to read the documentation for the webservice to understand what data needs to go into what places; but the documentation is much harder to read, and complex types are usually much harder to use (there's a tendency to model things as lots of complex xml, which are often hard to construct, instead of just a specified transform to a string)
Substitute "XML" for "JSON" and we've now come full circle.
The point about REST is that it is self-describing. And ideally should be using the same URIs as the version people clicking around in Firefox or Chrome see. The API is just the XML or JSON or whatever is flavour of the week version of the HTML version.
(Or we could use embedded data—microformats, microdata, RDFa—and get rid of that distinction.)
I agree. The promise of REST APIs is that will be self describing, but for that benefit to be realized, we need general purpose REST clients that can "discover" everything they need to know given just a root uri. Are there any such clients? And no, web browsers do not count.
But one of the things I did a little differently is that instead of writing the code and then the docs separately, I pass the validation information from the object itself.. so the API layer doesn't have to know any of it in advance. It can pass that along to the end clients.
I'm not convinced this is the solution but it mostly works for now and would love any & all feedback.
My next proof of concept will be to use Javascript to retrieve the required fields and decorate a simple html form.
Agreed. I came here to post something similar but I was going to mention working with SOAP[1] in addition to what you mentioned. It sounds like the OP is trying to do something which sounded very much like using SOAP and XML to me.
To paraphrase the OP, "Since so many APIs can be described in similar terms, why don't we have some sort of standard that one can look at to identify how to use the API instead of letting the API speak for itself?"
When you start going down this track, you're not only making things complicated on the client's end of things. On the server side, you're having to maintain two things for the API now. First: the ruleset, ensuring it's 100% to spec lest a client fail. Second: the code generating the response in the first place.
I've built clients and servers for both RESTful and SOAPy APIs and I can say I would take REST any day.
Yes my argument is simply that we publish API specs in a machine-readable format to avoid wasting time implementing clients repeatedly. WSDL and WADL had good intentions at heart, but XML is ugly. JSON is nice since it's human-readable and light. Why not publish JSON versions openly for REST APIs, reducing implementation cost for clients?
Yes, there are some dots you need to connect between his two statements. I'm assuming that he meant the following: those machine-readable specs also need to be read by humans at some point, just like code, and, since JSON is more readable, he proposes using it instead of XML.
XML is human readable. It was meant to be (this is why it's text instead of some binary format). It's just really, really verbose, which is what this JSON spec is going to end up being once it's dealt with all imaginable edge cases.
I don't know that either one is particularly close to the way I'd write down a list in real life, to be honest. This is a pretty trivial sample, and neither is especially hard to read/parse by a human. But the JSON still looks closer to line-noise to me. :-)
That may be so, but most opinions I hear in discussions about XML vs JSON say that JSON is more readable, probably because it's less redundant and similar to data structures found in some programming languages.
Really? Seems to me like Hypermedia APIs should move the problem from the wire protocol to the application protocol?
Hypermedia says "oh yeah, here's some markup, look there are URIs in it". For a human user, we're like "cool, I'll try and click these, see what they do".
But software is going to want "um...okay, how to I parse this markup, and how do I generate the submissions you want? And, okay, you can change URIs, but please don't change anything else about that operation, or I will break completely. That's right, we're not really decoupled."
So, even with hypermedia APIs, AFAIK you're still going to want some marshaling to/from host language. ...and so you're back to having a spec, and coupling, you've just moved it around.
(Rant on coupling, people seem to think it's always bad and you can make it go away. Reality: you can't make it go away, and sometimes just accepting it directly is a whole lot simpler than deceiving ourselves about it's existence by over-applying abstractions.)
We're replying to each other in separate comment threads :-), but this input form is still coupling--you can't add/remove/change the access_tokens/fields without clients breaking. Humans can handle that. Software can't.
That's why I think hypermedia makes all sorts of sense for explaining why the www is awesome--it change deal, users will adapt. But IMO it falls flat as some new paradigm for building client/server systems.
> Actually, you explicitly _don't_ want this. That's what hypermedia APIs are trying to remove.
> But IMO it falls flat as some new paradigm for building client/server systems.
I know of one company which you've absolutely heard of who has a 30-person team building a hypermedia API. They haven't talked about it publicly because they see it as a strategic advantage.
This year will be the year of code and examples; last time I was in San Fransisco, 5 different startups came up to me and told me that hypermedia is solving their problems. Expect to see more of this going on soon.
> Hm? I am skeptical...any links/explanations?
Mike Amundsen's "Building Hypermedia APIs in HTML5 and Node" has a pretty big section on this, and my book has a section entitled "APIs should expose workflows."
I actually have no idea why the hal people aren't writing hal specifications for existing services right now--it's not quite as nice as "native" support, but the format supports it, and it would be useful to see what a hal version of the Twitter API looked like, for example.
Yes, URIs in the response sounds amazingly cool, but it won't change anything.
The inline URLs of the web work because the consumers are humans who can deal with changes (more than just trivial URL changes, like added, removed features) and now click on this button or that button.
Software isn't that flexible, so it will be just as coupled as it is today--you're just moving the coupling around.
So this idea of a "robust client-server relationship" is a pipe dream IMO.
You've moved the coupling from "look at URL xyz" to "look for rel podcast_url". Okay, yes, you can now change the URL, that's cool, but I assert that's relatively trivial. You can't truly add/remove new functionality (or break existing contracts, like "look for rel=podcast_url") that some omnipotent client would suddenly start taking advantage of.
IMO this omnipotent realization/utilization of new/changed features is what hypermedia advocates get all excited about, without realizing that humans are really the only ones that can deal with that level of (true) decoupling.
The important part is that this is all documented in the media type. Of course, computers aren't able to just figure out what's up, that's why it's agreed upon beforehand.
> You can't truly add/remove new functionality
You can absolutely add new functionality, because rule #1 of clients that work this way is 'ignore things you don't understand.' Removal will obviously break things, so it's important to handle this in the appropriate way.
I guess ultimately my point is that these kinds of APIs are significantly different, and come with a very different set of constraints/goals/implementation details. It's like you're saying "well, I don't have a full list of function calls I can make!" because you're used to SOAP where there's a WSDL and 'RESTful' APIs don't have description documents. Of course! It's a different architecture!
With some conventions how the api is structured it's possible to have loose coupling the between client and server. This video demonstrates how it is possible to make changes in structure of the API that the client detects automatically:
Can you give an example of an acceptable implementation of a HATEOAS REST API (wow, that's a lot of letters) with an associated client that actually uses it?
My experience has been that you can't communicate much through HATEOAS that's actually beneficial to a human programmer writing a client. Sure, you can add all the hypermedia links you want in your API responses, but how does that make writing a client easier? Wouldn't it just be helpful to crawlers?
Not trying to put down the idea - I want to believe, but I just haven't seen any obvious examples using it in the real world yet.
It (should, I haven't tried it in a long while) work with both just fine. They both use the ALPS microblogging spec. yay generic clients!
As for people who have 'more real' ones: GitHub, Twilio (partially, more in the future), Balanced Payments (YC W11, iirc), Comcast (though that's internal :() Netflix has aspects, FoxyCart.
Interesting. I didn't realize that there were specs like ALPS for defining how to implement this kind of thing. Isn't it still a little difficult to do this for APIs that don't neatly fit into some kind of generic profile (e.g. microblogging)?
All that these sorts of description produce is a low-level API. That can be useful, but what's really needed are high-level APIs that provide meaningful semnatics:
my $me = Facebook->new( username => 'autarch' );
$me->post_status("I'm on Hacker News writing this comment");
my $friend = Facebook->new( username => 'imaginary' );
$me->post_on_wall( $friend, "Hey buddy, I am on Hacker News writing this comment" );
Here's another way to phrase it. A good API is based on the data and actions related to a specific domain of knowledge. Generic solutions produce APIs that are oriented around the communications protocol (REST).
On the client side, I don't really care (too much) if something is a POST or PUT, I want to send a message or update a repository's metadata or share a photo.
We've been experimenting with this at my office. We use yaml descriptions of all of our routes to generate test coverage. We plan to later generate our documentation and client libraries with the same docs.
Document-generated server behavior is something we're researching as well, to possibly represent business logic. We're hoping that patterns can be found and condensed into notations, like Regular Expressions do for string-parsing. We'll post about anything that we come up with.
One of my side projects us an Ajax library which allows javascript to respond to requests (LinkJS [1]). It has a helper object called the Navigator, which is like a miniature Web Agent. It retains a context, and uses the Link header from the response to populate the navigator with relations. It works out like this:
var nav = Link.navigator('http://mysite.com');
nav.collection('users').item('pfraze').getJson()
.then(function(res) {
console.log(res.body); // => { name:'pfraze', role:'admin' ...}
})
.except(function(err) {
console.log(err.message); // => 404: not found
console.log(err.response.status); // => 404
});
The advantage is that the link header is a relatively condensed representation of the resource graph. As a result, it's not a problem to send it and process it every time. You do gain latency, but the internet is only getting faster, and caching can be used. Meanwhile, the server can rewire links without interrupting their clients.
I have been puzzling over this API discovery issue on my current project (building out a reporting API).
I'm starting with self documentation for developers built in, not this (admittedly admirable) goal of a machine generated API mapping layer. I think the main issue is, you're trying to generate a generic interface to something that isn't, itself, generic.
How many versions of "RESTful" have you encountered?
Building a generic interface to non-generic interfaces is the domain of software engineers. Until we have machines building both sides of this equation, there will always be a need for human intervention.
I don't see how writing JSON REST API descriptions isn't practically the same as writing REST API clients anyway: they're still clients, just written declaratively rather than procedurally.
If the point is "stop writing procedural REST API clients and write them declaratively instead" then that advice is by no means restricted to REST API clients.
If the point is "hey, I noticed that REST API clients are another thing that we can now comfortably write declaratively" then OK.
I was thinking about this yesterday, but its seems like HN likes that kinda thing. Lots of frontpage article are direct orders from blogs with who knows what credibility.
I was just thinking that, I hate article titles that are phrased exactly as a command and it's merely a blog post that wants to change an entire body of thought that's well founded.
Writing using authoritative language is very common, and widely considered a best practice.
I highly recommend you simply accept it as what it is: the way some people communicate, especially online. It's not worth your time/attention/care to think about this.
Writing 'authoritatively' is used as a pop-culture substitute for reasoned argument. I find that it's a good litmus test for determining who I should ignore.
I read it as (I'd like to show you something that may help you) "Stop writing REST API clients". Imagine trying to visually scan HN article titles having to filter through useless pleasantries like that.
I take all direct orders as suggestions. Sometimes this gets me into trouble; The same way it does my 2 year old. Most of the time though, it's the way to go.
That's because the article does not argue against REST APIs. It argues against coding wrappers for them and instead proposes a solution that allows you to define API specific behavior in JSON and use one library to rule them all.
You aren't, but it's a mistaken thing to dislike: there's no value to rephrasing it as "I believe you should [do X]" or "I am arguing that you should [do X]" because that's necessarily always true anyway, i.e. no article can ever be anything other than what the author believes and argues for. So 'softening' the language would be inefficient - it would use more words while adding nothing of substance.
Json Schema is pretty good at describing data coming through JSON. But describing (REST) APIs requires more. For example a standard way to describe API endpoints with parameters and response types, errors, related models, default/allowable values etc. This was what the OP was referring to and this is what Swagger is trying to do. The Swagger Spec is here with some more details on whats required in addition to JSON Schema to document APIs: https://github.com/wordnik/swagger-core/wiki/API-Declaration Incidentally model/data specifications in swagger spec does map closely with json schema.
Well, it wouldn't be so bad if it were universal enough, but we'd probably have to wrap it in some sort of inter-ORB protocol for the internet or something...
Yes it would. Because making it universal enough makes it incredibly verbose and nasty to work with. JSON became popular because it was simpler than Web Services which became popular because it was simpler than CORBA which became popular because it was simpler than just talking over raw sockets using some sort of protocol ... oh, wait.
Actually not having a "universal" spec helps: it forces every provider to give some thought to how to make his API as lean as possible. Hopefully.
WSDL describes a pact between a client and a server so they can't screw it up. Comparing WSDL to WADL would make sense. JSON just describes objects in compact notation. Comparing JSON to XML would make sense when XML is only used to describe objects and nothing more. WSDL and WADL and HTML are all XML derivatives. WSDL and WADL could never exist or HTML for that matter if trying to use JSON to describe them since that's not what JSON was designed for.
WSDL is an object notation for objects that describe what a web service should do. This could just as well be done in JSON. It would be marginally less verbose, too :-)
Oh, and it's perfectly feasible to translate a well formed HTML document into some sort of JSON object. After all, HTML is just a set of tags with values, no?
136 comments
[ 3.5 ms ] story [ 180 ms ] thread[1]: http://developers.helloreverb.com/swagger/
[2]: https://github.com/wordnik/swagger-core/wiki
And then, we'll need a central location to store all of these API descriptors and UDDI [2] will be back with a vengeance.
[1] http://www.w3.org/TR/wsdl
[2] http://en.wikipedia.org/wiki/Universal_Description_Discovery...
The point about REST is that it is self-describing. And ideally should be using the same URIs as the version people clicking around in Firefox or Chrome see. The API is just the XML or JSON or whatever is flavour of the week version of the HTML version.
(Or we could use embedded data—microformats, microdata, RDFa—and get rid of that distinction.)
You can see links to the clients on the right.
OPTIONS /somePath?
REST means more than just GET, POST, PUT, DELETE.
Some may make the excuse that call OPTIONS /path isn't straightforward, but I have no clue how you could get more obvious than that.
I really wish there was a DESCRIBE verb that would return a structure of what is expected to be received/sent.
Then microformats could spring up around datatypes returned by DESCRIBE. This is of course very XML.
Edit: This would also allow for automatic discovery of new APIs.
https://github.com/caseysoftware/web2project-slim
But one of the things I did a little differently is that instead of writing the code and then the docs separately, I pass the validation information from the object itself.. so the API layer doesn't have to know any of it in advance. It can pass that along to the end clients.
I'm not convinced this is the solution but it mostly works for now and would love any & all feedback.
My next proof of concept will be to use Javascript to retrieve the required fields and decorate a simple html form.
To paraphrase the OP, "Since so many APIs can be described in similar terms, why don't we have some sort of standard that one can look at to identify how to use the API instead of letting the API speak for itself?"
When you start going down this track, you're not only making things complicated on the client's end of things. On the server side, you're having to maintain two things for the API now. First: the ruleset, ensuring it's 100% to spec lest a client fail. Second: the code generating the response in the first place.
I've built clients and servers for both RESTful and SOAPy APIs and I can say I would take REST any day.
[1] - http://en.wikipedia.org/wiki/SOAP
...
> ... XML is ugly. JSON is nice since it's human-readable and light.
???
XML:
JSON: I don't know that either one is particularly close to the way I'd write down a list in real life, to be honest. This is a pretty trivial sample, and neither is especially hard to read/parse by a human. But the JSON still looks closer to line-noise to me. :-)Hmmmm. Maybe JSON should just take pages from their book instead of reinventing the wheel?
Hypermedia says "oh yeah, here's some markup, look there are URIs in it". For a human user, we're like "cool, I'll try and click these, see what they do".
But software is going to want "um...okay, how to I parse this markup, and how do I generate the submissions you want? And, okay, you can change URIs, but please don't change anything else about that operation, or I will break completely. That's right, we're not really decoupled."
So, even with hypermedia APIs, AFAIK you're still going to want some marshaling to/from host language. ...and so you're back to having a spec, and coupling, you've just moved it around.
(Rant on coupling, people seem to think it's always bad and you can make it go away. Reality: you can't make it go away, and sometimes just accepting it directly is a whole lot simpler than deceiving ourselves about it's existence by over-applying abstractions.)
Actually, you explicitly _don't_ want this. That's what hypermedia APIs are trying to remove.
That's why I think hypermedia makes all sorts of sense for explaining why the www is awesome--it change deal, users will adapt. But IMO it falls flat as some new paradigm for building client/server systems.
> Actually, you explicitly _don't_ want this. That's what hypermedia APIs are trying to remove.
Hm? I am skeptical...any links/explanations?
> But IMO it falls flat as some new paradigm for building client/server systems.
I know of one company which you've absolutely heard of who has a 30-person team building a hypermedia API. They haven't talked about it publicly because they see it as a strategic advantage.
This year will be the year of code and examples; last time I was in San Fransisco, 5 different startups came up to me and told me that hypermedia is solving their problems. Expect to see more of this going on soon.
> Hm? I am skeptical...any links/explanations?
Mike Amundsen's "Building Hypermedia APIs in HTML5 and Node" has a pretty big section on this, and my book has a section entitled "APIs should expose workflows."
I previously commented more about it here: http://news.ycombinator.com/item?id=4951477
RPC APIs expose functions, SOAP/"REST" APIs expose objects, hypermedia APIs expose processes.
Stop creating REST APIs that are only level 1 or 2 (see http://martinfowler.com/articles/richardsonMaturityModel.htm... ).
Start writing HATEOS systems where the client is coupled to the semantic rather than the syntax.
Machine parseable interface descriptions might get rid of some boilerplate but it doesn't make for a more robust client-server relationship.
The inline URLs of the web work because the consumers are humans who can deal with changes (more than just trivial URL changes, like added, removed features) and now click on this button or that button.
Software isn't that flexible, so it will be just as coupled as it is today--you're just moving the coupling around.
So this idea of a "robust client-server relationship" is a pipe dream IMO.
<link href="http://example.com/low.rss rel="podcast_url" />
You've moved the coupling from "look at URL xyz" to "look for rel podcast_url". Okay, yes, you can now change the URL, that's cool, but I assert that's relatively trivial. You can't truly add/remove new functionality (or break existing contracts, like "look for rel=podcast_url") that some omnipotent client would suddenly start taking advantage of.
IMO this omnipotent realization/utilization of new/changed features is what hypermedia advocates get all excited about, without realizing that humans are really the only ones that can deal with that level of (true) decoupling.
> You can't truly add/remove new functionality
You can absolutely add new functionality, because rule #1 of clients that work this way is 'ignore things you don't understand.' Removal will obviously break things, so it's important to handle this in the appropriate way.
I guess ultimately my point is that these kinds of APIs are significantly different, and come with a very different set of constraints/goals/implementation details. It's like you're saying "well, I don't have a full list of function calls I can make!" because you're used to SOAP where there's a WSDL and 'RESTful' APIs don't have description documents. Of course! It's a different architecture!
http://oredev.org/2010/sessions/hypermedia-apis
And he's release a Java library for creating such a client:
https://github.com/cimlabs/hypermedia-client-java
My experience has been that you can't communicate much through HATEOAS that's actually beneficial to a human programmer writing a client. Sure, you can add all the hypermedia links you want in your API responses, but how does that make writing a client easier? Wouldn't it just be helpful to crawlers?
Not trying to put down the idea - I want to believe, but I just haven't seen any obvious examples using it in the real world yet.
We're all calling these "hypermedia APIs" these days.
> with an associated client that actually uses it?
I have written a toy client here: https://gist.github.com/steveklabnik/2187514
You can run it against this site, written in node: http://alps-microblog.herokuapp.com/
Or this site, written in Rails: https://rstat.us/
It (should, I haven't tried it in a long while) work with both just fine. They both use the ALPS microblogging spec. yay generic clients!
As for people who have 'more real' ones: GitHub, Twilio (partially, more in the future), Balanced Payments (YC W11, iirc), Comcast (though that's internal :() Netflix has aspects, FoxyCart.
This year will be the year of examples.
Mike Amundsen's terribly named but amazingly well written 'Building Hypermedia APIs in HTML5 and Node' is a really thorough examination of this topic.
On the client side, I don't really care (too much) if something is a POST or PUT, I want to send a message or update a repository's metadata or share a photo.
Document-generated server behavior is something we're researching as well, to possibly represent business logic. We're hoping that patterns can be found and condensed into notations, like Regular Expressions do for string-parsing. We'll post about anything that we come up with.
One of my side projects us an Ajax library which allows javascript to respond to requests (LinkJS [1]). It has a helper object called the Navigator, which is like a miniature Web Agent. It retains a context, and uses the Link header from the response to populate the navigator with relations. It works out like this:
The advantage is that the link header is a relatively condensed representation of the resource graph. As a result, it's not a problem to send it and process it every time. You do gain latency, but the internet is only getting faster, and caching can be used. Meanwhile, the server can rewire links without interrupting their clients.1 https://github.com/pfraze/linkjs
Most overused phrase right there.
Flippancy aside, maybe there's a need for a next generation of this that skips all the XML headaches after all.
JSON has just been lucky that its user base hasn't been the same enterprise architects that ruined Java.
I have been puzzling over this API discovery issue on my current project (building out a reporting API).
I'm starting with self documentation for developers built in, not this (admittedly admirable) goal of a machine generated API mapping layer. I think the main issue is, you're trying to generate a generic interface to something that isn't, itself, generic.
How many versions of "RESTful" have you encountered?
Building a generic interface to non-generic interfaces is the domain of software engineers. Until we have machines building both sides of this equation, there will always be a need for human intervention.
If the point is "stop writing procedural REST API clients and write them declaratively instead" then that advice is by no means restricted to REST API clients.
If the point is "hey, I noticed that REST API clients are another thing that we can now comfortably write declaratively" then OK.
I highly recommend you simply accept it as what it is: the way some people communicate, especially online. It's not worth your time/attention/care to think about this.
But it still looks like they pulled some rules out of nowhere and are enforcing them.
My naive impression is that JSON-Schema is trying to be just like XML Schema, but in JSON. Which doesn't seem like a good thing.
Actually not having a "universal" spec helps: it forces every provider to give some thought to how to make his API as lean as possible. Hopefully.
I think too many people consume REST APIs in different manners, utilizing different data in unique relations. This is the beauty of it.
Oh, and it's perfectly feasible to translate a well formed HTML document into some sort of JSON object. After all, HTML is just a set of tags with values, no?