This is ridiculous, we can't just let people have access to their data!... i mean OUR data. Just imagine, they may learn things we could have forced them to pay absurd fees to get! :P
As right as you are, making an API for your service so people can replace your crappy interface with their own good interface probably isn't the best business decision. Online banking sites can display ads. The bank can't inject ads into another client though.
Also consider that the bank would get blamed in popular opinion if someone wrote an insecure client that was determined to be leaking data.
But consider for example if Bank of America did this and someone created an awesome client app that basically became the main way anyone used their Bank of America account. If a bug is discovered in this application that means everyone's data is no longer secure, are users going to blame the app creator or Bank of America? Can Bank of America afford to take that chance?
You're right -- that's almost certainly the reason you don't see stuff like this happen.
Though, didn't Bank of America make that iPhone app a little while ago that allowed you to photograph a cheque in order to deposit it? They're obviously OK with sending financial data around. I guess maybe it was OK in that case because they were the ones that wrote the app?
In any case, you're right. Security is definitely the reason this kind of stuff hasn't been implemented. I still think that it's possible to write an open, secure interface to things. Email is the example I fall back on here. Like I mentioned above - sensitive information gets sent around in emails all the time, yet there are countless email clients and they're almost all secure (I say "almost all" to cover my ass if there's some that aren't). I think that if it can work for email, then maybe it can work in other domains.
Unfortunately, email is a terrible example. While it's possible to have a modicum of security in regards to receiving your e-mail, spoofing emails is trivially easy (unless you use cryptographic signing, which isn't well supported by the majority of clients afaik)
I don't see how that's related at all. I'm not saying Bank of America needs to work hard to prevent automated access to their online banking system. Right now, automated access counts as "undocumented behavior" at best. Bank of America is not likely to face public ridicule because someone hacked up an automated interface to their site which was not intended for that purpose and shot themselves in the foot. If, however, Bank of America says explicitally, "Hey, come use this API and applications built off it to access your account," and then one of those applications starts shooting people in the foot, people are going to wonder why Bank of America thought it was a good idea to let people access their accounts like that in the first place.
For my own personal use I think text based interfaces (with pipes) are superior, the higher level pluggable components in AppleScript/Automator seem more usable in general.
That said, there was an excellent article recently with the experiences of a trainer in getting people to use *nix, who found the minimalistic "command mode" more usable due to it's text oriented simplicity.
I don't personally know because I'm just a peon with a qwerty keyboard, but I think there are some great developers and talented architects out there who could figure it out and make it happen.
Don't captchas and open APIs fundamentally oppose each other?
The point of an API is to have a clear interface so that programs can be built on a service, and captchas exist to try to block bots from using services.
That would be an easy API to implement, in psuedo-code:
do:
server->return any errors
client->display errors
client->ask user for account name
client->server->request account name x
while ( server->x is invalid account name )
<< Repeat above loop for password >>
do:
server->return new captcha
client->display captcha
client->take user input
client->server->send captcha text
while(server->captcha does not match)
server->return success message
Agreed. I can't justify a use case for CAPTCHAs outside the browser, I was just pointing out that such an API is very conceivable. As silentbicycle said, a CAPTCHA is for preventing scripting, so you would (hopefully) still need a human somewhere in the process.
Easy. One of the arguments to the create_email call would be a phone number or IM or existing email. The captcha would be mms'ed/im'ed/emailed (along with an explanation of what it's for), and the account would be created when the person replied with the correct response.
As previously said, the financial incentive is simply not there. As much sense as it makes for you to have easy access to your information, there are too many businesses that rely on controlling that access.
On the other hand, tools to do this with almost any site already exist. RSS is common enough, and where it doesn't exist, screen scraping isn't that hard (and getting easier all the time). If I'm not mistaken, this is the exact problem that Yahoo pipes are trying to address.
Of course, sites that require a complicated login process (banking sites) are still out.
Having a good API for everything would be almost as good as The Singularity (tm), but making a good and secure API is very hard and comes with a massive hidden cost.
1) An API can require even more planning to design than the application or service itself.
2) Mistakes made in the design or implementation of the API turn into liabilities you can't solve without breaking users of the API.
3) The cost of maintaining an API so that it and its documentation doesn't fall out of date with the app/service can be as high (and sometimes even higher) as the cost of maintaining the app itself.
4) It widens the attack surface.
And those are just the ones off the top of my head.
I agree with all of your points, but I think many of these costs are minimized by the other point mentioned in the OP: following the UNIX philosophy. The smaller and simpler your service is, the better. If you have a service that can't be accessed with a simple API, you probably have more than one service, and would benefit from splitting it into multiple APIs.
Another thing worth mentioning is the possibility of building your API first, and then building your service on top of it. If you are providing a web service, then it [usually] boils down to data access and management. Figure out what data you are managing for your users and what you need to do with it, then build your API around that. If you plan to have a public facing API from the start, this approach makes sense (in the same way that using javascript only for progressive enhancement of a working site makes sense).
Of course, because you are an "internal user" of the API, you can bend the rules a little more than the average client, but the less you do this, the more useful your API is.
Regarding points 2 and 4: if public APIs for data access become the norm, this will simply become the cost of doing business. Secure API's aren't that difficult, and updating an API while maintaining backwards compatibility can be done, especially if you transition these changes and notify all broken clients.
For the UNIX philosophy I like the idea of small, useful tools loosely coupled. A big part of UNIX is that there are a lot of little tools that do one thing well. The smaller the tool is, less attack surface there will be, and each tool can interact with a common authentication/authorization interface to further simplify things.
So many web apps are already built on top of an API or framework provided by a RAD tool. So if you build another API atop that then you'd lose the ability to enjoy the benefits of the RAD tool for the actual application--since it's now two levels away from it.
The printing press enabled more people to read, though still few could write. The net has enabled more people to write, though still few can program. A net where everything has an API is an extremely helpful step in creating a system which enables more people to program.
The scenario descriptions are distinctly reminiscent of Berners-Lee's scenario descriptions for the 'semantic web' several years ago. Of course, that is now rather out of fashion, but not entirely justifiably. The 'semantic' angle was somewhat fanciful, but the core idea was the data web: It is one thing to put APIs on everything, but another for all the data types and formats to work together. That was more the aspect that RDF was addressing. It is neither thoroughly convincing nor completely apposite, but worth factoring in to the consideration anyway. (The essential RDF concept was elegant, whatever anyone says.)
Yes. I was just doing my college applications and thinking this exact thing. The Common Application would look like a piddly idea if the college application process just had an API.
I agree with 95% of what he says, but I'd make one change to the Unix philosophy: text files aren't the best primitive.
Text files have a super low-level API (stat, seek, read). I think the system he describes would be much more usable if he picked slightly higher-level data abstractions. I would recommend Python's built-in types: list, tuple, dict, set. Most data can be easily represented by these types and they provide a higher level API than streams (files), which means the barriers to using them are much lower.
That's probably fair. I honestly don't think we're at the point where we can abandon text files altogether, but you're right -- a lot of data could definitely be stored in slightly higher-level data structures.
The Unix idea has never been that text files are the best primitive, it's that they're the best approach until you have a solid reason to do otherwise, and you should start simple. (Unix is more pragmatic than that - "worse is better", etc.) As long as you use plain text, you can use grep, diff, various version-control systems, etc., without having to rewrite any tools for your custom format. Still, sometimes you need more elaborate formats.
SQLite is one good example of a portable, open, simple format that is complementary to flat files. JSON and Lua also seem very Unix-y to me. Unix typically also uses ELF, various compiled key-value databases, etc.
Unix's choice of data types was also influenced by being written for computer that had 32k of RAM. That's, what, as much as a bar of soap nowadays?
well, most data is exposed via a website, so if the api doesn't exist, just mechanize it. its not really feasible to rely on goodwill for these grand visions.
I want, I want, I want. How about you propose how this should be done? It's harder than it looks, and no, Unix is not the one-and-all answer to everything.
Take one of his example scenarios:
"When my alarm clock goes off in the morning, get the latest tech news podcast (downloaded overnight) from my computer via WiFi or a wired network connection. Play the podcast through the alarm clock’s speaker/s. If I hit “snooze” on the alarm clock, pause the podcast. Once I stop the alarm, remember where I was up to in the podcast and give that information back to my computer. Now when I go to listen to the podcast while I’m eating breakfast, it will continue playing where it left off."
This has a number of hidden assumptions:
* What is "the latest news podcast"? One user-selected source, or a blended filtered combination, or a crawled set selected based on previous behavior?
* How does a (not the!) computer know how to talk to the alarm clock? I hope I don't have to create a home network myself, cause I know my Mom certainly can't. How is this connection authorized? What happens if the power goes out on the computer - what will the clock do? What if the clock batteries fail - will the computer play the podcast instead?
* Is the saved position synced only the one source computer (I hope not). What if the network to the computer is down - can the alarm clock sync somewhere else?
* Can I resume listening to the podcast on a different computer at breakfast? My partner/sibling/friend's computer? How do I log in? How is the position data saved along with my identity? How does the breakfast computer address the first computer, or the alarm clock?
The problem here is not access to the external APIs (not that hard) but federation, identity management, authorization, privacy considerations, and addressing between these disparate systems. Proposal adopting existing standards for these requirements, or new proposals for standards are, of course, appreciated.
Here's the thing -- I could come up with individual answers to those questions based on arbitrary decisions that I make ("the user selects the source", "it's implied that a WiFi network already operates in the house", "the position is only synced to the source computer", "you can listen to the podcast at a different computer"), but you're totally right -- arbitrarily-decided-on answers doth not a standard make.
Like I said in another thread, I am just a peon with access to a keyboard. I trust that there are developers and system-setter-upper-ers much smarter than I who could put this into place.
On the existing standard front, there is Universal Plug and Play (http://www.upnp.org/) that aims to "to enable simple and robust connectivity among consumer electronics, intelligent appliances and mobile devices from many different vendors." I haven't played with this stuff at all, but there appears to be fair amount activity in this space: http://en.wikipedia.org/wiki/UPnP_AV_MediaServers
I don't know how much of the example scenario you could implement within the confines of UPnP, but perhaps it could be part of the solution.
43 comments
[ 3.0 ms ] story [ 85.5 ms ] threadAlso consider that the bank would get blamed in popular opinion if someone wrote an insecure client that was determined to be leaking data.
If we don't make APIs because someone might write an insecure client, would anyone make any APIs ever?
I do agree with you though :) It's probably not going to happen in the real world any time soon. C'est la vie.
But consider for example if Bank of America did this and someone created an awesome client app that basically became the main way anyone used their Bank of America account. If a bug is discovered in this application that means everyone's data is no longer secure, are users going to blame the app creator or Bank of America? Can Bank of America afford to take that chance?
Though, didn't Bank of America make that iPhone app a little while ago that allowed you to photograph a cheque in order to deposit it? They're obviously OK with sending financial data around. I guess maybe it was OK in that case because they were the ones that wrote the app?
In any case, you're right. Security is definitely the reason this kind of stuff hasn't been implemented. I still think that it's possible to write an open, secure interface to things. Email is the example I fall back on here. Like I mentioned above - sensitive information gets sent around in emails all the time, yet there are countless email clients and they're almost all secure (I say "almost all" to cover my ass if there's some that aren't). I think that if it can work for email, then maybe it can work in other domains.
That said, there was an excellent article recently with the experiences of a trainer in getting people to use *nix, who found the minimalistic "command mode" more usable due to it's text oriented simplicity.
The point of an API is to have a clear interface so that programs can be built on a service, and captchas exist to try to block bots from using services.
On the other hand, tools to do this with almost any site already exist. RSS is common enough, and where it doesn't exist, screen scraping isn't that hard (and getting easier all the time). If I'm not mistaken, this is the exact problem that Yahoo pipes are trying to address.
Of course, sites that require a complicated login process (banking sites) are still out.
As more companies become Google-esque data hogs, APIs will probably be encouraged.
Everything should talk to one another, but a service's name or url should never have to appear in any code. That's the user's job to input.
1) An API can require even more planning to design than the application or service itself.
2) Mistakes made in the design or implementation of the API turn into liabilities you can't solve without breaking users of the API.
3) The cost of maintaining an API so that it and its documentation doesn't fall out of date with the app/service can be as high (and sometimes even higher) as the cost of maintaining the app itself.
4) It widens the attack surface.
And those are just the ones off the top of my head.
Another thing worth mentioning is the possibility of building your API first, and then building your service on top of it. If you are providing a web service, then it [usually] boils down to data access and management. Figure out what data you are managing for your users and what you need to do with it, then build your API around that. If you plan to have a public facing API from the start, this approach makes sense (in the same way that using javascript only for progressive enhancement of a working site makes sense).
Of course, because you are an "internal user" of the API, you can bend the rules a little more than the average client, but the less you do this, the more useful your API is.
Regarding points 2 and 4: if public APIs for data access become the norm, this will simply become the cost of doing business. Secure API's aren't that difficult, and updating an API while maintaining backwards compatibility can be done, especially if you transition these changes and notify all broken clients.
So many web apps are already built on top of an API or framework provided by a RAD tool. So if you build another API atop that then you'd lose the ability to enjoy the benefits of the RAD tool for the actual application--since it's now two levels away from it.
Text files have a super low-level API (stat, seek, read). I think the system he describes would be much more usable if he picked slightly higher-level data abstractions. I would recommend Python's built-in types: list, tuple, dict, set. Most data can be easily represented by these types and they provide a higher level API than streams (files), which means the barriers to using them are much lower.
I still really like text files though :)
Great post by the way, very thought provoking.
SQLite is one good example of a portable, open, simple format that is complementary to flat files. JSON and Lua also seem very Unix-y to me. Unix typically also uses ELF, various compiled key-value databases, etc.
Unix's choice of data types was also influenced by being written for computer that had 32k of RAM. That's, what, as much as a bar of soap nowadays?
Take one of his example scenarios:
"When my alarm clock goes off in the morning, get the latest tech news podcast (downloaded overnight) from my computer via WiFi or a wired network connection. Play the podcast through the alarm clock’s speaker/s. If I hit “snooze” on the alarm clock, pause the podcast. Once I stop the alarm, remember where I was up to in the podcast and give that information back to my computer. Now when I go to listen to the podcast while I’m eating breakfast, it will continue playing where it left off."
This has a number of hidden assumptions:
* What is "the latest news podcast"? One user-selected source, or a blended filtered combination, or a crawled set selected based on previous behavior?
* How does a (not the!) computer know how to talk to the alarm clock? I hope I don't have to create a home network myself, cause I know my Mom certainly can't. How is this connection authorized? What happens if the power goes out on the computer - what will the clock do? What if the clock batteries fail - will the computer play the podcast instead?
* Is the saved position synced only the one source computer (I hope not). What if the network to the computer is down - can the alarm clock sync somewhere else?
* Can I resume listening to the podcast on a different computer at breakfast? My partner/sibling/friend's computer? How do I log in? How is the position data saved along with my identity? How does the breakfast computer address the first computer, or the alarm clock?
The problem here is not access to the external APIs (not that hard) but federation, identity management, authorization, privacy considerations, and addressing between these disparate systems. Proposal adopting existing standards for these requirements, or new proposals for standards are, of course, appreciated.
Like I said in another thread, I am just a peon with access to a keyboard. I trust that there are developers and system-setter-upper-ers much smarter than I who could put this into place.
I don't know how much of the example scenario you could implement within the confines of UPnP, but perhaps it could be part of the solution.