How did this float to the top of my front page so quickly. Are we all genuinely _that_ interested in new AWS products? or did this post receive some "extra" help
New product announcements from large tech companies make it to #1 before getting any comments on a regular basis. Submitting a duplicate URL counts as an upvote for the original submission, so it's pretty easy for something like this to make it to #1 just from people racing to be the first one to submit it.
Ah, I didn't know about the duplicate URL rule. That makes more sense. I've been here for quite a while and I haven't noticed such an aggressive advance before. So it triggered an alarm. Maybe I'll start seeing it more often
To answer your first question: Yes. Many of those who work on services are very interested in cloud-based systems, and Amazon happens to be a major leader in this.
Ancillary factors:
1. The title is clickworthy. While link-baity titles do work well on HN (compared to very vague titles)...this title sticks out because it is two words: One of them is "Amazon", which has very immediate connotations and brand recognition. The second is a proper noun. Those of us who haven't heard of "Cognito" are likely to think, immediately, "Whoa, new Amazon product, must click-through to see how it will impact my work". That it is two words also makes it visually distinctive regardless of what those words even are.
2. It's 11AM, which means almost every developer on the East Coast is awake, and West Coast developers, particularly the ones responsible enough to reserve HN viewing for before-work hours, are getting their coffee and checking HN. So, lots of potential upvoters.
Every time someone submits a link, the first posting gets a point. So if 20 people independently posted it to HN before it got to the front page, it would accumulate points pretty quickly without comments.
Amazon: "It is so difficult to merge user behavior data across different devices they use... I have an idea! Let's create a service which will do it for "them"!
Does Apple consider iCloud a competitor to other cloud services? You need an Apple device to sign up for it, so you can't move to it if you only have access to Android or Windows devices.
While this (and other similar) Backend-aaS look very appealing at first glance, it seems to suffer from the same problem as most AWS offerings - terrible platform lock-in. Show me that I could move all of this to another provider if I need to, and then we can talk seriously.
I also wonder how they "manage[s] the complexity of conflict resolution" without manual intervention.
The trick with your suggestion of making data more portable is that if Amazon or any other platform provider would do so, then it would create a race to the bottom in terms of price. This would make the business quickly unappealing for the likes of Amazon. This is why no one will make their platform formats portable. At least not while things are in flux and the battle field is still disputed. Once the market matures (see the mobile telephony of 2000s) then you'll be able to port your phone number.
At Fanout.io (not BaaS like Cognito, but another kind of microservice) we opened everything up to make us easy to migrate off of. Why wait for the market to mature when we can do things the way customers want from the start?
I'm a bit confused by your objection to "platform lock-in." Presumably a person or company who has chosen to use this service understands the technical limitations and features it entails. That includes understanding that AWS may in fact implement this service differently than other providers do and that a future move might entail some non-trivial effort. That isn't platform lock-in in any meaningful sense.
Besides, what makes you think you would be prevented from moving all this to another provider? From the FAQ (https://aws.amazon.com/cognito/faqs/), it seems that Cognito data is stored in a straight-forward Key-Value scheme. That's about as easy as it gets when it comes to data structure.
Moreover, the actual service itself would almost certainly be implemented differently with another provider. Why should Amazon, or anybody else who offers similar "menu of services" cloud computing, make it easy for you to do so (they make money off of you not moving)?
Because the best way to get a user to "move in" is to make it easy for them to "move out". It's actually in Amazon's financial interest to make it easy for people to move off of their offerings.
This is one of our core values and while we get odd looks from investors or MBA-types we know that it's the right move to make in the long run. This is the way things are moving and I hope that more companies will go this route.
The thing about casual discussions like that is it's easy to build a narrative without actually providing much substance. Joel's essay doesn't discuss Microsoft's "barrier(s) to exit" that Office itself didn't give up completely until last year (cf: Open Document), and Windows itself has yet to yield (it is still the dominant OS, by far, in the desktop and laptop markets).
Sure, "barriers to exit become barriers to entry" is applicable in some contexts, and for some part of a potential market it will be the barrier that keeps those customers away. I wouldn't dispute that. I would dispute that "barriers to exit become barriers to entry" be treated as a universal maxim applicable along all businesses and products. It's just as much of a strategy mistake to make as assuming the barrier can't exist in the first place. It ought to be considered; not treated as a key issue until or unless the data suggest it should be.
> Presumably a person or company who has chosen to use this service understands the technical limitations and features it entails.
That's a pretty big assumption to make. People, even technical people, make assumptions all the time that they'll be able to get their data out of a system and then aren't able to at a future date.
Is anyone delivering the equivalent with less lock-in? I can understand both sides of the argument but it seems that over time stuff always migrates towards more control (ie, people migrate OFF of Parse, Heroku, AWS, etc). I'd envision a much better set of images, Fabric/Chef recipes, to the point where you get your AWS but with vanilla OSS on generic hosting providers.
I'm trying out DreamFactory (dreamfactory.com) to see if I can accomplish the same goals but using something that's open source (Apache License). Sure I'll have to maintain my own VPS boxes but that's something I do anyway.
A couple of open source options that you can deploy to your own servers (or your cloud provider):
- Apache UserGrid (incubator project)
- DreamFactory
- BaaSBox
- LoopBack
- Helios
- Deployd
Couchbase Mobile using Sync Gateway is incredibly similar to Amazon Cognito. Even the native iOS and Android libraries have a similar feel to them (object-mapping with native objective-c objects, single call sync/replication function). Sync gateway has built in support for identity providers such as Facebook, but doesn't provide Google or Amazon identity support (this could easily be added with custom server code).
The big difference I am seeing here is Amazon Cognito doesn't yet have a story around syncing documents between multiple users/roles. Sync gateway has the notion of a sync function that providers developers with a hook to customize server-side syncing in a deeper way.
I'll be interested to see how Amazon developers the multi-user/role sync capabilities and integrates it with their native identity management solution. Could be very compelling down the road..
A buzzword used in Cognito and Zocalo is "secure". I'm assuming that means that it's server-side encrypted at rest.
What would be really impressive is if there was an option to client-side encrypt the data before sending it to AWS. Of course, that would mean you'd have to move your syncing logic to the client-side, too, but having a good client-side encryption option would be a real differentiator.
Wild guess: Amazon does nothing to solve sync conflicts, because it's highly correlated with your app business. Amazon gives you all conflicting versions and some flags telling your app "there's a conflict, you should fix it". Much like CouchDB.
So if it works like that, you absolutely can encrypt your payload client-side, and use Cognito as a mere transport.
Exactly. I don't know why people are so attracted to locked in solutions like Parse or now this when Couch has already been doing it for a long time, and there are mature third-party providers (Cloudant, Iriscouch, etc) that can help you get off the ground just as easily without ever being locked in with them.
I don't know how Amazon comes up with names for its products, but recently they all strike me as weird and stupid names. Do they confer with marketing first? Zocalo, Cognito? Really? These are the kinds of names you invent when you're looking for a cheap domain name for your startup. For Amazon, they don't make a lot of sense. They just seem like some executive out of their marketing depth and out of touch trying to be clever and anyone with sense afraid to speak up during the meeting when they come up with these names.
What cloud service names, off the top of your head, do you consider to be exemplary?
"Parse"? OK, sure...but that's not any more functionally descriptive than "Cognito"...and Parse is a more narrowly focused company with a primary service...In the case of "Cognito" and "Zocalo"..., Amazon has to come up with names that are distinctive because they provide so many core services...so many that even me, an engineer, get them confused. Imagine the difficulty of a fly-by marketing person trying to differentiate between similar sounding names.
Furthermore, these names are distinctive industry wide...presumably, if Zocalo becomes big, you will no longer have to say "AWS Zocalo", but just, "Zocalo"...just like you do with "S3" and "EC2".
To drive home the situation that Amazon is in, let's consider another great product name: Dropbox. Unlike Parse, it is very easy to relate to the core service. But consider the phrase, "Amazon Dropbox"..."Dropbox" is unique compared to the currently named AWS services...but it has a huge collision with Amazon's core service, which is, stocking and shipping products. If Dropbox.com didn't exist, and Amazon used it instead of "Zocalo"...there is a considerable risk that people associate "dropbox" with some kind of packaging thing.
Out of their recently-launched services, I liked the name "MatchBook" the most. It was intended to provide cheap or free Kindle versions of books which you'd previously purchased physical versions of. Unfortunately, it's a product that virtually no publishers have adopted.
They're unique while remaining loosely related to the products they describe.
The http://en.wikipedia.org/wiki/Z%C3%B3calo is the city center (of Mexico City's historical district), a simple and untaken metaphor for a service that "...provides users with a central location for both the documents and files they are...".
Zocalo, sure...but Cognito makes complete sense. The word incognito basically means anonymous. The prefix 'in-' frequently negates the meaning of the word it's attached to. So when you remove the negation from anonymous, you get a known identity, which seems very apropos to an identity service.
Cognito means known, recognized, identified; contrast with incognito. Is that really such a terrible, horrible, no good name for a service that verifies user identities and manages user profile data?
It's a shame Amazon can't afford to hire a marketing genius like you, but I think they're doing the best they can with the idiots they have to work with.
With so many new AWS features rolling out, I am worried that Amazon is spreading themselves too thin and will soon decide to stop supporting some of their less-popular services that I happen to be using.
As far as I can tell, there is no guarantee anywhere that they'll maintain any service for any length of time.
If that happens, it could waste weeks or months of engineering time trying to migrate to something else, or I'll just have to shut down my own services if I decide it's not worth migrating.
It's a valid concern. AWS has been around for many years now with numerous services. Has there been a single offering which they retired or stopped supporting? I'm not aware of one.
Amazon doesn't just create services, throws them over the wall and see if they stick. Rather, they almost always RESPOND to customer requests. Dr. Werner Vogels, AWS's CTO makes a point about that approach at about any keynote he gives. Similarly, they try to also avoid feature creep by only providing features which customers really ask for.
I am unaware of AWS modules discontinued by amazon, but it is also true that the oldest services are the most popular (S3, EC2) while some of the most recent ones have a (apparently?) smaller user base (SWF, Kinesis, AppStream).
I think I remember amazon people saying they are actually dogfooding this stuff so it should stay available for a long time anyway, rather than being a random Google Reader.
There is a strong culture of dog-fooding in Amazon. The advantages to it are quite straight forward :) If you've got a whole bunch of DBA experts already hired, why would you hire additional people to do DBA work for every team, when they could just use RDS? Along with that is the huge added benefit of a direct feedback loop, with consumers and producers able to directly meet and deal with feature requests, problems etc. etc.
Retiring any of the services that have been launched would cause not inconsiderable pain to other teams :)
AWS definitely dogfoods, to the point where people have to be careful internally to not create circular dependencies (this nearly happened to me once).
I think there's a notion of persistence in AWS' leadership that propagates through the teams. AWS is willing to fight through the early painful years for products to see them arrive at the stage that EC2/S3 have attained.
I am not too concerned about this – though I do think it's a valid concern considering the actions of other players in the space.
Sitting here in Seattle I see Amazon hoovering up all the local talent they can hire. And I see them putting up new tower after new tower to house them. I don't worry about them stretching themselves too thin as far as manpower goes.
And they don't have a history of closing down AWS products – for instance the first, (and not a runaway hit) Mechanical Turk is still with us.
Mturk is heavily used internally and in academia for a crazy variety of stuff. It's not S3, but it's hugely important for the people it serves; it would be incredibly foolish to shut it down.
Sounds like it saved you a lot of time getting your app up and running and I'm assuming they would give you a lot of time to migrate before just shutting it down. I wouldn't worry about this, however, this is something to consider any time you use a 3rd party to manage a service for you. I've had issues with fully managed RackSpace servers where the OS hit its EOL and it was on me to migrate everything to a new server. I've had this issue with fully managed database providers when they migrated to a new infrastructure and I had to move my data into a new database. There are other examples but most of them are pretty minor and none of them were too big of a deal.
There are many startups which do this (or at least similar) already. It's often referred to as "backend as a service" (BaaS). Some links below, but you'll find many more via Google.
Orando Labs (https://orandolabs.com) has EnduroSync and Identio. Striking similarities between the products. EnduroSync is a full object store and does not have the data size limitations.
For those wary of vendor lock-in, or for those wanting a self-hosted similar service: Just yesterday I spun up my own oauth.io oauthd server. It's working out fairly well. It's only third-party authentication, it doesn't handle guest users or local authentication.
The interesting thing about all these services is that they're trying to abstract away the same types of problems that all developers come across: Identity, connectivity, sync/backup (and to some extent, deployment).
A different (and more disruptive?) approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application. i.e something like the app-store model but it connects to the user's 'personal cloud' (or the desktops of old - if you prefer). Such a system needs to be FOSS at its core but with a way for developers to get paid for providing value to end-users (who themselves get control of their data/networks).
Of course, this won't happen overnight, but the alternative is that everyone ends up using proprietary silos, with huge lock-in and innovation suffers as the tech giants get distracted with lawsuits.
I'm working with others on the distributed systems infrastructure we need to make this possible [1], so that we can get to a place where everyone can have their own piece of the cloud. In fact, one of the major components of this is Mirage, which has been discussed quite a bit on HN recently [2].
Do users really care about this? My guess is that 99% don't and don't want to deal with these details. The project sounds cool and like something the HN crowd might want, but my intuition is that most users won't understand the implications.
It also adds an extra step in the data storage process within the app, which increases friction and seems like it would be hard to get developers to adopt. All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems.
Yes, users care about this but like most things you have to segment the market appropriately. I meet more and more people who are wary of putting all their trust/data/etc into only one provider, be that Google, Apple or anyone else. It's not just a privacy thing but a growing awareness of the sheer dependency on that one company. (Aside: I have to say FB is being pretty clever by buying up apps and letting them stay 'independent' - users don't notice that as much and reminds me of FMCG.).
We don't need most users in the beginning. Just those that are technically savvy enough to understand the landscape and the way the world is headed. From there we can develop more stuff and eventually get it into a state ready for the masses. For example, I can run my personal website as a unikernel [1], which is not suitable for mom-and-pop but great for anyone currently using any static site generators. As more libraries get released I can build on that an eventually run more of my infrastructure this way (my prototypical examples are mail, contacts and calendars).
In terms of the friction for developers, we simply have to make the libraries and tooling fun and easy to use. With well-designed systems it's not necessarily any more friction that what developers have to do now (i.e build their own backend).
> "All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems."
Futurists have a tendency to imagine a world of changed human behavior and it's compelling to do so. The reality is that the future rarely arrives as sweeping change, but rather as metaphor and specialization.
Whereas you can imagine others adopting new patterns of behavior because you understand the underlying reasons why such behavior is reasonable, the metaphor through which you explain this change is not readily understood. Why, as a User do I want this? If the answer is control and privacy, you might be barking up the wrong tree (time and again we've shown that those are not things consumers want or are willing to pay for).
If you want to drive dynamic change in the world, you have to change the underlying structure of complicated systems while steadfastly avoiding changes in user behavior. It turns out this is quite hard.
I applaud your efforts but encourage you to avoid the rabbit hole of endless specialization and to improve the marketing metaphor/rhetoric.
> time and again we've shown that those are not things consumers want or are willing to pay for
CITATION please. I think we have never seen such things. We've not had studies that control for all other factors and then conclude that users don't want control and privacy. I think your claim is flat out wrong. You're taking the fact that most users won't sacrifice by using a lower-quality, obscure thing for privacy and control, and leaping to the baseless conclusion that nobody cares about these things. I think you're completely wrong.
There have not been any case studies that have empirically demonstrated this to be true. If that is your basis for evaluating my statements then I cannot support my claim in a manner that you would find satisfactory.
I would however, anecdotally highlight the progression of social services from places of privacy to places of publicity. One could argue that sites like "The Well" were the original social networks but they did not grow (whether by choice is of course a matter for debate). Over time, each successive social startup defaulted to a more open stance and scaled an order of magnitude in users with each step. The trend suggests that all things being equal, users will say they want privacy but want the features publishing and syndication provides (voting purely with their registrations and advertising value).
I agree that privacy has value, but I would argue that the anecdotal evidence suggests that users want less private services (and yes I'm aware that correlation is not causation, but I think this observation is apt). This is not law or dogma but rather a thought and I'm sorry if it was communicated poorly previously.
> the progression of social services from places of privacy to places of publicity
I don't think that's a clear-cut anecdote. Facebook does actually feel very private, because it's isolated from Google and outside users. Look at the number of female profiles where everything but the name is hidden from non-friends. I don't remember MySpace or blog users ever being this paranoid.
And then, anecdotally, many people around me have started to live in WhatsApp/LINE gossip groups, which is something that could easily be decentralised.
I would look towards the trajectory of Path, and the fate of app.net for your evidence of the preferences of end users vis-a-vis paying a premium for privacy and security relative to free alternatives that make no such claims (Facebook and Twitter, respectively).
Please don't assume that a lock-in based on network effects must mean users are happy with (or even aware of) that trade. I don't like FB's practices, or the way LinkedIn deals with me, yet I maintain profiles on both sites.
Twitter was unambiguously non-private from the start so there were no broken expectations.
> I meet more and more people who are wary of putting all their trust/data/etc into only one provider, be that Google, Apple or anyone else.
I'd be wary of assuming from this data point that people are willing to be even slightly inconvenienced for it. Just take a look at Facebook: the entire history of the company has been one of disrespecting and subsequently pissing off users, and yet out of the huge amount of people I know who have repeatedly expressed anger at how Facebook treats their users, I know exactly one person who stopped using it or even reduced their usage.
I'm not sure whether your assumption that people's concern will translate into even the slightest bit of action is sound.
This is what I meant further up thread when I mentioned segmenting the market. I don't believe everyone will move to a new solution but I do believe that some will. I also believe that the first users will be the technical ones who already understand the trade-offs and that there's a continuum of inconvenience that they'd be willing to endure (I'm lazy, so I won't tolerate much).
For example, think of the number of people who maintain their own static websites, either on GitHub or EC2 or elsewhere. All those people could just use Tumblr or Wordpress.com, yet they chose not to.
I spent the last 3 years under this hypothesis [1][2][3]. In those 3 years I've come to realize that users don't actually care that much. Sure a subset of users care but the majority don't.
I think the "personal cloud" that makes sense both architecturally and in the minds of consumers looks very different from what we as developers understand it to be. When we took our software and began prototyping it on hardware devices (like NASes) and showed it to people their eyes lit up. They understood it. Finally.
This has a simple value proposition: Apple charges $100 for incremental 16/32/64 GB mobile storage. Users can buy 2TB worth of wifi storage for $200 and replacement firmware+source (http://www.hackseagatesatellite.com/) for $35.
I know Synology runs Linux so it should be straightforward to set up but we haven't done it yet.
This is the first time I've heard of this particular Seagate drive but the notion of having a drive you can connect to over wifi and which also fits in your pocket or bag is really interesting.
I do, but, granted, I am a typical HN reader, not a typical user. I want my data either to be encrypted by the key that only I have, before being sent to the online storage service, and/or being sent to the computer that I physically control (NAS at my home).
The reasoning that users do not want to maintain that infrastructure is a very valid one. However, it is like having a car vs. using public transportation. Most car owners do not maintain their car themselves, but at least you can remove your personal items from the trunk before you drive it to the mechanic.
Now the question is does this analogy extends to the advantages of using your own car vs public transportation. My guess is that need for privacy could be one big driver, but the company that can make home computing infrastructure an item of conspicuous consumption will be a big winner. Think Apple or Sonos, but in terms of what comes next (Augmented Reality, Voice HCI etc).
The problem isn't that users don't care, but that there's no revenue model. As a result none of these decentralized networking or platform companies can get funding.
Without funding, they don't have the resources to build truly compelling products. You can prototype something in your spare time, but to polish something to be truly competitive outside enthusiast circles you need to dedicate serious time to it.
Even if users did care there is no mechanism for them to pay for it, so it doesn't happen.
The big thing is each individual user would then be responsible for maintaining their own individual cloud. Otherwise, they'll just outsource that out and we're no better than before.
You probably don't. You also don't get grandma to run her own power plant, telephone switching network, etc. As long as her "stuff" is portable, it really doesn't matter who provides the service
If her "stuff" is that portable and the service provider has at least some kind of read access, how would one prevent the service provider from simply keeping a mirror of Grandma's data on their own servers?
> A different approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application.
Isn't that what App.net was supposed to be? By most standards that project was a huge failure. It's hard to proposition a user to start managing their own silo.
I'm really intrigued by this model of internet infrastructure. Would you mind helping me understand the relevant difference between what you, Sandstorm[1], and OpenPDS[2] are doing?
OpenPDS = US gov-defined policy (NSTIC) for access to personal data
Sandstorm = packaging & separation of web apps from policy-managed data
Nymote & Mirage = UK systems research (OS, storage) on strong enforcement of claims similar but not limited to those being made in the above two projects
Based on a brief skim of the Mirage docs, it looks like a really big difference between Mirage and Sandstorm is that Mirage wants apps to be written in OCaml based on a custom API whereas Sandstorm builds of Linux. Any server app that runs on Linux should be easy to port into Sandstorm, but it seems like a lot of things will need to be written from scratch on Mirage.
In fact it appears Mirage intends to replace the kernel entirely. I can certainly see the appeal of dumping legacy baggage, but it's going to take quite a bit of work to reach the point of practicality.
You can look through the ACM article to get a good overview [1]. Yes, it's taken a lot of work to get to this point and there's more to go. In order to enable a fully decentralised system, we need to understand and tackle the basic problems that all developers face, which is more involved than just Mirage (as described in my comment above).
All the code is also fully open source under a permissive license [2].
There was an MIT startup, Permabit, that tried to make a cloud based storage system which backed your own local disk. One idea was that your local storage is faster so you should use it for local things, and offline computation, but that if a shadow copy was being backed to the cloud in encrypted fashion. If your local storage fails, you just replace it, and the data dribbles back in from the cloud to your new local device.
They also had some hack whereby large common files (like app executables) where compressed using their hash code or something, but that is not the important idea in my opinion.
I think having that transparent local/cloud duality is one piece of the puzzle for a service which is both fast and local and also cloud based. You shouldn't have to care where your data lives. The other pieces of the puzzle are how to grant fine grained shared access without confusing users and introducing security holes.
I was actually thinking something like this for dropbox would be a no brainer.. like a cloudmail client for all your accounts, or an igoogle/ireader service that's hosted and uses your dropbox for its' data.
This sounds quite similar to what the (now more-or-less defunct) Singly was trying to accomplish. Having talked to a former employee about it at some length, the TL;DR is that 1) the technology and ideas are cool and 2) uptake was disappointing. That said, execution matters more than ideas when it comes to success.
I'm definitely finding it interesting to see Amazon releasing more and more offerings to make life as a developer easier. Will be interesting to see whether they can leverage some of these services to make developing for the Fire phone extra appealing.
Is there anything like this in the enterprise space? Many of our customers want authentication against their internal Active Directory or other single sign-on solution. It would be nice if there was a company that exported an api, and did all the work of connecting up the various types of auth, so that we can focus on our own product rather than redo integration work that has probably been done by many other companies already, probably even for the customer's other services.
Thanks for the pointer. This particular service is useful to know about, but a bit more mobile focused than is appropriate for our current product. Noted for future reference tho!
There's a pretty big market for this/enterprise level identity management in general. At least in Germany, companies usually prefer inhous/more control over hosted solutions though (especially in the light of the last 1+ year). SAP IdM is the big dog here (it's somewhat messy though since SAP bought the technology and it shows that it's somewhat of a hodgepodge imo) and the only one I've worked with. I'm sure all the big ERP companies provide this in some way.
Not really the API you're looking for but I thought it was worth mentioning.
Am I missing something or could this be useful for non-mobile applications as well like HTML/JS apps if they provided an API outside of the iOS/Android SDKs?
165 comments
[ 6.8 ms ] story [ 238 ms ] threadAncillary factors:
1. The title is clickworthy. While link-baity titles do work well on HN (compared to very vague titles)...this title sticks out because it is two words: One of them is "Amazon", which has very immediate connotations and brand recognition. The second is a proper noun. Those of us who haven't heard of "Cognito" are likely to think, immediately, "Whoa, new Amazon product, must click-through to see how it will impact my work". That it is two words also makes it visually distinctive regardless of what those words even are.
2. It's 11AM, which means almost every developer on the East Coast is awake, and West Coast developers, particularly the ones responsible enough to reserve HN viewing for before-work hours, are getting their coffee and checking HN. So, lots of potential upvoters.
I also wonder how they "manage[s] the complexity of conflict resolution" without manual intervention.
That's why I always advocate for building your own backend, if you have a long-run plan and/or expect to generate a lot of data for the app.
See you at the race to the bottom!
Besides, what makes you think you would be prevented from moving all this to another provider? From the FAQ (https://aws.amazon.com/cognito/faqs/), it seems that Cognito data is stored in a straight-forward Key-Value scheme. That's about as easy as it gets when it comes to data structure.
Moreover, the actual service itself would almost certainly be implemented differently with another provider. Why should Amazon, or anybody else who offers similar "menu of services" cloud computing, make it easy for you to do so (they make money off of you not moving)?
This is one of our core values and while we get odd looks from investors or MBA-types we know that it's the right move to make in the long run. This is the way things are moving and I hope that more companies will go this route.
Sure, "barriers to exit become barriers to entry" is applicable in some contexts, and for some part of a potential market it will be the barrier that keeps those customers away. I wouldn't dispute that. I would dispute that "barriers to exit become barriers to entry" be treated as a universal maxim applicable along all businesses and products. It's just as much of a strategy mistake to make as assuming the barrier can't exist in the first place. It ought to be considered; not treated as a key issue until or unless the data suggest it should be.
That's a pretty big assumption to make. People, even technical people, make assumptions all the time that they'll be able to get their data out of a system and then aren't able to at a future date.
Any other OSS out there for this?
Any feedback on it?
The big difference I am seeing here is Amazon Cognito doesn't yet have a story around syncing documents between multiple users/roles. Sync gateway has the notion of a sync function that providers developers with a hook to customize server-side syncing in a deeper way.
I'll be interested to see how Amazon developers the multi-user/role sync capabilities and integrates it with their native identity management solution. Could be very compelling down the road..
What would be really impressive is if there was an option to client-side encrypt the data before sending it to AWS. Of course, that would mean you'd have to move your syncing logic to the client-side, too, but having a good client-side encryption option would be a real differentiator.
So if it works like that, you absolutely can encrypt your payload client-side, and use Cognito as a mere transport.
Exactly. I don't know why people are so attracted to locked in solutions like Parse or now this when Couch has already been doing it for a long time, and there are mature third-party providers (Cloudant, Iriscouch, etc) that can help you get off the ground just as easily without ever being locked in with them.
"Parse"? OK, sure...but that's not any more functionally descriptive than "Cognito"...and Parse is a more narrowly focused company with a primary service...In the case of "Cognito" and "Zocalo"..., Amazon has to come up with names that are distinctive because they provide so many core services...so many that even me, an engineer, get them confused. Imagine the difficulty of a fly-by marketing person trying to differentiate between similar sounding names.
Furthermore, these names are distinctive industry wide...presumably, if Zocalo becomes big, you will no longer have to say "AWS Zocalo", but just, "Zocalo"...just like you do with "S3" and "EC2".
To drive home the situation that Amazon is in, let's consider another great product name: Dropbox. Unlike Parse, it is very easy to relate to the core service. But consider the phrase, "Amazon Dropbox"..."Dropbox" is unique compared to the currently named AWS services...but it has a huge collision with Amazon's core service, which is, stocking and shipping products. If Dropbox.com didn't exist, and Amazon used it instead of "Zocalo"...there is a considerable risk that people associate "dropbox" with some kind of packaging thing.
The http://en.wikipedia.org/wiki/Z%C3%B3calo is the city center (of Mexico City's historical district), a simple and untaken metaphor for a service that "...provides users with a central location for both the documents and files they are...".
It's a shame Amazon can't afford to hire a marketing genius like you, but I think they're doing the best they can with the idiots they have to work with.
As far as I can tell, there is no guarantee anywhere that they'll maintain any service for any length of time.
If that happens, it could waste weeks or months of engineering time trying to migrate to something else, or I'll just have to shut down my own services if I decide it's not worth migrating.
Amazon doesn't just create services, throws them over the wall and see if they stick. Rather, they almost always RESPOND to customer requests. Dr. Werner Vogels, AWS's CTO makes a point about that approach at about any keynote he gives. Similarly, they try to also avoid feature creep by only providing features which customers really ask for.
I think I remember amazon people saying they are actually dogfooding this stuff so it should stay available for a long time anyway, rather than being a random Google Reader.
Retiring any of the services that have been launched would cause not inconsiderable pain to other teams :)
I think there's a notion of persistence in AWS' leadership that propagates through the teams. AWS is willing to fight through the early painful years for products to see them arrive at the stage that EC2/S3 have attained.
Sitting here in Seattle I see Amazon hoovering up all the local talent they can hire. And I see them putting up new tower after new tower to house them. I don't worry about them stretching themselves too thin as far as manpower goes.
And they don't have a history of closing down AWS products – for instance the first, (and not a runaway hit) Mechanical Turk is still with us.
http://www.kinvey.com/ https://backendless.com/ https://www.parse.com/ http://www.baasbox.com/
A different (and more disruptive?) approach to this would be to put more control of these things into the hands of end-users such that they provide the 'backend' into which you (the developer) load your application. i.e something like the app-store model but it connects to the user's 'personal cloud' (or the desktops of old - if you prefer). Such a system needs to be FOSS at its core but with a way for developers to get paid for providing value to end-users (who themselves get control of their data/networks).
Of course, this won't happen overnight, but the alternative is that everyone ends up using proprietary silos, with huge lock-in and innovation suffers as the tech giants get distracted with lawsuits.
I'm working with others on the distributed systems infrastructure we need to make this possible [1], so that we can get to a place where everyone can have their own piece of the cloud. In fact, one of the major components of this is Mirage, which has been discussed quite a bit on HN recently [2].
[1] http://nymote.org/blog/2013/introducing-nymote/
[2] https://news.ycombinator.com/item?id=8020125 and https://news.ycombinator.com/item?id=8005130 and https://news.ycombinator.com/item?id=7726748
It also adds an extra step in the data storage process within the app, which increases friction and seems like it would be hard to get developers to adopt. All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems.
We don't need most users in the beginning. Just those that are technically savvy enough to understand the landscape and the way the world is headed. From there we can develop more stuff and eventually get it into a state ready for the masses. For example, I can run my personal website as a unikernel [1], which is not suitable for mom-and-pop but great for anyone currently using any static site generators. As more libraries get released I can build on that an eventually run more of my infrastructure this way (my prototypical examples are mail, contacts and calendars).
In terms of the friction for developers, we simply have to make the libraries and tooling fun and easy to use. With well-designed systems it's not necessarily any more friction that what developers have to do now (i.e build their own backend).
> "All that said, I'd love to be proven wrong and see an implementation that is killer enough to solve these problems."
We're working on it.
[1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li...
Futurists have a tendency to imagine a world of changed human behavior and it's compelling to do so. The reality is that the future rarely arrives as sweeping change, but rather as metaphor and specialization.
Whereas you can imagine others adopting new patterns of behavior because you understand the underlying reasons why such behavior is reasonable, the metaphor through which you explain this change is not readily understood. Why, as a User do I want this? If the answer is control and privacy, you might be barking up the wrong tree (time and again we've shown that those are not things consumers want or are willing to pay for).
If you want to drive dynamic change in the world, you have to change the underlying structure of complicated systems while steadfastly avoiding changes in user behavior. It turns out this is quite hard.
I applaud your efforts but encourage you to avoid the rabbit hole of endless specialization and to improve the marketing metaphor/rhetoric.
"Bitcoin, billions of phones, and NSA eavesdropping are among trends that may eventually result in cloud successor: data-local computing"
https://twitter.com/balajis/status/487403585445978112
CITATION please. I think we have never seen such things. We've not had studies that control for all other factors and then conclude that users don't want control and privacy. I think your claim is flat out wrong. You're taking the fact that most users won't sacrifice by using a lower-quality, obscure thing for privacy and control, and leaping to the baseless conclusion that nobody cares about these things. I think you're completely wrong.
I would however, anecdotally highlight the progression of social services from places of privacy to places of publicity. One could argue that sites like "The Well" were the original social networks but they did not grow (whether by choice is of course a matter for debate). Over time, each successive social startup defaulted to a more open stance and scaled an order of magnitude in users with each step. The trend suggests that all things being equal, users will say they want privacy but want the features publishing and syndication provides (voting purely with their registrations and advertising value).
I agree that privacy has value, but I would argue that the anecdotal evidence suggests that users want less private services (and yes I'm aware that correlation is not causation, but I think this observation is apt). This is not law or dogma but rather a thought and I'm sorry if it was communicated poorly previously.
I don't think that's a clear-cut anecdote. Facebook does actually feel very private, because it's isolated from Google and outside users. Look at the number of female profiles where everything but the name is hidden from non-friends. I don't remember MySpace or blog users ever being this paranoid. And then, anecdotally, many people around me have started to live in WhatsApp/LINE gossip groups, which is something that could easily be decentralised.
Twitter was unambiguously non-private from the start so there were no broken expectations.
I'd be wary of assuming from this data point that people are willing to be even slightly inconvenienced for it. Just take a look at Facebook: the entire history of the company has been one of disrespecting and subsequently pissing off users, and yet out of the huge amount of people I know who have repeatedly expressed anger at how Facebook treats their users, I know exactly one person who stopped using it or even reduced their usage.
I'm not sure whether your assumption that people's concern will translate into even the slightest bit of action is sound.
For example, think of the number of people who maintain their own static websites, either on GitHub or EC2 or elsewhere. All those people could just use Tumblr or Wordpress.com, yet they chose not to.
I think the "personal cloud" that makes sense both architecturally and in the minds of consumers looks very different from what we as developers understand it to be. When we took our software and began prototyping it on hardware devices (like NASes) and showed it to people their eyes lit up. They understood it. Finally.
[1] https://www.kickstarter.com/projects/jmathai/openphoto-a-pho...
[2] http://blog.trovebox.com/post/36911180540/all-cloud-storage-...
[3] https://github.com/photo
This has a simple value proposition: Apple charges $100 for incremental 16/32/64 GB mobile storage. Users can buy 2TB worth of wifi storage for $200 and replacement firmware+source (http://www.hackseagatesatellite.com/) for $35.
Most of these NAS devices run some version of Linux which makes it trivial to set up our software on them. Some are easier than others though.
Beyond that I can't say much regarding what we've been tinkering with internally but it's pretty damn cool :).
This is the first time I've heard of this particular Seagate drive but the notion of having a drive you can connect to over wifi and which also fits in your pocket or bag is really interesting.
The reasoning that users do not want to maintain that infrastructure is a very valid one. However, it is like having a car vs. using public transportation. Most car owners do not maintain their car themselves, but at least you can remove your personal items from the trunk before you drive it to the mechanic.
Now the question is does this analogy extends to the advantages of using your own car vs public transportation. My guess is that need for privacy could be one big driver, but the company that can make home computing infrastructure an item of conspicuous consumption will be a big winner. Think Apple or Sonos, but in terms of what comes next (Augmented Reality, Voice HCI etc).
Without funding, they don't have the resources to build truly compelling products. You can prototype something in your spare time, but to polish something to be truly competitive outside enthusiast circles you need to dedicate serious time to it.
Even if users did care there is no mechanism for them to pay for it, so it doesn't happen.
How do you get grandma on her own personal cloud?
User Dependency Injection!
[1] https://sandstorm.io/ [2] http://openpds.media.mit.edu/
Sandstorm = packaging & separation of web apps from policy-managed data
Nymote & Mirage = UK systems research (OS, storage) on strong enforcement of claims similar but not limited to those being made in the above two projects
Based on a brief skim of the Mirage docs, it looks like a really big difference between Mirage and Sandstorm is that Mirage wants apps to be written in OCaml based on a custom API whereas Sandstorm builds of Linux. Any server app that runs on Linux should be easy to port into Sandstorm, but it seems like a lot of things will need to be written from scratch on Mirage.
In fact it appears Mirage intends to replace the kernel entirely. I can certainly see the appeal of dumping legacy baggage, but it's going to take quite a bit of work to reach the point of practicality.
All the code is also fully open source under a permissive license [2].
[1] http://queue.acm.org/detail.cfm?id=2566628
[2] https://github.com/mirage/
They also had some hack whereby large common files (like app executables) where compressed using their hash code or something, but that is not the important idea in my opinion.
I think having that transparent local/cloud duality is one piece of the puzzle for a service which is both fast and local and also cloud based. You shouldn't have to care where your data lives. The other pieces of the puzzle are how to grant fine grained shared access without confusing users and introducing security holes.
Not really the API you're looking for but I thought it was worth mentioning.