The cloud is the most perplexing place on the internet. The cost savings are so dramatic, it's a waste of time to do ROI. Applications up and running in a week that used to take 6 months.
The Cloud seems too good to be true, but it's not. It's real and it is better.
I remember reading the Steve Jobs interview by Playboy and he said he was on the edge of the petroleum revolution, starting the PC revolution. Twenty years later, the internet revolution and now here is cloud computing that is going to make the internet just blow up.
Web application development is made orders of magnitude easier with cloud computing.
The cloud is an inevitability. FUD is holding us back. The Cloud is the way of the future.
The cloud is indeed real - but not necessarily better. There are real risks associated with it, not just FUD. Outsourcing every aspect of your business can potentially increase your focus on your "core competencies", but you run the very real risk of having a provider close their doors, forcing you to re-evaluate at a moments notice. The "cloud" isn't some magical thing where hardware failures don't happen, it's a real infrastructure managed by someone else. When you hide everything behind the term "cloud" (as in: But we don't have to evaluate their architecture - it's in the cloud!), you risk relying a little too much on someone else's ability to manage a datacenter/network/infrastructure.
Sometimes this is a good decision, sometimes this is not. IT people hate the lack of clarity, because when something fails and they can't explain why or what they did to prevent it (nothing! it's the cloud!), it's their ass. And that's a very real risk.
This is why I like EC2, as opposed to something like Azure. If Azure goes away, you're out of luck. If EC2 goes away... bring up your own Linux, Windows, or OpenSolaris instances, and run your apps there.
Granted, that's not true of S3, SQS, or SDB... but seriously, how hard is it to program against an abstraction layer? Easy enough to rewrite your own "store_photo" function to use a replacement for S3.
"but you run the very real risk of having a provider close their doors, forcing you to re-evaluate at a moments notice."
The solution, of course, is to build your own cloud atop the clouds from a few different providers so that you're never reliant on any one of them specifically. ;P
The only real advantage that I can see is that cloud computing makes it possible to scale quickly on demand without the associated capital expenditure for 'stuff', you just pay for usage.
That's got a downside too, at the moment you reach a steady state it is probably cheaper again to buy the actual hardware and colocate instead of using cloud services (which after all exist to make money for their owners).
If you look at the Platform as a Service space, a lot of development work is done automatically. For example, upload a spreadsheet and the data persistence layer is inferred from the data itself, no need to manually create the tables, the pojos, etc. Just drag and drop.
On top of that, all the security is built in. Email notifications, built in. All this in addition to the scalability of the cloud infrastructure. No need to build in clustering, failover, etc -- all handled automagically.
I've noticed a lot of people who talk about "the cloud" really have no idea what they are talking about. I've been trying to move parts of my agencies' infrastructure to the Cloud for some time so let me dispel a few myths.
(For the record, contrary to this article's title the bottleneck for me has been management all the way)
Cloud Computing is not easy: The person below who claims you can get a cloud application up in a week has clearly never tried to put a mission critical cloud application up at all. Managing virtual servers can be as hard if not harder than actual servers in many cases because you have to automatically track a bunch of items that would normally be static (is the server active, does it have an ip, if so what is that ip, if not you need to assign an ip and then automatically track that ip, etc...)
The Cloud is not necessarily standardized: The above example looks at environments such as Amazon which are hard to maintain because of all the virtual up and down happening. The other end of the spectrum would be services like Microsoft Azure. Azure is easy to use but it's a platform that has to be written to. You write an application in Azure and you have to rewrite significant parts of it to take it somewhere else.
The Cloud is not always cost efficient: The Cloud is only cost efficient up to the point where you can afford to fill a server person's time. Once that happens having a full time server person becomes more cost effective.
Management IS NOT going to be the one pushing this: The article claims IT should adopt Cloud Computing or Management will. Unlikely. In my experience Cloud computing makes people very, very, very nervous. Let's put all our data on someone else's server does not go over well on its own. Add that to the fact that management types have been trained by the world to think "outsourcing = more expensive" (which is true in non-tech areas) and you have a very steep uphill battle.
Sorry for the long comment but a lot of this Cloud hype is getting out of hand. I'm in favor of Cloud computing but we have to remember it's a tool to solve some specific problems. NOT a tool to solve every problem.
> The Cloud is not always cost efficient: The Cloud is only cost efficient up to the point where you can afford to fill a server person's time. Once that happens having a full time server person becomes more cost effective.
Actually, it's a server team's time. Server persons sleep, go on vacation, and so on.
As the tools get better, that "amount to fill" keeps getting larger and larger.
And, what are the odds that your team is as good as Amazon's?
That said, your team cares about your stuff more than Amazon's team does.
Fair enough on the team concept. Though I think you could get by with one server person when you have a technical staff that could fill in during vacations et al. But you're right, it certainly requires more than one.
> And, what are the odds that your team is as good as Amazon's?
S3 was down unscheduled more in 2008 than we were. (I only know about their major published widely outages, and know about all of ours that lasted more than 15 seconds.)
I realize that's not a perfect indicator, maybe it was an especially bad year for S3/good year for us, nor is their app apples-to-apples against ours, but one of the primary reasons we CAN'T afford to go even to S3 for "cheap" file storage is that it represents a serial reliability problem: both "them" and "us" have to be up for us to make money.
When we price it out, the 3-year fully burdened cost of our in-house storage vs S3 pricing is about a wash. (It's 5-10% in favor of in-house, but I'm sure we're not perfectly fairly accounting for all the tiny costs [someone has to file purchase orders, do shipping/receiving, calculate depreciation and file tickets to let EMC techs into our DCs to replace failed drives, etc, etc] in our model, but that is after accounting for the IT team dedicated to storage and the on-call rotation portion allocated to storage.)
If we were a lot smaller, it would be a no-brainer to try to use cloud services where it made sense, but as Tom said and anamax clarified, once you're at scale and have access to good IT staff in-house, the cloud offerings are not particularly compelling cost-savings measures.
(Edit: I should perhaps clarify. We do use S3 now, in production, for one of our offerings that is not core to our business, and one for which an arbitrary 8 hours of downtime would not present a problem. We also built two years ago the ability to overflow our bulk/cold image uploads to S3 if we run out of space on our in-house storage. Here again, if we suddenly find ourselves having substantial unforecast demand, it's better to go out to a third party than to turn customers away because the disk is full.
Excellent points! One possible development is 'self hosted clouds', using the technlogy that goes in to cloud architecture but run it on your own in-house or colocated hardware.
Most privacy policies would be hard to live up to anyway storing critical data in the 'cloud', so a large portion of business-it will always be hosted locally or on hardware controlled by the corporation whose responsibility it is to safeguard the data.
The more I read about The Cloud(tm), the more I'm convinced it's just Far East outsourcing all over again. I worked with an Indian outsourcing firm as one of the few American programmers, and I can tell you Cloud computing is going to hit the same wall: communication. No matter how much money you save upfront or ease of prediction that the contract could give you, you WILL lose time and effectiveness due to the increased communication the inserted organizational layer will introduce.
At least from my perspective as an engineer, AWS in particular removes huge layers of nonsense, and there are no increased layers of communication.
It's cheaper than buying machines (by a huge amount!), cheaper than keeping them running, and I can make changes on the fly that would otherwise take a week to grind through netops.
If load spikes (as it did recently for one of our customers), the conventional answer is "we can't buy 20 more servers in 2 days"! My response was "why not use EC2?"
For a new project, rather than submitting a PO for $20k+ in hardware, I can say "it'll cost $140 for the first month". Much easier to approve -- I can put that through on expenses!
Finally, if you consider that most large companies outsource datacenter management, whether truly outsourced to a services company, or "internally outsourced" to another division, using a cloud system that at least acknowledges your right to remotely administer your own service is a step forward, not a step back.
I read stuff like this, and the same questions that I've put to Microsoft and Amazon come round again... can data placed in the cloud be kept within a geographic/legal set of borders through policy and how do we approach compliance rules for archiving, freedom of information and data protection?
To these questions, deafening silence or waffle. Never a serious and practical response.
The cloud is immature in terms of it's enterprise applications until these kind of questions are being addressed with multiple answers.
For small business the cloud may be a godsend, but to the enterprise the cloud is something for the small guys to play with that might make them better customers/partners.
The docs for Amazon S3 state that a bucket can either be kept in the US or in the EU. How precise a location are you needing? (They're not going to have multiple data centers per zip code.)
Aside from that, I'm not sure what role you would expect them to take for archiving - isn't it your job to decide what data needs storing, and how to organize it?
21 comments
[ 2.7 ms ] story [ 60.5 ms ] threadThe Cloud seems too good to be true, but it's not. It's real and it is better.
I remember reading the Steve Jobs interview by Playboy and he said he was on the edge of the petroleum revolution, starting the PC revolution. Twenty years later, the internet revolution and now here is cloud computing that is going to make the internet just blow up.
Web application development is made orders of magnitude easier with cloud computing.
The cloud is an inevitability. FUD is holding us back. The Cloud is the way of the future.
Sometimes this is a good decision, sometimes this is not. IT people hate the lack of clarity, because when something fails and they can't explain why or what they did to prevent it (nothing! it's the cloud!), it's their ass. And that's a very real risk.
Granted, that's not true of S3, SQS, or SDB... but seriously, how hard is it to program against an abstraction layer? Easy enough to rewrite your own "store_photo" function to use a replacement for S3.
The solution, of course, is to build your own cloud atop the clouds from a few different providers so that you're never reliant on any one of them specifically. ;P
Drink kool-aid much?
Just curious: what exactly do you mean by this? i.e. - how does cloud computing help with web app development?
That's got a downside too, at the moment you reach a steady state it is probably cheaper again to buy the actual hardware and colocate instead of using cloud services (which after all exist to make money for their owners).
On top of that, all the security is built in. Email notifications, built in. All this in addition to the scalability of the cloud infrastructure. No need to build in clustering, failover, etc -- all handled automagically.
(For the record, contrary to this article's title the bottleneck for me has been management all the way)
Cloud Computing is not easy: The person below who claims you can get a cloud application up in a week has clearly never tried to put a mission critical cloud application up at all. Managing virtual servers can be as hard if not harder than actual servers in many cases because you have to automatically track a bunch of items that would normally be static (is the server active, does it have an ip, if so what is that ip, if not you need to assign an ip and then automatically track that ip, etc...)
The Cloud is not necessarily standardized: The above example looks at environments such as Amazon which are hard to maintain because of all the virtual up and down happening. The other end of the spectrum would be services like Microsoft Azure. Azure is easy to use but it's a platform that has to be written to. You write an application in Azure and you have to rewrite significant parts of it to take it somewhere else.
The Cloud is not always cost efficient: The Cloud is only cost efficient up to the point where you can afford to fill a server person's time. Once that happens having a full time server person becomes more cost effective.
Management IS NOT going to be the one pushing this: The article claims IT should adopt Cloud Computing or Management will. Unlikely. In my experience Cloud computing makes people very, very, very nervous. Let's put all our data on someone else's server does not go over well on its own. Add that to the fact that management types have been trained by the world to think "outsourcing = more expensive" (which is true in non-tech areas) and you have a very steep uphill battle.
Sorry for the long comment but a lot of this Cloud hype is getting out of hand. I'm in favor of Cloud computing but we have to remember it's a tool to solve some specific problems. NOT a tool to solve every problem.
Actually, it's a server team's time. Server persons sleep, go on vacation, and so on.
As the tools get better, that "amount to fill" keeps getting larger and larger.
And, what are the odds that your team is as good as Amazon's?
That said, your team cares about your stuff more than Amazon's team does.
S3 was down unscheduled more in 2008 than we were. (I only know about their major published widely outages, and know about all of ours that lasted more than 15 seconds.)
I realize that's not a perfect indicator, maybe it was an especially bad year for S3/good year for us, nor is their app apples-to-apples against ours, but one of the primary reasons we CAN'T afford to go even to S3 for "cheap" file storage is that it represents a serial reliability problem: both "them" and "us" have to be up for us to make money.
When we price it out, the 3-year fully burdened cost of our in-house storage vs S3 pricing is about a wash. (It's 5-10% in favor of in-house, but I'm sure we're not perfectly fairly accounting for all the tiny costs [someone has to file purchase orders, do shipping/receiving, calculate depreciation and file tickets to let EMC techs into our DCs to replace failed drives, etc, etc] in our model, but that is after accounting for the IT team dedicated to storage and the on-call rotation portion allocated to storage.)
If we were a lot smaller, it would be a no-brainer to try to use cloud services where it made sense, but as Tom said and anamax clarified, once you're at scale and have access to good IT staff in-house, the cloud offerings are not particularly compelling cost-savings measures.
(Edit: I should perhaps clarify. We do use S3 now, in production, for one of our offerings that is not core to our business, and one for which an arbitrary 8 hours of downtime would not present a problem. We also built two years ago the ability to overflow our bulk/cold image uploads to S3 if we run out of space on our in-house storage. Here again, if we suddenly find ourselves having substantial unforecast demand, it's better to go out to a third party than to turn customers away because the disk is full.
99.90% is >>> 0.00%. :) )
Most privacy policies would be hard to live up to anyway storing critical data in the 'cloud', so a large portion of business-it will always be hosted locally or on hardware controlled by the corporation whose responsibility it is to safeguard the data.
It's cheaper than buying machines (by a huge amount!), cheaper than keeping them running, and I can make changes on the fly that would otherwise take a week to grind through netops.
If load spikes (as it did recently for one of our customers), the conventional answer is "we can't buy 20 more servers in 2 days"! My response was "why not use EC2?"
For a new project, rather than submitting a PO for $20k+ in hardware, I can say "it'll cost $140 for the first month". Much easier to approve -- I can put that through on expenses!
Finally, if you consider that most large companies outsource datacenter management, whether truly outsourced to a services company, or "internally outsourced" to another division, using a cloud system that at least acknowledges your right to remotely administer your own service is a step forward, not a step back.
To these questions, deafening silence or waffle. Never a serious and practical response.
The cloud is immature in terms of it's enterprise applications until these kind of questions are being addressed with multiple answers.
For small business the cloud may be a godsend, but to the enterprise the cloud is something for the small guys to play with that might make them better customers/partners.
Aside from that, I'm not sure what role you would expect them to take for archiving - isn't it your job to decide what data needs storing, and how to organize it?