> Because looking elsewhere in the PaaS space, instance-based pricing — or something similar — is still the norm.
That's not the point. The point is that AppEngine boosted their pricing from between 5x-25x depending on the project. People aren't mad that AppEngine is using instance-based pricing, they're mad that they bought into a proprietary system that has a lot of lock-in and then the system increased their pricing by a ludicrous amount.
What does it matter if they boosted by 10% or 2500%? If they boosted by 2500%, could it be that the previous price was really low?
What if they weren't charging any money before, and then started charging? It would be an ∞% raise. Is it bad that they offered their service for free for a while?
PS: feel free to point out what's wrong with my questions.
We can answer this by comparing prices to other providers. For example:
Amazon (a standardised but not a cheap provider) used to charge $0.15/GB bandwidth out but now charges $0.12/GB (and does not charge for bandwidth in): http://goo.gl/n1ZxZ And this is tier one; it goes down to $0.05/GB or less for bulk bandwidth.
Appengine used to charge $0.12/GB out, but under the new pricing will charge $0.15/GB out (and still charges $0.10/GB in): http://goo.gl/QgA3n
Does this suggest that Google was running a charity, that it is merely covering it's costs, and that Google's world-renowned infrastructure costs more than Amazon's?
Thanks for the answer. I think that if it turns out to be too expensive, people will find alternatives. Maybe people who already use it and think that the development cost in order to migrate is too high will suck it up, but new customers will just go to other providers.
True. I could have lived with instance based pricing if it was actually reasonable. But its a real shame, because resource-based pricing really attracted me to the platform. Now they've just thrown in the towel. I'd rather they added metrics (and costs) for the missing resources (which appears to be RAM) than just give up and say "instances". Now we have to guess which resource we are over-using to cause another instance to start up.
What's worse, not being able to easily migrate your GAE specific code to another provider or suddenly having to pay through your nose?
No one is trying undermine the fact that Google has to turn a profit, but they should've have either been frank about the pricing from the very beginning so that people would have made an informed choice, or retooled the platform so that it didn't require all that custom plumbing and would not have done all that optimization to use the least resources thats now useless if you want to migrate elsewhere.
This can be called a classic bait and switch if it was done intentionally. At the very least Google should offer grandfathered pricing for people who already depend on GAE.
Python, Java and Go are all "open" languages and don't limit you to Appengine, and Google allows you to download your data without restriction. The only lock-in is the APIs, and even there the datastore API is the only one that isn't built on a very standard interface. Given the datastore is distributed and therefore can't expose an exact replica of the MySQL API, how could Google lock you in LESS? Oh, and lets not forget that you can run your code UNCHANGED on Appscale or Typhoon AE on any IAAS or VPS you like.
This article is right - Google needed to make Appengine pay for itself. And when you factor in how much it costs to provide a scalable service from redundant servers, Appengine is still good value. For example, most of the grumblers are muttering about migrating to AWS, but when they realise that they need at least two instances in diverse locations; or make that two instances in each location if their service might become too popular for one server to handle; and ELB to route requests to the servers; and how tricky it is to keep their data consistent on all those servers; and what a hassle a serious backup regime is; and how much time it takes to manage the whole thing... I'm picking they'll suddenly go rather quiet.
Or maybe they're happy to run their site on a single AWS instance. In which case, they should do just that - Appengine is not the right tool for them anyway.
I run several profitable apps on Appengine, and until now paid cents a month to Google. I've always known that isn't sustainable for them, and that they'd have to get serious about pricing someday. I for one am happy to pay a couple of orders of magnitude more for the certainty that my platform will persist.
Actually Goolge and the GAE team are very much about app/data portability. They even sponsored our open source App Engine project AppScale (http://appscale.cs.ucsb.edu/sponsors.html) which lets users run their app on their own hardware or on EC2.
I can't help but think their pricing policies will ultimately undermine the platform. People that haven't already started developing on GAE are surely watching this, and probably glad they didn't start. Nobody likes bait and switch.
How will Google encourage a new developers to start working with GAE when platform-neutral providers (AWS, linode) are getting cheaper and better all the time?
I think the real competitors are Heroku et al. Places where you don't have to think about file systems and upgrading and security and deployment as much as with e.g. AWS.
Oh, it's far worse than that. Until Google---a company who's focus is far from this product---can demonstrate years of responsible management of these sorts of products they're going to be radioactive for anyone who has other options.
Among the core changes is a switch from true resources-based pricing to instance-based pricing, which means that applications written to be as efficient as possible to consume minimal resources are now relatively meaningless
The author is completely clueless here.
The GAE change is that "true resources-based pricing" switched from an overabundant, almost irrelevant resource (CPU time) to the actual resource in short supply - RAM.
For web servers, it doesn't matter how much CPU time you use, it only matters how much RAM you occupy. Linode, Rackspace, and most other VPS providers* charge this way. Most server CPUs are sitting around blocked on I/O - especially ones running single-threaded synchronous web servers (I'm looking at you, Rails).
The new GAE pricing is still based on true resources - the ones that matter, not the ones that are irrelevant.
* AWS is fairly unique with a hybrid billing approach.
If RAM is the one GAE charging for, then they are massively overcharging the apps. Spawning a new process off a parent process takes very little memory. Most of the memory of the child processes are shared with the parent process. That's how Python/Ruby/Perl/whatnot web apps work. They spawn a new process to handle each request because it's really cheap on CPU and RAM to do so.
The first GAE instance process on a server would take up fair amount of RAM. The next one takes very little. Yet Google charges for the second one fully.
This hasn't been true for a very long time. Back in the olde days of C programs with large code/data segments and tiny heaps it would be true. But things have changed:
* In modern interpreted/JITed languages, the "code" is actually data that gets malloced. These segments are not shared.
* Modern interpreted/JITed applications eat VAST quantities of heap. They constantly produce garbage, most of which (hopefully) gets culled by the garbage collector.
On GAE, even small Python instances typically grow to 30-40MB. Java instances typically hover around 90MB. The actual per-instance reservation must be significantly larger because boxes can never be allowed to go into swap. Figure that Google allocates ~128MB for every single active instance of every single application. It's a lot.
The GAE change is that "true resources-based pricing" switched from an overabundant, almost irrelevant resource (CPU time) to the actual resource in short supply - RAM.
Except that thats just not true! We aren't charged by MB-hour. We are charged by instance hour. Do I pay less if my app is using only a small amount of memory? Not at all.
You may argue that instances are a proxy for ram. Certainly they are. They just aren't a proxy that has useful, meaningful metrics. The rules for when or why the infamous scheduler decides to spin up a new instance is pretty much fucking voodoo right now. Before, I had a metric, and it was directly related to me writing good code. Now, I might as well read tea leaves.
So, specifically, I am not complaining about being billed for RAM. If this is the route they want to go (and I understand why), then I my complaint is precisely that I am not being billed for RAM.
I'm going to speculate that all instances reserve a fixed amount of RAM irrespective of actual heap usage. I can't speak authoritatively about GAE, but this is how I would design the system.
The problem is that while your instance may only use 30MB now, it could grow to 100MB in an instant without warning. Tipping a box into swap would effectively crash all instances present. Oversubscribing RAM is a dangerous game, and I suspect Google doesn't try to play it - they have enough trouble debugging performance issues as it is.
So basically, an instance represents a fixed about of RAM. Instance-hours and RAM-hours are the same.
I'd rather have Google say "in order for GAE to stay alive, we've had to raise pricing on our current metrics to X", I can understand that, I can justify that, I can compare it and decide how to deal with it/optimize for it, but instead we got, "based on user popular feedback we're radically changing the way we measure usage, then charging some significant multiplier to whatever everybody is already paying so that it no longer makes sense to operate on our platform and you can't even really compare it in such a way as to make sense of the change, then we'll systematically not address anybody who asks us about this in forums or via other communications but we're going to do all this in the most frustratingly passive aggressive way possible" [1][2] Can anybody really believe that popular feedback to Google was "we're not being gouged nearly enough, Google, charge us 2000% more!"
"Over the last three years, we’ve collected great feedback from our customers and now believe that the biggest thing we can do to help our customers is to graduate App Engine from preview status. "
"Adding business features will help App Engine meet a broader set of needs and the new, more transparent pricing model will help customers better align their App Engine investment with their business goals."
"App Engine graduating from Preview later this year!
Based on this feedback we’ve decided to make some fairly large changes:"
"In order to become an official Google product we must restructure our pricing model to obtain sustainable revenue. Based on customer feedback this means focusing on usage-based pricing and placing per-user, per-app pricing on hold until further notice."
Google screwed up by trying to charge for CPU time in the first place. CPU time is an overabundant resource in the cluster, RAM is precious. This change was inevitable.
Look at Linode, Rackspacecloud, even EC2. They all charge primarily by the megabyte. I don't like it either, but it's the unfortunate economic reality.
Google doesn't seem to think that RAM is the limiting resource:
"Most frontend instances don't come near their RAM limit, and most of their RAM consumption is on shared data like packages." -- nickjohnson http://goo.gl/SOeSF
Actually, if you look at the context, this reinforces my point that RAM is the limiting factor. Nick mentioned this to reassure some other user worried about Google lowering the RAM limit. As long as Google is being conservative WRT oversubscribing RAM, the instance RAM limit is the true measure of the cost of an instance.
It's not about how much you use, it's about how much you could use.
All this "we aren't a non-profit" and "oh poor us we have to make some money" and "its better than if we turn it off", just deliberately misses the point:
It was news to us that they are losing so much money.
It was news because they spent the last three years telling us to jump through insane hoops, locking ourselves thoroughly into a for-profit company, on the sole belief that doing so would allow efficiencies and scale not to be found elsewhere.
The complaint is not "you shouldn't make money off us". The complaint is what the fuck was all that bullshit we've been doing for three fucking years if it didn't make any fucking difference - as demonstrated by your "viable billing prices"?
And what does it say about their competence? After three years, they only discovered in May that they were measuring the wrong thing? Google? The analysis company. Only in May someone decided to look at how all the "brilliant GAE Way" actually impacted performance and resources? Nobody thought, "this cpu billing is all very well, but we are actually running 20 bazillion machines to serve 5 web pages per second - maybe our model might be wrong".
For the tiny guys, the free plan is viable. Perhaps for the giant guys, (I don't know) the plan is viable. For everyone in the middle, it is better to go to EC2. The only people who say they are staying are those who do not have the competence to move to EC2 or Heroku, i.e. non engineers every one of them.
So GAE used to be all "Hey, engineers, come code to our awesome paradigm shifting api, and you can be efficient and scale."
Now GAE is, "Hey, MBAs and Muggles, have your monkeys write generic java/python and we'll make it scale no matter what the cost (to you)!"
gregd, project manager: "We will also introduce a new pricing structure for App Engine based on more transparent usage-based pricing." http://goo.gl/d78vS
jonmac, engineer: "The system is large and complex, its hard to understand and hard to explain." http://goo.gl/6NzF1
Customers were upset about the pricing changes then, but fervor has picked up in intensity lately because the new App Engine model goes into effect later this month.
No, the fervor picked up in intensity because they only now gave us the new metrics, and the price of them, and they are fucking insane.
Well, before google built it into the billing, nobody could really know how many 'instance-hours' they were using, and it turned out to be a lot more than expected.
wow. This is a little shocking; prices just don't go up in the hosting industry. I mean, if you are charging for hardware/bandwidth, just stop lowering your prices and your margin goes up rather quickly.
34 comments
[ 3.1 ms ] story [ 40.1 ms ] threadThat's not the point. The point is that AppEngine boosted their pricing from between 5x-25x depending on the project. People aren't mad that AppEngine is using instance-based pricing, they're mad that they bought into a proprietary system that has a lot of lock-in and then the system increased their pricing by a ludicrous amount.
What if they weren't charging any money before, and then started charging? It would be an ∞% raise. Is it bad that they offered their service for free for a while?
PS: feel free to point out what's wrong with my questions.
We can answer this by comparing prices to other providers. For example:
Amazon (a standardised but not a cheap provider) used to charge $0.15/GB bandwidth out but now charges $0.12/GB (and does not charge for bandwidth in): http://goo.gl/n1ZxZ And this is tier one; it goes down to $0.05/GB or less for bulk bandwidth.
Appengine used to charge $0.12/GB out, but under the new pricing will charge $0.15/GB out (and still charges $0.10/GB in): http://goo.gl/QgA3n
Does this suggest that Google was running a charity, that it is merely covering it's costs, and that Google's world-renowned infrastructure costs more than Amazon's?
No one is trying undermine the fact that Google has to turn a profit, but they should've have either been frank about the pricing from the very beginning so that people would have made an informed choice, or retooled the platform so that it didn't require all that custom plumbing and would not have done all that optimization to use the least resources thats now useless if you want to migrate elsewhere.
This can be called a classic bait and switch if it was done intentionally. At the very least Google should offer grandfathered pricing for people who already depend on GAE.
This article is right - Google needed to make Appengine pay for itself. And when you factor in how much it costs to provide a scalable service from redundant servers, Appengine is still good value. For example, most of the grumblers are muttering about migrating to AWS, but when they realise that they need at least two instances in diverse locations; or make that two instances in each location if their service might become too popular for one server to handle; and ELB to route requests to the servers; and how tricky it is to keep their data consistent on all those servers; and what a hassle a serious backup regime is; and how much time it takes to manage the whole thing... I'm picking they'll suddenly go rather quiet.
Or maybe they're happy to run their site on a single AWS instance. In which case, they should do just that - Appengine is not the right tool for them anyway.
I run several profitable apps on Appengine, and until now paid cents a month to Google. I've always known that isn't sustainable for them, and that they'd have to get serious about pricing someday. I for one am happy to pay a couple of orders of magnitude more for the certainty that my platform will persist.
How will Google encourage a new developers to start working with GAE when platform-neutral providers (AWS, linode) are getting cheaper and better all the time?
The author is completely clueless here.
The GAE change is that "true resources-based pricing" switched from an overabundant, almost irrelevant resource (CPU time) to the actual resource in short supply - RAM.
For web servers, it doesn't matter how much CPU time you use, it only matters how much RAM you occupy. Linode, Rackspace, and most other VPS providers* charge this way. Most server CPUs are sitting around blocked on I/O - especially ones running single-threaded synchronous web servers (I'm looking at you, Rails).
The new GAE pricing is still based on true resources - the ones that matter, not the ones that are irrelevant.
* AWS is fairly unique with a hybrid billing approach.
The first GAE instance process on a server would take up fair amount of RAM. The next one takes very little. Yet Google charges for the second one fully.
This is a natural way to prevent hot instances from overwhelming a single box in the cluster.
* In modern interpreted/JITed languages, the "code" is actually data that gets malloced. These segments are not shared.
* Modern interpreted/JITed applications eat VAST quantities of heap. They constantly produce garbage, most of which (hopefully) gets culled by the garbage collector.
On GAE, even small Python instances typically grow to 30-40MB. Java instances typically hover around 90MB. The actual per-instance reservation must be significantly larger because boxes can never be allowed to go into swap. Figure that Google allocates ~128MB for every single active instance of every single application. It's a lot.
Except that thats just not true! We aren't charged by MB-hour. We are charged by instance hour. Do I pay less if my app is using only a small amount of memory? Not at all.
You may argue that instances are a proxy for ram. Certainly they are. They just aren't a proxy that has useful, meaningful metrics. The rules for when or why the infamous scheduler decides to spin up a new instance is pretty much fucking voodoo right now. Before, I had a metric, and it was directly related to me writing good code. Now, I might as well read tea leaves.
So, specifically, I am not complaining about being billed for RAM. If this is the route they want to go (and I understand why), then I my complaint is precisely that I am not being billed for RAM.
The problem is that while your instance may only use 30MB now, it could grow to 100MB in an instant without warning. Tipping a box into swap would effectively crash all instances present. Oversubscribing RAM is a dangerous game, and I suspect Google doesn't try to play it - they have enough trouble debugging performance issues as it is.
So basically, an instance represents a fixed about of RAM. Instance-hours and RAM-hours are the same.
Primary complaint. We are all speculating. We went from measurement to speculation. I can't write performance applications on speculation.
[1]http://googlecode.blogspot.com/2011/05/google-app-engine-new...
"Over the last three years, we’ve collected great feedback from our customers and now believe that the biggest thing we can do to help our customers is to graduate App Engine from preview status. "
"Adding business features will help App Engine meet a broader set of needs and the new, more transparent pricing model will help customers better align their App Engine investment with their business goals."
[2]http://googleappengine.blogspot.com/2011/05/year-ahead-for-g...
"App Engine graduating from Preview later this year! Based on this feedback we’ve decided to make some fairly large changes:"
"In order to become an official Google product we must restructure our pricing model to obtain sustainable revenue. Based on customer feedback this means focusing on usage-based pricing and placing per-user, per-app pricing on hold until further notice."
Look at Linode, Rackspacecloud, even EC2. They all charge primarily by the megabyte. I don't like it either, but it's the unfortunate economic reality.
"Most frontend instances don't come near their RAM limit, and most of their RAM consumption is on shared data like packages." -- nickjohnson http://goo.gl/SOeSF
It's not about how much you use, it's about how much you could use.
It was news to us that they are losing so much money.
It was news because they spent the last three years telling us to jump through insane hoops, locking ourselves thoroughly into a for-profit company, on the sole belief that doing so would allow efficiencies and scale not to be found elsewhere.
The complaint is not "you shouldn't make money off us". The complaint is what the fuck was all that bullshit we've been doing for three fucking years if it didn't make any fucking difference - as demonstrated by your "viable billing prices"?
And what does it say about their competence? After three years, they only discovered in May that they were measuring the wrong thing? Google? The analysis company. Only in May someone decided to look at how all the "brilliant GAE Way" actually impacted performance and resources? Nobody thought, "this cpu billing is all very well, but we are actually running 20 bazillion machines to serve 5 web pages per second - maybe our model might be wrong".
For the tiny guys, the free plan is viable. Perhaps for the giant guys, (I don't know) the plan is viable. For everyone in the middle, it is better to go to EC2. The only people who say they are staying are those who do not have the competence to move to EC2 or Heroku, i.e. non engineers every one of them.
So GAE used to be all "Hey, engineers, come code to our awesome paradigm shifting api, and you can be efficient and scale."
Now GAE is, "Hey, MBAs and Muggles, have your monkeys write generic java/python and we'll make it scale no matter what the cost (to you)!"
gregd, project manager: "We will also introduce a new pricing structure for App Engine based on more transparent usage-based pricing." http://goo.gl/d78vS
jonmac, engineer: "The system is large and complex, its hard to understand and hard to explain." http://goo.gl/6NzF1
No, the fervor picked up in intensity because they only now gave us the new metrics, and the price of them, and they are fucking insane.
We knew how much instances would cost, it just looks like nobody bothered to do the math until Google built it into the billing.