Seems like "precise prediction and execution of Amazon Glacier operations" might be a niche product people would pay for (and probably already exists for enterprise use cases?)
That's something that generally keeps me from using AWS and many other cloud services in many cases: the inability to enforce cost limits. For private/side project use I can live with losing performance/uptime due to a cost breaker kicking in. I can't live with accidentally generating massive bills without knowingly raising a limit.
How would you script a protecting against the issue described in the article? Unless you make sure to check the cost before each single request you can't stop it (and if you accidentally send one massive retrieval request even that isn't enough)
For other services it is easier, but even then, setting up and managing my own cost control mechanism is a level of complexity (and risk of failure) I'd really want to avoid, esp. since I probably use AWS to avoid management overhead.
You can't in this particular use case, but I can't envisage AWS providing a cost-control system that would stop this, either. It doesn't make sense--they're not calculating costs as they go. What I'm saying is that what Amazon would provide you is not functionally different from what you can do yourself right now.
I would be a lot more worried about a risk of over-charging myself if AWS wasn't incredibly good about refunding accidental overages.
I have not tried variety of AWS services because I have no idea what it would cost me if something went haywire on my server.
If I could simply deposit to Amazon a prepaid amount and it would just use this deposit until it's depleted, after which the services I have would grind to halt. This would be a perfect way for me to try it.
Pricing plans FOR CONSUMERS should always be straightforward and easy to understand. This is not a consumer product.
Pricing plans FOR B2B should model, as effectively as possible, the underlying costs -- this allows the provider to offer the lowest possible pricing for the services that cost them the least to provide, with expensive services priced accordingly. As others have mentioned on this thread, utilities are really, really good at this -- they come up with extremely complex rate plans for their largest customers that help them achieve whatever economies they are aiming for, for example incentivizing customers to provide level-loading (which is effectively what Amazon is doing in this retrieval scheme).
I think the reality is that most cloud customers are approximately consumers. Some big/sophisticated customers may want precise knobs to tune their spend versus "capability" but many businesses just want you to store their data. Pricing models like Glacier's are scary as hell to any CFO, unless they're shown a plan that says "So we're going to save $XXM on this, 100% certain".
Adding on your utility company meme, there are different rate schedules for residential and business customers in most places. In Cloud, I think negotiated contracts with advanced customers probably make more sense than complex, unfriendly pricing models for everyone.
That's a manufactured excuse for a fundamentally bad product api and pricing structure. When dropbox is more useful than AWS, amazon has screwed the pooch (which they do pretty often). Segmenting users by arbitrary circuitous logic into "consumers" (can't find a good use for it) and "enterprise" (can find a good use for it) isn't constructive. Both classes should avoid it, because it's not even an inexpensive choice, for what you get.
How is Dropbox more useful? Dropbox and AWS Glacier are vastly different products and show exactly the divide between consumer and enterprise that you say doesn't exist.
Glacier pricing has to be the most convoluted AWS pricing structure and can really screw you.
Google Nearline is a much better option IMO. Seconds of retrieval time and still the same low price, and much easier to calculate your costs when looking into large downloads.
Can this tool offer deduplication (so that changing a folder name does not re-upload thousands of files) or is that something I would have to code into my own backup solution?
They may also take a while to have the same level of software support, since their API isn't compatible with the S3 API. I know of at least one online backup software provider that works with multiple storage backends but which when asked did not intend to support B2 (since Backblaze is technically a competitor).
They're both difficult to grok but for different reasons. (Well, the same underlying reason - bandwidth - just accounted for differently).
Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs.
Nearline is difficult to grok because it simply caps your bandwidth at 4 MB/s per TB stored. It starts at 1 MB/s.
Basically, both are optimized to make retrieval slow and/or expensive because the cost optimizations they're doing internally are incentivizing very cold data, very big data, or both.
Note that with "on-demand I/O" you can basically remove this I/O throttle (best effort though) at an increased cost. Nearline is still cheaper than dealing with glacier (or even S3 Infrequent Access) particularly for small files.
Disclaimer: I work on Compute Engine (GCE) but not Cloud Storage (GCS).
>Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs.
It does go against their slogan "only pay for what you use", though. You're not using that 15GB/hour for 744 hours, and haven't explicitly selected provisioned IOPs.
Yes, there are Debian, Ubuntu, Centos, SUSE, CoreOS, RHEL and Windows Server images available for launch. You should see them in your cloud dashboard as choices when you go to launch an instance.
Entering exactly the phrase "Does Google cloud have any plain VMs (running CentOS for example) available" into Google search results in, as a first result, a link explaining exactly how to launch one of those [0].
Your comment doesn't really relate to the parent, to this thread and is a question that is trivially easy to answer. There's no point in hundreds-to-thousands of HN readers seeing your comment which can be answered in under a second and anyone who's used GCE already knows the answer to.
Please considering querying Google, or a relevant knowledge base, before querying a tangentially related HN comment chain.
You're absolutely right, my apologies. -- I recall looking for this quite a while ago without success, but I should have done a fresh search before posting.
Yes, Compute Engine is what you're after (https://cloud.google.com/compute/). Those predefined things are getting started / "click to deploy" for people that don't want to bother.
Since I can't find a way to set my account status to "personal" I'm somehow personally responsible to pay VAT (I wouldn't even know how to do that) if I don't want to risk getting problems with our version of the IRS. That doesn't sound like a nice way to do my personal backups.
On AWS that's not a problem, Amazon bills and pays the VAT for me.
That's all any of us on Google Cloud can usefully say about the damn VAT issue. We've got people working on it, but it's a Google wide issue (and I honestly don't understand the last mile enough to know why we don't do exactly what you describe AWS doing).
As a side reply to the poster below, it's not true that we're trying to exclude individuals. There are lots of happy, individual customers all over the world. We unfortunately just didn't get out ahead of this personal service / VAT thing as a company. So again, Sorry! And someone is working on it, but don't expect any immediate fixes when taxes/regulations/laws are involved.
So pass this up the chain if you can: I've used both Azure and (continue to use) AWS, but I've never seriously even considered Google Cloud Platform because I feel like Google's service offering is not stable. I know inherently that this is because of the way Google treats its consumer offerings, but I can't help but imagine that it applies to the business offerings as well.
I hear you. Those of us that care regret the shutdown of Reader, as it has almost certainly cost us more business dollars than running it in perpetuity may have.
That said, Google (like most businesses?) hasn't cancelled any product that is making lots of money. At Google, that means Ads, Apps, and yes Cloud. We're here to stay, and unlike on the consumer side Cloud has an explicit deprecation policy for products that have "gone GA" (graduated from Beta).
Big picture items like Compute Engine, Google Cloud Storage, etc. aren't going anywhere. We might deprecate our v1 API after we're on v2 of course, but we're not cancelling the product (again, it makes us money, and that's a pretty key distinction from the consumer products Google has historically shuttered).
Disclaimer: I work on Compute Engine and haven't personally cancelled anything yet ;).
It's almost as cheap as Glacier, but requires no waiting and has no complicated hidden costs, just simply somewhat higher request pricing, a minimum 30 days of storage and an extra $0.01 per GB for data retrievals.
> Standard - IA has a minimum object size of 128KB. Smaller objects will be charged for 128KB of storage.
Rounding up all small files to 128 KB can be a huge deal. I for example use Nearline to directly "rsync" my NAS box for offsite backup (yeah I know, I should use something that turns it into a real archive or something, but I'm lazy and Synology has this built in). If those hundreds of thousands of (often) small files were rounded up, S3-IA would easily be more expensive than S3/GCS.
Disclaimer: I work at Google on Compute Engine and am a happy Nearline customer myself.
You can also try the duplicity backup tool. It uses tar files to store the primary backup and its binary incremental deltas. It supports a number of backup back-ends, but you can also just generate the backup to a local disk and then rsync it to a remote location.
Come on, don't downvote someone for learning something (you think is) well known for the first time. Especially when their response is "that's great and I'm going to use it". https://xkcd.com/1053/
The comment is noise - no different from "+1" or "me too". If you found a comment helpful and want to thank someone for it, the way to do that is an upvote.
I've had some big Glacier bills in the past, even the upload pricing has gotchas[1]
These days the infrequent access storage method is probably better for most people. It is about 50% more than Glacier (but still 40% of normal S3 cost) but is a lot closer in pricing structure to standard S3.
Only use glacier if you spend a lot of time working out your numbers and are really sure your use case won't change.
[1] - 5 cents per 1000 requests adds with with a lot of little files.
I use Infrequent Access Storage for backups, through a tool called duplicity (or more aptly, I use a GUI front-end for that tool called Deja Dup). Instead of uploading every individual file, it gathers them into 25 mb .tar files & uploads those along with an index describing where each actual file is. That has made the requests negligible for me.
This is a simple case of spending more than you should have because you didn't understand the service you were using. It's impacted a little worse by how silly the whole endeavor is, given the preponderance of music streaming services.
This is why I break my large files uploaded to Glacier into 100MB chunks before uploading. If I ever need them, I have the option of getting them in a slow trickle.
This is no longer necessary, they now allow you to specify ranges of partial files. So you can split a single large file into multiple requests to keep you within allowance or within budget.
Nice. I also upload par2 checksum files for each chunk so changing things now would involve a bit of script rewriting, but that's good to know for the future.
No, it's not necessary at all, I hope. I probably should have called them just "Par2" files not Par2 checksum files. They aren't just checksums. They allow reconstruction of the original file even after fairly extensive damage. Which isn't likely, so, yes, not strictly necessary, but who knows, could be useful in the what-if scenario. The checksums you mention only help with checking, not with repair.
The retries don't cost extra. Sending off all 150 retrieval requests for all the data at once set the retrieval rate and the price, they should have been staggered over a lot of time to keep the rate low.
I have a strong feeling that he would get a refund if he contacted Amazon support, considering it was caused by a bug in the official SDK and he didn't ACTUALLY use the capacity he's being asked to pay for.
Multiple requests didn't cause the issue. It was asking for all of his data to be queued in such a small window of time. The same thing would have happened without the bug.
So depending on how the "average monthly storage" is computed you could get 20x more data in one month and then retrieve the 5% (previously 100%) that you care about for free, and then delete the additional data?
The only use case I would be willing to commit to glacier would be legal-hold or similar compliance requirement.
The idea would be that the data would either never be restored or you could compel someone else to foot the bill or using cost sharing as a negotiation lever. (Oh, you want all of our email for the last 10 years? Sure, you pick up the $X retrieval and processing costs)
Few if any individuals have any business using the service. Nerds should use standard object storage or something like rsync.net. Normal people should use Backblaze/etc and be done with it.
Back when I worked in banking we had requirements like that (though we didn't use glacier)
We had a legal requirement to be able to product up to 7 years worth of bank statements upon receipt of a subpoena.
Not "reproduce the statements from your transactions records" but "give us a copy of the statement that you sent to this person 6.5 years ago"
We had operational data stores that could generate a new statement for that time period, but if we received the subpoena then we needed to be able to produce the original, that included the (printed) address that we sent it to, etc.
We had (online) records of "for account 12345, on 27th October 2011, we sent out a statement with id XYZ", we'd just need a way to pull up statement XYZ.
There's no way(^) we'd ever get subpoenaed for more than 5% of our total statement records in a single month, so something like Glacier would have been a great fit.
We had other imaging+workflow processes where we'd receive a fax/letter from a client requesting certain work be undertaken (e.g a change of address form). 90 days after the task was completed, you could be pretty sure that you wouldn't need to look at the imaged form again, but not 100% sure. We could have used glacier for that.
We use case that would have cost us (rare, but we needed to plan for it) was "We just found that employee ABC was committing fraud. Pull up the original copies of all the work they did for the 3 years they worked here, and have someone check that they performed the actions as requested."
Depending on circumstances & volume that might trigger some retrieval costs, but the net saving would almost certainly still be worth it.
(^) Unless there was some sort of class action against us, but that's not a scenario we optimised for.
I'm happy enough to use it as a "third copy" - for never-expected-to-be-used recovery if both my local and remote backups fail.
I know it'll take either a lot of time or money to restore from Glacier, but if my home and work backups have both gone I'll either not care about my data any more, or I'll be perfectly happy to throw a grand or so at Amazon to get my stuff back (or, more likely, be happy to wait up to 20 months for the final bits of my music and photo collections to come back to my own drives).
Even with the large data retrieval bill he still saves ~$100 vs the price of keeping that data in S3 over the same time period. Reading this honestly makes me think glacier could be great for a catastrophic failure backup.
I looked up the Google drive pricing and the costs increase quickly over that 100GB level. I'm going to use 3TB of data for my example because that is approximately the amount of data I would be backing up at work. The cost for Google Drive would be the $99 a month 10TB plan, Amazon Glacier is $21.51 a month. This is before you get into things like having the enterprise AWS ecosystem with IAM versus a single user Gmail account. Remember I am only talking about retrieval in the case of a catastrophic failure, the data is already backed up elsewhere. As long as I can manage to go a year without destroying all my backups Glacier comes out on top over Drive even taking into account the retrieval fees. In the best case scenario I never ever retrieve that data.
I briefly used Glacier for daily backups as a failsafe if our internal tape backups failed when we needed them. The 4 hour inventory retrieval when I went to test the strategy and the bizarre pricing quickly make me look at other options.
This sounds a lot like demand-billing [1] [2] that's common with electric utilities, particularly commercial, and increasingly, people with grid-tied solar installations. [citation needed]
You pay a lower per-kilowatt-hour rate, but your demand rate for the entire month is based on the highest 15-minute average in the entire month, then applied to the entire month.
You can easily double or triple your electric bill with only 15 minutes of full-power usage.
I once got a demand bill from the power company that indicated a load that was 3 times the capacity of my circuit (1800 amps on a 600 amp service). It took me several days to get through to a representative that understood why that was not possible.
Impressive that Amazon can choose to serve a request at 2x the bandwidth you need, with no advance notice, and charge you double the price for the privilege.
Glacier's pricing structure is complicated, but fortunately it's now fairly straightforward to set up a policy to cap your data retrieval rate and limit your costs. This was only introduced a year ago, so if like Marko you started using Glacier before that it could be easy to miss, but it's probably something that anyone using Glacier should do.
> fortunately it is fairly strait forward to cap your data retreival rates.
Amazon has done a great job with this feature. By doing a poor job implementing something for an extremely narrow usecase, in a technology that is outdated and then providing the most complicated pricing structure surrounding every aspect of the product one can't helpbut use the feature: any other provider or service.
Like, wtf would be the usecase for amazon glacier in 2016? I dont think I would put hubdreds of petabytes of sata into 20 year cold storage, and the author of this post certainly wouldnt use it again. The fact that i need to read 2 pages of pricing docs and then the 2 pages you linked to control them because I cant estimate them myself, is a sure sign this is absurd
Tape storage is still the most optimal form of long-term storage. If you need to store things for an exceptionally long time, such as financial data, scientific data, etc, then you're going to get the most bang for your buck on tape.
Depends. Having an in-house key and shipping everything to Amazon encrypted means that you have all the infrastructure there and waiting, and not capable of being read. Additionally, that tape library would need to be stored, and periodically tested so that tapes can be rotated out as needed. Sending that data to a service like glacier means you've shown due diligence, but at the same time, don't need to maintain a schedule of testing every disk every year.
this was what I meant. thanks. You can buy a 5tb hard drive for ~138.00. You could likely buy several of them at discount to get started. As you go forward in time, these will become much cheaper, allowing you to continue purchasing them on demand from the market for much less money.
This allows you to trivially share, copy, move and retrieve that data quickly as well as fully control who has access and when.
I am sure there are use cases for this but in a situation where you have petabyte scale data, it is often the case that you also have the infrastructure to save it. How many places would need to store >5tb of data a week that
* don't have this capability in house
* will almost never need to access it again.
* will not need it in a timely manner, if they do need to access it again.
* don't have the money to implement their dedicated server and storage on site for this purpose.
I am not saying that this rules everyone out, but the prices are so low, and tape must be so annoying, I couldn't imagine why they keep offering this. Obviously, some peopel must be using it but in 2016 with storage prices being so low already, i don't know how many places have this amount of data and meet the above requirements.
The post states he paid $150 yo retrieve 60gb of data. For $150 you can buy a 5tb hdd drive.
What usecase could the price delta make sense to have a 4 hour feedback loop and all of your important data locked in someone elses data center.
the usecase where your data is so properly massive that this makes sense && you don't have the storage infrastructure in place, is so narrow that it doesnt make sense.
It is basically one research student's crawl data
Edit: also, s3 is pretty cheap. So again, i dont really see the usecase here. How much room is in the market between your own physical or digital system and amazon s3 or an equivalent. you would have to have a massive amount of data you dont care about and be very price sensitive.
The data I care about is already backed up on two different multi-TB drives at home, and another one at work.
Glacier is the contingency for "something took out the original data and all three backups in two different locations 7 or 8km apart - if I'm still alive after whatever just happened, I'll consider whether or not to pay Amazon a grand or so to retrieve it quickly from Glacier, or wait ~20 months to get it all for single-digit-dollars".
Right the services cost money but retrieval is free. Going by the cost of the harddrives amortized out, it'll probably be the same or less. You get far more durability and less complexity with universal web access.
I believe there are other similar services for Linux or you can just use browser to upload files with Amazon.
You don't have to pay $150 for retrieval of 60GB. And you don't do long-term storage for X TB / 5 TB * $150. You might have to rent space in someone elses datacenter to put your own external backup... or you could pay Amazon for Glacier and not deal with maintenance etc. Might be worth it even if you have infrastructure for all data that isn't glacier-cold.
Lots of businesses have data retention requirements, and it can be difficult and time consuming to make sure this data is backed up in a way that is secure and can survive a catastrophe.
The author's use case (and most other personal use cases) might not be a good fit for Glacier, but he's not the target market.
SOX compliance, legal requirements to save communications, etc. There are a lot of places where there are needs to maintain a huge amount of information that you're probably never going to need again.
Not all products are for all people. If you foresee a need to recover a large amount of data all at once, then glacier's not for you. If you might occasionally need a filing from 6 years ago, then glacier would be great.
It's not about recovering a large amount of data; its about recovering a large % of your own stored data.
Amazon starts to charge you extra anytime you exceed restoring 5% of the data.
If for example you save all your tax-related documents in Glacier, then you are audited then the accounts department or the government will want all the information. Not 5% of it. Not 10% of it. Everything. At that point Amazon will have you over a barrel, because getting the data out at a reasonable time frame will cost exponentially more than dripping out the data over the course of 20 months.
>If for example you save all your tax-related documents in Glacier, then you are audited then the accounts department or the government will want all the information. Not 5% of it. Not 10% of it. Everything.
Are you sure about that? I haven't worked with tax litigation specifically, but I've worked with e-discovery w.r.t. e-mail and I can assure you that no one ever asks for all the e-mails sent by a particular company over all time. It's always a matter of asking for the e-mails sent by or received by a select group of people, over a fairly discrete time period. For something like this, a Glacier store might make sense, if it was coupled with an online metadata cache stored in e.g. S3.
With tax litigation the issue is that you have to prove you didn't simply shift money and accounting briefs around, and the only way to realistically prove it is to show all the statements in the time period that you'd required to keep them (I think that's the last 6 years).
The government basically comes to you and says they think you owe X, and you have to prove that false to their satisfaction. The more data you give your CPA to work with, the better.
> Amazon starts to charge you extra anytime you exceed restoring 5% of the data.
Isn't one way to get past this... increasing your data usage by 20x? If OP used less than a $1 a month, then if he uploaded $20 of junk data, he can get the 5% original data back "for free". Sure, it's $20, but it beats out $150+.
A better way would be if you had 20+ categories of data that are totally unrelated, like; your tax-stuff, your code, your diary from 1995-2010, ... Since these are very unrelated, you are not likely to need every one of at the same time, ASAP.
Though it's hard for me to imagine having so many categories of unrelated, useful and important data.
It looks like it is even more complicated than that. You can get 5% out per month at no charge, but that it only if you spread it out across the entire month. The extra charges happen the first time you exceed 5%/30 in a single day.
Wouldn't it be better for the OP to simply upload 20 * 60GB (= 1.8TB) of random data, wait a month (paying less than 20 USD), and then download the initial 60GB within his 5% monthly limit?
I don't like how the title and article reads like a hit piece on Amazon Glacier. It's great at what it is intended for. In addition it seems he still saved money because over 3 years because the $9 a month savings added up to more than the $150 bill for retrieval.
I'm surprised that this aspect has not been mentioned here in the comments yet:
> I was initiating the same 150 retrievals, over and over again, in the same order.
This was the actual problem that resulted in the large cost.
At my old job we would get a lot of complaints about overage charges based on usage to our paid API. It wasn't as complicated of pricing as a lot of AWS services, just x req / month and $0.0x per req after that, but every billing cycle someone would complain that we overcharged them. We would then look through our logs to confirm they had indeed made the requests and provide the client with these logs.
> I was initiating the same 150 retrievals, over and over again, in the same order.
>> This was the actual problem that resulted in the large cost.
That is not true, according to the article itself. The first request for the full 60.8gb already results in a $154.25 bill, regardless of the ones that follow. From that point on he can continue to retrieve 15.2GB/hour for the rest of the month without incurring further costs.
> > I was initiating the same 150 retrievals, over and over again, in the same order.
This was the actual problem that resulted in the large cost.
Except that it wasn't. The repeated requests were free, because he already set the maximum rate with the first wave of requests. Surprising?
Also, it really is not a hit piece. It's an honest report of what he did (and what he did wrong) and that he thinks the docs aren't as clear as they could be.
Thanks, I guess I glossed over that part of the article but when I got to the end and saw the original quote I assumed the worst. The title alone is pretty inflammatory in my opinion.
Are you trying to say that the OP was not allowed to back up their own CDs? I have several boxes of space wasting CDs of my own that have all been ripped in lossless, and feel somewhat insulted by that notion. Note: I live in Australia, a somewhat less insane country for DRM than the US (AFAIK).
CDs don't even have DRM in the general case. However, local laws can still forbid ripping (e.g. here in the UK where after a brief period of legality it is once again illegal to rip a CD).
EDIT
removed material that might have derailed things further.
Roger that; I used the DRM term incorrectly, meaning copyright law. Most of my collection was bought in Norway, but I did the ripping a decade ago plus minus a few years in Australia. For the time being we have much better consumer protection here.
I don't even use my collection now that I have Spotify Premium. The only music I've bought lately is some 24bit high bitrate stuff.
Also in Australia, and its not that black-and-white in my understanding:
This is known as "Format Shifting" — taking one copyrighted medium and converting it to another. In Australia, you are explicitly not allowed to do this with CDs, DVDs and Blu-rays.
You are only allowed to keep a digital copy if you continue to retain the original — a backup. If the original is lost or destroyed, your digital copies must be discarded.
For example, you can rip a CD and put it on your iPod, or computer, as long as you continue to own the CD. The issue here is that in both cases you also control the device you are copying it to. You don't control but rather lease space on Amazon's servers — so it introduces a grey area on whether you are allowed to backup to such places and whether putting data on those servers constitutes distribution of the copyright material.
Realistically, none of this is black-and-white and Amazon could flag it as infringing content and remove it just to cover themselves against DMCA complaints anyway. This is true in both Australia and the US, regardless of the differences in copyright law (Australian copyright law offers far fewer protections than the US, incidentally) because both having similar DMCA laws.
1) Buys music CD.
2) Rips CD to own computer. Shares with own devices [via personal cloud] for continued listening.
3) Destroys original CD.
4) Continues to listen to music which has been paid for; not sharing files with anybody else.
I for one fail to see the problem here. But then again, I'm probably not the brightest bulb in the pack either.
I have yet to meet a yo e who destroys their originals. Do you? Most people sell or gift them to someone else, which is not only outside of the letter of current copyright laws, but the spirit of those laws as well.
It would be easier to download pirated copies off the internet and exactly as legal.
Mine are all in a box shoved against the wall under my bed, where they don't get it the way but I can prove that I still deserve consideration under the First Sale doctrine.
Glacier is more comfortable to use through S3, where you upload and download files with the regular S3 console, and just set their storage class to Glacier with a lifecycle rule. I've used the instructions in here to do it: https://aws.amazon.com/blogs/aws/archive-s3-to-glacier/
Amazon makes it very easy to transition data from S3 to Glacier. But, if you want to get it back, you will still have to go through all of the byzantine Glacier rules and fees, including the 4-hour wait.
And, there is no "transition from Glacier to S3": if you want to do that, you have to:
1. restore it to S3 (and incur the fees and 4-hour wait)
2. copy the restored S3 object to a new S3 object
3. delete the restored object (or wait for it to timeout)
The post is a useful cautionary tale, and he's not alone in getting burned by Glacier pricing. Unfortunately it was OP not reading the docs properly.
Yes, the docs are imperfect (and were likely worse back in the day). And it was compounded by the bug, apparently. But it's what everyone on HN has learned in one way or another... RTFM.
Was it mentioned in the article that the retrieval pricing is spread over four hours, and you can request partial chunks of a file? Heck, you can retrieve always all your data from Glacier for free if you're willing to wait long enough.
And if it's a LOT of data, you can even pay and they'll ship it on a hardware storage device (Amazon Snowball).
Anyone can screw up, I'm sure we all have done, goodness knows I have. But at the very least, pay attention to the pricing section, especially if it links to an FAQ.
Like you said, a good cautionary tale. Even this incident, although definitely rough, isn't that bad compared to what can happen if you scale a full infrastructure improperly, or just overlook something involving a software-based business. Just a couple of months ago, a test EC2 instance that had been forgotten about got spun up by a very archaic piece of code, and ended up costing the guys I'm working with quite a bit. Always RTFM (especially AWS pricing docs) carefully, or feel the burn.
Paying 87c a month for a couple of years, then 52c a month for a few more years, to back up 60+GB, and then getting a one off fee of $150 still averages out at around $2/mo. Hardly getting shafted.
Depends on your budget. If you don't have $150 disposable income to blow on a screwup, it's an expensive mistake.
It's like getting a parking ticket for parking somewhere you've unknowingly been parking illegally for years. Yeah, if you average it out, it's cheap. But having to stump up the cash still hurts.
60Gig is what - maybe 200 CDs losslessly compressed? That's a couple of grand's worth of disposable income he entrusted to Amazon and his own management skills (assuming he no longer has access to the original disks to re-rip them (and ignoring the copyright implications of that)).
I guess he always had Linus's backup strategy open to him - "Only wimps use music backups: real men just upload their important tunes on Bittorrent, and let the rest of the world seed it ;)"
Meta, Response to sub-comments[meta enough yet?]: Ah, that makes way more sense. Sorry for labelling the parent a troll. This was clearly my mistaken interpretation. It was weird. As I tried to check out the math on the parent's assertions, they came out an order of magnitude too high like 3 times in a row. So I concluded that the parent was intentionally inflating the relevant costs to misdirect the discussion. However, it was really just my failure to consider the perspective of the parent, aligned with a loose enough situation that the mistake was possible. Again, I apologize for the misapplied label. I'll leave the original comment below so as not to rewrite history on how I was (due to sincere misunderstanding, I maintain) kind of a jerk :/
A couple grand? 500GB hdds have been less than $100 for a long time now, certainly since before Amazon Glacier was a product.
Just because you can formulate a large valuation for something doesn't mean that it's a reasonable valuation.
Even $10/cd is a ridiculous estimate of the cost of blank CDs. If you insisted on using blank CDs for backup, you could do 200 CDs for $40.
Edit: And what are CDs... 640mb? that's like 12 CDs, not 200... I'm realizing that I've fallen into a troll trap...
Meta: I guess I'll leave this comment up as a cautionary tale.
Indeed. I'm guessing 200-ish albums, purchased at ~$10 each, would take up 60G when losslessly compressed. (As pointed out, 640MB/disk would mean just under 100 "full" audio CDs as bit for bit copies, so 200 is probably a quite low estimate).
I bought my ~3000 disc CD collection here in Australia, many of them still have price stickers of $30 and more on them. While CDs were "a thing" here, I figure I spent about a quarter of a cheap house on them. (I don't regret a cent though. I'm also "that guy" who thinks "You only had 60Gig of music to back up? Wow!", but is mostly socially-aware enough to keep that sort of reaction to himself.)
>Glacier is designed with the expectation that retrievals are infrequent and unusual, and data will be stored for extended periods of time. You can retrieve up to 5% of your average monthly storage (pro-rated daily) for free each month. If you choose to retrieve more than this amount of data in a month, you are charged a retrieval fee starting at $0.01 per gigabyte. Learn more.
If you want more than 5% of your data in a month, the minimum it will cost you is $0.011 per gigabyte. Click here to see how to work out how much it will cost.
It's certainly not unreadable, although to be fair, it's nowhere near as clear as it should be.
It's definitely sleazy of Amazon to hide the pricing info like that, but I can't imagine seeing that sentence while making a purchase and not clicking through to see how the pricing actually works. But I guess that's just me.
Not just you. But read the page referred by that "Learn more" link. It's very unclear. I mean, or course, it has unambiguous meaning. But to understand it you need to read a lengthy sheet of prose, and only deep in it, after many definitions, you must notice a phrase "we multiply your peak hourly billable retrieval rate" ... "by the number of hours in a month". What concentration of attention, and how much time a reader need to spend, to understand this important nuance?
As far as I see, in all other places they specify their pricing "per GB", and only this small phrase uncovers real meaning, which is not per actual GBs you transferred, but your peak rate multiplied by number of hours in month. IMHO this should be one of the first phrases describing the pricing model.
> The post is a useful cautionary tale, and he's not alone in getting burned by Glacier pricing. Unfortunately it was OP not reading the docs properly.
Well, it was that and also the docs being knowingly and deliberately set up to trick incautious readers. The user does have a responsibility to read the fine print, but that doesn't excuse Amazon being openly evil about it. This is no different than ISPs that advertise "up to 50Mb/s" when they know very well that their network won't deliver more than 5.
Even after people have RTFM, many are still misunderstanding the pricing model. For instance, people now think you can store 2TB of data and download 100GB of it at once (5%) for free. Nope! You can request a max of 0.16% per day of data for free. Once you exceed 0.16% of your data in any given day, you start getting charged. I think it's fair to say that even with the intent to RTFM, the pricing model is unusually strange and AWS should really put more effort into explaining it.
I suppose part of the neat trick of it is that because we don't have to know, Amazon can switch it out for something else anytime it's convenient for them, or some new tech comes up. Or split thing up among several methods and compare costs. As long as they structure their operations such that nothing is ever lost and everything can be retrieved with a few hours notice at any time, they can try anything they want.
Whatever information became public, people would write scripts and the like with those assumptions baked in. Even if it wasn't officially documented, it'd be bad PR for Amazon to break things people were relying on.
I'm sure that was a commercial decision, not a technical one.
They could be running Glacier storage at cost (or even a slight loss).
But they make their profit when you try to get your data out.
I'm not implying anything nefarious - more along the lines that Amazon (could have!) looked at the market and compared demand to what they were offering. Then found something that scratched an itch...
In most of these cases, the technical decisions impact the commercial. If writing megabytes were "technically" cheaper than reading, you can either choose to pass that cost onto customers or simplify the pricing model assuming some read-to-write ratio.
They didn't just set the prices (or the pricing model) to amuse themselves.
231 comments
[ 929 ms ] story [ 4557 ms ] threadThat's something that generally keeps me from using AWS and many other cloud services in many cases: the inability to enforce cost limits. For private/side project use I can live with losing performance/uptime due to a cost breaker kicking in. I can't live with accidentally generating massive bills without knowingly raising a limit.
For other services it is easier, but even then, setting up and managing my own cost control mechanism is a level of complexity (and risk of failure) I'd really want to avoid, esp. since I probably use AWS to avoid management overhead.
I would be a lot more worried about a risk of over-charging myself if AWS wasn't incredibly good about refunding accidental overages.
were linked elsewhere in the comments.
Edit: By user?id=re.
I have not tried variety of AWS services because I have no idea what it would cost me if something went haywire on my server.
If I could simply deposit to Amazon a prepaid amount and it would just use this deposit until it's depleted, after which the services I have would grind to halt. This would be a perfect way for me to try it.
Pricing plans FOR B2B should model, as effectively as possible, the underlying costs -- this allows the provider to offer the lowest possible pricing for the services that cost them the least to provide, with expensive services priced accordingly. As others have mentioned on this thread, utilities are really, really good at this -- they come up with extremely complex rate plans for their largest customers that help them achieve whatever economies they are aiming for, for example incentivizing customers to provide level-loading (which is effectively what Amazon is doing in this retrieval scheme).
Adding on your utility company meme, there are different rate schedules for residential and business customers in most places. In Cloud, I think negotiated contracts with advanced customers probably make more sense than complex, unfriendly pricing models for everyone.
Big disclaimer: I work on Compute Engine.
That's a manufactured excuse for a fundamentally bad product api and pricing structure. When dropbox is more useful than AWS, amazon has screwed the pooch (which they do pretty often). Segmenting users by arbitrary circuitous logic into "consumers" (can't find a good use for it) and "enterprise" (can find a good use for it) isn't constructive. Both classes should avoid it, because it's not even an inexpensive choice, for what you get.
Google Nearline is a much better option IMO. Seconds of retrieval time and still the same low price, and much easier to calculate your costs when looking into large downloads.
https://cloud.google.com/storage/docs/nearline?hl=en
Their pricing is great (0.5c/month) but I'm a little worried about their single DC.
This makes their offer very interesting.
Considering though that before that you couldn't even upload a directory, only files, this is a huge step. :)
Is there any source for the single DC?
EDIT: Nevermind. I did some more googling & found one:
https://www.backblaze.com/blog/b2-cloud-storage-frequent-que...
Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs.
Nearline is difficult to grok because it simply caps your bandwidth at 4 MB/s per TB stored. It starts at 1 MB/s.
Basically, both are optimized to make retrieval slow and/or expensive because the cost optimizations they're doing internally are incentivizing very cold data, very big data, or both.
Disclaimer: I work on Compute Engine (GCE) but not Cloud Storage (GCS).
It does go against their slogan "only pay for what you use", though. You're not using that 15GB/hour for 744 hours, and haven't explicitly selected provisioned IOPs.
Not a problem all you can get is a crappy 8 Mbps ADSL. Oh well, let me whine... ;)
Your comment doesn't really relate to the parent, to this thread and is a question that is trivially easy to answer. There's no point in hundreds-to-thousands of HN readers seeing your comment which can be answered in under a second and anyone who's used GCE already knows the answer to.
Please considering querying Google, or a relevant knowledge base, before querying a tangentially related HN comment chain.
[0]: https://cloud.google.com/compute/docs/operating-systems/linu...
Disclaimer: I work on Compute Engine.
This write up is a pretty good summary of our differences on the compute side: https://news.ycombinator.com/item?id=10922105
Disclaimer: I work on Compute Engine, and specific to that article launched Preemptible VMs.
Since I can't find a way to set my account status to "personal" I'm somehow personally responsible to pay VAT (I wouldn't even know how to do that) if I don't want to risk getting problems with our version of the IRS. That doesn't sound like a nice way to do my personal backups.
On AWS that's not a problem, Amazon bills and pays the VAT for me.
From that same page: "Google Cloud Platform services can be used only for business purposes in the European Union."
That's all any of us on Google Cloud can usefully say about the damn VAT issue. We've got people working on it, but it's a Google wide issue (and I honestly don't understand the last mile enough to know why we don't do exactly what you describe AWS doing).
As a side reply to the poster below, it's not true that we're trying to exclude individuals. There are lots of happy, individual customers all over the world. We unfortunately just didn't get out ahead of this personal service / VAT thing as a company. So again, Sorry! And someone is working on it, but don't expect any immediate fixes when taxes/regulations/laws are involved.
That said, Google (like most businesses?) hasn't cancelled any product that is making lots of money. At Google, that means Ads, Apps, and yes Cloud. We're here to stay, and unlike on the consumer side Cloud has an explicit deprecation policy for products that have "gone GA" (graduated from Beta).
Big picture items like Compute Engine, Google Cloud Storage, etc. aren't going anywhere. We might deprecate our v1 API after we're on v2 of course, but we're not cancelling the product (again, it makes us money, and that's a pretty key distinction from the consumer products Google has historically shuttered).
Disclaimer: I work on Compute Engine and haven't personally cancelled anything yet ;).
It's almost as cheap as Glacier, but requires no waiting and has no complicated hidden costs, just simply somewhat higher request pricing, a minimum 30 days of storage and an extra $0.01 per GB for data retrievals.
> Standard - IA has a minimum object size of 128KB. Smaller objects will be charged for 128KB of storage.
Rounding up all small files to 128 KB can be a huge deal. I for example use Nearline to directly "rsync" my NAS box for offsite backup (yeah I know, I should use something that turns it into a real archive or something, but I'm lazy and Synology has this built in). If those hundreds of thousands of (often) small files were rounded up, S3-IA would easily be more expensive than S3/GCS.
Disclaimer: I work at Google on Compute Engine and am a happy Nearline customer myself.
These days the infrequent access storage method is probably better for most people. It is about 50% more than Glacier (but still 40% of normal S3 cost) but is a lot closer in pricing structure to standard S3.
Only use glacier if you spend a lot of time working out your numbers and are really sure your use case won't change.
[1] - 5 cents per 1000 requests adds with with a lot of little files.
And then a customer wanted all their files rather than just one or two. Although that was billed back to the them.
https://en.wikipedia.org/wiki/Parchive
The idea would be that the data would either never be restored or you could compel someone else to foot the bill or using cost sharing as a negotiation lever. (Oh, you want all of our email for the last 10 years? Sure, you pick up the $X retrieval and processing costs)
Few if any individuals have any business using the service. Nerds should use standard object storage or something like rsync.net. Normal people should use Backblaze/etc and be done with it.
We had a legal requirement to be able to product up to 7 years worth of bank statements upon receipt of a subpoena.
Not "reproduce the statements from your transactions records" but "give us a copy of the statement that you sent to this person 6.5 years ago"
We had operational data stores that could generate a new statement for that time period, but if we received the subpoena then we needed to be able to produce the original, that included the (printed) address that we sent it to, etc.
We had (online) records of "for account 12345, on 27th October 2011, we sent out a statement with id XYZ", we'd just need a way to pull up statement XYZ.
There's no way(^) we'd ever get subpoenaed for more than 5% of our total statement records in a single month, so something like Glacier would have been a great fit.
We had other imaging+workflow processes where we'd receive a fax/letter from a client requesting certain work be undertaken (e.g a change of address form). 90 days after the task was completed, you could be pretty sure that you wouldn't need to look at the imaged form again, but not 100% sure. We could have used glacier for that.
We use case that would have cost us (rare, but we needed to plan for it) was "We just found that employee ABC was committing fraud. Pull up the original copies of all the work they did for the 3 years they worked here, and have someone check that they performed the actions as requested." Depending on circumstances & volume that might trigger some retrieval costs, but the net saving would almost certainly still be worth it.
(^) Unless there was some sort of class action against us, but that's not a scenario we optimised for.
I know it'll take either a lot of time or money to restore from Glacier, but if my home and work backups have both gone I'll either not care about my data any more, or I'll be perfectly happy to throw a grand or so at Amazon to get my stuff back (or, more likely, be happy to wait up to 20 months for the final bits of my music and photo collections to come back to my own drives).
Does s/he substantiate this claim in any way? AFAIK glacier's precise functioning is a trade secret and has never been publicly confirmed.
https://news.ycombinator.com/item?id=4416065
You pay a lower per-kilowatt-hour rate, but your demand rate for the entire month is based on the highest 15-minute average in the entire month, then applied to the entire month.
You can easily double or triple your electric bill with only 15 minutes of full-power usage.
I once got a demand bill from the power company that indicated a load that was 3 times the capacity of my circuit (1800 amps on a 600 amp service). It took me several days to get through to a representative that understood why that was not possible.
[1] http://www.stem.com/resources/learning
[2] http://www.askoncor.com/EN/Pages/FAQs/Billing-and-Rates-8.as...
http://docs.aws.amazon.com/amazonglacier/latest/dev/data-ret... https://aws.amazon.com/blogs/aws/data-retrieval-policies-aud...
Amazon has done a great job with this feature. By doing a poor job implementing something for an extremely narrow usecase, in a technology that is outdated and then providing the most complicated pricing structure surrounding every aspect of the product one can't helpbut use the feature: any other provider or service.
Like, wtf would be the usecase for amazon glacier in 2016? I dont think I would put hubdreds of petabytes of sata into 20 year cold storage, and the author of this post certainly wouldnt use it again. The fact that i need to read 2 pages of pricing docs and then the 2 pages you linked to control them because I cant estimate them myself, is a sure sign this is absurd
So you trust Glacier or Google Nearline (or any similar provider) without testing? No testing ever???
I wouldn't feel happy with my critical data floating around in the cloud, without my checking it at least once a year to make sure it really exists.
And once you do start verifying that data, you will incur all sorts of charges to access it.
This allows you to trivially share, copy, move and retrieve that data quickly as well as fully control who has access and when.
I am sure there are use cases for this but in a situation where you have petabyte scale data, it is often the case that you also have the infrastructure to save it. How many places would need to store >5tb of data a week that
* don't have this capability in house
* will almost never need to access it again.
* will not need it in a timely manner, if they do need to access it again.
* don't have the money to implement their dedicated server and storage on site for this purpose.
I am not saying that this rules everyone out, but the prices are so low, and tape must be so annoying, I couldn't imagine why they keep offering this. Obviously, some peopel must be using it but in 2016 with storage prices being so low already, i don't know how many places have this amount of data and meet the above requirements.
What usecase could the price delta make sense to have a 4 hour feedback loop and all of your important data locked in someone elses data center.
the usecase where your data is so properly massive that this makes sense && you don't have the storage infrastructure in place, is so narrow that it doesnt make sense.
It is basically one research student's crawl data
Edit: also, s3 is pretty cheap. So again, i dont really see the usecase here. How much room is in the market between your own physical or digital system and amazon s3 or an equivalent. you would have to have a massive amount of data you dont care about and be very price sensitive.
The data I care about is already backed up on two different multi-TB drives at home, and another one at work.
Glacier is the contingency for "something took out the original data and all three backups in two different locations 7 or 8km apart - if I'm still alive after whatever just happened, I'll consider whether or not to pay Amazon a grand or so to retrieve it quickly from Glacier, or wait ~20 months to get it all for single-digit-dollars".
That gives you 2 backup providers that can durably store everything and it's free and quick to access. Why deal with all the harddrives and glacier?
I believe there are other similar services for Linux or you can just use browser to upload files with Amazon.
You don't have to pay $150 for retrieval of 60GB. And you don't do long-term storage for X TB / 5 TB * $150. You might have to rent space in someone elses datacenter to put your own external backup... or you could pay Amazon for Glacier and not deal with maintenance etc. Might be worth it even if you have infrastructure for all data that isn't glacier-cold.
Lots of businesses have data retention requirements, and it can be difficult and time consuming to make sure this data is backed up in a way that is secure and can survive a catastrophe.
The author's use case (and most other personal use cases) might not be a good fit for Glacier, but he's not the target market.
Not all products are for all people. If you foresee a need to recover a large amount of data all at once, then glacier's not for you. If you might occasionally need a filing from 6 years ago, then glacier would be great.
Amazon starts to charge you extra anytime you exceed restoring 5% of the data.
If for example you save all your tax-related documents in Glacier, then you are audited then the accounts department or the government will want all the information. Not 5% of it. Not 10% of it. Everything. At that point Amazon will have you over a barrel, because getting the data out at a reasonable time frame will cost exponentially more than dripping out the data over the course of 20 months.
Are you sure about that? I haven't worked with tax litigation specifically, but I've worked with e-discovery w.r.t. e-mail and I can assure you that no one ever asks for all the e-mails sent by a particular company over all time. It's always a matter of asking for the e-mails sent by or received by a select group of people, over a fairly discrete time period. For something like this, a Glacier store might make sense, if it was coupled with an online metadata cache stored in e.g. S3.
The government basically comes to you and says they think you owe X, and you have to prove that false to their satisfaction. The more data you give your CPA to work with, the better.
Legal issues, at least in the US, do not have those requirements.
I have had situations in the past 12 months where recovering past data would be worth >$1k for the right 100k of data.
Isn't one way to get past this... increasing your data usage by 20x? If OP used less than a $1 a month, then if he uploaded $20 of junk data, he can get the 5% original data back "for free". Sure, it's $20, but it beats out $150+.
Though it's hard for me to imagine having so many categories of unrelated, useful and important data.
I'm surprised that this aspect has not been mentioned here in the comments yet:
> I was initiating the same 150 retrievals, over and over again, in the same order.
This was the actual problem that resulted in the large cost.
At my old job we would get a lot of complaints about overage charges based on usage to our paid API. It wasn't as complicated of pricing as a lot of AWS services, just x req / month and $0.0x per req after that, but every billing cycle someone would complain that we overcharged them. We would then look through our logs to confirm they had indeed made the requests and provide the client with these logs.
>> This was the actual problem that resulted in the large cost.
That is not true, according to the article itself. The first request for the full 60.8gb already results in a $154.25 bill, regardless of the ones that follow. From that point on he can continue to retrieve 15.2GB/hour for the rest of the month without incurring further costs.
This was the actual problem that resulted in the large cost.
Except that it wasn't. The repeated requests were free, because he already set the maximum rate with the first wave of requests. Surprising?
Also, it really is not a hit piece. It's an honest report of what he did (and what he did wrong) and that he thinks the docs aren't as clear as they could be.
EDIT removed material that might have derailed things further.
I don't even use my collection now that I have Spotify Premium. The only music I've bought lately is some 24bit high bitrate stuff.
This is known as "Format Shifting" — taking one copyrighted medium and converting it to another. In Australia, you are explicitly not allowed to do this with CDs, DVDs and Blu-rays.
You are only allowed to keep a digital copy if you continue to retain the original — a backup. If the original is lost or destroyed, your digital copies must be discarded.
For example, you can rip a CD and put it on your iPod, or computer, as long as you continue to own the CD. The issue here is that in both cases you also control the device you are copying it to. You don't control but rather lease space on Amazon's servers — so it introduces a grey area on whether you are allowed to backup to such places and whether putting data on those servers constitutes distribution of the copyright material.
Realistically, none of this is black-and-white and Amazon could flag it as infringing content and remove it just to cover themselves against DMCA complaints anyway. This is true in both Australia and the US, regardless of the differences in copyright law (Australian copyright law offers far fewer protections than the US, incidentally) because both having similar DMCA laws.
http://www.lifehacker.com.au/2013/01/format-shifting-101-wha...
Also backed up by: http://copyright.com.au/about-copyright/exceptions/ - which states we're allowed to "space shift" music.
And then posting this fact to hacker news. Not the brightest bulb in the pack.
1) Buys music CD. 2) Rips CD to own computer. Shares with own devices [via personal cloud] for continued listening. 3) Destroys original CD. 4) Continues to listen to music which has been paid for; not sharing files with anybody else.
I for one fail to see the problem here. But then again, I'm probably not the brightest bulb in the pack either.
It would be easier to download pirated copies off the internet and exactly as legal.
Mine are all in a box shoved against the wall under my bed, where they don't get it the way but I can prove that I still deserve consideration under the First Sale doctrine.
[1] https://therub.org/2015/11/18/glacier-costlier-than-s3-for-s...
And, there is no "transition from Glacier to S3": if you want to do that, you have to:
1. restore it to S3 (and incur the fees and 4-hour wait) 2. copy the restored S3 object to a new S3 object 3. delete the restored object (or wait for it to timeout)
Yes, the docs are imperfect (and were likely worse back in the day). And it was compounded by the bug, apparently. But it's what everyone on HN has learned in one way or another... RTFM.
Was it mentioned in the article that the retrieval pricing is spread over four hours, and you can request partial chunks of a file? Heck, you can retrieve always all your data from Glacier for free if you're willing to wait long enough.
And if it's a LOT of data, you can even pay and they'll ship it on a hardware storage device (Amazon Snowball).
Anyone can screw up, I'm sure we all have done, goodness knows I have. But at the very least, pay attention to the pricing section, especially if it links to an FAQ.
Paying 87c a month for a couple of years, then 52c a month for a few more years, to back up 60+GB, and then getting a one off fee of $150 still averages out at around $2/mo. Hardly getting shafted.
It's like getting a parking ticket for parking somewhere you've unknowingly been parking illegally for years. Yeah, if you average it out, it's cheap. But having to stump up the cash still hurts.
I guess he always had Linus's backup strategy open to him - "Only wimps use music backups: real men just upload their important tunes on Bittorrent, and let the rest of the world seed it ;)"
A couple grand? 500GB hdds have been less than $100 for a long time now, certainly since before Amazon Glacier was a product.
Just because you can formulate a large valuation for something doesn't mean that it's a reasonable valuation.
Even $10/cd is a ridiculous estimate of the cost of blank CDs. If you insisted on using blank CDs for backup, you could do 200 CDs for $40.
Edit: And what are CDs... 640mb? that's like 12 CDs, not 200... I'm realizing that I've fallen into a troll trap...
Meta: I guess I'll leave this comment up as a cautionary tale.
I bought my ~3000 disc CD collection here in Australia, many of them still have price stickers of $30 and more on them. While CDs were "a thing" here, I figure I spent about a quarter of a cheap house on them. (I don't regret a cent though. I'm also "that guy" who thinks "You only had 60Gig of music to back up? Wow!", but is mostly socially-aware enough to keep that sort of reaction to himself.)
Wouldn't 2 x 100GB Blu-ray discs be better for the same price? Less physical storage space required and less time to burn the data.
If you want more than 5% of your data in a month, the minimum it will cost you is $0.011 per gigabyte. Click here to see how to work out how much it will cost.
It's certainly not unreadable, although to be fair, it's nowhere near as clear as it should be.
'Use Amazon Glacier, it costs about a cent!' is the essential pitch.
As far as I see, in all other places they specify their pricing "per GB", and only this small phrase uncovers real meaning, which is not per actual GBs you transferred, but your peak rate multiplied by number of hours in month. IMHO this should be one of the first phrases describing the pricing model.
Well, it was that and also the docs being knowingly and deliberately set up to trick incautious readers. The user does have a responsibility to read the fine print, but that doesn't excuse Amazon being openly evil about it. This is no different than ISPs that advertise "up to 50Mb/s" when they know very well that their network won't deliver more than 5.
Snowball currently supports importing data to AWS. Exporting data out of AWS will be supported in a future release.
They could be running Glacier storage at cost (or even a slight loss).
But they make their profit when you try to get your data out.
I'm not implying anything nefarious - more along the lines that Amazon (could have!) looked at the market and compared demand to what they were offering. Then found something that scratched an itch...
They didn't just set the prices (or the pricing model) to amuse themselves.
Disclaimer: I work on Compute Engine.