It looks like an amount of space hard to fill with meaningful data. I wonder if they would prevent people from filling it with random junk just for fun.
The fact that the offer at its minimum level (which is only free for 1 month, its an "up to 6 months" free deal) requires a commitment to maintain at least 1PB in the system for 12 months (which means you are on the hook for paying for it for the non-free portion of that commitment period), is probably a constraint on sending in junk data just for fun.
As is the fact that you have to migrate the data from non-Google servers -- either your own or some other cloud provider -- to qualify, since you're going to pay for the sending the data, junk or otherwise, you are dumping into it.
Cold storage is absolutely a necessity when you reach the scale of imgur, unless you want to run the service incredibly inefficiently.
imgur keeps photos forever [0]. To do so without some kind of cold storage infrastructure in place for all the millions of photos that will probably never be seen again, or perhaps at a rate of once per year, would be infrastructure cost suicide.
Facebook faces a similar challenge and has gone in to detail about how it developed a cold storage strategy to deal with it[1].
I would say that for a service like imgur there is probably a level of parity between the amount of importance they need to place on content delivery and on cold storage.
Only hard to fill if you're dealing with text only. Say you're building a video copyright detection crawler and you want to store videos from all over the internet.
You can't upload this data from one place, but if you have lots of installations of something, each of which is independently generating a lot of data, it's feasible.
From the details (in an image, some in fine print on the image) on the offer: that's if you migrate at least 1PB into Google Cloud Nearline in the first 3 months, and maintain at least 1PB in Google Clound Nearline for at least 12 months.
And its only 1 month free at that level of commitment; anything more than 1 month requires a migration commitment to more than the 1PB.
EDIT: At the time this comment was written, the thread title referred to the 100PB free for up to 6 months offer, not the fact that Nearline was now in GA. While the content of the comment is still true, the title that is being clarified by the comment is no longer present. (Similar things seem to be true of a number of the other top level comments.)
Once the free period is over the 100PB will cost $1 million per month in storage fees alone. Will the free month(s) even cover the retrieval and transfer fees to take 100PB out of another cloud?
To store 100PB, one can always buy 50.000 hard drives. Would only cost $2-3 million of initial payment. I wonder how much maintenance would cost to provide same availability with near storage.
6 months of free storage is a no brainer business strategy. Once you get anywhere near 100PB of data loaded, are you really going to move it elsewhere after 6 months?
We (rsync.net) maintain 's3cmd' and 'gsutil' in our environment, so you can do direct cloud <--> cloud transfers from either amazon or google to and from rsync.net.
We don't have a cold storage product, so perhaps not relevant for nearline ... but it is relevant for their online options. You do have the ability to migrate to another provider.
Two random notes:
- we're announcing zfs send/recv[1] as a transport in the next few days and there will be s3-competitive pricing for this product at TB+ levels.
- "HN readers discount" is still available all these years later. Just email.
If you have tons of TBs, you don't want to care about random failure with some third party. Amazon S3 has the reputation that they never lost a file, AFAIK. On their scale that provides an argument to just fork over the cash and forget about it.
While I think ZFS is great, it's just not a solution for really large storage.
Also, if you are still using s3cmd, that only tells me how inexperienced you are.
I am absolutely blown away with the 100 PB for every customer... Meaning all together they must have somewhere in the exabyte to zettabyte range or are completely betting on the fact that no one will be able to/actually be willing fill this/their cloud. I guess I am just not used to the fact that on average most people have at least a 1 TB hard drive or hard drives that add up to 1 TB...
> I am absolutely blown away with the 100 PB for every customer...
Its 100PB free for 1 month, if you commit to transfer in at least 1PB in the first three months, and commit to keep at least 1PB for 12 months (Its "up to 6 months" free of 100PB, but more than 1 month free requires greater than 1PB commitment.) At $0.01/GB/mo, 1PB of data for 11 months (12 months minus 1 month free) is a $110k commitment.
Its not "100PB for every customer", at least not free, its a variable size discount that covers up to 100PB for 1-6 months of a 12 month commitment period for customers able to make up-front commitments of $110k or more.
1. It's only free for "up to" 6 months, and a client that moved 100 PB is not wanting to migrate away after 6 months merely to play on the free temporary space (at $0.01 per GB per month, that's what a million bucks a month starting on the 7th month ? That's a great incentive to give it free as a starter)
2. It's only if you upload at least 1 PB of data in the first three month, which removes a lot of companies and only leaves the serious heavy data potential customers
Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 13x premium to pull data out.
For instance, lets say you have a 50GB backup that you automatically verify via a testing process. You don't want to burden the disks of your primary datastore longer than you have to [the source of the backup] so your workflow is:
Database Server -> Create Backup Locally -> Push Backup to Object Store Bucket [$.50/month to store the backup, assume you store 7 days] -> Spin up VM which pulls from Object Store Bucket and verifies the backup [$.06/day for an 4GB Linode for an hour to do this which is plenty of RAM and time ] -> Pull the Backup & Verify [$.50/Day]
30 Days Backup Cost:
$20.30/month to maintain a week of verified backups with RunAbove = (7 * 50 * .01) + (30 * .06) + (30 * .01 * 50)
$200.30/month to maintain a week of verified backups with Nearline = (7 * 50 * .01) + (30 * .06) + (30 * .13 * 50)
I don't know why anyone would willingly pay an order of magnitude more for cold storage.
If I am reading the pricing page[1] correctly, you still get hit with the egress charges on top of the $0.01/GB charge to retrieve from Nearline. Thus, for less than 1TB of data retrieval is $0.13/GB ($0.01 Nearline charge + $0.12 egress charge) if you are not in China or Australia.
Yeah, that is how I read the pricing page as well and its consistent with how all other Google and Amazon products of this nature work.
The bandwidth cost is simply too high. They really need a "low quality" v. "standard quality" option. Plenty of people sell bandwidth at $.01-$.04 a GB with 99%-99.99% availability
However, the egress charge only applies to data moved off Google Cloud Services. If you use a Google compute instance to verify the backup (same size as the Linode at $0.05/hr), the cost becomes:
You forgot disk pricing for apples to apples. You also lose multi-provider redundancy [e.g. Everything must be on Google cloud or you are maintaining multiple builds of your production database machines ]
Are there any well accepted theories as to how these (Nearline & Glacier) systems function? I know with Glacier, I've heard powering down hard-drives, advanced compression (explains 3hr retrieval), and blu-ray storage (explains 3 month minimum).
With Nearline bringing access down to 3secs for the same cost, and now with On-Demand IO, I really don't know what to speculate. Assuming I/O is limited due to spread across media, how could that be scaled up on-demand?
(disclosure: am work for Google).
Not sure on AWS.
Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition:
- Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler.
- Google Compute Engine is a VPC out of the box, even cross-region. You can carve out your own sub-VPCs easily using firewall rules.
- Traffic from Google to end-user is not being dumped off of Google network as soon as possible. Google carries your traffic as close to the end-user as possible.
But you are much more expensive than S3 & CloudFront when it comes to request pricing
Cloud Storage considers retrieving an object through its HTTP url is considered a class B XML request type which is priced at
$0.01/10,000 ops. This is 1.3x-2.5x more expensive than cloudfront and s3 respectively. This is also not well documented and caused me some trouble. I didn't expect HTTP GET requests to be counted as XML API requests.
What did you expect GET requests to be counted as then? As an aside, I think it's fairly well documented: https://cloud.google.com/storage/pricing#operations-pricing (lists "GET Object" under class B) [but disclosure: I work for Google]
It did not occur to me that HTTP GET will be considered as an XML API operation. So I didn't expect HTTP GET requests to be charged because everywhere I read (including that page you listed) the pricing is for XML API operation.
That + it being more expensive than cloudfront and s3 came as a rude shock :(
In general I find google cloud documentation and the service much better and more pleasant to work with than AWS but in this case it is not. Both S3 and Cloudfront have much clearer pricing (and positioning). In network pricing,
both S3 & Cloudfront are significantly cheaper
“Networking is a red alert situation for us right now,” explained Hamilton. “The cost of networking is escalating relative to the cost of all other equipment. It is Anti-Moore. All of our gear is going down in cost, and we are dropping prices, and networking is going the wrong way. That is a super-big problem, and I like to look out a few years, and I am seeing that the size of the networking problem is getting worse constantly. At the same time that networking is going Anti-Moore, the ratio of networking to compute is going up.”
it i kind of topologically clear - with linear growth of the number of nodes, the fabric network architectures have to grow more than linear (and you have to have fabric if you'd like to have performance growing linear with number of nodes - otherwise there wouldn't be any sense in adding nodes after some point)
To me, "SLA" is an enterprise oriented term, which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind.
99% for a typical non-scaling enterprise app is crap.
Because at least part of the target market for this product cares about quantifying the guarantee, whatever it is.
> To me, "SLA" is an enterprise oriented term,
I think that's excessively narrow in terms of who cares about it, but Nearline is clearly in large part an enterprise-targeted offering, so, even if it was a purely enterprise-oriented term, its appropriate.
> which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind.
Knowing expected failure characteristics can be an important input to intelligently architecting with failure in mind.
> 99% for a typical non-scaling enterprise app is crap.
Nearline isn't an app, its one of a set of closely related storage offerings that, by design, would usually be used in coordination with each other and possibly other storage systems by an app. For its role in that stack, 99% doesn't seem immediately unreasonable, to me.
I am a bit puzzled by the lack of durability/data loss statistics.
In comparison, Amazon Glacier states that it "is designed to provide average annual durability of 99.999999999% for an archive."
57 comments
[ 4.1 ms ] story [ 56.7 ms ] threadAs is the fact that you have to migrate the data from non-Google servers -- either your own or some other cloud provider -- to qualify, since you're going to pay for the sending the data, junk or otherwise, you are dumping into it.
Cold storage is absolutely a necessity when you reach the scale of imgur, unless you want to run the service incredibly inefficiently.
imgur keeps photos forever [0]. To do so without some kind of cold storage infrastructure in place for all the millions of photos that will probably never be seen again, or perhaps at a rate of once per year, would be infrastructure cost suicide.
Facebook faces a similar challenge and has gone in to detail about how it developed a cold storage strategy to deal with it[1].
I would say that for a service like imgur there is probably a level of parity between the amount of importance they need to place on content delivery and on cold storage.
[0] https://help.imgur.com/hc/en-us/articles/201476457-How-long-...
[1] https://code.facebook.com/posts/1433093613662262/-under-the-...
100 PB = 102400 TB = 104857600 GB = 107374182400 MB.
You have 365/2 * 24 * 3600 seconds in 6 months, that's 15768000 seconds.
So you need to upload ~7 GB/sec, that's 60 Gb/sec constant upload.
And its only 1 month free at that level of commitment; anything more than 1 month requires a migration commitment to more than the 1PB.
EDIT: At the time this comment was written, the thread title referred to the 100PB free for up to 6 months offer, not the fact that Nearline was now in GA. While the content of the comment is still true, the title that is being clarified by the comment is no longer present. (Similar things seem to be true of a number of the other top level comments.)
So the bigger news here is Google Nearline Storage graduating to general availability.
We don't have a cold storage product, so perhaps not relevant for nearline ... but it is relevant for their online options. You do have the ability to migrate to another provider.
Two random notes:
- we're announcing zfs send/recv[1] as a transport in the next few days and there will be s3-competitive pricing for this product at TB+ levels.
- "HN readers discount" is still available all these years later. Just email.
[1] over SSH
While I think ZFS is great, it's just not a solution for really large storage.
Also, if you are still using s3cmd, that only tells me how inexperienced you are.
I'm not sure you understand how s3cmd is being used in the above context ... the idea is this:
Also, we've been doing this longer than Amazon has.Can we have a brief (one or two sentence) summary of what people should be using instead?
Its 100PB free for 1 month, if you commit to transfer in at least 1PB in the first three months, and commit to keep at least 1PB for 12 months (Its "up to 6 months" free of 100PB, but more than 1 month free requires greater than 1PB commitment.) At $0.01/GB/mo, 1PB of data for 11 months (12 months minus 1 month free) is a $110k commitment.
Its not "100PB for every customer", at least not free, its a variable size discount that covers up to 100PB for 1-6 months of a 12 month commitment period for customers able to make up-front commitments of $110k or more.
2. It's only if you upload at least 1 PB of data in the first three month, which removes a lot of companies and only leaves the serious heavy data potential customers
That is a pretty steep hit when you can use a lower quality object store:
https://www.runabove.com/storage/object-storage.xml
For $.01/GB and $.01/GB to pull.
Yeah, it may not be as highly available as Nearline but for the 99% of the time it is available...you don't have a 3 second delay and you aren't paying a 13x premium to pull data out.
For instance, lets say you have a 50GB backup that you automatically verify via a testing process. You don't want to burden the disks of your primary datastore longer than you have to [the source of the backup] so your workflow is:
Database Server -> Create Backup Locally -> Push Backup to Object Store Bucket [$.50/month to store the backup, assume you store 7 days] -> Spin up VM which pulls from Object Store Bucket and verifies the backup [$.06/day for an 4GB Linode for an hour to do this which is plenty of RAM and time ] -> Pull the Backup & Verify [$.50/Day]
30 Days Backup Cost:
$20.30/month to maintain a week of verified backups with RunAbove = (7 * 50 * .01) + (30 * .06) + (30 * .01 * 50)
$200.30/month to maintain a week of verified backups with Nearline = (7 * 50 * .01) + (30 * .06) + (30 * .13 * 50)
I don't know why anyone would willingly pay an order of magnitude more for cold storage.
[1] https://cloud.google.com/storage/pricing#storage-pricing
The bandwidth cost is simply too high. They really need a "low quality" v. "standard quality" option. Plenty of people sell bandwidth at $.01-$.04 a GB with 99%-99.99% availability
(7 * 50 * .01) + (30 * .05) + (30 * .01 * 50) = $20.00/month
https://cloud.google.com/compute/pricing#localssdpricing
$0.113
So its:
[apples to apples; local ssd]
$186.95 = (7 * 50 * .01) + (30 * .05) + (30 * .1213 * 50)
[persistent provisioned ssd]
$54 = (7 * 50 * .01) + (30 * .05) + ( ( ( (.17 * 96) / 720) + .01) * 50 * 30)
Do they have fixed costs or is this one of the last ways they have to to make money on their platforms?
Google's Network is a vast, secure, and performant Global SDN. This allows us to do some cool things. Therefore, Google has a fundamentally unique value proposition:
- Traffic between zones/regions at Google never leaves Google Network. Therefore, no need to setup VPC or VPN between zones/regions. Makes deployments much much simpler.
- Google Compute Engine is a VPC out of the box, even cross-region. You can carve out your own sub-VPCs easily using firewall rules.
- Traffic from Google to end-user is not being dumped off of Google network as soon as possible. Google carries your traffic as close to the end-user as possible.
Edit: take a look at this as well.. http://googlecloudplatform.blogspot.com/2015/06/A-Look-Insid...
Cloud Storage considers retrieving an object through its HTTP url is considered a class B XML request type which is priced at $0.01/10,000 ops. This is 1.3x-2.5x more expensive than cloudfront and s3 respectively. This is also not well documented and caused me some trouble. I didn't expect HTTP GET requests to be counted as XML API requests.
That + it being more expensive than cloudfront and s3 came as a rude shock :(
In general I find google cloud documentation and the service much better and more pleasant to work with than AWS but in this case it is not. Both S3 and Cloudfront have much clearer pricing (and positioning). In network pricing, both S3 & Cloudfront are significantly cheaper
“Networking is a red alert situation for us right now,” explained Hamilton. “The cost of networking is escalating relative to the cost of all other equipment. It is Anti-Moore. All of our gear is going down in cost, and we are dropping prices, and networking is going the wrong way. That is a super-big problem, and I like to look out a few years, and I am seeing that the size of the networking problem is getting worse constantly. At the same time that networking is going Anti-Moore, the ratio of networking to compute is going up.”
http://www.enterprisetech.com/2014/11/14/rare-peek-massive-s...
Amazon has Import/Export which lets you ship drives, is this the best option?
50 TB / 0.000000625 TBps = 80000000 seconds = 925.92 days
AWS Import can probably import at 100MBps or ~6 days.
[0] https://cloud.google.com/storage/docs/early-access
To me, "SLA" is an enterprise oriented term, which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind.
99% for a typical non-scaling enterprise app is crap.
Because at least part of the target market for this product cares about quantifying the guarantee, whatever it is.
> To me, "SLA" is an enterprise oriented term,
I think that's excessively narrow in terms of who cares about it, but Nearline is clearly in large part an enterprise-targeted offering, so, even if it was a purely enterprise-oriented term, its appropriate.
> which a lot of folks (us) don't care about because we're scaling horizontally and architect with failure in mind.
Knowing expected failure characteristics can be an important input to intelligently architecting with failure in mind.
> 99% for a typical non-scaling enterprise app is crap.
Nearline isn't an app, its one of a set of closely related storage offerings that, by design, would usually be used in coordination with each other and possibly other storage systems by an app. For its role in that stack, 99% doesn't seem immediately unreasonable, to me.