I loaded 5 tonnes of sand into my 4runner and now it's ruined, Toyota claims a 4Runner is not a 5 tonne truck won't give me my money back or honor the warranty, I swear the mechanics were laughing at me.
BB handles many PBs of data. Actually I think it's ZBs.
However, I'm not sure how many of their clients have single drives with 17TB of data. Perhaps this is an edge case that they didn't adequately test for.
And since a vast majority of their customers will not have more than 3TB on any single drive, I can see why this might not be well tested.
i think you meant exabytes, not zettabytes, and as they state on their website "350 million GB stored ... (and counting)" that means there at about 1/3 of an exabyte now
Yev from Backblaze here -> we have customer for Backblaze Online Backup with over 40TB of data, and customers with 100s of TBs if not Petabytes on B2 Cloud Storage, so we can certainly handle it.
Not sure what happened in this particular case as I'm not familiar with it, but I'd venture to guess that something occurred on the computer that caused the client to think it needed to re-upload those 13TB of data, so it scheduled them for re-upload.
And as I recall, BB charges something like $5/mo for unlimited backups. The fact that they won't refund him is both dumb on their part (because really, what is $5) and pointless to mention on his part (because really, how is $5 going to compensate him?)
Yev from Backblaze here -> we did offer them a refund when they cancelled their account. In general we have a pretty generous refund policy for our yearly and bi-yearly plans, but giving refunds for the $5-rate, is a bit cumbersome.
I just wonder what someone could have 17TB at home of. In 2010 the Library of Congress text database was 20TB. I know videos can take up some room but do people back up their video libraries to the cloud?
Well according to another post[0] "Breaking bad in 4k is 1.7Tb" or "star wars 200gb for first 6 movies. thats just 1080p". So I can imagine that much data being saved. Also, the subreddit is called datahoarder, so I imagine gathering bytes is important and "easy".
if you have 17TB of data. Where is the best place to back up that much data? Other then redundant back up drives... Is there a suitable cloud solution for this? If not is there enough demand to build one?
I dont own that much data. I cant imagine having to keep track of that much data, I hate having more than 1 TB of data to keep track of.
Backblaze and Crashplan both have (or had) a thing where they'd mail you a large hard drive to load your data on for a deposit of a few hundred dollars. Although at 13 TB I guess you'd still need to prioritize what you really wanted backed up straight away.
The best thing to do is probably to buy an NAS, seed it with your 17TB of files and then colocation it at a colo facility or a friend's house and then use some sort of one way sync utility (rsync, rclone, carbon copy cloner, bittorrent sync, etc) to update the remote NAS as you add/delete files to the source.
This is the way I would go took, although with two NAS boxes. One local and a similarly sized one off site. Then you backup to the local one and you sync the state to the remote one.
Yev from Backblaze here -> Cloud Storage services are better suited for that much data. I wrote a post recently where I went in to the differences between Sync, Backup, and Storage as it relates to the cloud -> https://www.backblaze.com/blog/sync-vs-backup-vs-storage/.
I'm not sure what happened in this user's case - but my guess is that something happened on their machine where our backup client decided that data on the machine didn't match with what is in the DC and restarted backing up the data so that we'd have a clean copy, resulting in 13TB of data getting re-scheduled for backup.
I wouldn't say that, it's just that I don't have a lot of data about this particular case to go on. In some cases if you overwrite our index files, you can get something that's known as a Safety Freeze. Again - I have no idea what happened with this customer, but I would venture to guess that since this occurred on their C: drive, something similar to what causes a Safety Freeze (https://help.backblaze.com/hc/en-us/articles/217666178-Safet... happened here, and they needed to re-upload the data (in most cases, previously uploaded data is still available for restore - but with 13TB of data to re-upload and not a lot of bandwidth, it would have taken them way longer than our 30-day history to do that).
So much facepalm here. A Backblaze rep is in that thread confirming that they do have a failure case where they can lose backup data -- or at least, it'll become inconsistent. Ok, that's bad. I don't think they're idiots, but it'd be interesting to know what's so technically challenging about the index problem that they haven't been able to solve it.
But ... 17 TB total? For online backups? That's just not what they're designed for, unless you've got one hell of an internet connection. The complainant says it's going to take over a year to re-upload the 13TB. That's insane.
I think you have to give up on online backups for that use case and settle for a pretty beefy home rig in a fire-resistant box. Moving that kind of data up and down an internet connection just doesn't work real well yet.
I don't have any kind of special connection, but when I switched computers had to go through the process of deleting my entire online backup in order to use my new machine. Took a few weeks (not a year) for 5TB from scratch, all the while on pins and needles hoping nothing happened to my local disk.
I love Backblaze, but their confusing and weird policy re: switching machines made for a nail biting and slow, painful process. After reading this, I'm now considering alternative offsite backups.
Yev from Backblaze here - as far as the technical aspects, it's tough for me to say b/c I don't know what happened in this case, but we do have use-cases where we need to "Freeze" backups (more info: https://help.backblaze.com/hc/en-us/articles/217666178-Safet...). I would venture to guess that since this occurred on their C: drive, something similar to that happened here, and they needed to re-upload the data (in most cases, previously uploaded data is still available for restore - but with 13TB of data to re-upload and not a lot of bandwidth, it would have taken them way longer than our 30-day history to do that).
The nature of the data isn't given, but ~20TB of video wouldn't be unthinkable. A video operation, backing up original masters for safe keeping, offsite is pretty reasonable for a variety of scenarios.
>>> It took 8 months or so for my initial backup and will definitely take over a year with the added data now [upload 13 TB again] so I just cancelled and they gave me a refund for the last payment.
>>> So, all in all just a waste of time and money.
Dude wanted to backup 18 TB of backup on a home connection until he realized it's not going well.
Messaged the customer support to complain => he got a timely reply, then decided to leave the service and got a refund.
That looks like good service to me.
P.S. I feel sorry for companies who have to deal with that kind of customers.
Seed drive service. ~20 hours to copy 10T of local data to a few drives, few days to ship, ~20 hours to copy the data to the cloud. But I guess some people think that cost is greater than 8 months of uploading? Whatever...not my data.
Let's just be real. It's assholes like this that take advantage of the buffet line, thereby costing all of the rest of us. And then they complain when they can't go back for the 45th plate of food.
People like this are why Amazon's unlimited cloud storage just became limited. The people out there abusing the spirit of the offer know who they are. The finance world is FULL of those people, and it's a good part of why the economic world is really fucked up.
Be a team player. Be thoughtful of your peers and neighbors. We'll all end up better off, even you. /rant
This is why I hate services that offer "unlimited" of some resource, like Dreamhost and AWS, so I will never use them. Everyone is worse off. People who use less than a ridiculous amount are paying for a few outliers using 100x more. People who use a ridiculous amount can no longer be sure whether their service will be active the next day because an employee deems their use "ridiculous".
And I don't think they're at fault like you're suggesting. Suppose I have a legitimate use case for storing 17TB of data and need to do it on a budget. During my shopping, I discover ten $1000/yr services for 20TB storage and three $100/yr services for unlimited storage. To me, the $100 services seem like scams or that they have no idea what they're doing. Amazon, Backblaze, and Dreamhost are literally giving off these vibes! Regardless, I'm on a budget so I purchase Backblaze and upload my 17TB. But in the back of my mind I'm nervous that they will shut off my service, despite the contract I entered with them to keep an "unlimited" amount of data for the price I paid. Or, if the contract says they have the right to terminate my service if they deem the amount to be too much, that would make nervous even as a normal user of the service, since the definition of "ridiculous" might be decided by some judge in a court if the service decides to cancel the contract and my company decides to sue. Would you really want a judge to decide that, or would you prefer a hard limit to be placed on the website and the contract in the first place? I truly don't understand the appeal of "unlimited" plans compared to simple hard limited, tiered plans.
> Suppose I have a legitimate use case for storing 17TB of data and need to do it on a budget.
Can you give me a theoretical example? Seems like saying, "Suppose I have a legitimate case for needing 45 trips to the buffet line."
I get your points, but I think outliers are outliers, and correspondingly they should be thinking as such. So if I had, for whatever reason, a need to store 17TB of data, I would need to be as clever about how I stored it as I obviously was clever enough to even possess 17TB.
There could be many reasons. In order to run simulations at my work, I request databases of input data and results of previous experiments. Each experiment is 1-100GB of xray images and dumps from other instruments. Mostly the experimenters dump the data they don't need, so often my team becomes the only possessor of the raw data. It would be convenient to have an off-site backup in case someone rm -rf's the drives.
When I was younger I worked at a recording studio. Each song consumed about 10GB, and it was convenient to return to the sources at a future date once the mix was delivered to the artist. Each artist recorded about 10 songs, and in a year we could do 50 artists.
I can't imagine what animation and video production houses go through per day. Having a backup is completely necessary for these folks. I could go on, but I feel 17TB is pretty common for businesses, so I'm not sure why you're asking the question. These days you can get 17TB for only $700.
I can give you a practical example. I do a lot of photography/videography in my spare time (not professional, have never been paid for it). I shoot RAW photos on a Sony A7RII. Every photo is ~85MB and I can shoot 700 photos in a day. Assuming I shoot one day every weekend, that's 3TB of data a year (that's actually a close estimate when I account for vacations and such too). Then there's video. I own a Panasonic GH5 and record on an Atomos Ninja Inferno, which outputs 4K 10-bit ProRes video at 60fps. This consumes ~1TB per hour and I tend to store it for archival purposes since it only costs ~$25 to do so for pretty much ever.
For a less extreme case, one of my old school teachers has been doing photography as a hobby for ~15 years. She's amassed ~10TB of photos over that time and currently stores them on piles of HDDs and a few spindles of DVDs scattered around her house.
And yes, I absolutely accept that I'm an outlier but any service that offers an unlimited plan is explicitly advertising itself to outliers. They're not putting themselves out there as someone with a 1TB, 2TB, 10TB or even "enough for 99.9% of people!", they're saying that they're giving storage with zero limits.
Your buffet comparison by the way isn't a good one. A buffet isn't truly unlimited, it has a time restriction, a condition that you can only eat what you take, you can't leave with it and the fact that a human being can only physically eat so much. They haven't defined a limit but there certainly is one.
How is he an asshole? He was sold the service, and he used it properly, the terms of service were followed. If anything, this story shows that Backblaze sucks at providing said service.
There's the letter of the law and the spirit of the law. People know the difference. And the few people who don't know the difference don't get a free pass for ignorance.
Unlimited bandwidth, free drinks, unlimited storage, etc., are all offered with some expectation of limits, with occasional spikes in use.
But let's back up a moment. 17TB is a lot of data. If you were a professional photographer or videographer, I could possibly imagine that need. Or if you were doing some truly big data work, or some financial algo backtesting (on a really grand scale), maybe. Those people should be able to afford storage prices in line with their amount of data.
Secondly, assume you lost your local drive. How are you going to restore 17TB? Whose network (that you're taking advantage of) are you going to saturate with your restore?
To lower that with a significant up front cost, you could buy the drives yourself, perhaps shucking external ones and then
send them over to Vapornode https://vapornode.com/storage-vps so they can host it at 10 USD a month per disk. If you have a Fry's close by, they have 8TB units for 200, recently Best Buy had them for 180 but I think that ended. So that's 200 + 10 / month vs 4.5 * 8 = 36 USD / month, you are saving 26 USD a month, it takes eight months for this to be cheaper. I would say it's not worth it because replacing dead disks are on you but it's up to you. Also getting many smaller disks make for better RAID.
Finally, lowendtalk regularly has discussions on bring your own disk providers. You need to know what you are buying but it's doable.
You can get lower than that even, if you need a lot of data.
Buy a used SC846 second hand, that costs ~$900 inclusive of rails and a SAS2 backplane upgrade. Then buy drives. You can get 8TB drives for ~$230 (Seagate Archive or shucked Reds) or $0.027/GB. Add cheap colocation for say $70/mo.
Assuming the hardware lives about 5 years, total cost is $900 + (70x12x5 = $4200) + num_gb x 0.027.
If you fill all the bays you're looking at 192TB for 5 years for $10284, or $171.4/month or $0.90/TB/mo.
I just checked out Time4VPS. Their storage VPS's have a 200 IO/s limit. At the typical 4K per IO, that's 800K bytes/sec, 1.6MB/s, or 2.1TB per month. It's a little misleading that they advertise a 10TB bandwidth limit per month on a 100Mbit/s port, when the IO limit kicks in way before that.
I wonder what the overhead is like on 13TB of data.
I have about 80GB with Backblaze right now -- just documents, photos and music on my laptop -- and the metadata it stores in /Library/Backblaze.bzpkg takes up 46GB, or about half the size of the actual backup. Most of this is in uncompressed plain-text log files showing every file which has been backed up, going back years. On a 256GB SSD that overhead hurts.
I asked their support, but their only solution was to delete the backup, reinstall the client, and start from scratch, which seems... less than optimal.
(On the other hand I do understand the impulse to squirrel away terabytes of data... I now make a point of archiving all of my mail accounts and PDFs of papers I read, knowing that with good search, they'll be valuable for years to come. But I haven't quite read 13TB worth of papers yet.)
I've stated this before in a previous thread, but I had previously used backblaze on a few computers but found it was taking up almost 8GB of space on my laptop for the backup list of files. This seemed very odd to me so i reached out to backblaze and the response was to make a new account to re-upload everything. Given the time it takes to re-upload everything (i didn't quite have 13 TB, but i had enough) i just terminated my account.
What the heck are you storing that you've used 20-30TB? I don't really get it.
UHD Movies? Why would you save those in the age of streaming? Family pictures/videos? At 20TB, you must have hundreds of thousands of pictures. There's not enough time in your life to look at all those! The only thing I can think of is that you shoot photo/video professionally, or maybe do some kind of 3D/video rendering?
But I agree with you - when you have this much data, why the hell are you transferring it to the cloud? Buy a couple harddrives, load them up, and stick them in a safe.
Who said I'm storing anything? A handful of machines with the cheap default spinning rust HDs in them and you get to 30tB very quickly. A typically family of three probably has double digits tB just in their Costco purchased laptops.
Heck, my parents have >10tB of raw storage between their home and work computers. Even though they aren't data hoarders, between software, important documents and email and videos and pictures of grandkids and friends they probably easily sit at 4 or 5 tB of total storage that deserves backup.
77 comments
[ 4.2 ms ] story [ 140 ms ] threadHowever, I'm not sure how many of their clients have single drives with 17TB of data. Perhaps this is an edge case that they didn't adequately test for.
And since a vast majority of their customers will not have more than 3TB on any single drive, I can see why this might not be well tested.
Not sure what happened in this particular case as I'm not familiar with it, but I'd venture to guess that something occurred on the computer that caused the client to think it needed to re-upload those 13TB of data, so it scheduled them for re-upload.
[0] https://www.reddit.com/r/DataHoarder/comments/6j06qp/how_do_...
17TB is a lot to send over the wire. If you've got a residential ISP connection it'll probably take months.
You don't want to deal with cloud on a home connection, it's gonna take forever.
If you are a company, you pay for storage on S3, Google Storage, OVH or IBM Softlayer. It's great quality yet simple and moderate costs.
Companies keeping data in house pay for NetApp, NetBackup and tape hardware.
Lose data locally, restore from local NAS
Lose data on local NAS, restore from remote NAS
Lose data on remote NAS, restore from local NAS
I'm not sure what happened in this user's case - but my guess is that something happened on their machine where our backup client decided that data on the machine didn't match with what is in the DC and restarted backing up the data so that we'd have a clean copy, resulting in 13TB of data getting re-scheduled for backup.
Not the cloud if you have a typical home internet connection
But ... 17 TB total? For online backups? That's just not what they're designed for, unless you've got one hell of an internet connection. The complainant says it's going to take over a year to re-upload the 13TB. That's insane.
I think you have to give up on online backups for that use case and settle for a pretty beefy home rig in a fire-resistant box. Moving that kind of data up and down an internet connection just doesn't work real well yet.
I don't have any kind of special connection, but when I switched computers had to go through the process of deleting my entire online backup in order to use my new machine. Took a few weeks (not a year) for 5TB from scratch, all the while on pins and needles hoping nothing happened to my local disk.
I love Backblaze, but their confusing and weird policy re: switching machines made for a nail biting and slow, painful process. After reading this, I'm now considering alternative offsite backups.
This isn't too much for online backup in general. 17t is not so bad in a lot of cases.
It might be too much for Backblaze. I don't know. I don't know if they can effectively back up at line rate. They couldn't for me.
3 days on a 500 mbit/second connection---which is about what you'd really be able to get if you had a 1-gigabit line.
It's fairly easy to get them in any US metro area so long as you are willing to spend. It's also percolated pretty well through out Japan and Korea.
>>> So, all in all just a waste of time and money.
Dude wanted to backup 18 TB of backup on a home connection until he realized it's not going well.
Messaged the customer support to complain => he got a timely reply, then decided to leave the service and got a refund.
That looks like good service to me.
P.S. I feel sorry for companies who have to deal with that kind of customers.
I mean it's not an ideal circumstance, but not the end of the world. He really just needs a faster connection to support that much data.
People like this are why Amazon's unlimited cloud storage just became limited. The people out there abusing the spirit of the offer know who they are. The finance world is FULL of those people, and it's a good part of why the economic world is really fucked up.
Be a team player. Be thoughtful of your peers and neighbors. We'll all end up better off, even you. /rant
And I don't think they're at fault like you're suggesting. Suppose I have a legitimate use case for storing 17TB of data and need to do it on a budget. During my shopping, I discover ten $1000/yr services for 20TB storage and three $100/yr services for unlimited storage. To me, the $100 services seem like scams or that they have no idea what they're doing. Amazon, Backblaze, and Dreamhost are literally giving off these vibes! Regardless, I'm on a budget so I purchase Backblaze and upload my 17TB. But in the back of my mind I'm nervous that they will shut off my service, despite the contract I entered with them to keep an "unlimited" amount of data for the price I paid. Or, if the contract says they have the right to terminate my service if they deem the amount to be too much, that would make nervous even as a normal user of the service, since the definition of "ridiculous" might be decided by some judge in a court if the service decides to cancel the contract and my company decides to sue. Would you really want a judge to decide that, or would you prefer a hard limit to be placed on the website and the contract in the first place? I truly don't understand the appeal of "unlimited" plans compared to simple hard limited, tiered plans.
Can you give me a theoretical example? Seems like saying, "Suppose I have a legitimate case for needing 45 trips to the buffet line."
I get your points, but I think outliers are outliers, and correspondingly they should be thinking as such. So if I had, for whatever reason, a need to store 17TB of data, I would need to be as clever about how I stored it as I obviously was clever enough to even possess 17TB.
When I was younger I worked at a recording studio. Each song consumed about 10GB, and it was convenient to return to the sources at a future date once the mix was delivered to the artist. Each artist recorded about 10 songs, and in a year we could do 50 artists.
I can't imagine what animation and video production houses go through per day. Having a backup is completely necessary for these folks. I could go on, but I feel 17TB is pretty common for businesses, so I'm not sure why you're asking the question. These days you can get 17TB for only $700.
So either the guy has a legitimate business reason (in which case he's definitely chosen the wrong service (level), or he's not using it for business.
I can give you a practical example. I do a lot of photography/videography in my spare time (not professional, have never been paid for it). I shoot RAW photos on a Sony A7RII. Every photo is ~85MB and I can shoot 700 photos in a day. Assuming I shoot one day every weekend, that's 3TB of data a year (that's actually a close estimate when I account for vacations and such too). Then there's video. I own a Panasonic GH5 and record on an Atomos Ninja Inferno, which outputs 4K 10-bit ProRes video at 60fps. This consumes ~1TB per hour and I tend to store it for archival purposes since it only costs ~$25 to do so for pretty much ever.
For a less extreme case, one of my old school teachers has been doing photography as a hobby for ~15 years. She's amassed ~10TB of photos over that time and currently stores them on piles of HDDs and a few spindles of DVDs scattered around her house.
And yes, I absolutely accept that I'm an outlier but any service that offers an unlimited plan is explicitly advertising itself to outliers. They're not putting themselves out there as someone with a 1TB, 2TB, 10TB or even "enough for 99.9% of people!", they're saying that they're giving storage with zero limits.
Your buffet comparison by the way isn't a good one. A buffet isn't truly unlimited, it has a time restriction, a condition that you can only eat what you take, you can't leave with it and the fact that a human being can only physically eat so much. They haven't defined a limit but there certainly is one.
Unlimited bandwidth, free drinks, unlimited storage, etc., are all offered with some expectation of limits, with occasional spikes in use.
But let's back up a moment. 17TB is a lot of data. If you were a professional photographer or videographer, I could possibly imagine that need. Or if you were doing some truly big data work, or some financial algo backtesting (on a really grand scale), maybe. Those people should be able to afford storage prices in line with their amount of data.
Secondly, assume you lost your local drive. How are you going to restore 17TB? Whose network (that you're taking advantage of) are you going to saturate with your restore?
We should be grateful these early outliers exist and break things for us in advance.
To lower that with a significant up front cost, you could buy the drives yourself, perhaps shucking external ones and then send them over to Vapornode https://vapornode.com/storage-vps so they can host it at 10 USD a month per disk. If you have a Fry's close by, they have 8TB units for 200, recently Best Buy had them for 180 but I think that ended. So that's 200 + 10 / month vs 4.5 * 8 = 36 USD / month, you are saving 26 USD a month, it takes eight months for this to be cheaper. I would say it's not worth it because replacing dead disks are on you but it's up to you. Also getting many smaller disks make for better RAID.
Finally, lowendtalk regularly has discussions on bring your own disk providers. You need to know what you are buying but it's doable.
https://www.backblaze.com/b2/cloud-storage-pricing.html
But it's very funny how it is practically the same 4 EUR a month a terabyte as the rent-a-server-for-its disks solutions.
Buy a used SC846 second hand, that costs ~$900 inclusive of rails and a SAS2 backplane upgrade. Then buy drives. You can get 8TB drives for ~$230 (Seagate Archive or shucked Reds) or $0.027/GB. Add cheap colocation for say $70/mo.
Assuming the hardware lives about 5 years, total cost is $900 + (70x12x5 = $4200) + num_gb x 0.027.
If you fill all the bays you're looking at 192TB for 5 years for $10284, or $171.4/month or $0.90/TB/mo.
Sia = $2/TB/month.
Amazon Glacier = $4/TB/month.
Time4VPS Storage = $4/TB/month.
Backblaze B2 = $5/TB/month.
Backblaze B2
I have about 80GB with Backblaze right now -- just documents, photos and music on my laptop -- and the metadata it stores in /Library/Backblaze.bzpkg takes up 46GB, or about half the size of the actual backup. Most of this is in uncompressed plain-text log files showing every file which has been backed up, going back years. On a 256GB SSD that overhead hurts.
I asked their support, but their only solution was to delete the backup, reinstall the client, and start from scratch, which seems... less than optimal.
(On the other hand I do understand the impulse to squirrel away terabytes of data... I now make a point of archiving all of my mail accounts and PDFs of papers I read, knowing that with good search, they'll be valuable for years to come. But I haven't quite read 13TB worth of papers yet.)
Crap, just between the main drives on computers in my house I easily have 20-30tB of raw disk space.
It might be a lot to transfer...yeah. At 50Mbps that's what...a month of sustained transfer? But how valuable is your data?
1 - https://www.newegg.com/Product/Product.aspx?Item=N82E1682217...
UHD Movies? Why would you save those in the age of streaming? Family pictures/videos? At 20TB, you must have hundreds of thousands of pictures. There's not enough time in your life to look at all those! The only thing I can think of is that you shoot photo/video professionally, or maybe do some kind of 3D/video rendering?
But I agree with you - when you have this much data, why the hell are you transferring it to the cloud? Buy a couple harddrives, load them up, and stick them in a safe.
Heck, my parents have >10tB of raw storage between their home and work computers. Even though they aren't data hoarders, between software, important documents and email and videos and pictures of grandkids and friends they probably easily sit at 4 or 5 tB of total storage that deserves backup.