What Backblaze is doing here is so underrated. This a large scale, practical, in-datacenter real data on essential hardware infrastructure that is available almost nowhere else, and they provide it, and their excellent analysis, completely for free.
I miss this culture and I admire leadership that allows it to not only exist, but thrive. I fear the day a stockholder meeting occurs and someone wringing their hands see the decommissioned pennies they can save by limiting or stopping these reports.
While I find this data interesting it isnt usually very actionable.
The skus with the lowest number immediately get bought out(if they are still available, which they are not always) and will never be available. You also always run the risk of "getting a bad batch" or just getting some drives that got beat up in shipping.
Usually this data is only useful for keeping an eye on your own stuff and prioritizing replacements when the time comes.
When buying drives I just look at the sizes I need and the performance then get 1/3rd from each of the manufacturers.
Any sort of long term testing is like this. You can't know what the long term reliability of something is when you buy it. You can estimate from reliability of similar items made in the past, but even if you bought some of everything and kept it on the shelf for X years and then only used the best, the stuff aged on the shelf.
Reports like this might help drive planning for failures. It might also help validate your experience if you've had a bunch of failures with some model and they have too.
IIRC, there have been a couple models that seemed to hit a big bathtub curve style end of life (I think 6TB drives in particularl); that could be a pre-failure indicator for you if you have that model.
Otherwise, yeah, mostly not actionable, but very nice to see the data.
> When buying drives I just look at the sizes I need and the performance then get 1/3rd from each of the manufacturers.
This is a good plan, you should avoid most correlated failures from firmware and manufacturing (although there's a lot of shared supply chain, so you might not avoid all correlated failures if some common component was made improperly during a long enough time period that all three drive makers would be using it in your purchase).
I've been following this report for many years, but Backblaze, as a backup service (traditionally), has very different IO patterns than many users. They originally started with consumer drives, which we found to be far too unreliable. In my experience, the BER and write cycles have a dramatic impact on overall drive performance. The MTBF declines sharply as write cycles increase, both as a percentage of IO and overall IO.
Backblaze changed IO patterns with B2, but that would be the key data for me to make this more useful: failure rate as a percentage of bytes read/written, etc.
My takeaway... The specific model plays a huge role in the failure rate.
A great model has a MTBF of 250 years.
A bad model might have a MTBF of just 5 years.
I suspect if you had a need for reliable storage which couldn't be met with the usual RAID approach, buying 2nd hand drives from eBay of a model and batch proven to be really reliable is probably your best bet.
7 comments
[ 0.25 ms ] story [ 35.3 ms ] threadI miss this culture and I admire leadership that allows it to not only exist, but thrive. I fear the day a stockholder meeting occurs and someone wringing their hands see the decommissioned pennies they can save by limiting or stopping these reports.
Hopefully not, given the performance one was just newly added!
The skus with the lowest number immediately get bought out(if they are still available, which they are not always) and will never be available. You also always run the risk of "getting a bad batch" or just getting some drives that got beat up in shipping.
Usually this data is only useful for keeping an eye on your own stuff and prioritizing replacements when the time comes.
When buying drives I just look at the sizes I need and the performance then get 1/3rd from each of the manufacturers.
Reports like this might help drive planning for failures. It might also help validate your experience if you've had a bunch of failures with some model and they have too.
IIRC, there have been a couple models that seemed to hit a big bathtub curve style end of life (I think 6TB drives in particularl); that could be a pre-failure indicator for you if you have that model.
Otherwise, yeah, mostly not actionable, but very nice to see the data.
> When buying drives I just look at the sizes I need and the performance then get 1/3rd from each of the manufacturers.
This is a good plan, you should avoid most correlated failures from firmware and manufacturing (although there's a lot of shared supply chain, so you might not avoid all correlated failures if some common component was made improperly during a long enough time period that all three drive makers would be using it in your purchase).
Backblaze changed IO patterns with B2, but that would be the key data for me to make this more useful: failure rate as a percentage of bytes read/written, etc.
A great model has a MTBF of 250 years.
A bad model might have a MTBF of just 5 years.
I suspect if you had a need for reliable storage which couldn't be met with the usual RAID approach, buying 2nd hand drives from eBay of a model and batch proven to be really reliable is probably your best bet.