lol my money is on Seagate.. I thought it amusing how they rebadged their crap drives as Samsungs when they bought the name because of their very good rep in the sector.
From what I understand, at this point all 3 major hard disk vendors have very similar failure rates. The thing to look out for is that certain runs of drives may be "contaminated" during the manufacturing process, but knowing which ones is pretty unpredictable.
Also some combinations of devices can create problems. We had a RAID controller that randomly rebuild mirrored sets because the drive was 'dead'. After the rebuild the drive seemed fine. We concluded there was some firmware delay that caused the controller to time out some operation and call the drive 'failed'.
We replaced the drives with a different brand and the 'failures' went away.
> We concluded there was some firmware delay that caused the controller to time out some operation and call the drive 'failed'.
There is indeed a firmware setting on how long the drive spends checking for errors after it detects one, during which the drive doesn't respond. Sometimes this is too long for the RAID controller, so it drops it.
It used to be you could buy WD Caviar Black drives and tweak that timeout setting on the controller to effectively have WD RE drives (enterprise version of the WD Black drives). They removed that "feature" a few years ago.
We are constantly looking at new hard drives, evaluating them for reliability and power consumption. The Hitachi 3TB drive (Hitachi Deskstar 5K3000 HDS5C3030ALA630) is our current favorite for both its low power demand and astounding reliability. The Western Digital and Seagate equivalents we tested saw much higher rates of popping out of RAID arrays and drive failure. Even the Western Digital Enterprise Hard Drives had the same high failure rates. The Hitachi drives, on the other hand, perform wonderfully.
In the second article they say that WD-RED are 2nd in reliability (WD-RED did not exist 2011). I'm happy that I've got a cheap Hitachi Ultrastar. But who knows.
As a personal anecdote: WD-Green failure rates are huge here. 24/7 Desktop machines, 240 drives. I've replaced in last 12 month at least 20 drives.
While WD has taken over Hitachi, they had to give Toshiba Hitachi's desktop 3.5" HDD operations in part of the deal of taking over Toshiba's (TSDT) Thailand factory. So if you want Hitachi's good 'ol reliability, buy Toshiba desktop drives.
Yev on Backblaze | We actually hope to see if we can collect enough data to release a "per manufacturer" report on had drive failure. We're currently collecting that data, but as always, follow the Backblaze Blog for info on that type of stuff!
Damn, my last two harddrives have failed in around 3 years exactly. Did I have bad luck or am I being too mean to them?
My computer is on mostly all the time and is often reading files throughout the night (for slow uploads for example). Does it make a difference how often you read/write a drive or only the spun-up time? One died suddenly without any warning in the SMART data and the other got badblocks and started to struggle reading data.
The data presented in the article only makes sense if you buy a large quantity of hard drives; if you only have had a handful then you were just unlucky.
I suspect the reason why people do "burn in" tests on hard drives is to make drives that suffer from early failure ("infant mortality" as described in the article) fail early enough that you can RMA them with the manufacturer. Apart from that, I don't think there's much you can do to improve your chances.
Backblaze may not know because they are "a company that keeps more than 25,000 disk drives spinning all the time". After 3-5 years, you'd better have a back-up of a drive you choose to spin down. Every drive I've lost (in the last 10-15 years and ignoring two failed controllers subject to a close lightning strike) failed to start back up when I had powered the machine off for maintenance.
I've bought perhaps 50 drives in the past 20 years, and maybe 10 of them died, the others mostly becoming obsolescent. I only started taking serious logs about 6 years ago.
Drives have died for me both in 24/7 powered systems and through power cycles. Drives have reported intermittent failures for many months, but still lived for years without any actual data loss. The oldest drive I still have spinning is a 200G IDE containing the OS for my old OpenSolaris zfs NAS; must be getting on for 9 years.
I advise having a back-up of every drive you own, preferably two. I built a new NAS last week, 12x 4G drives in raidz2 configuration; with zfs snapshots, it fulfills 2 of the 3 requirements for backup (redundancy and versioning), while I use CrashPlan for cloud backup (distribution, the third requirement). Nice thing about CrashPlan is my PCs can back up to my NAS as well, so restores are nice and quick, pulling from the internet is a last resort.
The one thing to know about cheap consumer cloud backup solutions like CrashPlan and Backblaze is that they only have one copy of your data. So if their RAID array where your data is stored dies and cannot be rebuilt, it's all gone. You can Google for a few disaster stories about both companies.
Like I said, they are only one third of my backup strategy. My house burning down, or someone breaking in and going into my attic to steal my 30kg 4U server, should be the only two realistic scenarios in which I will need to rely on CrashPlan.
I do same, but in reverse. I use cloud servers, with versioning backups, but still beam additonal backups back to office. Just to survive total data center destruction disaster.
Ideally in most circumstances you should have at least one cold (non-raid, non-connected) backup of all data, and an offsite one. More being much better.
Backblaze doesn't use RAID, they use blob-level replication. I'm sure they can lose data, the question is how likely they are to lose data simultaneous with you losing your HD and local backup.
One pure storage disk I use has been alive with top S.M.A.R.T. values for over 8 years now. The one with more regular reads & writes is 5 years old now. And while I have local backups I'm now finally in the progress of uploading 800GB of (semi-) important data to backblaze. I'll probably be done in another month.
It depends on the warranty period of the drive. Your hard drive manufacturer knows precisely how long the drive will last and sets their warranty period to expire right before your drive gives up the ghost.
Yeah, if that was true that would be completely against EU regulations,so you could sue manufacturers and win millions. So if you have any proof that they are doing that, you are practically a millionaire, I don't know why are you not on your way to the court yet.
How so? That is how warrantees work. They're designed to protect you against premature failure. Unless it's a necessary competitive advantage no company is going to warrantee something past its expected lifetime. And in that case it's going to be included in the price.
Designing a device to fail after a pre-determined amount is very much against the law. Expected lifetime is different, but this is not what the previous poster was suggesting.
The manufacturer has some good information about the statistical likelihood of your drive failing within a certain time period and tunes the warranty accordingly. They do not know exactly when your specific drive will fail as this would require some form of time travel.
I wonder how many will die before they age out of usefulness. If you're still spinning 250Gb hard disks which are using the same power and space that a 4Tb drive could be using - it might not actually be economically sensible to keep running them.
Certainly the old 9.1Gb SCSI disks that were so popular 10 years ago are well past be justifiable to give power to now.
I suspect the overheads of collection, shipping and distribution would dwarf the savings from not buying the optimal price/GB drive du jour.
Hard drive space per dollar grows exponentially, and they're big weighty things. The window of time where it would be economical to reuse would be short, and value dubious.
That's a horrible idea! The logistics to collect and ship the old tech to Africa alone would probably cost more then just buying it bulk from China and ship in one batch.
But there are other considerations:
* This would also result in a big pile of waste in Africa, as their recycling infrastructure is limited.
* They need food, shelter, stable politics and functional education before they can make any use of computers.
* They have limited energy supply. Low powered tablets / laptops are much more useful.
Agreed. Better yet, keep them in a Google data center. Its far more efficient to make cloud storage available at a reasonable price. How about a diskless laptop that boots from cloud storage? That would be a sweet spot economically (depending on network costs I guess).
>How about a diskless laptop that boots from cloud storage?
While this is a cool idea, how much bandwidth would you need to boot at roughly the same speed you do today? Some SSDs have 500 MB/s (~4 Gbit) read. You'd need to have gigabit networking with almost zero latency for that to perform well.
I suppose a smaller OS like Chrome OS would be perfect for that. Even if this worked on fiber, how would you boot over a cellular network? Aside from costing you a ton of money, it would take forever to download.
You could not believe the price they have to pay for computer. It is easily 3x, 4x the price we pay for the same hardware. And I they do need computer.
But, it is possible that the logistic/shipping is more expensive than the production of new hardware.
Also, cheaper more energy efficient hardware could be more cost effective.
source: A friend of mine worked for ong in Burkina Faso
I do believe you. It's economics of scale: they don't have it.
While most of the western world has huge fright ship docks that can load/unload and ship relatively price efficient, Africa does not.
This means, the primary means to ship stuff there is via airplane, which is notoriously weight constrained, which in turn makes shipping bulk goods expensive.
This makes shipping newer, lighter tech even more cost effective than old hardware.
p.s. This is probably true for anything they need to import, not just tech.
I don't think that's an horrible idea, because I routinely donate old computer tech to non-profits, which do make use of it. Usually to assemble computers for children. Granted, this has negligible cost to me, as we are in the same country.
When one has access to cheap tech from the BestBuy just around the corner, it is difficult to imagine how expensive it can be in third world countries. It may also be difficult to understand how incredibly old are some of the computers that do exist.
I can't believe that it would be so expensive to ask people and companies to donate their old "junk", fill up a container and ship it.
> This would also result in a big pile of waste in Africa, as their recycling infrastructure is limited.
That may be so. That much tech waste could also create the necessary conditions for a recycling industry to start. Assuming that it doesn't already.
> They need food, shelter, stable politics and functional education before they can make any use of computers.
Is that so? Can't computers help them achieve those goals?
> They have limited energy supply. Low powered tablets / laptops are much more useful.
Yes, many locations completely lack power and ordinary desktop computers wouldn't work. But I don't think that's true for all african countries.
You can use default warranty information to figure out the lower bound on useful life. Companies price the value of the warranty into the product and perform statistical QA to ensure that 95%-99% of all products released will work correctly for the length of the standard warranty. Also added warranties aren't worth the cost. Just replace the product when it breaks.
Agreed. Pretty much they assume you won't return a failed drive either. Since they last about as long as the warranty, you have only a few weeks at best to remember to send it in for replacement.
I've been hit-and-miss, gotten a few drives replaced, had a few warranties expire. But pretty much every disk drive fails eventually.
Think about it - its a commodity. If it lasted much longer than the warranty, they spent too much on robustness for the price.
I think the floods has probably been a factor in reduced reliability. It took forever for prices to come down to where they were and the manufacturers are probably cutting corners everywhere to save costs. Why ramp up factories when the tech itself is on the way out?
I think this is most evident in the reduced warranty periods compared to before when 5 years was quite normal.
Yev from Backblaze | Yes, prices still haven't come back down to pre-flood levels unfortunately. They used to drop about 4% per month in price (over time) but now it's going down more slowly.
It'd be very useful to have more detailed information about read/write volume and capacity, MTBF should vary a lot depending on those. Until then I'll keep being paranoid.
Part of the problem is the move by manufacturers to have consumers basically burn-in test their products for them as cost reduction and shift the expense to retailers.
Well ultimately it's the consumer/user who does the work. I've considered buying second-hand disks just to get around this problem. Although that usually comes with other issues...
We dont know, but backblaze sure knows how to:
1. print backblaze's brand as often as linguistically possible in the same article.
2. Get to the top of HN over a 50 yrs old technology's failure rate without clustering for brand spin speed density etc. (read not much)
Am I unaware that there are new paid spots on the first page of HN? (it would make sense I guess, from a business perspective)
TIA to anyone that can be of help on this,
cheerio, (and good luck to Blackblaze, backblaze a path to a backblazing success!)
There's always been paid articles. This is not one of them. They get to the front page because they're usually well written, contain interesting information, and are relevant to the HN user base.
Yev from Backblaze | One might say that we are "Backblazing a trail"? To my knowledge we've never paid for a HN top article. People tend to like our pod and storage related stuff and we're always thrilled to see it on here, so we come and chat about it along with everyone. The discussions are awesome. I'll chat with the writing team to see if we can take out one or two Backblaze's from future posts ;-)
Hey, it was tongue-in-cheek. Your service looks awesome, and as an awareness piece it was bang on. Although I don't recall hearing about backblaze before, the name has been ringing in my ears all day. Keep up the good work and good luck!
The hard disk drive quality has dropped over the last few years.
* Most consumer drives over 2TB have extremely poor reliability. Just check any Amazon or Newegg review (DOA and early mortality show up with more frequency). Yes, I know using reviews are not accurate but since there is no public information of drive failure rates there is not really much to go on.
* The reduction of manufacturer warranty since Thailand floods. Surprise, they never changed it back to the original 3 year warranty.
If you have a large array of disks, there is nothing to really to worry about. If you have a small set of drives, spend a little extra and get the "Black" or RE drives with 5 year warranty. Avoid any "Green" drive.
I have to suspect its because bleeding-edge drives have to be over-rated to compete. They have better margins which should mean they could afford better quality. But they can barely make the drive at all; they want to ship before reliability is quite there; so top-end drives are sketchy.
The RED drives you are refering to actually have some of the worst reviews around compared to even normal consumer drives. (Source: newegg reviews you mentioned)
About half of the Newegg reviews are 3-eggs or lower. And even some of the 5-egg reviews mention dead drives! They only got 5 eggs because of Newegg's easy return policy. The drives are clearly unreliable.
Why avoid the green drives? I assumed those were less power hungry and spinning slower so more reliable. I've been ordering them for RAID5 arrays and not had too many issues yet.
Some drives like WD's Green series have variable RPM to save power (and therefore heat). I don't think the OS is involved at all with controlling how fast those disks spin (so long as they are spinning at all).
At the very bottom, in small, grey text, you'll find "IntelliPower" defined as:
"A fine-tuned balance of spin speed, transfer rate and caching algorithms designed to deliver both significant power savings and solid performance. For each WD Green drive model, WD may use a different, invariable RPM."
The host can spin down a drive manually, but most often it's done autonomously be the drive.
‣ In Linux you can manually ...
• check the power state of your drive using: hdparm -C /dev/sda
• manually spin down the drive (standby) using: hdparm -y /dev/sda (it will immediately spin up at the first attempt to read a sector)
‣ Or you configure the automatic standby of the drive (which also does not involve the OS)...
• hdparm -S n /dev/sda will configure the timeout of the drive to a value encoding the time to spin-down on a non-linear scale, check the manpage
• hdparm -B n /dev/sda will configure another type of power management which doesn't specify a fixed timeout, but rather a vendor-defined type of arbitrary power saving measures on a scale of 1..254 (1: waste power, 254: conserve power, n>128 allows spin down)
The latter two options are handled internally by the drive and (as far as I know) even stored non-volatile.
Greens have had problems with aggressive head parking. If you have an idle set of them you can go through their design limit of head parks in a couple of months and start to get failures shortly after. Done that.
Check your S.M.A.R.T data. Look at the head park number. (Load cycles I think it is called, can't look it up now). If it is a six digit number, you are in trouble. For a server you want if to be in the same order as to power ups. Anything else and you have to explain to yourself "why?"
Edit: adding. The 1TB and smaller greens were disasters. I ruined a lot of them. I was told all of the 2TB and up greens didn't have head park issues, but spent part of last week replacing a storage unit populated with 2TB greens when a spindle failed (>200 unrecoverable blocks) and found that some of the 2TB greens were load cycling into the 200000 range, others weren't running up. They were all identical models purchased at the same time. Maybe they had different firmware? I replaced hem with REDs. They aren't supposed to park and they won't try to recover a bad sector for more than a few seconds so the don't hang your RAID when they get bad sectors.
This might be anecdotal but external WD (MyLife) drives are usually from the Green series and I had 2 different ones fail on me after about 1 year of use. Same happened to 2 friends of mine. I blamed it on the constant head parking (it went idle after 10 minutes of unuse).
I have 4 WD greens, one from an external enclosure the others being internal drivers, the first being maybe 3 years old now, all still going fine, also anecdotal
As someone who inherited 240 24/7 running WD-Greens: http://idle3-tools.sourceforge.net/ works fine but disabling the timer has negative performance impact. 3000 seconds is fine through. But you need a complete powercycle before the changes take effect. No more parking. Does make a difference in longevity in my not very scientific opinion.
I can second the 200> bad blocks. Sometimes they still work fine after using badblocks -w a few times on them and raising the timer.
I assume you mean Start_Stop_Count. A quick check on two servers, each with 4 green drives in RAID6 and RAID5 setups tells me this hasn't been a problem. Both have Start_Stop_Count's below 100 (on the order of the number of boots the servers have had). I don't see any other number that could be the head park.
The number I have been finding to be high is Hardware_ECC_Recovered (values between 1036555546 and 2699460003). Not sure if that's normal. I've also had two 1.5TB drives now end up with unrecoverable sectors. RAID recovers from that just fine but I've been replacing them as it keeps reoccurring and is supposed to be a signal of failing disks. These 1.5TB drives are a 3+´years old and I've been thrashing them a bit lately. I'd have expected them to last longer though.
I may have spoke too soon. One of my servers has 2 Samsung Green and 2 WD Green drives in RAID6. Here's the SMART value that you seem to be discussing:
$ for dev in `ls /dev/sd?`; do echo $dev; sudo smartctl -a $dev | grep Load_Cycle_Count; done | cut -d " " -f 2,40
/dev/sda
Load_Cycle_Count 24
/dev/sdb
Load_Cycle_Count 24
/dev/sdc
Load_Cycle_Count 1947798
/dev/sdd
Load_Cycle_Count 1907706
sda and sdb are the samsungs and sdc and sdd are the WDs. I also just replaced a failing Samsung Green drive in another machine with a WD and it already has a Load_Cycle_Count in the 10000s. I guess I need to start avoiding Green WDs at least, maybe the Greens altogether.
I'm curious, why run 4 disks in RAID6 instead of RAID10? You loose two disks worth of capacity to RAID in either case, but with RAID6 there's also parity overhead, slower recovery, slower performance, especially in degraded mode?
Yep, that's it. The servers I run are all personal and their main workload is keeping my files safe. Being able to survive 2 drive failures in all situations is important. I just discovered in this thread that on that RAID6 array 2 drives are actually suffering from excessive head-parking. So having used RAID6 and bought 2 different types of drives bought me some insurance against the simultaneous failure of the two drives. The performance is fine anyway.
And for anyone that doesn't think this is worthwhile:
We recently had 3 servers have two drives each fail within hours of each other, with about two weeks between each of the 3 servers. These were 3 out of 4 servers that had been configured at the same time, with drives from the same delivery - clearly something had gone wrong.
Usually we try to drive types, but we didn't have enough suitable drives when we had to bring these up. Thankfully we do have everything replicated multiple times and very much specifically avoided replicating things from one of the new servers to another.
When we brought them back online we got a chance to juggle drives around, so now they're nicely mixed in case we get more failures.
For my private setup, I've gone with a mirror + rsync to a separate drive with an entirely different filesystem + Crashplan. Setups like that seems paranoid until you suffer a catastrophic loss or near loss...
My first big scare like that was a decade or so ago when we had a 10 or 12 drive array of IBM Deathstar (Deskstars) that started failing, one drive after the other, about a week apart, and the array just barely managed to rebuild... Particularly because it slowed the array down so much during the rebuild, that we were unable to complete a backup a day while running our service too, and taking downtime was unacceptable. So our backups lagged further and further behind while we waited for the next drive failure.. Those were some tense weeks.
That's a great example of what I worry about. On my first server I bought 4 identical drives when I built it and then when I needed more space I again bought 4 identical larger drives. Since this was all on RAID5 the risk was actually pretty high. On my second server I bought two of each manufacturer and used RAID6 so now I can survive a whole batch going wrong at the same time. Next time I need to build one from scratch I may even go for 4 different drives (mixing red/green/etc).
What I am doing now as soon as I get unrecoverable errors from the drives is to replace them one at a time with whatever is the best cost/TB drive. Whenever all the drives have been upgraded I can resize the array to the new minimum drive size.
I recently went through all my old 3.5" floppies I could find and most of them were still quite readable. This seems like a bit of hysteria to me. I didn't preserve as many of my 5.25" floppies, but I also don't have a drive for them. My 3.5" drive is an old USB one that came from a Toshiba laptop circa 2000. It was in a drive bay that was replaceable with a cdrom drive.
Anecdotal data-point: a few years ago I rescued the data off my 3.5" floppies from the early 90's (IBM PC and Atari ST formats) and they all seemed to be fine.
I don't have any hardware to read my pile to 5.25 Atari 800 disks.
Just out of curiosity, I wonder what the savings would amount to if this company used something like SpinRite to fix / recover failed drives? Although I've never used it from what I hear its pretty good at saving drives...
It's actually not that good at saving drives, however it sometimes saves your data. Lately I've used it to recover a 250 GB drive that was part of a failed RAID-5 array, and miraculously got the data back.
So SpinRite may be handy, but throw the drive away after use.
Extremely useful info here. Most of my HDs have been running for years and still work fine, but you go online, and all you read about is that HDs are horrendously unreliable and all fail after a year or two. (manufacturer propaganda, anyone?)
The optical drives I've had, on the other hand, are actually unreliable. They all seem to break down after about four years, and I don't use them all that often!
Your disks are going fine, do you feel the urge to report about it in the reviews on amazon or newegg or where you bought it from?
There is an inherent bias in the reviews. Which makes the backblaze report so interesting, they have less of a bias though they do not report actual disk vendors and models to really draw direct inference only the general trend.
Spot on, I have experienced my drives to either fail quite soon, or "never". I am still running one of my very early 40GB hard disks. It must be like 8 years old.
These numbers line up nicely with what I've experienced on much smaller scales (I've never personally cared about more than a few hundred spinning drives at once), which is that in a nice mix of old, middle-aged, and new drives, 5-10% go kaput each year.
Incidentally, about "consumer-grade drives", the last time I looked into this, I was led to believe that if it's SATA and 7200RPM (or less), there's no hardware distinction. It's just firmware. Consumer drives try very hard to recover data from a bad sector, while Enterprise/RAID drives have a recovery time limit to prevent them being unnecessarily dropped from an array (which will have its own recovery mechanisms). That's it.
Well Intel tells us different story[1] that promises
1. More performance
2. Less vibration (improves performance)
3. ECC Memory
There is a long feature reference that mentions things like: higher RPM, more quality, larger magnets, air turbulance control, dual processors, etc.
I'm not a spec in hard drives, just that I remember reading this stuff when trying to figure out do I need it. In the end, For my small-scale corporate file server, I chose zfs raidz with consumer grade disk drives.
A marketing team at Intel tells us a vague story about what is either a very vague or very specific set of drives, or may be about an entirely hypothetical set of drives. It's not clear.
They even admit to the problem themselves at the end:
"Some hard drive manufactures may differentiate enterprise from desktop drives by not testing certain enterprise-class features, validate the drives with different test criteria, or disable enterprise-class features on a desktop class hard drives so they can market and price them accordingly. Other manufacturers have different architectures and designs for the two drive classes. It can be difficult to get detailed information and specifications on different drive modes."
That PDF tells me nothing interesting. It's marketing crap for clueless executives, not a technical analysis. (Given their absurd obsession with "Higher RPM" as some sort of defining characteristic, it's not even relevant to the statement I made in the first place.)
If some of them live a long, long time, it makes it hard to compute the average. Also, a few outliers can throw off the average and make it less useful.
> Proper statistical analysis would help you there.
Yes, if you know the probability distribution. If you don't know the distribution, you can not calculate your confidence, and thus can not do a proper statistical analysis.
And, guess what, nobody knows the probability distribution of hard drive failures. That's exactly what they are trying to find out.
There are actually many methods in survival analysis -- just as in the rest of statistics -- that do not impose strict distributional assumptions, and account for the fact that many drives are still operational. But as someone else mentioned, the median is also a good statistic to report :)
Approximately 40 PB raw storage in our datacentres here, half of them Supermicro servers with whatever disk that came, half HP Proliant with $$$ HP Enterprise class disks, all < 5 years old, so quite comparable to the Backblaze situation.
80% drives surviving after 5 years seems right, this is what we're seeing as well. The hardware is decommissioned faster then the drives fail.
tl;dr: Here's some statistically significant data on how 25000 drives have worked over 4 years, please now comment on how the 3 drives you've owned died.
161 comments
[ 3.6 ms ] story [ 207 ms ] threadWe replaced the drives with a different brand and the 'failures' went away.
There is indeed a firmware setting on how long the drive spends checking for errors after it detects one, during which the drive doesn't respond. Sometimes this is too long for the RAID controller, so it drops it.
It used to be you could buy WD Caviar Black drives and tweak that timeout setting on the controller to effectively have WD RE drives (enterprise version of the WD Black drives). They removed that "feature" a few years ago.
We are constantly looking at new hard drives, evaluating them for reliability and power consumption. The Hitachi 3TB drive (Hitachi Deskstar 5K3000 HDS5C3030ALA630) is our current favorite for both its low power demand and astounding reliability. The Western Digital and Seagate equivalents we tested saw much higher rates of popping out of RAID arrays and drive failure. Even the Western Digital Enterprise Hard Drives had the same high failure rates. The Hitachi drives, on the other hand, perform wonderfully.
In the second article they say that WD-RED are 2nd in reliability (WD-RED did not exist 2011). I'm happy that I've got a cheap Hitachi Ultrastar. But who knows.
As a personal anecdote: WD-Green failure rates are huge here. 24/7 Desktop machines, 240 drives. I've replaced in last 12 month at least 20 drives.
The article actually makes this point (about anecdote), but their data suggests that failure rates do rise substantially after three years.
I suspect the reason why people do "burn in" tests on hard drives is to make drives that suffer from early failure ("infant mortality" as described in the article) fail early enough that you can RMA them with the manufacturer. Apart from that, I don't think there's much you can do to improve your chances.
http://static.googleusercontent.com/external_content/untrust...
Drives have died for me both in 24/7 powered systems and through power cycles. Drives have reported intermittent failures for many months, but still lived for years without any actual data loss. The oldest drive I still have spinning is a 200G IDE containing the OS for my old OpenSolaris zfs NAS; must be getting on for 9 years.
I advise having a back-up of every drive you own, preferably two. I built a new NAS last week, 12x 4G drives in raidz2 configuration; with zfs snapshots, it fulfills 2 of the 3 requirements for backup (redundancy and versioning), while I use CrashPlan for cloud backup (distribution, the third requirement). Nice thing about CrashPlan is my PCs can back up to my NAS as well, so restores are nice and quick, pulling from the internet is a last resort.
Not everyone lives in the EU. In fact, the majority of people don't.
Outside your regulation happy haven, warranty periods aren't random and do indicate durability under normal use.
Care to back that up with any real data instead of baseless consumer speculation relying on time travel?
Certainly the old 9.1Gb SCSI disks that were so popular 10 years ago are well past be justifiable to give power to now.
But these drives will still be useful. What about, say, shipping them to ONGs located in Africa?
Hard drive space per dollar grows exponentially, and they're big weighty things. The window of time where it would be economical to reuse would be short, and value dubious.
But there are other considerations:
* This would also result in a big pile of waste in Africa, as their recycling infrastructure is limited.
* They need food, shelter, stable politics and functional education before they can make any use of computers.
* They have limited energy supply. Low powered tablets / laptops are much more useful.
While this is a cool idea, how much bandwidth would you need to boot at roughly the same speed you do today? Some SSDs have 500 MB/s (~4 Gbit) read. You'd need to have gigabit networking with almost zero latency for that to perform well.
I suppose a smaller OS like Chrome OS would be perfect for that. Even if this worked on fiber, how would you boot over a cellular network? Aside from costing you a ton of money, it would take forever to download.
But, it is possible that the logistic/shipping is more expensive than the production of new hardware.
Also, cheaper more energy efficient hardware could be more cost effective.
source: A friend of mine worked for ong in Burkina Faso
While most of the western world has huge fright ship docks that can load/unload and ship relatively price efficient, Africa does not.
This means, the primary means to ship stuff there is via airplane, which is notoriously weight constrained, which in turn makes shipping bulk goods expensive.
This makes shipping newer, lighter tech even more cost effective than old hardware.
p.s. This is probably true for anything they need to import, not just tech.
Many countries on the continent appear to have rather robust capacity for their economies.
http://ports.com/browse/africa/map-view/
I don't think that's an horrible idea, because I routinely donate old computer tech to non-profits, which do make use of it. Usually to assemble computers for children. Granted, this has negligible cost to me, as we are in the same country.
When one has access to cheap tech from the BestBuy just around the corner, it is difficult to imagine how expensive it can be in third world countries. It may also be difficult to understand how incredibly old are some of the computers that do exist.
I can't believe that it would be so expensive to ask people and companies to donate their old "junk", fill up a container and ship it.
> This would also result in a big pile of waste in Africa, as their recycling infrastructure is limited.
That may be so. That much tech waste could also create the necessary conditions for a recycling industry to start. Assuming that it doesn't already.
> They need food, shelter, stable politics and functional education before they can make any use of computers.
Is that so? Can't computers help them achieve those goals?
> They have limited energy supply. Low powered tablets / laptops are much more useful.
Yes, many locations completely lack power and ordinary desktop computers wouldn't work. But I don't think that's true for all african countries.
I've been hit-and-miss, gotten a few drives replaced, had a few warranties expire. But pretty much every disk drive fails eventually.
Think about it - its a commodity. If it lasted much longer than the warranty, they spent too much on robustness for the price.
I think this is most evident in the reduced warranty periods compared to before when 5 years was quite normal.
http://www.ssrc.ucsc.edu/Papers/xin-mascots05.pdf
Am I unaware that there are new paid spots on the first page of HN? (it would make sense I guess, from a business perspective)
TIA to anyone that can be of help on this, cheerio, (and good luck to Blackblaze, backblaze a path to a backblazing success!)
Don't like it? Don't vote for it.
* Most consumer drives over 2TB have extremely poor reliability. Just check any Amazon or Newegg review (DOA and early mortality show up with more frequency). Yes, I know using reviews are not accurate but since there is no public information of drive failure rates there is not really much to go on.
* The reduction of manufacturer warranty since Thailand floods. Surprise, they never changed it back to the original 3 year warranty.
If you have a large array of disks, there is nothing to really to worry about. If you have a small set of drives, spend a little extra and get the "Black" or RE drives with 5 year warranty. Avoid any "Green" drive.
http://www.newegg.com/Product/Product.aspx?Item=N82E16822236...
http://www.amazon.com/WD-Red-NAS-Hard-Drive/dp/B008JJLZ7G
been running 3x of these in a raid-5 NAS, no issues so far (not that it's any kind of indicator on a system which idles as a backup all day)
Some of them were also crippled in firmware so you couldn't use them in RAID1 arrays, but this might have changed.
http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-771...
At the very bottom, in small, grey text, you'll find "IntelliPower" defined as:
"A fine-tuned balance of spin speed, transfer rate and caching algorithms designed to deliver both significant power savings and solid performance. For each WD Green drive model, WD may use a different, invariable RPM."
‣ In Linux you can manually ...
• check the power state of your drive using: hdparm -C /dev/sda
• manually spin down the drive (standby) using: hdparm -y /dev/sda (it will immediately spin up at the first attempt to read a sector)
‣ Or you configure the automatic standby of the drive (which also does not involve the OS)...
• hdparm -S n /dev/sda will configure the timeout of the drive to a value encoding the time to spin-down on a non-linear scale, check the manpage
• hdparm -B n /dev/sda will configure another type of power management which doesn't specify a fixed timeout, but rather a vendor-defined type of arbitrary power saving measures on a scale of 1..254 (1: waste power, 254: conserve power, n>128 allows spin down)
The latter two options are handled internally by the drive and (as far as I know) even stored non-volatile.
http://linux.die.net/man/8/hdparm
(Edit: fixed my broken English ;-) )
Check your S.M.A.R.T data. Look at the head park number. (Load cycles I think it is called, can't look it up now). If it is a six digit number, you are in trouble. For a server you want if to be in the same order as to power ups. Anything else and you have to explain to yourself "why?"
Edit: adding. The 1TB and smaller greens were disasters. I ruined a lot of them. I was told all of the 2TB and up greens didn't have head park issues, but spent part of last week replacing a storage unit populated with 2TB greens when a spindle failed (>200 unrecoverable blocks) and found that some of the 2TB greens were load cycling into the 200000 range, others weren't running up. They were all identical models purchased at the same time. Maybe they had different firmware? I replaced hem with REDs. They aren't supposed to park and they won't try to recover a bad sector for more than a few seconds so the don't hang your RAID when they get bad sectors.
I can second the 200> bad blocks. Sometimes they still work fine after using badblocks -w a few times on them and raising the timer.
The number I have been finding to be high is Hardware_ECC_Recovered (values between 1036555546 and 2699460003). Not sure if that's normal. I've also had two 1.5TB drives now end up with unrecoverable sectors. RAID recovers from that just fine but I've been replacing them as it keeps reoccurring and is supposed to be a signal of failing disks. These 1.5TB drives are a 3+´years old and I've been thrashing them a bit lately. I'd have expected them to last longer though.
We recently had 3 servers have two drives each fail within hours of each other, with about two weeks between each of the 3 servers. These were 3 out of 4 servers that had been configured at the same time, with drives from the same delivery - clearly something had gone wrong.
Usually we try to drive types, but we didn't have enough suitable drives when we had to bring these up. Thankfully we do have everything replicated multiple times and very much specifically avoided replicating things from one of the new servers to another.
When we brought them back online we got a chance to juggle drives around, so now they're nicely mixed in case we get more failures.
For my private setup, I've gone with a mirror + rsync to a separate drive with an entirely different filesystem + Crashplan. Setups like that seems paranoid until you suffer a catastrophic loss or near loss...
My first big scare like that was a decade or so ago when we had a 10 or 12 drive array of IBM Deathstar (Deskstars) that started failing, one drive after the other, about a week apart, and the array just barely managed to rebuild... Particularly because it slowed the array down so much during the rebuild, that we were unable to complete a backup a day while running our service too, and taking downtime was unacceptable. So our backups lagged further and further behind while we waited for the next drive failure.. Those were some tense weeks.
What I am doing now as soon as I get unrecoverable errors from the drives is to replace them one at a time with whatever is the best cost/TB drive. Whenever all the drives have been upgraded I can resize the array to the new minimum drive size.
Odd question, but I've always been wondering. These things just seem to hast forever.
[0] http://ascii.textfiles.com/archives/3191
I don't have any hardware to read my pile to 5.25 Atari 800 disks.
So SpinRite may be handy, but throw the drive away after use.
The optical drives I've had, on the other hand, are actually unreliable. They all seem to break down after about four years, and I don't use them all that often!
There is an inherent bias in the reviews. Which makes the backblaze report so interesting, they have less of a bias though they do not report actual disk vendors and models to really draw direct inference only the general trend.
Incidentally, about "consumer-grade drives", the last time I looked into this, I was led to believe that if it's SATA and 7200RPM (or less), there's no hardware distinction. It's just firmware. Consumer drives try very hard to recover data from a bad sector, while Enterprise/RAID drives have a recovery time limit to prevent them being unnecessarily dropped from an array (which will have its own recovery mechanisms). That's it.
There is a long feature reference that mentions things like: higher RPM, more quality, larger magnets, air turbulance control, dual processors, etc.
I'm not a spec in hard drives, just that I remember reading this stuff when trying to figure out do I need it. In the end, For my small-scale corporate file server, I chose zfs raidz with consumer grade disk drives.
[1] Enterprise-class versus Desktop-class Hard Drives: http://download.intel.com/support/motherboards/server/sb/ent...
They even admit to the problem themselves at the end:
"Some hard drive manufactures may differentiate enterprise from desktop drives by not testing certain enterprise-class features, validate the drives with different test criteria, or disable enterprise-class features on a desktop class hard drives so they can market and price them accordingly. Other manufacturers have different architectures and designs for the two drive classes. It can be difficult to get detailed information and specifications on different drive modes."
That PDF tells me nothing interesting. It's marketing crap for clueless executives, not a technical analysis. (Given their absurd obsession with "Higher RPM" as some sort of defining characteristic, it's not even relevant to the statement I made in the first place.)
Proper statistical analysis would help you there.
Yes, if you know the probability distribution. If you don't know the distribution, you can not calculate your confidence, and thus can not do a proper statistical analysis.
And, guess what, nobody knows the probability distribution of hard drive failures. That's exactly what they are trying to find out.
80% drives surviving after 5 years seems right, this is what we're seeing as well. The hardware is decommissioned faster then the drives fail.