>I still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates.
You will still need to set up either BTRFS or ZFS to avoid silent Data corruption which is happening more often than most people know. And you have to set it up to scrub it from time to time. It isn't automatic.
Arguably the cheapest solution that hit the above bar would be Synology DS223J at $180, their newest entry model with BTRFS support ( on a fairly old Linux Kernel, I think it is 5.10 that will never get updated again in the life time of the product ). You will still need to buy 2 HDD in redundant RAID and set it up yourself. And it is quite bulky as well.
Hardly consumer friendly. And that is before 99.9999% on HN jumping on you, telling you need another backup for this NAS, or another off line, third location devices....
It is sad, not only just consumer. Judging by the number of comment here plenty in the tech industry simply uses an additional copy, or external SSD devices as backup. And I can understand why, but there simply isn't a simple, easy to use consumer solution that offer enough security for a reasonable price.
And I have been stating this for more than a decade already. Your Kids's video are worth a lot more than you can imagine. Tried explaining it to your wife some of those video are gone.
My favorite trick for this lately has been to use a stripe of mirrored pairs, where each pair is a WD Gold and a WD Red Pro (of matching size).
They're very similar drives (AFAICT, WD Gold is just a Red Pro but with a higher lifetime r/w rating) but it makes me feel a bit better that they're not from too similar of batches without having to actually temporally space out my purchases. I can buy 1 of each in the same cart.
And then for truly unrecoverable data, I back it up to S3 Deep Archive which seems to be one of AWS's only reasonably priced offerings. Don't ask me how much it's cost in egress bandwidth to pull it all back though. I'm hoping if I end up needing it I'll be grateful enough for my foresight to justify paying it.
in my case i use 2.5" 5TB disks. there is only one model from one manufacturer making those.
i bought one each from two different shops located in different parts of the country 5 years ago. i used them in a mirror raid. one of them died a year ago. then i got two more drives, one of which was second hand slightly used. the price was good and i figured it was a good way to ensure i'd get drives from two very different batches.
I'm not sure what exactly you mean - they're different lines of drives. And different SKUs on Amazon. The probability of me getting sibling drives out of that - and that's assuming the only difference between red pros and golds are binning - is infinitesimally small.
Heh, me too, but just a reminder raid is not a backup. raid can keep your data online in the face of some types of hardware failure. It won't protect against mistakes, malice, or disaster. backups will.
What I do is build one good drive array then each machine gets a cheap local drive and can access the good drive via network and I don't have to spend too much putting redundancy in each machine. Now... "looks around embarrassedly" time to work on that backup system.
Uh... is one broken drive such a big issue? Hardware will fail, usually when it is very new or very old (Bathtub curve). Or are there more such events the article didn't mention?
It'll be true for sufficiently short periods of time, or for ages if you never put any data on it.
About 1/4 of hard drives I buy die in the first 24 hours since I unsportingly run badblocks on them to see whether they're on the start of the failure curve.
badblocks has a destructive write mode. It writes numbers to disk then reads them back a while later and tells you if they differ. That seems to work hard drives fairly hard - they warm up at least. A day is usually how long my patience lasts.
There are probably better stress test approaches out there. The basic idea is to find out if your drive is one that'll die early before putting a load of data you want to keep on it.
I don't do this to ssd as they have lowish write limits anyway, so I just assume any data written to a ssd will vanish at some point. Hard drives in a network share, ssd in dev machines treated as scratch storage.
If you read the article, the first paragraph explains that this was the replacement drive that WD sent him when his previous one failed.
So it’s at least 2 drives in a row in a short period of time. Statistically that will happen, but it’s like gambling - if you win too much you get shot.
Which points to a firmware bug in the SSD that the use-case managed to trigger repeatedly. Firmware bugs were around in abundance when performant SSDs first came to market more than a decade ago, and while manufacturers have gotten better at testing SSD firmware, it is still inevitable that bugs will occasionally make it into shipping hardware. Moral of the story: don't trust your valued data to any one storage device. I've tried to use drives from different vendors in RAID 1 mirrors to minimize exposure to firmware bugs, and I've still seen SSDs die at unexpected times in the last few years.
Yes I bought a new WD 1TB a year ago and one day it was damaged beyond repair. I did not do anything with it. Shop technician's diagnosis was "several tiny scratches on the disk". Not sure what that meant, I assume the whole build of the drive is just bad?
I just lost a 4 years worth of my kid's videos. I will never forgive WD.
Edit: I just forgot to add that my old WD HD (bought around 2009-ish) is still working. But reading the contents will take hours. But the point is the quality of WD has gone downhill.
>I just lost a 4 years worth of my kid's videos. I will never forgive WD.
I'm sorry, but your fate was sealed the moment you decided that one copy was sufficient. Drives can fail randomly, so keeping valuable data on one without redundancy will virtually guarantee that you will lose data at some point in time. Just be thankful that you learned your lesson and it only cost 4 years worth of videos. If it happened 3 years down the line it could easily have been 7 years worth of videos.
I would strongly recommend signing up for either Backblaze, or using something like iCloud Photos or Google Photos.
My entire 100,000 photos, 20 year library is now in iCloud photos. It's constantly synced and backed up to icloud. Then I have it pulled down in full to a mac that then backs up to backblaze.
Lots of ways to do it, but thats one. :)
For $5-10/month you can get unlimited backup from a machine with Backblaze including your external drives.
You should never trust valuable data to a single HD; always make backups.
Personally, I use a portable HD (spinning rust kind, not SSD) to do regular backups of my PC, but the portable HD itself is also backed up onto another portable HD.
Even with backups, drive failure rate matters! If you keep n backups, and a drive dies, then you will at least temporarily have n - 1 backups, which is presumably one less than you wanted.
Make sure to read the full article on this one, not just the beginning. I was initially rolling my eyes--yes, loosing data sucks, but all drives can die and there's such a thing as bad luck--but the issue really does appear to be larger than that.
I blame tech journalism like the verge for this failure.
SSD manufacturers are well aware of what their drives should do. But they are also aware that to get sales they need to win benchmarks, and that to do that they cut corners with data integrity.
A benchmark of a drive should involve a heavy workload with a random drive hard and unplanned poweroff every 30 seconds. The drive should never lose a single fsync'ed byte.
The drive should also be put in hot/cold cycling chambers. And in radiation chambers. And journalists should publish which drives meet their obligations despite all the adverse (yet still in spec) conditions they're put through.
I bet no ssd for sale today could survive all 3 tests above for a month and still get okayish performance without data loss.
Journalists could go even further. Have robustness tests where they drop the drive in salt water, shoot it with a gun, chuck it in a fire etc., And then see how much data can be recovered with data recovery tools.
…right. Don’t blame the manufacturer for making shitty drives, blame tech review sites for testing those drives with the tools available and reporting the results of those tests. This is some 4D level victim blaming shit. I’m almost impressed at the absurdity of this argument.
This exact SSD failed on me in a similar way. They also recently had a firmware issue that borked a bunch of drives, and they only released a fix tool for Windows users (I guess macOS/Linux users can pound sand?)
I will say that the warranty service was quite good, but given the below-zero level of trust I have in SanDisk post WD acquisition, the replacement drive is going to be basically fulfilling the role of “paperweight and very oversized temporary USB drive for ephemeral sneakernet transfers”.
Bought a pair of Samsung T7’s to replace it, no issues with them so far.
I have this exact drive; when plugged into my desktop PC it instantly wedges the USB root hub, causing USB to not work until I power cycle the entire machine. (Intel, W10)
Yet the same drive works flawlessly in an Intel laptop. I think the m.2 to USB bridge chip inside might be the issue.
51 comments
[ 3.1 ms ] story [ 120 ms ] threadI still use spinning rust as my primary back up --- arranged in a redundant array with offset manufacture dates.
You will still need to set up either BTRFS or ZFS to avoid silent Data corruption which is happening more often than most people know. And you have to set it up to scrub it from time to time. It isn't automatic.
Arguably the cheapest solution that hit the above bar would be Synology DS223J at $180, their newest entry model with BTRFS support ( on a fairly old Linux Kernel, I think it is 5.10 that will never get updated again in the life time of the product ). You will still need to buy 2 HDD in redundant RAID and set it up yourself. And it is quite bulky as well.
Hardly consumer friendly. And that is before 99.9999% on HN jumping on you, telling you need another backup for this NAS, or another off line, third location devices....
It is sad, not only just consumer. Judging by the number of comment here plenty in the tech industry simply uses an additional copy, or external SSD devices as backup. And I can understand why, but there simply isn't a simple, easy to use consumer solution that offer enough security for a reasonable price.
And I have been stating this for more than a decade already. Your Kids's video are worth a lot more than you can imagine. Tried explaining it to your wife some of those video are gone.
They're very similar drives (AFAICT, WD Gold is just a Red Pro but with a higher lifetime r/w rating) but it makes me feel a bit better that they're not from too similar of batches without having to actually temporally space out my purchases. I can buy 1 of each in the same cart.
And then for truly unrecoverable data, I back it up to S3 Deep Archive which seems to be one of AWS's only reasonably priced offerings. Don't ask me how much it's cost in egress bandwidth to pull it all back though. I'm hoping if I end up needing it I'll be grateful enough for my foresight to justify paying it.
What I do is build one good drive array then each machine gets a cheap local drive and can access the good drive via network and I don't have to spend too much putting redundancy in each machine. Now... "looks around embarrassedly" time to work on that backup system.
In your small sample size maybe.
About 1/4 of hard drives I buy die in the first 24 hours since I unsportingly run badblocks on them to see whether they're on the start of the failure curve.
There are probably better stress test approaches out there. The basic idea is to find out if your drive is one that'll die early before putting a load of data you want to keep on it.
I don't do this to ssd as they have lowish write limits anyway, so I just assume any data written to a ssd will vanish at some point. Hard drives in a network share, ssd in dev machines treated as scratch storage.
It's on the TrueNAS forum, but it'll work on basically any Linux system. I'm currently using Unraid, which is based on Slackware.
(I must be luckier than the parent, because I've only had a single drive in a dozen or more die during testing - and that was a used one off ebay!)
https://www.theverge.com/2023/5/22/23733267/sandisk-extreme-...
https://arstechnica.com/gadgets/2023/05/sandisk-extreme-ssds...
So it’s at least 2 drives in a row in a short period of time. Statistically that will happen, but it’s like gambling - if you win too much you get shot.
I just lost a 4 years worth of my kid's videos. I will never forgive WD.
Edit: I just forgot to add that my old WD HD (bought around 2009-ish) is still working. But reading the contents will take hours. But the point is the quality of WD has gone downhill.
I'm sorry, but your fate was sealed the moment you decided that one copy was sufficient. Drives can fail randomly, so keeping valuable data on one without redundancy will virtually guarantee that you will lose data at some point in time. Just be thankful that you learned your lesson and it only cost 4 years worth of videos. If it happened 3 years down the line it could easily have been 7 years worth of videos.
My entire 100,000 photos, 20 year library is now in iCloud photos. It's constantly synced and backed up to icloud. Then I have it pulled down in full to a mac that then backs up to backblaze.
Lots of ways to do it, but thats one. :)
For $5-10/month you can get unlimited backup from a machine with Backblaze including your external drives.
Personally, I use a portable HD (spinning rust kind, not SSD) to do regular backups of my PC, but the portable HD itself is also backed up onto another portable HD.
SSD manufacturers are well aware of what their drives should do. But they are also aware that to get sales they need to win benchmarks, and that to do that they cut corners with data integrity.
A benchmark of a drive should involve a heavy workload with a random drive hard and unplanned poweroff every 30 seconds. The drive should never lose a single fsync'ed byte.
The drive should also be put in hot/cold cycling chambers. And in radiation chambers. And journalists should publish which drives meet their obligations despite all the adverse (yet still in spec) conditions they're put through.
I bet no ssd for sale today could survive all 3 tests above for a month and still get okayish performance without data loss.
When the review sites only review one thing, corners get cut to make that one thing look good.
Any honest manufacturer who did what they thought was best for the customer, not what got them the best reviews, went bankrupt long ago.
I will say that the warranty service was quite good, but given the below-zero level of trust I have in SanDisk post WD acquisition, the replacement drive is going to be basically fulfilling the role of “paperweight and very oversized temporary USB drive for ephemeral sneakernet transfers”.
Bought a pair of Samsung T7’s to replace it, no issues with them so far.
Bad drive runs happen. They’ve been happing a LOOONG time. If it’s important, dupe or raid.
Otherwise, expect to lose the data. It’s just a matter of time.
Lots of film and audio gear has a double recording option for just this reason.
I’m sorry the author has to relearn this lesson the hard way!
Yet the same drive works flawlessly in an Intel laptop. I think the m.2 to USB bridge chip inside might be the issue.