119 comments

[ 0.22 ms ] story [ 13.2 ms ] thread
100% everything here. That level of tamper evidence should be good enough for the reasonable individual, IMHO.

On the enterprise scale, the question I think everyone should be asking is: Do they have LTO tapes on a shelf? Fundamentally the same issue (but larger), in terms of needing to ensure it's physically separate, and secure, and tamper evident. =)

Anything in the cloud or connected to your network can be hacked. The backup of last resort must be on a shelf.

Do they have LTO tapes on a shelf?

Most enterprise gigs I've worked sent them to Iron Mountain or equivalent.

This seems a little over the top to me.

Just use a magnetic hard disk and your data will be fine for many decades if not centuries. You don't need M-Blu Rays (which have very limited capacity anyway). A hard disk is cheap and stores several Terabytes.

You obviously need to store the data in a physically remote place like your shed, car or safe deposit box in case your house burns down.

Never, ever use a SSD if you want your data to be safe for any amount of time. What's worse, SSDs can fail unexpectedly with almost zero chance of data recovery.

With the decline in quality I am no longer confident my HDDs are actually a save backup strategy.

I have so many photos of vacations and people who I've buried over the years, that I don't want to lose.

So I was thinking about using Blu-ray M-Disks as well, these last couple of months.

Hard disks aren't even rated for one decade.
Just picking one at random: The Seagate Exos X20 series of hard disk drives is rated[1] with a MTBF of 2,500,000 hours. That's 285 years.

[1]: https://www.seagate.com/files/www-content/datasheets/pdfs/ex...

MTBF is for hardware failure, not data loss.

This recent study I found [1] shows almost 4% HDD loss after only 2.5 years for consumer hardware, enterprise products being a bit better.

[1] https://research.cs.wisc.edu/adsl/Publications/latent-sigmet...

If the data storage hardware cannot read a block that it has stored within itself, then has the hardware not failed?
You could say so, but that is not what the MTBF you quoted measures in practice. Not an opinion.
I prepended that chapter with a question: If the data storage hardware cannot read a block that it has stored within itself, then has the hardware not failed?

I then presented practical examples of how I understand failures to behave in data storage.

Can you answer the initial question?

Or are you just here to brow-beat your version of what you believe to be correct without the benefit of citation, or logic, or reason?

(Remember, kind HN user: I did not dismiss you at all. I engaged on all cylinders. And I'm here to learn. If you have something to teach, then: Please teach it.

I'm not interested in logical fallacies, though. Please spare me of those if doing so is within you divine capabilities.)

And MTBF is with active use. Electronics which are not powered on can last almost indefinitely and under a huge range of environmental conditions. From freezing cold to scorching heat.

You just need to make sure you let the electronics acclimatize before powering them on. I always let my external HD acclimatize for 24 hours before powering it on after I grab it from my shed where it's been lying for months in freezing conditions. No problem whatsoever.

Offline HDDs are not a reliable long term data storage. Not even for a few years.
> Just use a magnetic hard disk and your data will be fine for many decades if not centuries.

A few months ago I needed to pull from an offline backup. The disk, which used CMR and was purchased new from an actual computer supply company (i.e. not Amazon), was burnt in for about 72 hours before it was loaded up with data and shoved in a padded box. Ten years later, the drive was pulled, and there were so many media surface errors that most of the data was unrecoverable. (I haven’t bothered opening the drive to see if there is some obvious sign of what exactly failed.)

In my case the backups were created in multiples using two different brands for exactly this reason, and the second copy was fine. But I wouldn’t say it’s the case a single magnetic disk can take the place of BD.

Mechanical disks are also getting expensive. The disk purchased to replace the failed backup was being sold at three times its original release MSRP. Some of this is due to the AI hysteria but some of it is also that the market is shrinking and becoming K-shaped.

Oh for sure don't trust any single copy. But I wonder, how many surface errors? How much of the data would have been lost if all your drives degraded that much?

For example, if most files are multiple gigabytes, the drive has 3 million tracks, and 3 thousand of those tracks are ruined: one degraded copy leads to most files being broken, two degraded copies leads to only a single digit number of files being broken, and three degraded copies means >99% odds of recovering everything.

Ten years in general is way too long to leave any piece of electronics unpowered, let alone a fine mechanics device like a HDD with moving parts. They need regular exercise.
You should always let electronics acclimatize before powering them up. I'd say at least 24 hours.
> Just use a magnetic hard disk and your data will be fine for many decades if not centuries

It is absolutely not safe for many decades. If you haven't encountered any drive failures yet — you're lucky.

> A hard disk is cheap and stores several Terabytes

You should check prices again, I wouldn't say cheap considering price bump during the last year (or around so). M-Disks are way less convenient, but pretty reasonable for their cost.

I've had one drive for almost 20 years (don't know how long exactly). It is powered up once in a while and stored inside the house. No read errors of any kind as far as I can tell.
Tell that to my circa-2000s WD MyBook that clicks and fails to initialize after ~17 years in climate-controlled storage in its original packaging. I'm about to attempt surgery on the heads to see if I can get it going again.

I also have a binder full of DVD-Rs, of about the same age and containing mostly the same data, that all read perfectly with no issues.

You're asking for trouble if you count on just one backup solution to keep your data safe.

You have the option of recovering the data through specialized shops that perform surgery on the drive.
Hard drives are a safe option if you keep them redundant, online, and scrub them regularly. They are not a good long-term offline archive format.
Just use a magnetic hard disk and your data will be fine for many decades if not centuries.

Comically untrue, and trivial to verify that it is.

You obviously need to store the data in a physically remote place like your shed, car

Let's make it worse by keeping them someplace with terrible environmentals.

If you're gonna be wrong, don't be just a little wrong...

Even if perfectly held still in a sealed box with perfect conditions, hard disk drives need controllers; they need the chipset to drive them, which has ROM and other stored microcode that is susceptible to bit flips due to cosmic ray impingement. I've traced faults in a few 20+ year old HDD's due to failed controllers likely due to such loss (either in the main controller fail, a ROM/BIOS chip fail, or less likely but still possible the cache IC, or motor controller). The data is there on the spinning rust, you just won't have a way to access it.

The failure of SSD's you warn of and the way the HDD's fail due to a dead controller aren't that different.

This won't happen with a physically written media like a CD, DVD, or Blu Ray disc. With physical media, you're separating out the media from the reader (the drive). If your disc reader dies or you cant find it, etc, you can go find any another one on ebay even decades from now, very likely.

Doing surgery on an HDD controller PCB to replace a dead controller, while possible, sucks, and may be incredibly difficult if not impossible to source parts for in the future.

Are Verbatim's M-Disc Blu-Rays still really M-Disc? There was something about how they changed the recipe a few years ago, and the counter-argument was that Blu-Ray discs are non-organic anyway, which was the main feature of M-Disc DVD.
There's very few companies now making new Blu-Ray drives too. Pioneer has supposedly stopped producing them, and the ASUS drive I bought earlier this year was one of the only SKUs on the market. Sony stopped producing high-capacity recordable BDXL discs too which means they aren't much of an option now for large backups.
If by "really M-Disc" you mean "substantially different from standard HTL BD-Rs", this has never been the case. See this review from when they first came out in 2015: https://goughlui.com/2015/10/16/review-tested-verbatim-lifet...

> While the immediate burn quality seems relatively satisfactory, I was still not satisfied as to whether these discs were truly worth their premium over commodity blank media. Looking at the discs, it seemed that the recording layers were likely made of similar materials based on the visual aspect, unlike the obvious difference with DVDs. Also, it seemed that regular recorders were intended to be able to record to the BD-R’s (according to the Verbatim text).

...

> Digging further around the M-disc site, it is noted that their section on longevity testing actually features no data about the BD-R discs at all. Indeed, if there were tests, the reports were not actually featured on the site at all. Further poking around in M-Disc Technology pages shows only references to their DVD products. In fact, it seems their patented “etched in stone” material may only apply to their DVD products.

...

> From my research into the data, it seems likely that the M-Disc BD-R is not substantially different from other quality BD-R discs on the market as they all use inorganic recording compounds (with the exception of BD-R LTH of course). From a visual inspection, the disc appears the same bright chocolate-brown, unlike the obvious distinction with regular dye based DVDs and the M-Disc DVD which was a silver-grey. But my main concern lies in the demonstrated lifetimes of quality BD-Rs which are similar to the M-Disc DVD (to which the M-Disc BD-R is claimed to have also achieved). In essence, it appears that the MKM/Verbatim BD-R DL has already reached the same level of endurance as the M-Disc DVD had claimed without being “special” at all. It is thus, hard to justify paying more for an M-Disc BD-R, and I cannot really recommend it.

The "not really M-Disc" stuff comes from a post somebody made on the "Data Hoarder" Reddit board a few years ago. The discs were changed to enable higher burning speeds which resulted in a slight difference in appearance, but he/she didn't seem to have realized that Blu-Ray M-Discs were never substantially different from BD-Rs and instead assumed that the tweaked discs were more of a scam than the originals.

https://en.wikipedia.org/wiki/Joseph_Smith_Papyri

https://en.wikipedia.org/wiki/Golden_plates

https://en.wikipedia.org/wiki/M-DISC#History

Basically your media only needs to outlive your drives, retrieval mechanisms, software interfaces, and drivers.

This is not a difficult achievement. The people marketing long-term archival storage media are capitalizing on ignorance of people who have worked with books and photographs, which are "readable" by human senses. Books and photographs are media which can be preserved and fundamentally produced on archival media from the get-go.

Backup media, since it is necessary to have machines read and interpret them, are an entirely different beast, subject to an entire chain of things to preserve. As I've commented before regarding BD-RE XL, my media and the Blu-Ray drive itself outlasted the system it was backing up, and therefore did their job, without any investment or wasting money on M-DISC "Engraved in Stone Tablets" for the gentiles.

My understanding is that the original M-Disc pitch mattered much more for DVD because it replaced the organic dye layer, while BD-R already uses an inorganic recording layer
I debated doing this recently but ultimately spent a lot of money on an LTO-6 tape drive instead, just due to the cost efficiency over time for larger amounts of data. I justified it to myself as I needed to either expand my NAS or clear stuff off it, and the drive was the same price as buying two new HDDs to expand the NAS thanks to the obscene price inflation at the moment. As I’d likely need to further expand my NAS next year (I’m shooting a lot of photo and video) I think this works out a much more cost effective way of storing this data long term when I likely don’t need to access it much.
[delayed]
Perhaps for some use-cases, the value of the data degrades at a rate proportional to its persistence layer, and doing a re-migration is not useful.
Looks like I'm a bit out of date, but older generations of LTO drives were able to read tapes from up to two generations ago, and write tapes from one generation back. Looks like this isn't the case since LTO-8. It looks for 8 and 9 theycan read and write to tapes back one generation. 10 breaks the pattern and can only read LTO-10 tapes.

https://www.lto.org/lto-generation-compatibility/

If you have money for LTO-10 drives and tapes, then compatibility shouldn't really be a problem. The drive alone is $20,000 to $30,000. If you can afford and LTO-10 setup, then an array of lower LTO drives almost seems affordable.
> is how closed-down the ecosystem for the LTO drives themselves seems to be.

Linear Tape Open is remarkably Open for an enterprise product with Open in the name.

Personally, I feel like archival storage of personal digital data is not really solvable. I feel pretty confident about keeping my online storage going while I'm able. After that, there's nothing effective to preserve it when I'm gone. That's not a huge deal for me, I don't expect my heirs to want or need these things... Photobooks with annotations are more likely to last and have some chance of being apprechiated.

EBay has a bunch of LTO-3 tape drives for sale for <$300. LTO-3 is 22 years old and can read LTO-1, originally released 26 years ago. I expect them to still be for sale 20 years from now, albeit fewer in number. I see no reason why LTO-7/8/9 drives won’t be readily available for the foreseeable future.
I guess the practical strategy is to treat migration as part of the backup plan rather than something you deal with in 15 years
I am also considering an LTO solution of some kind, but I also found out that LTO drives seemingly need some TLC and maintenance if you don’t want your tapes to jam up. I feel like at least with optical media, it’s nearly impossible to screw it up.

I really wish a multi-terabyte optical media solution took off, because that would be the path I’d ultimately take, instead of going with expensive LTO devices and media.

I bought a used LTO5 tape drive from ebay, and I've been getting used LTO5 tapes. I've never had a problem. The tape drive lives out in my garage inside my backup server. I have a 10Gbit link so I can write 140MB/s over ethernet from the server inside the house. It works perfectly, has for years, nothing "jams up" (so far). I've had to use a tape drive cleaner cartridge once since I got this thing years ago for routine maintenance. I verify writes for anything important.

Optical just didn't make sense to me. Overall it was way more expensive, and a lot slower to burn BluRay discs.

The max BluRay disc size is 128GB (which is smaller than a lot of SD cards these days), and the fastest you can write to a BluRay disc is 18MB/s. That's a lot of of swapping discs to burn my many Linux distros.

And those 128GB BluRay discs cost between $30 and $40 each. Used tapes (with a lot of life left in them) on ebay are far less expensive per TB - the lowest I've paid is about $1.40/TB for LTO5 tapes without a case. I typically pay about $4/TB for unused LTO5 tapes with a case on ebay. It's getting a bit more difficult to find those deals, but they come up every now and then.

In bulk the 128gb blu ray discs aren't nearly as expensive -- I picked up a 25 pack for about $230 18 months ago

The bigger problem is finding a drive that can read and write them -- the 128gb format support is a bit harder to find. I thought I would be good with my nice Pioneer BDRW drive but it didnt have the support so I had to buy an LG unit. So now my desktop has two BDRWs -- how very 2006!

I recently got 90 LTO5 tapes for $200 total. That's 135TB. A 25 pack of Blu ray will get you to 3.2TB.

The LTO5 drive cost me about $150 on ebay, and an HBA card was $30. Yay for datacenter hand-me-downs.

At least where I am, Bluray disks are currently cheaper (in bulk) per megabyte than hard drives are.

Tape reigns supreme in cost per megabyte, but the infrastructure costs are astronomical.

SD card is an atrocious archival/backup format as data retention is rated in low single digit years - though I am continuously amazed I can store 100GB or more on something so tiny. I would love a true archival SD card format.

>Tape reigns supreme in cost per megabyte, but the infrastructure costs are astronomical.

A used LTO5 tape drive cost me $150, an HBA card cost me $30. That's literally the only "infrastructure" I needed to acquire. The tape drive has been running fine for several years. It lives in a tower PC, so I'm not sure what astronomical cost you are talking about. Sure, a new LTO-10 drive costs $20,000+ but I don't need the latest and greatest. I just need cheap, fast storage.

I've had very mixed success procuring used tape drives in the past, with a very high rate of failures. Looking at LTO-5 drives available to buy secondhand in Australia, they range from $300-$1000, predominantly shipped from the US or China. Maybe it is just because I've been burned before with used tape drives, but that is probably more than I am willing to spend.

Unfortunately I don't have either a tower PC or a server in which to install it, being predominantly an iMac/Laptop user - so would need to factor in an external enclosure which are certainly not cheap.

It feels to me the broader issue is that consumer archival media is dead, with the (dying) exception of optical disk. I would love consumer tape drives to become a thing again - lower capacity, smaller, and USB-C connected.

I've had exactly one bad tape out of several hundred. YMMV
As a person instead of a company, can't you just buy a stack of hard drives instead of a stack of tapes?

Even with the inflated prices nowadays 3tb 3.5" sata drives are cheaper than 2.5tb LTO-6 tapes.

FWIW both CD- and DVD-RW are less reliable than the read-only counterparts. M-Disc is supposedly very good.
I have used tape (of various flavours, QIC, DAT, LTO) and DVD+R and DVD-R for decades. Bluray has been the most reliable of all of them for me overall.

Tape is amazing, and resilient, but the drives are fragile and expensive.

The sad part is that Panasonic/Sony/Pioneer have pulled out of making drives (I believe Panasonic still make some for their own vertically integrated products), which just leaves Hitachi-LG.

I see several used LTO-6 tape sales on Ebay, for about $13 per tape. The cheapest used SAS drive I see is about $30. I'd trust a used tape over a used hard drive every time. Tapes have a 30 year life span, I can't imagine a hard drive would come close.
I'm not sure I'd trust a used tape. They're generally only rated for a couple hundred or so writes. They'll totally sit on a shelf for 30 years, but writes tear them up.
If I write and verify the write, then that's good enough for me. I don't only buy used tapes, I have about 100 that are new/unused that my important data go to, and really important stuff goes on at least 2 tapes.
Ignoring the cost of a tape drive, tapes are cheaper per TB, yes. I wouldn’t write off used hard drives though - buying only the cheapest per TB used enterprise drives, I’ve only had a ~5% failure rate across roughly 200 drives with 2-5 years of runtime (while I’ve owned them - they usually have 4-6 years when I get them). I usually get rid of them once they hit 8-10 years of uptime (or if any get any uncorrectable sectors).

If you only need ~50TB it’s probably both cheaper and easier to just buy hard drives, even today, factoring in the cost of the tape drive. Maybe even 100TB. I’m personally backing up ~225TB off-site these days and I still find AWS S3 glacier deep archive to be cheap enough - at current prices I’ve decided the hassle of doing tape again isn’t worth my time unless I’m over 300TB.

Blu-rays are way more durable than CDs and DVDs for some reason
The Blu-ray spec demands a better coating.
The biggest advantage of LTO is that it's designed to be offline. Hard drives start to get weird when they've been sitting still for too long.
> The biggest advantage of LTO is that it's designed to be offline.

It's been a while since I did enterprise backup, but when I did, most backup software had a 'bootstrap procedure' that you could use even if you were completely destroyed.

First, most enterprise software does not need a license for restorations, only for backups. In a disaster recovery scenario you're unlikely to have the keys.

At end of every backup run/day, an e-mail is sent out with the tapes and offsets that have the indexes and configuration backups. You can send this message to an e-mail address that is separate from corporate e-mail (in case that gets wiped as well); back in the day you could send it to /dev/lp0 (and stick a copy in the box that the offsite tapes are transported in).

So even if your entire infrastructure is wipe (e.g., see NotPetya and Maersk), there's still a starting point for restoring your restore-backup server, and rebuilding up your infrastructure.

Remember: your backups have to be inaccessible to attackers.

The new type type of laser heat assisted drives might change that. Laser and heat to write data. Kind of similar to old mini discs
After reading reading the "Environmental and shipping specifications for LTO tape cartridges" [1] I came to the conclusion that tapes are not that great.

    Environmental   Allowable       Recommended 
    Factor          Storage         Storage
    Temperature     16 to 32°C      16 to 25°C
                    (61 to 90°F)    (61 to 77°F)
In the summer it's not unusual to reach temperatures over 30°C in the house.

There's also this story [2]:

> I had a room full of servers, many with 16-36 disks per server and had an AC failure. Things turned off as planned, but it hit 115°F [46°C] for a bit. None of the disks died, majority of the tapes did.

[1]: https://www.ibm.com/docs/en/storage-deep-archive/1.3.1?topic...

[2]: https://news.ycombinator.com/item?id=21066981

Anecdotal, but...

I doubt that the recommended storage conditions for qic tapes were much different from the above; and I've seen a system boot from a qic-120 tape that had been kept in a non-climate controlled ministorage locker in Santa Fe NM for 17 years.

Holy cow, thank you for this. On that same IBM[1] link, it reads:

Maximum wet bulb temperature (Only LTO-8 and earlier cartridges) / Maximum Dew Point temperature (LTO-9 and later cartridges), all across the board, allowable / recommended / shipping: 26°C (79°F).

In Florida, we like to refer to 79°F dew points as "Tuesday."

More to the point, the time I want those LTO cartridges to earn their keep is during a disaster scenario. The most likely disaster scenario--other than fire--is hurricane. High temperature and humidity, and no power to run the HVAC to mitigate those. (Unless you have generators with fuel to run for days, and your HVAC, generators, and fuel tanks survived.)

Oh very interesting! This isn't (currently) too much of a concern for me as I live in Wales, and though it's ridiculously humid here there's not much risk of it hitting those temperatures inside my house the vast majority of the year. I shoot a lot of photographic film as well so I'm used to storing these sorts of media, though I probably won't be shoving the tapes in the fridge beside the film. Currently they sit in my windowless office.
Here in the north of Scotland where it never really gets below 5°C or above 25°C making it t-shirt weather all year round, I think the tapes will be okay in a dry reasonably-insulated storage room.
They’re engineered like this because 99.9% of tape storage media is sold to corporations who store their backups in climate controlled buildings with backup generators.
I can see this making a lot more sense for photo/video. The Holocron approach is neat when your truly irreplaceable data fits on a handful of discs, but I would not want to burn and manage dozens of them every year
Yeah I'm tending to shoot around 1TB of data per wedding I cover now that video is in fashion for it. Thankfully it's exactly the sorta thing tape is suitable for - once I'm done I can just archive a couple of weddings off per tape. It's generally very unlikely I get requests for that data later (once they've ordered prints) so doesn't make much sense keeping it on spinning disks in my NAS.
> ultimately spent a lot of money on an LTO-6 tape drive instead

I'm semiseriously considering getting the old LTO3 library from work and putting it to use. Sure, it's "only" 400GB per tape but I have a literal vanload - the racking takes up the same volume as a mid-size Ford Transit - of new and used LTO3 tapes to dispose of too.

A petabyte would only be 800kg or so!

The problem with optical disks is that nowadays they have a too small capacity in comparison with magnetic tapes or magnetic disks, so they are useful only for archiving small amounts of data, of a few TB at most.

If you want to archive up to 100 TB or even 200 TB, the cheapest way to do that is with HDDs, but you must keep in mind that you must use at least duplicate HDDs for everything, preferably stored in different locations, to avoid losing data due to files that become corrupted after a long term storage or due to some accident. Moreover, you must migrate the data to new HDDs periodically, e.g. every 5 years.

If you want to archive several hundred TB of data or more, magnetic tapes become cheaper and they are more reliable than HDDs. You still need to migrate the data, e.g. every 10 years, but much less because of the risk of corruption or mechanical failures, but because the tape format may become obsolete and appropriate tape drives that can read it might no longer be available.

Now that I'm at ~$350 a year on Backblaze I need to look at this again. Or the LTO-6 tapes.
Does anyone use it for family photo backup for generation to pass on? I am struggling to come up with a good back up solution that last ages.
Buy a new hard drive every year, make it your primary drive. The previous one becomes your backup.

Or just keep it attached to your system, so spyware, adware, and the NSA will make free backups for you!

This has been my system for a while now. The funny side is that, I have yet to have a drive failure so the amount of local backups is getting silly.

The other thing I did was I ended up collecting HDD's out of old DVR's people were throwing out. Those drives are used for cold storage but not for anything important because they already have tens of thousands of hours on them and in not the best temperature conditions. Again, non have failed yet.

Funnily enough, the only HDD I have failing is a 2.1GB Quantum Fireball from 1997 in my Pentium 1! Really should swap that for an SD card via IDE.

I've had drive failures now and then. I once dropped a 10T drive on the floor, and that was the end of that!
Oh no, that is an awful IT horror story.
Not that bad, I had a backup.

I've been programming for 50 years now. I've had all sorts of failures, including two incidents of my desktop computer burst into flames.

I now set the desktop on a piece of sheet steel. I also only use steel enclosures for them.

I have a Carver amp that runs 24/7 in my office, for the last 40 years. It's now emitting a 60 cycle hum, which I learned is due to decaying capacitors. I guess I need to get a cap kit and fix it.

I had a 4 TB drive recently that didn't really get hit, dropped, or anything drastic. Just lightly knocked on the SATA connector. Which snapped off.
A couple of years ago, I repaired one of a pair of drives (I think they were 750GB) that had the connectors mysteriously snapped off[1] while I was poking around in a client's system.

It was a special system, Linux-based, running proprietary software for expensive proprietary hardware.

I apologized profusely. I got them some new hard drives coming right away. I found a forgettable donor drive in the boneyard that had a close-enough footprint for the connector, harvested it with hot air, and put its connector onto one of the broken client drives.

A little bit of dd and some mdadm magic later, and they were back in business with that RAID-1 array like it never happened (other than just a few days of downtime on a system they only used occasionally).

---

A different time, 25 or more years ago now, I somehow broke a 4-pin Molex[2] power connector off of one of my own IDE drives. I don't remember the details, but I do remember that I fixed it by soldering a Y cable power splitter into the holes on the board. This gave me a drive that worked, and that had power pass-through. It was kind of nice to have (and it worked for years and years after that).

---

[1] I wasn't rough with anything; I was just moving wiring out of the way, as one does, to attack the dust bison that had accumulated over the previous decade or more with some canned air.

I somehow broke the connectors of both drives, anyway. I can't explain it and I don't have to, so I guess I won't bother trying. They were fine when I started and they were broken when I finished.

[2]: Those aren't Molex connectors. Everyone calls 'em that, like everyone calls adhesive bandages Band-Aids, but the old drives used a design characterized by the Amphenol Mate-N-Lok connector design which had nothing to do with Molex.

I do this, alongside other methods.

The problem with archiving data is you always need to be prepared to use the "next big thing" once your preferred method becomes unobtainable.

I currently have :

- Photos in iCloud (Live working copy)

- Backup at home on NAS (updated nightly, snapshots)

- Backup in pCloud (lifetime, mine or theirs, no illusions, hopefully lasts a couple of months if I die unexpectedly)

- Yearly archive on external HDDs, identical copies stored in separate locations, clearly labeled "Photo backup". No encryption, no archiving, just plain APFS formatted drives that can be plugged into a computer and read.

- Yearly archive M-disc blu-ray discs, again, identical copies, stored in separate locations. One disc per year labeled as such, ie. "Photo Backup 2020". The idea being you restore them in chronological order and have a complete set.

For family photos, my solution is as follows:

- A photo album extended with a curated set of photos each year. The real deal!

- NAS at home and blu-ray off site (parents home) for all photos

Your backup plan is to store the asset in [checks actuarial tables] the place most likely to be withdrawn first?
Sorry if it was not clear: The NAS is at my house but houses may burn down. THE discs are at my parents house for off-site backup. Like the sibling poster says, I would be the one going through the things anyway. But if unsure you can also ask a friend to keep them, one blu-ray can hold tons of photos and doesnt take much space.
I love curating my photo library but after I kick the bucket I have to assume the number of photos that my future relatives care about will number in the dozens.
I've talked to many deceased estate cleaners over the years, I have seen them in action many times. The amount of stuff people want to keep from relatives is a rounding error of 0%.

Whole lives of collecting sentimental goods all end up going in a giant bin, a whole house cleaned out in less than 2 hours. Photo albums are rarely kept, they just go in with it all and off to the tip usually in a single morning.

What you think is important to you, is basically just trash to your relatives/kids. They will probably have want they want before you are gone.

Same here. 240,947 photos in there right now and organized enough that I can find stuff (of course a single nice social gathering or outdoor adventure can be 100+ photos - cheap to store so why get too picky).

So why is it that photos >20 years old just aren't that interesting any more? Because photos need to be augmented by memories, or by stories from others. Once they're just pictures... and once I'm gone... no more interest.

We'd never throw out old physical photo albums. We have a cupboard full of them that my mom left behind. Some of the photos in them are of people we didn't even know, and now won't even know the names of. We keep them because we still have a little bit of connection to them - we knew the people to whom they were significant. But soon they'll just be landfill.

That's how human memory must work. We can't have every generation leave behind hundreds of thousands of pictures, piling up forever. Memories of past generations get purged until 2-3 generations back they're forgotten.

Of course there are exceptions to every rule. An uncle was persuaded to write his memoirs about his childhood on the farm in the middle of the last century. That will still be a good read after he, those who knew him as a person, are gone. Assuming the books survive - it was a small, private printing only.

No large photo collection is complete until you plot all the photos with location on a map.
It is important to me that my kids have memories of their childhood after I am gone. Something I have very few of for myself.

So my archiving needs are in the decades, not the centuries. If my kids can access them in 20 or 30 years, I will be happy.

I am skeptical of cloud services for such purposes because cloud services require ongoing maintenance: contractually, financially and technically.

If you want to pass stuff on stuff so it will last for ages, then: Get it done already. Print it, email it, post it somewhere and share links, have it framed and then delivered by a singing courier -- however it is done, then just do it and get it over with.

Now it's their problem to take care of, ignore, make fun of, or whatever, which was always ultimately going to be the case anyway.

This allows you to back with living your life while you still have one.

There's little point to bothering with predicting what the future will look like after you're dead, and even less point to trying to influence how others will behave on your behalf.

To extend:

If you really want to have a lasting legacy of documentation, then build it and send it. Send it early, and send it often. Be the most interesting person on Earth today for your group.

Chances are fantastic that nobody will want to volunteer to build it for you after you're gone. But if it's still important for you today, then make it easy on them: Send it right now.

...or don't. Really.

Telling some great stories at gatherings while perhaps sharing a couple of visual-reinforcement photos on a phone is enough unless you're a Kennedy or something. A wonderfully-interactive story is worth a thousand photographs. If you've got interesting stuff to share, then it will be interesting to hear, and to interact with, and it will remembered.

For awhile.

The stories will be retold and will ebb and flow, and will subsequently die in a generation or three at most, but that's OK: You'll be dead, too. Get over it. By generation 4, those kids won't know who Uncle Fred was anyway except as kind of sleepy myth about the carefully-preserved BD M-Disc archives that were binned by Aunt Sally a decade prior because even Aunt Sally didn't care anymore, and couldn't be stuffed to keep them.

(Except! In the odd chance that you are a Kennedy or something, then: Hire biographers early, and often. Get that money, and publish that stuff while you can still direct the proceeds.)

(comment deleted)
Each year I select about 100 representative photos and get a photo book printed. It’s not a perfect solution but at least they might get something to see 100 years from now.
Am I unusual in not having looked at my parents or grandparents photo collections?

I occasionally look at my own, but that's about it.

Nope, I think it's really hard to assemble an album with sufficient context to warrant your time and interest.

If an album is well annotated with dates and locations and names and descriptions, it might be fun, but I suspect the number of old photo albums with this kind of effort put in them is rare.

No. People in my grandparents' time and place couldn't afford cameras. There are maybe one or two photos of a person. Some siblings are only notes on an old church ledger.
My dad created/curated a family archive dating back to the 19th century. Since I moved it to immich people sometimes look at it! In fairness, though, they mostly look at the new stuff.
Have you considered carving them into the walls of a pyramid?
Data durability is one thing.

Being even remotely relevant to future generations is quite another.

I had so many of my users ask your question that I wrote this up: https://photostructure.com/guide/how-do-i-safely-store-files...

But a month or two after I initially wrote that, my brother and I found a couple boxes of photos from my parents and grandparents. No notes. No dates. No albums. Just envelopes and envelopes of undated photos.

Taking the time to make an annotated printed photo book, including dates, people and place names, and any other context is going to make those images a lot less irrelevant to future generations, and printed media, stored well, should easily last 50-100 years.

I would not make a bet that you'll be able to procure a bdxl reader in that time period.

Print them out on archival photo paper.
Yes. AVCHD for video (or BDMV), and JPEGs in folders. Labelled. They play in every BD player I've tried them in from 2006 onwards.

And the crazy thing is - we actually watch home movies in a linear format as a consequence. Rather than picking a clip here and a clip there amongst a sea of photos, video can be laid out in a linear timeline and watched without editing.

(I still use a professional Panasonic video camera that shoots AVCHD, which makes this super easy. Because editing video is not something I'm ever likely to have time to do)

I'm surprised this is not more common. I have been doing this since 2016 (started with DVDs and quickly moved to Bluray since writable DVDs are more prone to failure).
Do you periodically re-read the old discs to check them? I burn a par2 recovery set (around 10 percent) onto each disc alongside the data, and once a year run a full verify with dvdisaster or just par2 verify. A disc that starts throwing correctable errors gets re-burned while it can still be read cleanly, which matters more to me than the rated lifetime of any single disc.
Sealed pelican case has decent precedent. A small regional bank I was aware of used basically this exact strategy. There were couriers going between branches daily, so it was easy to cross-ship a sealed DR/backup box to a remote branch every day. It stayed in the remote vault until its return journey
Having long-term backups like this I wonder if bundling a known-working Blu-ray reader or even a combination writer is a good idea. Hypothetically a reader would be hard to come by. With a reader I would either need a adapter to make it work.
> In case of a catastrophic fire for example, both the computer and backup drive could be destroyed.

I just keep my offsite backup drives in my office drawer. There is a note on them that says they are personal backup and not to be plugged into company equipment. If you have an office this is probably the easiest solution.

If a disaster wipes out my home and my office then there are bigger things to worry about than my data.

(comment deleted)
Did this experiment and it works but with caveats:

1. The disks bigger than 50GB require a BDXL drive to read, which means spending $300 on a disk player. Which also means rare to find in the future. So if you want larger compatibility use 50GB or smaller M-disk.

2. Japan sells supposedly original MDisk but they look rather black than gold, so they are not the original. Still strength and longevity is quite good on Blu-ray.

3. Storing the disk on materials that degrade like cloth/plastic sleeves may be counterproductive. I got a hard case but it takes effort and is increasingly hard to find.

Still the 50GB play nicely even on old players.

For something explicitly meant to be recovered in a worst-case scenario 10–20 years from now, optimizing for boring, widely available readers seems more valuable than squeezing another 50GB onto each disc
And squeezing 50GB out of two layers seems like it might have better long term reliability that squeezing 100GB out of three.
I would include a Blu-ray drive in the case. Who knows how long will they be available.
Neither TFA or the comments seem to start with the two most obvious rules: Digital data can be perfectly replicated and it can be checksummed. Those should be the pillars of any backup. Regular transfer to new physical media and check.
I have created an archive on 100-Gb M-disk sealed in a tempered glass enclosure. Placed it in on a tiny island in the White sea.

https://www.tautology2.net/2022/12/14/where-will-the-arrow-o...

Great post, so I'm here to nitpick it:

> They needn’t be particularly beautiful, either. Better not to stir strong aesthetic or religious feelings with inscriptions and the like.

I remember a discussion on architecture where the point was that, while newer buildings are built with more sustainable techniques, they are also ugly and therefore no one cares if you want to demolish one. Buildings perceived to be pretty, on the other hand, are more likely to stay around longer and therefore have a lower environmental impact in the long run.

Making your M-Disk aesthetically attractive may be a way to ensure that they are around for a longer time, if only because someone may confuse them with electronic waste and take them somewhere away from nature to ensure they are properly destroyed.

Sure, both is true. I didn't want to make the object too attractive, so that it makes visitors want to take it with them or disclose its location. And not too ugly to provoke an urge to break it or throw into the water.
I would be so afraid of breaking the disc if I ever tried to break the glass to retrieve it. Hopefully there is a mechanical engineer around when someone wants to retrieve it.
This is a tempered glass that should break into small pieces around 1cm each
I've gone all in on M-Disc (and even regular HTL inorganic bluray). I've been using it as an archival medium for about five years.

With hard drive prices being what they are, I feel like my decision is increasingly justified. Storing large amounts of personal data (documents/photos etc) either in the cloud or "hot" on local drives, where the access patterns are infrequent, has simply stopped scaling.

Having discipline around burning disks has also made more disciplined about archiving.

The burn speeds of 6x Bluray media aren't substantially slower than SMR hard drives. During a recent archiving session, my SMR 4TB hard drive was sustaining 60MB/s. My Bluray burner sustains 27MB/s.

A pack of 20 BD-R DL Blurays (50GB each - a total of 1TB) from Japan was less expensive than a 1TB 2.5" external hard drive.

Cloud storage isn't tenable either. Despite having a 1000/400 fibre connection, with international links from Australia being down, backing up to Hetzner only sustains 5MB/s - backing up 5TB to AWS or equivalent would be faster, but likely to cost around $700 a year. This is on top of the $140 a month for business grade internet with reasonable upload speeds.

I'd love an LTO drive, but that requires a server or appropriate desktop, and seems overkill for 5-10TB. Even for secondhand drives, I'd be looking at multiple thousands for a a complete setup.

Continually upgrading the NAS with more bays and more drives is also not sustainable.

I index and catalogue each disk with Neofinder (great piece of software) that allows offline browsing and searching of each disk. The searching and browsing is more performant than the software provided on my Synology.

And as an upside, videos and photos of the kids can be viewed by anyone capable of sticking a disc in the drive, without needing access to servers or cloud providers (I burn some in BDMV format with no DRM, so it should play in any Bluray player or computer).