I wonder what the maintenance costs would be to keep a digital document storage system running robustly and effectively for 800 years (as long as the Magna Carta document lasted)?
(Not trying to imply that vellum should be the only storage method but that sounds like cheap robustness, and I'd not want to throw that away just because you want to be capital-D Digital.)
The other thing is it is real of course, the intent of it isn't for tourists, neither are the 'castles and queens' and the branding as you put it.
Not saying it doesn't help tourism, its just not the intent of it all & doesn't have much bearing on the decisions being taken as much as the simple act of continuing traditions.
The closest equivalent I can think of immediately is Amazon Glacier, which is intended for 'long-term' storage where they take care of transferring the data as necessary to preserve its accessibility. Obviously they're thinking years-to-decades, not centuries, but as a ballpark it may work.
Their current pricing is $0.0007 per GB-month, or $0.000064 per KB for 800 years in today's dollars. Since the Latin text of Magna Carta is 28 KB of UTF-8 text, that works out to $0.0018 to store the text for 800 years. You'd obviously also want a high-resolution scan of the original document. I don't know how big that would be. If it was a 100MB image, it would cost $6.66 to store for 800 years at Amazon's current Glacier prices.
Obviously we're making immense assumptions about the reliability of the service, of price stability to deliver the quality of service, and so on. And perhaps the biggest issue with this analysis is that we're also relying on huge economies of scale for it to be profitable to deliver an archival service like this over centuries.
If this sounds crazy cheap to you, you're not alone. As technologists we're used to dealing with huge volumes of data, so it's easy to forget that historically these volumes simply did not exist.
> Obviously we're making immense assumptions about the reliability of the service
Even assuming the corporate entity will survive that long is dicey. To ensure survival of a document on such as service you can not just look at the cost of a storage service, but the cost of infrastructure to care for it. E.g. just the matter of keeping an organization alive for the duration to ensure an account is kept active and live and prepared to ensure each and every monthly bill is paid.
The Magna Carta survived wars and revolutions and changes of government and economic collapses and the Black Death. Glacier may struggle with a long electrical blackout.
Hard to say, considering tech will continue to evolve over that time. Who knows, 100 years from now we might have found an extremely reliable way to store digital information long-term, thus we'd only have to do the "copy from one medium to another" dance for a few generations. Or we might not.
It would require a trust, a fund and a dedicated organisation, possibly using lawyers. Legal documents relating to land and titles have survived for hundreds of years in this manner.
It's a social not a technological problem.
But how to do it is much easier to solve.
Why to do it is the actual problem. What to keep and when to keep it without knowing the future.
How does anyone know that the 100 photos that a person from Morocco put onto Flickr 5 years ago would have value in the future? Why should those be archived? In 200 years these photos could be immensely valuable.
How can this potential future value ensure the archiving of the photos today? Is there economics for this?
Our research group is looking at this exact question, and it turns out that there are several factors that effect the balance between cost and reliability/longevity in an archive.
Paper: http://www.ssrc.ucsc.edu/Papers/gupta-mascots14.pdf
If it's laser toner, then it's resin and carbon fused to the page at a high heat. Carbon is an element so can't fade, and resin's only problem is ultraviolet. Pretty stable combo. I did a lot of research on this subject and concluded that acid-free rag + laser should last 500+ years.
I recall reading about recovering writing that had been effaced from a palimpsest and overwritten. The ink from the first writing had iron oxide in it as a pigment. This allowed the former text to be detected magnetically, even though it was no longer discernible to the eye.
Carbon black and iron(ii,iii) oxide would certainly be very long-lasting pigments, and would likely remain legible until the substrate disintegrates.
It's the color printing that is hard to preserve. Gold leaf, of course, lasts as long as the page. Organic dyes tend to fade over time and with exposure to heat, oxygen, and UV light, whereas inorganic pigments are more stable.
Vellum works and has been proven to work for hundreds of years. Archival paper is interesting but largely unproven tech. On paper it works (no pun intended) but we really haven't seen if it will last even 200 years, and we should before we start printing our most treasured documents on it.
Digital long term storage is just, no. Look, all key UK documents are ALREADY digital they're available to download right now. However we aren't talking about digital Vs. non-digital, we're talking about what to store BACKUPS on.
Digital long term storage is something people who frankly know little about technology point to. It is up to IT-types to figure out the costs and the "how?" from then on out. There is no digital storage media which will still exist reliably in 200-400 years, not CDs, not HDDs, and not SSDs. Tapes are the closest and they won't last that long.
So we aren't talking about slapping it on a CD and storing it in some archive, we're talking about slapping it on a piece of digital media today and then moving it every one hundred years without fail, or we lose it forever... If one government or generation loses interest then future generations suffer (see the current government as an example: moving away from reliable Vellum to unproven papers).
Plus with digital you also have to worry about: solar flares/EMP, format knowledge (both of the file system & file format), media protocol knowledge (SATA or CDs might be forgotten technology), and the ability to alter historical documents without detection (not that vallum is immune, but it requires more skill/time).
PS - The "Digital Preservation Coalition" are a group of historians and archivists. They aren't technologists, they just point blindly at digital without explaining the nitty gritty of HOW.
Entirely agree with you about the current lack of longer term stable data storage options.
That said, waiting the full duration of a material's projected life span before actual use of that material would limit options hugely, especially with the massively increasing corpus of generated data.
In this sort of context (amount of data) one can spread your bets and use vellum, as well as other methods, until other more stable forms of storage are easily available.
Archiving things for future civilizations to dig up is one thing. Then you may be talking about hundreds or thousands of years that the data just sits there. Then you have to worry about the longevity of bits, and whether the technology and knowledge to read the data will exist in the future. Digital is very hard for that use case.
On the other hand, storing data for an active/continuously used archive doesn't seem so hard. You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten. Keeping digital data around for a decade is pretty easy. If you do that 100 times in a row you have kept it for a thousand years and you never had a file format or a storage media older than a decade. You could "forget" to upgrade the archive, but you could also "forget" to scribble on the calfs some time.
> You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten.
But you need to avoid the mistakes that people have already made: using some weird format that dies, and then having all your data locked away in a semi-readable file.
Off the cuff, I'd point to any format for which you'd have to fire up an emulator to run the software to read it. I would also include any format which is not openly documented or which requires a propriety tool to view it. When you look at archiving digital formats, you have to look at the whole toolchain and realize that the substrate of the medium itself will degrade, the electrical or magnetic field will degrade, the storage format will quickly become obsolete, the digital file format will be and so forth. More relevant to the discussion at hand, supposing you do intend to store important documents in a digital format as a long-term archive. What format do you store the instructions for reading and decoding the archive? Wouldn't carbon pigment on vellum be stable enough?
Edit: On further consideration, I think we have to regard digital media and formats like a form of encryption (a simile, I don't mean this literally) which gets stronger over time. Time will bury extant evidence of what is contained on the media and how it is stored. Why should not make the information as explicit (plain-text) as possible? People in 800 years will still have to spend their efforts decoding early 21st century English legalese.
What are these "weird formats" that keep getting referenced here? LZH? ARC? PKZIP? WordPerfect? Filesystems formatted in FAT16? All of these formats are still very much readable.
Some formats that are no longer usable in general, such as punch cards, were obsoleted consciously. And you might struggle to find a LaserDisc reader. Otherwise, we should be fine. Or am I missing something?
Yes, you're missing the hindsight bias. People using punch cards and laserdiscs weren't warned ahead of time that things stored on them would become unretrievable soon after. Nobody knows in advance which formats/media will be "obsoleted consciously".
The overlap between punch cards and other formats was decades.
"During the 1960s, the punched card was gradually replaced as the primary means for data storage by magnetic tape, as better, more capable computers became available. ... [P]unched cards were still commonly used for data entry and programming until the mid-1980s when the combination of lower cost magnetic disk storage ... made punched cards obsolete ..."[1]
Yes, I listed wikipedia as a primary source, deal with it.
And remember, punch cards and laserdiscs are famous obsolete storage formats. What if you put your important information on an Iomega Jaz cassette, or stored it in a Bernoulli Box cartridge?
MUMPS still sees a lot of active use today -- and MUMPS databases store data in plaintext, in ways that tend to be pretty well-documented and relatively easy to understand. EBCDIC, Betamax, laserdisc, Zip drives, and HD-DVD are probably better examples.
Didn't you say that MUMPS is an obsolete or generally weird format, not that it's something you wouldn't use?
But yes, I use it, both professionally and personally. Legacy MUMPS is not pleasant -- the VA's code looks like someone left their telegraph out in the rain -- but the language is astonishingly good at string-handling, mostly due to certain constructs that make it very hard to translate into normal languages.
So I've got an HTML file and a PDF file, one was generated off the other. In the generated file, when I got to highlight some portions, I discover that it has been converted into a column format and it pastes wrong. It is easy to imagine that PDF being converted into a third format, and that conversion happening slightly wrong, from the aforementioned weirdness.
So imagine you have generations of digital files, converted from format to format to format. Why? Because software engineers are assholes who keep inventing new formats, for stupid goddamn reasons. A hundred generations from now, you'll be looking at copies of copies of copies which aren't just checksumable bit-wise copies but transcriptions, with transcription errors. If our descendants are lucky, all of those different versions will be preserved as well, but that just means that if you notice a possible transcription error, you'll have the opportunity to dig into two hundred year old character and file encodings to try to figure out what the original text was.
Which is not that different of a situation from trying to guess whether a scribe three copies ago misread the scribe four copies ago's atrocious f for a t.
Yeah converting from one format to another that isn't completely compatible might be an issue in the future. Even how to preserve websites isn't exactly intuitive because as you saw the conversion to PDF was faulty, and doing "file" > "save as" would not yield the same HTML because the browser modifies the DOM. We have to start using formats specifically designed for archiving such as WARC for webpages: http://www.digitalpreservation.gov/formats/fdd/fdd000236.sht.... (relevant and interesting site in general, run by the Library of Congress)
Here's an example from Brian Moriarty's retrospective on the game "Loom" [1] in 2015, a mere 27 years after work on it began in 1988:
"The third disk here is the only known copy of the original design documents for Loom. It's an 800K Macintosh floppy employing a proprietary format readable only by a vintage Macintosh drive (thank you Apple). In preparing for this lecture, I obtained a dusty old Mac which had not been turned on since 1997. After reseating all the cables and boards I got it to boot and determined that the files on that disk are still intact and still fully readable. A few moments after making this happy discovery however, and before I could actually retrieve those files, the hard drive in the Mac crashed and died permanently. I'll recover the files eventually, but for now you'll have to rely on my failing memory for how Loom was conceived."
We have a much better idea of what digital technologies and formats will last into the future than we did in the 1980s. Pick reasonably open formats and standards that have been around for a long time, and are used every day. HTML has been heavily used for more than 20 years, and it's only getting more widely used. UDF (the DVD file system) has been around for almost 20 years, and ISO 9660 (CD file system) even longer. Better yet: mix, match, and do all of them!
The problem is not making sure you upgrade, nor is it about technology. It's identifying the things to keep without knowing the future.
50 years ago and all those letters that were burnt or composted because no one bothered to keep the sentimental letters of soldiers in a world war.
20 years later and all those millions of photos that were on Flickr but no one bothered to upgrade.
100 years later and all those GPS logs from taxi cabs that were logged by firms are lost as no one thought they would be interesting or useful.
600 years later and these floppy disks with meeting minutes on them about an early internet organization are lost but might well be worth more than they are now.
It's not free though so every company with data makes decisions about what is worth saving. Sure the costs are always falling but you have to pay (at best) today's prices to store today's data.
Roll-over to new tech is safe. The content of discussion is inherently reliable due to its digital nature.
Keeping multiple copies is necessary practice and the long-time trend is that the next copy you produce has no greater cost than the previous copy (if you wait years, it will be significantly cheaper).
I thought the idea that it would be difficult was a misunderstanding of the task to begin with. No one is trying to archive digitally for a thousand years. That's hard. Archiving for 10 years is easy. As many times as you want.
>You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten.
"Just". Have fun justifying the cost of re-encoding and re-archiving all of the laws of the UK every few years. The advantage of vellum is that once it's written, that's it. You don't need to go back and re-encode and upgrade to the latest storage technology. It's there, and it'll last literally a thousand years if it's stored in a reasonably dry, cold, and dark location.
How much is it? One hard drive, max. Write it on a few hard drives and move it to something else in 10 or 20 years before the standard is outdated. I doubt we won't be able to read text data any time soon. It's not handling itself but the risk of data loss feels at least as low as someone stealing or sabotaging a single physical document.
While there's much truth in your statement, I wouldn't be so quick to discount the knowledge and experience of archivists, who have collectively as a profession put a tremendous amount of time and effort into researching archival theory and practice, both physical and digital, and who count among them extremely knowledgeable technologists. They are acutely aware of the fact that you can't just burn a cd and call it a day. With that said, I don't know anything about the Digital Preservation Coalition; you may be right about them in particular. [Edited for clarity]
Doing this doesn't cost much, and it's a basic archive of the business of the people worth keeping in human readable form.
At the end of the day, tech is good, but the business of people needs to be human readable, accessible in a court, trusted by all. This is a basic requirement humans have, and without it in place, we are forced to trust and depend on tech for law, and it's real consequences.
Nothing wrong with using tech, mind you. We can, should and do. It's more efficient.
But, say something bad happens. That's what the vellum is for.
That's a good choice. Laser-marking stainless steel sheet is a common operation with modern laser cutters. That should last thousands of years. It will survive water, fire, and handling.
Long Now is doing something like this but with a nickle disk instead of stainless and etching and electroforming instead of engraving. It allows much smaller features vs engraving. They're engraving 13000 pages about 1500 languages onto a small 3inch disk.
> Archival paper is interesting but largely unproven tech. On paper it works (no pun intended) but we really haven't seen if it will last even 200 years, and we should before we start printing our most treasured documents on it.
No no. This doesn't make sense. If we're not going to put our treasured documents on archival paper until we've proved that archival paper really will last for centuries, they're never going to be put on archival paper, no matter how great it might or might not be, because we'll all be dead.
You wanted to say we should see how archival paper pans out before we stop printing our treasured documents on other types of material.
Yes, also makes complete sense from a cultural perspective. If there is any sort of process involved in making a copy on new formats as they emerge, then you have to maintain a culture of people that are willing and able to execute the process. Far simpler to use a technology that is proven to work over several hundred years. Exhibit A: The Magna Carta [1]. With care, you can pick it up and read it, even today,
Automation won't hack it either. At the moment, the most complex machine that we know how to make persist into deep time is a very simple clock [2], and even that requires human intervention.
In Anathem [3], Neal Stephenson speculates on what this combo of culture and deep-time technology might look like.
> Archival paper is interesting but largely unproven tech.
Archival paper is tested by artificially aging it (e.g. exposure to heat and moisture to simulated passage of time.) Some long-term tests are underway to determine whether accelerated aging tests accurately simulate the natural aging process.[1] And there are associated standards, e.g. ISO 5630-3.[2] But you are correct, we really have no proof of how modern archival paper will perform over a multi-century timeline.
Actually, archivists and technologists have been working together to solve the problems of digital preservation for a long time. There is an ISO Standards for digital preservation systems - ISO 14721:2012 originally authored by and still maintained by The Consultative Committee for Space Data Systems (CCSDS, which part of NASA). There are commercial systems that were developed to these standards, among them Preservica (http://preservica.com/) which is used in many archives in Europe.
Archivists are acutely aware of the obsolescence of digital formats and media. And the solution cannot be just printing them out on vellum or archival paper because increasingly records are being created digitally that cannot be meaningfully or adequately converted into printed formats. I am talking about records that are audio, video, high resolution photographs (like those that NASA is seeking to preserve digitally) etc. Even emails when printed out onto paper would be missing a lot contextual information that would have been kept if you preserved it as a digital format like EML or MSG.
So, existing digital preservation systems may still be imperfect, but nevertheless we still need find improved solutions for preserving them digitally.
I don't work for CCSDS or Preservica. I work for a national archive as a specialist on digital records and archives, but my background is in IT having spent about 15 years in software development.
There is no such medium as digital. Digital is the encoding method. All media have a lifespan, a certain durability. That determines its future survivability. A digital format could be coded onto vellum, but plain text is more reliable. This is the typical enthusiasm for "technology" by leaders who know only how to get re-elected.
It's not pedantry. "Digital" says nothing about the physical medium, which is a crucial issue if what you're talking about is the viability of long-term archives. Much better to say "magnetic" or "optical" or what have you, but merely to say "It's the 21st century, we should be digital now!" marks an argument as poorly considered.
This is a good point, for more reasons that mere pedantry. Describing storage methods as "digital" also encourages people two conflate two equally important but distinct concerns:
1. Durability of the physical medium over long periods of time, and
2. Our ability to decode extract information from the media even if they do physically survive. (I remember this being a hot issue several years ago, but one that seems to have faded a bit recently.)
Reminds me of the 'leafs' from Anathem[0]. for sure none of the current digital storage artifacts will be readable in +500 years, like scrolls. Contiguous upgrade required by digital will always have a cost and only 'certain' items will end up surviving, where the criteria for choosing will vary with fashion, religion, politics. Wonder is archaeologist in 2500 CE will reveal "500 years old file discovered in digital archives" and then, year later, announce the succesful decoding of an original cca. 2016 animated cat GIF...
Anathem was inspired by the Long Now which is trying to build the 10,000 year Clock and Library. They've been doing a lot of work to try to preserve things for that long. One attempt is the Rosetta Project which is text and info on 1500 languages in 13000 pages etched and electroformed onto a nickle platter. [1]
There was a post here just a few days ago about the difficulties in accessing the data from the BBC's Domesday Project just 30 years later. The original Domesday Book was written on vellum and copies have survived almost a millennium.
This has really got me wondering: What would be a format that is fundamentally readable by both humans and computers? And what does that question really mean?
Archiving isn't a one and done. Anyone that sells a single solution and doesn't have it including costs for migration over time have never dealt with data for very long or don't have any foresight at all.
First job out of college, I worked as a oil data digital archivist, recovering data from old tapes. The data was geological shot data (seismic data collected by blowing dynamite, ground thumping trucks and explosive gas ships).
There are likely thousands of different combinations of hardware, data and formats that we could support.
This was just from a 50 year period.
I've no doubt that data rot is one of the hardest challenges for the future.
If it's really important then you better put it on something physical that lasts.
It seems to me that a lot of the issues people are bringing up in relation to this story were handled in the age of vellum's primacy by orders of monks or the like?
It would seem to me that the place to look for knowledge about digital archiving is those who have taken up that call in modern times, such as the American examples of ibiblio.org and archive.org. I'm fairly confident that there exist European equivalents, as well. Why not look to the research, as opposed to breathless naysaying?
It's certainly not a solved problem, but it's definitely not one that's being handwaved away, either.
87 comments
[ 4.6 ms ] story [ 149 ms ] thread(Not trying to imply that vellum should be the only storage method but that sounds like cheap robustness, and I'd not want to throw that away just because you want to be capital-D Digital.)
Not to mention supporting the UK's tourism brand... castles and queens and whatnot.
The Situation Room is part of the Whitehouse's brand whether tourists get to see it or not.
Not saying it doesn't help tourism, its just not the intent of it all & doesn't have much bearing on the decisions being taken as much as the simple act of continuing traditions.
Their current pricing is $0.0007 per GB-month, or $0.000064 per KB for 800 years in today's dollars. Since the Latin text of Magna Carta is 28 KB of UTF-8 text, that works out to $0.0018 to store the text for 800 years. You'd obviously also want a high-resolution scan of the original document. I don't know how big that would be. If it was a 100MB image, it would cost $6.66 to store for 800 years at Amazon's current Glacier prices.
Obviously we're making immense assumptions about the reliability of the service, of price stability to deliver the quality of service, and so on. And perhaps the biggest issue with this analysis is that we're also relying on huge economies of scale for it to be profitable to deliver an archival service like this over centuries.
If this sounds crazy cheap to you, you're not alone. As technologists we're used to dealing with huge volumes of data, so it's easy to forget that historically these volumes simply did not exist.
Even assuming the corporate entity will survive that long is dicey. To ensure survival of a document on such as service you can not just look at the cost of a storage service, but the cost of infrastructure to care for it. E.g. just the matter of keeping an organization alive for the duration to ensure an account is kept active and live and prepared to ensure each and every monthly bill is paid.
It's a social not a technological problem.
But how to do it is much easier to solve.
Why to do it is the actual problem. What to keep and when to keep it without knowing the future.
How does anyone know that the 100 photos that a person from Morocco put onto Flickr 5 years ago would have value in the future? Why should those be archived? In 200 years these photos could be immensely valuable.
How can this potential future value ensure the archiving of the photos today? Is there economics for this?
The end result is almost indistinguishable from the paper version of the same, unless you touch it.
http://carlos.bueno.org/2010/09/paper-internet.html
Carbon black and iron(ii,iii) oxide would certainly be very long-lasting pigments, and would likely remain legible until the substrate disintegrates.
It's the color printing that is hard to preserve. Gold leaf, of course, lasts as long as the page. Organic dyes tend to fade over time and with exposure to heat, oxygen, and UV light, whereas inorganic pigments are more stable.
Digital long term storage is just, no. Look, all key UK documents are ALREADY digital they're available to download right now. However we aren't talking about digital Vs. non-digital, we're talking about what to store BACKUPS on.
Digital long term storage is something people who frankly know little about technology point to. It is up to IT-types to figure out the costs and the "how?" from then on out. There is no digital storage media which will still exist reliably in 200-400 years, not CDs, not HDDs, and not SSDs. Tapes are the closest and they won't last that long.
So we aren't talking about slapping it on a CD and storing it in some archive, we're talking about slapping it on a piece of digital media today and then moving it every one hundred years without fail, or we lose it forever... If one government or generation loses interest then future generations suffer (see the current government as an example: moving away from reliable Vellum to unproven papers).
Plus with digital you also have to worry about: solar flares/EMP, format knowledge (both of the file system & file format), media protocol knowledge (SATA or CDs might be forgotten technology), and the ability to alter historical documents without detection (not that vallum is immune, but it requires more skill/time).
PS - The "Digital Preservation Coalition" are a group of historians and archivists. They aren't technologists, they just point blindly at digital without explaining the nitty gritty of HOW.
That said, waiting the full duration of a material's projected life span before actual use of that material would limit options hugely, especially with the massively increasing corpus of generated data.
In this sort of context (amount of data) one can spread your bets and use vellum, as well as other methods, until other more stable forms of storage are easily available.
Interesting piece released yesterday related to long term data storage; a projected lifespan of 13.8 billion years doesn't sound too bad... University of Southampton - http://www.southampton.ac.uk/news/2016/02/5d-data-storage-up...
On the other hand, storing data for an active/continuously used archive doesn't seem so hard. You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten. Keeping digital data around for a decade is pretty easy. If you do that 100 times in a row you have kept it for a thousand years and you never had a file format or a storage media older than a decade. You could "forget" to upgrade the archive, but you could also "forget" to scribble on the calfs some time.
But you need to avoid the mistakes that people have already made: using some weird format that dies, and then having all your data locked away in a semi-readable file.
Edit: On further consideration, I think we have to regard digital media and formats like a form of encryption (a simile, I don't mean this literally) which gets stronger over time. Time will bury extant evidence of what is contained on the media and how it is stored. Why should not make the information as explicit (plain-text) as possible? People in 800 years will still have to spend their efforts decoding early 21st century English legalese.
Compare and contrast: https://en.wikipedia.org/wiki/Domesday_Book#Custodial_histor...
Some formats that are no longer usable in general, such as punch cards, were obsoleted consciously. And you might struggle to find a LaserDisc reader. Otherwise, we should be fine. Or am I missing something?
"During the 1960s, the punched card was gradually replaced as the primary means for data storage by magnetic tape, as better, more capable computers became available. ... [P]unched cards were still commonly used for data entry and programming until the mid-1980s when the combination of lower cost magnetic disk storage ... made punched cards obsolete ..."[1]
Yes, I listed wikipedia as a primary source, deal with it.
[1] https://en.wikipedia.org/wiki/Punched_card
But yes, I use it, both professionally and personally. Legacy MUMPS is not pleasant -- the VA's code looks like someone left their telegraph out in the rain -- but the language is astonishingly good at string-handling, mostly due to certain constructs that make it very hard to translate into normal languages.
So imagine you have generations of digital files, converted from format to format to format. Why? Because software engineers are assholes who keep inventing new formats, for stupid goddamn reasons. A hundred generations from now, you'll be looking at copies of copies of copies which aren't just checksumable bit-wise copies but transcriptions, with transcription errors. If our descendants are lucky, all of those different versions will be preserved as well, but that just means that if you notice a possible transcription error, you'll have the opportunity to dig into two hundred year old character and file encodings to try to figure out what the original text was.
Which is not that different of a situation from trying to guess whether a scribe three copies ago misread the scribe four copies ago's atrocious f for a t.
I'm not sure if I'll ever be able to open them. I don't even have a way of getting them off the floppy.
"The third disk here is the only known copy of the original design documents for Loom. It's an 800K Macintosh floppy employing a proprietary format readable only by a vintage Macintosh drive (thank you Apple). In preparing for this lecture, I obtained a dusty old Mac which had not been turned on since 1997. After reseating all the cables and boards I got it to boot and determined that the files on that disk are still intact and still fully readable. A few moments after making this happy discovery however, and before I could actually retrieve those files, the hard drive in the Mac crashed and died permanently. I'll recover the files eventually, but for now you'll have to rely on my failing memory for how Loom was conceived."
[1]: http://ludix.com/moriarty/loom.html (I transcribed the passage above from the video)
50 years ago and all those letters that were burnt or composted because no one bothered to keep the sentimental letters of soldiers in a world war.
20 years later and all those millions of photos that were on Flickr but no one bothered to upgrade.
100 years later and all those GPS logs from taxi cabs that were logged by firms are lost as no one thought they would be interesting or useful.
600 years later and these floppy disks with meeting minutes on them about an early internet organization are lost but might well be worth more than they are now.
Roll-over to new tech is safe. The content of discussion is inherently reliable due to its digital nature.
Keeping multiple copies is necessary practice and the long-time trend is that the next copy you produce has no greater cost than the previous copy (if you wait years, it will be significantly cheaper).
Ah, the classic sign of someone about to say something rash about technology.
"Just". Have fun justifying the cost of re-encoding and re-archiving all of the laws of the UK every few years. The advantage of vellum is that once it's written, that's it. You don't need to go back and re-encode and upgrade to the latest storage technology. It's there, and it'll last literally a thousand years if it's stored in a reasonably dry, cold, and dark location.
It is more like the way the Long Now Foundation wants to build a clock that will run unattended for 10.000 years (http://longnow.org/clock/)
I personally would go for clay tablets, rather than vellum.
Doing this doesn't cost much, and it's a basic archive of the business of the people worth keeping in human readable form.
At the end of the day, tech is good, but the business of people needs to be human readable, accessible in a court, trusted by all. This is a basic requirement humans have, and without it in place, we are forced to trust and depend on tech for law, and it's real consequences.
Nothing wrong with using tech, mind you. We can, should and do. It's more efficient.
But, say something bad happens. That's what the vellum is for.
http://rosettaproject.org/disk/concept/
No no. This doesn't make sense. If we're not going to put our treasured documents on archival paper until we've proved that archival paper really will last for centuries, they're never going to be put on archival paper, no matter how great it might or might not be, because we'll all be dead.
You wanted to say we should see how archival paper pans out before we stop printing our treasured documents on other types of material.
Automation won't hack it either. At the moment, the most complex machine that we know how to make persist into deep time is a very simple clock [2], and even that requires human intervention.
In Anathem [3], Neal Stephenson speculates on what this combo of culture and deep-time technology might look like.
[1] https://en.wikipedia.org/wiki/Magna_Carta [2] https://en.wikipedia.org/wiki/Clock_of_the_Long_Now [3] https://en.wikipedia.org/wiki/Anathem
Archival paper is tested by artificially aging it (e.g. exposure to heat and moisture to simulated passage of time.) Some long-term tests are underway to determine whether accelerated aging tests accurately simulate the natural aging process.[1] And there are associated standards, e.g. ISO 5630-3.[2] But you are correct, we really have no proof of how modern archival paper will perform over a multi-century timeline.
[1]https://www.loc.gov/preservation/scientists/projects/100-yr_... [2]http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_...
Archivists are acutely aware of the obsolescence of digital formats and media. And the solution cannot be just printing them out on vellum or archival paper because increasingly records are being created digitally that cannot be meaningfully or adequately converted into printed formats. I am talking about records that are audio, video, high resolution photographs (like those that NASA is seeking to preserve digitally) etc. Even emails when printed out onto paper would be missing a lot contextual information that would have been kept if you preserved it as a digital format like EML or MSG.
So, existing digital preservation systems may still be imperfect, but nevertheless we still need find improved solutions for preserving them digitally.
I don't work for CCSDS or Preservica. I work for a national archive as a specialist on digital records and archives, but my background is in IT having spent about 15 years in software development.
http://dictionary.reference.com/browse/digital
5. available in electronic form; readable and manipulable by computer: Scan these two pages so you'll have them as a digital document.
1. Durability of the physical medium over long periods of time, and
2. Our ability to decode extract information from the media even if they do physically survive. (I remember this being a hot issue several years ago, but one that seems to have faded a bit recently.)
[0] https://en.wikipedia.org/wiki/Anathem
[1] http://rosettaproject.org/disk/concept/
There are likely thousands of different combinations of hardware, data and formats that we could support.
This was just from a 50 year period.
I've no doubt that data rot is one of the hardest challenges for the future.
If it's really important then you better put it on something physical that lasts.
It would seem to me that the place to look for knowledge about digital archiving is those who have taken up that call in modern times, such as the American examples of ibiblio.org and archive.org. I'm fairly confident that there exist European equivalents, as well. Why not look to the research, as opposed to breathless naysaying?
It's certainly not a solved problem, but it's definitely not one that's being handwaved away, either.