My wife and I used to get prints of all of our photos when we first started dating. Then we switched to digital, a lot of which has been lost on old hard drives. When we got married we had a really hard time finding photos of us as a couple because most had been lost.
Now that we have kids we back up our photos in multiple places, but we also make sure to print out the ones we really like, just in case. The really good ones we print on canvas and hang on the walls.
But it's true, most of records would be lost if there were a global cataclysm that took out all digital records. That being said, I'm pretty sure our mortgage is only digital so at least we would own our house free and clear!
> That being said, I'm pretty sure our mortgage is only digital so at least we would own our house free and clear!
Very similar to what happens in Mr. Robot where the plan is to encrypt all of the biggest worldwide bank's records so that they're inaccessible, thus wiping out the majority of debt.
New people are being born all the time who haven't had the opportunity to watch the entire history of television (or even just the best 10%). By spoiling a show that you've watched, you're denying them the opportunity to experience the joy that you had.
The logical conclusion of this attitude: "Therefore, all conversation of any form, especially any crossing generational boundaries, must cease immediately."
There is some information that people want to learn through conversation, some that people don't mind learning through conversation, and some that people actively want to avoid learning through conversation.
I don't know why you'd reach the conclusion that all information falls into the latter category.
Don't worry, that's far from the most interesting plot point in Mr. Robot! Now, if the above commenter had spoiled any of the circumstances surrounding the described hack, then you'd definitely want to be angry. Knowing that hack happens is like knowing that there are dragons in Game of Thrones.
(This might be a controversial opinion, but if you've seen Fight Club, it's hard to watch Mr. Robot and not know exactly where things are going anyways. Of course, it's definitely a fun watch regardless!)
Oh man, I hear you: I had a mess just like that, tried to find something to automatically sweep everything into a neat pile, couldn't find it, and ended up writing PhotoStructure: https://photostructure.com/about/introducing-photostructure/
This looks fantastic. I have a lot of comments about my old photos stored in 4DOS Descript.ion files, since I was using ACDsee to organize them. Can you speculate how to save those?
Huh, I've never heard of that format. If you send an example to hello@photostructure.com I can take a look: best case, ExifTool will be able to read them already, and you can just add .ion as a sidecar extension.
This is just what I wanted for a long time, amazing to randomly come across it because I'd already given up thinking it doesn't exist.
The only feature it's missing - but I know this would be complicated and advanced - would be to scan for similar images and duplicates that don't exactly match. Ideally it would present them all and then I could manually mark for deletion what I want to get rid of.
So many duplicates. We were all so naive when digital photography first became a thing. I remember back in the day everyone recommended "keep all files, delete nothing, storage is cheap".
One of the worst data losses of photos I had was when apple switched from iPhoto (which I'd organized meticulously) to Photos.
It wasn't until a few hard drive crashes later did I realize how poorly Photos imported the pictures. I found old iPhoto data files on an old Time Machine backup and was able to restore some, but anything that was linked was long gone.
So most of the digital photos from my 20's are gone.
I met somebody once who told me that digital drives expire and will lose data unless they are “fired up” from time to time. Because of that, his company had a system of taking out and doing reads on their old storage in a clunky and manual way throughout the year. I wasn’t able to find out why this was necessary or the time frames involved. Can anybody confirm or deny or explain?
> FLASH memory works by essentially placing some electrons on an insulated island of material through quantum tunnelling and then "trapping" them there. However, there is some probability that they can tunnel off, and thus the stored charge does leak out over time. How long it takes for enough charge to leak off that a "read" can't tell the difference (signal to noise wise) depend on the manufacturer and specifications but 10 years is definitely in the danger zone. There's also a stress degradation issue but that probably doesn't apply here.
Yeap. And we don't have a reliable Data Storage solution. Given the extreme low chances of disaster like fire, having a physical item is still so much better.
We need a cheap storage solution that is reliable. Doesn't need to be fast. Even DVD dont last longer than 10 years.
Someday this whole thing is going to back fire and we move back to hybrid / photobooks.
> Microsoft and Warner Bros. have collaborated to successfully store and retrieve the entire 1978 iconic “Superman” movie on a piece of glass roughly the size of a drink coaster, 75 by 75 by 2 millimeters thick.
> The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong.
> “One big thing we wanted to eliminate is this expensive cycle of moving and rewriting data to the next generation. We really want something you can put on the shelf for 50 or 100 or 1,000 years and forget about until you need it,” Rowstron said.
But I can't buy it for love or money, so AFAIC, it might as well not exist. Ditto with the mosquito-zapping laser that was supposed to change the world.
This is really the storage technology I want. It's what CD-R's were promised to be (but came nowhere near delivering - organic dye is, well, organic).
The big question I suppose is whether you can consumerize it. Though I suppose you could imagine the return of something like mall photobooths: you buy an organic dye substrate which something like a CD drive can write to, then drop it off to be "developed" into an impervious substrate by a harsher process (guessing there's an NH4F etch in that process).
That is the problem. Even if I am willing to pay $1K, which is a lot from a consumer perspective, it is tiny in the business world and the cost to commercialisation.
"What are my children or any potential grandchildren […] going to do with the 400 pictures of my pet that are on my phone?" Nothing, they don't need it. You sit your grandkid on your lap and look in their eyes and tell them stories about the family dog. They'll be happy to listen, they don't gain much by seeing your 400 photos.
I guess it's a natural human impulse to try to leave a mark for the future. But the future will be okay, they don't need our crap data dumps. There's way more useless fluff being generated than what needs to be preserved. People survived when there was no photography.
Yes, preserve your wedding photos, a few important life events here and there, that can be fun to look at for your kids. The full camera roll on your phone? Who cares? Let's not burden the coming generations with all this stuff. The future is theirs, not ours.
For the few things that you want to preserve, make high quality physical prints. Their huge benefit is that they don't break silently. It's utterly obvious when looking at the physical object that it still "holds your data", as opposed to an HDD, DVD or tape which you need to put in a machine to check if it's still readable, which you won't reliably do. Analog printouts are a very robust preservation format. They can burn, but DVDs and HDDs aren't fireproof either (backup printouts are a solution too). Yes, prints may degrade in color, but it's a transparent process.
But overall, forests need to clean themselves by fires, there needs to be a clean slate. I don't want to know who my great-great-grandparents were. I don't want their family photos. Maybe they were bad people, criminals etc. Maybe there was domestic abuse, trauma, tragedy etc. I don't need to know. I want to live my life with the people that are alive in the now. I don't want my ancestors to haunt me from the grave. And I don't want to haunt my descendants.
So true. When I had to clean out my parents' house after my mom died, there were boxes and boxes of photos. Just in boxes, not really organized. Had been saved for years. I gave some thought to scanning them, but ultimately was honest and told myself that nobody had ever looked at these and nobody ever would. So they just got thrown away. Nobody really cares, years later, about day to day minutia preserved like this.
On the other hand, I know the names and a couple dates of my ancestors going back to the 1700s. But more than two generations back, I know nothing about them. Except a famous one, Aaron Burr (a brother, not direct). Yes, that guy!
I started at the Library of Congress 20 years ago this month working on their digitization effort. It was HUGE even then - an estimated 50TB/day needed to be digitized - and that did not address the 200+ years of backlog or the advent of blogging (rare at the time), streaming media, and everything else online.
Our mission was to ensure that the assets we were working with were usable by someone ~100 years from now.
We would take something like an LP (Ray Charles was one of my favorites) and play it and record it in a lossless wav at a high bitrate. Then we'd take the physical record, the entire sleeve, and anything else included and scan that at 300dpi in tiff. Of course, we'd document every piece of equipment we used (down to the serial number) with every setting and who did the work. If we later realized a given piece of equipment was misconfigured, we could detect what had been created with it and adjust or re-create as necessary. I wrote some of the standards (XML schemas) for metadata collection which were still in use as late as 2015. (They may still be, I haven't checked.)
Whew.
Then we'd take a version of the audio and images to create mp3s and jpgs for listening+viewing in the reading rooms. The original physical media would go into nitrogen filled vaults in Culpepper, Virginia to prevent oxidization and further deterioration.*
We had similar processes for other media like books, wire spool recordings (popular in WW2), wax cylinder recordings, and pictures around the Civil War. Many of them were exceptionally fragile and required special equipment. My least favorite but most fascinating were Thomas Edison's first motion pictures.
Pop quiz: What does cotton and silver nitrate degrade into? ;)
But in all of those cases, we could go back to the original media which - hopefully - hadn't deteriorated beyond recovery. In a digital-first or digital-only environment, we're counting on fragile components storing rotting file formats on deprecated operating systems in physical locations that may suffer disasters.
* At the time, the RIAA was complaining endlessly and we may have made jokes about RIAA execs and those vaults but I can neither confirm nor deny any such proposals.
This excessive detail seems nutty to me, and ultimately counterproductive. It's counterproductive in that it reduces the amount of things that can be preserved. What's more important - audio detail that cannot be heard, or more records?
The best way to archive data long term is to make diverse copies. The easiest way to make diverse copies is to simply make them available to download off of the internet. Don't restrict it to "reading rooms".
Me, no, I can't think of a use for the super hi fi version. Heck, just the snap crackle pop on the best vinyl pressings destroys any utility of super hi fi.
Is it just the music itself? Does it include the physical media and its condition? If the media has deteriorated, how does that affect it?
Does it include the cover art? Does it include the liner notes? Does it include the notes through the production process?
For the recording of a live performance, does it include the recording equipment and the venue? Does it include the audience reaction?
Does it include documenting everyone who performed backup vocals, accompanied via instruments, produced it, and the label itself?
While normal listeners (99.9% of the public) would stop at the music, the audiophiles will want to understand the physical media and performance conditions, and the researchers/archivists want everything because it's a snapshot of a particular work at a particular time & place via particular means.
But part of your comment is spot on.. diverse sources. It's not feasible for one Library to digitize absolutely everything in the world (or a country), so we participated in an exchange with a bunch (20? not sure anymore) of other libraries, museums, etc who could digitize their own collections and then we could all share them at scale with as little DRM as possible.
Think of it as Napster for libraries. I advocated for no DRM (especially for old, out of production works) but that's why we butted heads with the RIAA.
I strongly suspect that storage of data is far cheaper than the costs of digitization, or actually storing physical objects, regardless of how high the resolution.
I once put all my photos through JPEG2 compression, because I thought saving space was a good idea... I've regretted it ever since.
The function of an archive is to preserve artifacts of the past with as much fidelity and context as possible. You can't possibly know what is going to be important to people in the future. Saving 2 cents on storage per century could be the equivalent of using shorter dates/times to save memory, the tradeoff that lead to the Y2K and Y2.038k problems.
Yes, that was the goal but they were prioritizing works that may be lost due to physical deterioration.
Because in theory they already have everything:
"Mandatory deposit (17 U.S.C. section 407) requires the owner of copyright or of the exclusive right of distribution to deposit in the U.S. Copyright Office for the use of the Library of Congress two complete copies of the best edition within 3 months after a work is published. Copies of all works under copyright protection that have been published or distributed in the United States must be deposited with the Copyright Office within 3 months of the date of first publication."
And final tidbit from me on this.. Yes, they get submissions of all forms of copyrighted material which means they have a pornography collection. When I was there, that group was down the hall and the lead person was a kind old lady that I chatted with every day for months before she revealed her area of focus. It was.. odd.
Odds are the technical information about the recording setup is fairly constant, and thus doesn't require much manual input, nor storage space to consume.
They've also mentioned that always knowing the exact technical setup can help track down problems, which reduces the chances of loss as things are archived down the road.
Been saying this about Wayback Machine for awhile. They've archived every HTML page ever, but it's weighed down by high-res images, javascript, CSS, and more recently video. All of this is a liability to preserving the HTML itself. I'm not saying that graphics and video aren't valuable, but it would be nice if they took a more defensive posture to the collapse of the internet. If they are in danger of shutting down entirely someday, will anyone be able to "at least" grab all of the PDFs and HTML and TXT docs that IA has archived? Or will we have to copy tons and tons of extraneous stuff too?
> This excessive detail seems nutty to me, and ultimately counterproductive. It's counterproductive in that it reduces the amount of things that can be preserved. What's more important - audio detail that cannot be heard, or more records?
The guy was working in a pretty important national archive, on what sounds like are rare or one-of a kind artifacts. Archives aren't just about preservation, but preservation in a way that allows research to be performed. Research will likely need high quality copies of good provenience, and all the record-keeping could help explain some phenomenon that was introduced during the digitization process.
So, if they just did a quick an dirty rip of as much stuff as they could, they could probably "preserve" more, but that data might be useless to some future research project after the original has degraded into uselessness.
And in any case, my bet is most well-preserved physical objects (at least paper ones) will long outlast any digital copies that are made today.
The excessive detail is a necessity and a precaution. Researchers need that information (textual genetics or any art history for instance, maybe future archeology). A message can't be separated from the form it embodied; you need both to understand how an artwork has been received.
Besides, you always want to preserve more information than what is needed today, just in case new ways to exploit data appear later. That's what archeology does when it refuses to exploit some sites and preserves them for later : maybe the current techniques destroy data, so we save some sites for later - just in case.
There was a Woody Guthrie recording (that I have not heard myself) on wire spools that was released in the 2000s. If I recall correctly, the audio restoration process was only possible by recording the audio with a 384kHz sampling rate, because the wire spools had picked up ambient EMI that could be used to correct for speed variations. If the only available samples were MP3s, then the audio restoration could not have been done.
Similarly, other historic audio recordings might contain forensic clues outside the ranges of human hearing that will be useful to future historians.
Finally, storing lossless formats makes sense for an archive, because raw bits are likely to be easier to recover if knowledge is lost, and localized damage to the data will result in only localized damage of the output.
I know the professional archivists strongly disagree with me on this. I know there are cases like the one you mention where higher precision turned out to be useful.
But I also know of the HP archives that burned down while awaiting funding for a pixel perfect proper archiving of it. The result was there's nothing left for future historians.
The Vatican Library is another. It's full of one-of-a-kind documents. They can't afford to do a pixel perfect archive of them, so nothing gets done. It's all at huge risk of fire.
An imperfect copy is infinitely better than ashes.
> Scott says he thinks there should be legal or regulatory requirements on companies that give people the option to retrieve their data, for a certain period – say, five years – after an online service is due to shut down.
I get the motivation here, but in addition to keeping the stored data, this means one must keep account and authentication systems, maintain a service and UI for people to request this data, etc for years. That seems like an excessive obligation, depending on what the product was which took in the data to begin with. What data should be eligible? If I ordered delivery from a service which folded, should I really be able to request my order records years later?
The idea of protecting data against catastrophic events reminds me of the KEO satellite project, which I think is not yet officially dead but has been delayed to the point that it seems doomed. But they aspired to put a lot of data in a very durable format on a satellite, to act as a kind of time capsule, intended to be recovered tens of thousands of years into the future. I think the intent was to use optical media, but made of glass.
Clearly this isn't yet an example of successfully preserving data. But the mindset of setting a goal to make something readable in the very distant future seems admirable.
> She studied maths at university and has copies of her handwritten notes. “There’s a point when I started taking digital notes and I can’t find them,” she says with a laugh.
I've lost much of my handwritten notes from university.
I used to handwrite code in notebooks before typing it in. That's all gone.
I'm always amazed that people actually think "online/digital is forever". I NEVER has been and never will be. And as we each "create" more data, the life span will decrease as fast.
Even websites can't be assumed to remain at the other end of the bookmark you keep. You MUST download it if you really want an archive you can count on.
If you aren't buying a new mass storage device EVERY year and moving anything older than 3 years to the new storage, you are taking a risk.
This is why I still buy physical books rather than Kindle et al. This is why I convert personal documents and momentos like photos to paper, et al. if it actually matters. I still have photos of my ancestors from the 1900s-1920s only because they are on paper.
And I've worked on most of the advanced technologies in use today over the last few decades - high priest level contact. But the truth you learn if you learn anything is: you should not put technology first in your life! It's a tool but not even the best tool.
One side of my family lost the photos from a leak in the roof. The other side lost the photos because the house was destroyed. (The handful left were ones that were copies given to friends and relatives.)
I know this is an unusual view, and I'm not endorsing it for others, but please delete my data. I want to have as little as possible. Obviously, if it's something precious and needs saving, do the due diligence of backups and hard copies when applicable. But I fear becoming a digital hoarder. I fear it becoming a cross to bear. Not to mention, the less you keep, the less you stand to lose.
BBC often embed links in their articles as references. I am a little surprised they don't have an archiving mechanism to manage this content, especially things like tweets (although maybe a bad example b/c twitter should be banned from journalism altogether, kind of like separation of church and state).
One upside of this is that the nonconsensual "revenge porn" that used to plague many victims is sometimes actually disappearing from the Internet for good. I had a friend whose ex-boyfriend uploaded a consensually filmed video in the early 2000s of her giving him oral sex, with her real name, onto porn sites. She went the police and they said they couldn't do anything. She would file takedowns with the sites and the video would always pop back up later. It wasn't the ex-boyfriend doing it because he was in jail for some of that time. Online trolls sent the video to her parents, to her employers, and to her friends. It seemed like she would never get away from this video, but in recent years it seems to have actually disappeared. It doesn't pop back up. Hasn't for a long time, and even if it does again, it's been some good years of relief for her.
71 comments
[ 3.3 ms ] story [ 140 ms ] threadNow that we have kids we back up our photos in multiple places, but we also make sure to print out the ones we really like, just in case. The really good ones we print on canvas and hang on the walls.
But it's true, most of records would be lost if there were a global cataclysm that took out all digital records. That being said, I'm pretty sure our mortgage is only digital so at least we would own our house free and clear!
Very similar to what happens in Mr. Robot where the plan is to encrypt all of the biggest worldwide bank's records so that they're inaccessible, thus wiping out the majority of debt.
If you didn’t watch it yet, were you ever really going to?
Imo, spoilers without warnings are fair after 3 months time+ for TV shows and movies. Maybe even sooner.
I don't know why you'd reach the conclusion that all information falls into the latter category.
Yes. I have the whole thing saved and did in fact plan on watching it eventually.
(This might be a controversial opinion, but if you've seen Fight Club, it's hard to watch Mr. Robot and not know exactly where things are going anyways. Of course, it's definitely a fun watch regardless!)
The only feature it's missing - but I know this would be complicated and advanced - would be to scan for similar images and duplicates that don't exactly match. Ideally it would present them all and then I could manually mark for deletion what I want to get rid of.
So many duplicates. We were all so naive when digital photography first became a thing. I remember back in the day everyone recommended "keep all files, delete nothing, storage is cheap".
https://old.reddit.com/r/askscience/comments/5cexmj/how_stab...
Thanks for the link!
We need a cheap storage solution that is reliable. Doesn't need to be fast. Even DVD dont last longer than 10 years.
Someday this whole thing is going to back fire and we move back to hybrid / photobooks.
> Microsoft and Warner Bros. have collaborated to successfully store and retrieve the entire 1978 iconic “Superman” movie on a piece of glass roughly the size of a drink coaster, 75 by 75 by 2 millimeters thick.
> The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong.
> “One big thing we wanted to eliminate is this expensive cycle of moving and rewriting data to the next generation. We really want something you can put on the shelf for 50 or 100 or 1,000 years and forget about until you need it,” Rowstron said.
https://news.microsoft.com/innovation-stories/ignite-project...
The big question I suppose is whether you can consumerize it. Though I suppose you could imagine the return of something like mall photobooths: you buy an organic dye substrate which something like a CD drive can write to, then drop it off to be "developed" into an impervious substrate by a harsher process (guessing there's an NH4F etch in that process).
"What are my children or any potential grandchildren […] going to do with the 400 pictures of my pet that are on my phone?" Nothing, they don't need it. You sit your grandkid on your lap and look in their eyes and tell them stories about the family dog. They'll be happy to listen, they don't gain much by seeing your 400 photos.
I guess it's a natural human impulse to try to leave a mark for the future. But the future will be okay, they don't need our crap data dumps. There's way more useless fluff being generated than what needs to be preserved. People survived when there was no photography.
Yes, preserve your wedding photos, a few important life events here and there, that can be fun to look at for your kids. The full camera roll on your phone? Who cares? Let's not burden the coming generations with all this stuff. The future is theirs, not ours.
For the few things that you want to preserve, make high quality physical prints. Their huge benefit is that they don't break silently. It's utterly obvious when looking at the physical object that it still "holds your data", as opposed to an HDD, DVD or tape which you need to put in a machine to check if it's still readable, which you won't reliably do. Analog printouts are a very robust preservation format. They can burn, but DVDs and HDDs aren't fireproof either (backup printouts are a solution too). Yes, prints may degrade in color, but it's a transparent process.
But overall, forests need to clean themselves by fires, there needs to be a clean slate. I don't want to know who my great-great-grandparents were. I don't want their family photos. Maybe they were bad people, criminals etc. Maybe there was domestic abuse, trauma, tragedy etc. I don't need to know. I want to live my life with the people that are alive in the now. I don't want my ancestors to haunt me from the grave. And I don't want to haunt my descendants.
It would be fun to know more.
Our mission was to ensure that the assets we were working with were usable by someone ~100 years from now.
We would take something like an LP (Ray Charles was one of my favorites) and play it and record it in a lossless wav at a high bitrate. Then we'd take the physical record, the entire sleeve, and anything else included and scan that at 300dpi in tiff. Of course, we'd document every piece of equipment we used (down to the serial number) with every setting and who did the work. If we later realized a given piece of equipment was misconfigured, we could detect what had been created with it and adjust or re-create as necessary. I wrote some of the standards (XML schemas) for metadata collection which were still in use as late as 2015. (They may still be, I haven't checked.)
Whew.
Then we'd take a version of the audio and images to create mp3s and jpgs for listening+viewing in the reading rooms. The original physical media would go into nitrogen filled vaults in Culpepper, Virginia to prevent oxidization and further deterioration.*
We had similar processes for other media like books, wire spool recordings (popular in WW2), wax cylinder recordings, and pictures around the Civil War. Many of them were exceptionally fragile and required special equipment. My least favorite but most fascinating were Thomas Edison's first motion pictures.
Pop quiz: What does cotton and silver nitrate degrade into? ;)
But in all of those cases, we could go back to the original media which - hopefully - hadn't deteriorated beyond recovery. In a digital-first or digital-only environment, we're counting on fragile components storing rotting file formats on deprecated operating systems in physical locations that may suffer disasters.
* At the time, the RIAA was complaining endlessly and we may have made jokes about RIAA execs and those vaults but I can neither confirm nor deny any such proposals.
Bright light if you look at it wrong?
The best way to archive data long term is to make diverse copies. The easiest way to make diverse copies is to simply make them available to download off of the internet. Don't restrict it to "reading rooms".
Take one of Ray Charles' records..
Is it just the music itself? Does it include the physical media and its condition? If the media has deteriorated, how does that affect it?
Does it include the cover art? Does it include the liner notes? Does it include the notes through the production process?
For the recording of a live performance, does it include the recording equipment and the venue? Does it include the audience reaction?
Does it include documenting everyone who performed backup vocals, accompanied via instruments, produced it, and the label itself?
While normal listeners (99.9% of the public) would stop at the music, the audiophiles will want to understand the physical media and performance conditions, and the researchers/archivists want everything because it's a snapshot of a particular work at a particular time & place via particular means.
But part of your comment is spot on.. diverse sources. It's not feasible for one Library to digitize absolutely everything in the world (or a country), so we participated in an exchange with a bunch (20? not sure anymore) of other libraries, museums, etc who could digitize their own collections and then we could all share them at scale with as little DRM as possible.
Think of it as Napster for libraries. I advocated for no DRM (especially for old, out of production works) but that's why we butted heads with the RIAA.
> the audiophiles will need to understand the physical media and performance conditions,
They really don't :-)
> and the researchers/archivists want everything because it's a snapshot of a particular work at a particular place in time via particular means
Phooey. There are far more interesting things that need archiving.
I once put all my photos through JPEG2 compression, because I thought saving space was a good idea... I've regretted it ever since.
The function of an archive is to preserve artifacts of the past with as much fidelity and context as possible. You can't possibly know what is going to be important to people in the future. Saving 2 cents on storage per century could be the equivalent of using shorter dates/times to save memory, the tradeoff that lead to the Y2K and Y2.038k problems.
Because in theory they already have everything:
"Mandatory deposit (17 U.S.C. section 407) requires the owner of copyright or of the exclusive right of distribution to deposit in the U.S. Copyright Office for the use of the Library of Congress two complete copies of the best edition within 3 months after a work is published. Copies of all works under copyright protection that have been published or distributed in the United States must be deposited with the Copyright Office within 3 months of the date of first publication."
Ref: https://www.copyright.gov/help/faq/mandatory_deposit.html
And final tidbit from me on this.. Yes, they get submissions of all forms of copyrighted material which means they have a pornography collection. When I was there, that group was down the hall and the lead person was a kind old lady that I chatted with every day for months before she revealed her area of focus. It was.. odd.
But the choice remains:
1. archive in as agonizingly irrelevant detail as technically possible
2. archive as much as possible before it decays irretrievably
Can't have both.
They've also mentioned that always knowing the exact technical setup can help track down problems, which reduces the chances of loss as things are archived down the road.
The guy was working in a pretty important national archive, on what sounds like are rare or one-of a kind artifacts. Archives aren't just about preservation, but preservation in a way that allows research to be performed. Research will likely need high quality copies of good provenience, and all the record-keeping could help explain some phenomenon that was introduced during the digitization process.
So, if they just did a quick an dirty rip of as much stuff as they could, they could probably "preserve" more, but that data might be useless to some future research project after the original has degraded into uselessness.
And in any case, my bet is most well-preserved physical objects (at least paper ones) will long outlast any digital copies that are made today.
The medium is the message. (h/t Marshall McLuhan)
The interaction between tools, artists, and expression is endlessly fascinating.
Similarly, other historic audio recordings might contain forensic clues outside the ranges of human hearing that will be useful to future historians.
Finally, storing lossless formats makes sense for an archive, because raw bits are likely to be easier to recover if knowledge is lost, and localized damage to the data will result in only localized damage of the output.
But I also know of the HP archives that burned down while awaiting funding for a pixel perfect proper archiving of it. The result was there's nothing left for future historians.
The Vatican Library is another. It's full of one-of-a-kind documents. They can't afford to do a pixel perfect archive of them, so nothing gets done. It's all at huge risk of fire.
An imperfect copy is infinitely better than ashes.
Is there any published technical document about the choices made by LoC to achieve that goal?
I get the motivation here, but in addition to keeping the stored data, this means one must keep account and authentication systems, maintain a service and UI for people to request this data, etc for years. That seems like an excessive obligation, depending on what the product was which took in the data to begin with. What data should be eligible? If I ordered delivery from a service which folded, should I really be able to request my order records years later?
Clearly this isn't yet an example of successfully preserving data. But the mindset of setting a goal to make something readable in the very distant future seems admirable.
I've lost much of my handwritten notes from university.
I used to handwrite code in notebooks before typing it in. That's all gone.
Even websites can't be assumed to remain at the other end of the bookmark you keep. You MUST download it if you really want an archive you can count on.
If you aren't buying a new mass storage device EVERY year and moving anything older than 3 years to the new storage, you are taking a risk.
This is why I still buy physical books rather than Kindle et al. This is why I convert personal documents and momentos like photos to paper, et al. if it actually matters. I still have photos of my ancestors from the 1900s-1920s only because they are on paper.
And I've worked on most of the advanced technologies in use today over the last few decades - high priest level contact. But the truth you learn if you learn anything is: you should not put technology first in your life! It's a tool but not even the best tool.
One side of my family lost the photos from a leak in the roof. The other side lost the photos because the house was destroyed. (The handful left were ones that were copies given to friends and relatives.)
Paper rots and burns.
Kinda like how conceiving a child is hard if you want one and easy if you don't.
The "war stories for hours" is very strong with this statement, I feel.
I figure that a lot of them can sadly never be told :( but I'd like to firmly +1 the idea of considering what can be shared. Sign me up :P