I can relate to this, although I to do this only on a very moderate level. As a side note, despite the technological advances of the last years, you can meet quite a lot of collectors in private direct connect hubs these days.
The client which today is mostly used in private hubs is AirDC. It is optimized for efficient file transfer and comfortable file organization. For instance, AirDC lets you define different share profiles for different hubs, which is tremendous useful to satisfy different share rules for different hubs. To do so, AirDC makes heavy usage of the ADC protocol (http://adc.sourceforge.io), and they implemented nice extensions for it over the years. Recently, build a web interface. Now you can run AirDC on your server, and access it via browser from your smartphone. Otherwise, DC++ is still a legit choice. Both clients are active and maintained. There are other clients as well: ApexDC++ (modern client, but runs still on WinXP, as far as I know); ncdc (https://dev.yorhel.nl/ncdc) and eiskaltdc (https://github.com/eiskaltdcpp) both run at Linux and are maintained.
Concerning hubs: There are hublists, which include open hubs; but don't except quality and safety there. Most great hubs from the past went off the grid at least a decade ago. To go there, you need an invite. You also need to obey to draconic rules, like 24/7 uptime or 6 TB min high quality share.
Somewhat related... I'm starting to think the fear we had 20 years ago about formats going obsolete is largely overblown. I know for sure that SOME formats will become unusable in another few decades, but the major things we all use for this type of stuff will pretty much never die. I bet we'll be able to open a GIF, JPG, PNG, MP4 in 100 years just as easily as we can today. There's just way too many of these things around now.
I know libraries and museums are full of oddball things like wire recorders and wax cylinders that are used as examples, but I'm just not sure that's applicable to most (not all) digital files now. I just can't imagine there will ever come a day where we'll say "It's time to convert these 1 billion PNGs we have saved to the latest greatest format or we'll never be able to use them". Hopefully I'll be alive in 30 years to see if I'm wrong :-)
But that's not what this is about. It's about saving the original source for later use. Be it FLIC files, GIFs, or w/e the next format is. Archiving is fairly important.
There are already digital file formats that are difficult to read - often made worse by them having design quirks that are not documented anywhere and only exist in the original implementation, which of course won't run on modern systems. The same can happen for particular (non-standard) variations of documented formats.
Digital data formats have the same problem as large and popular websites (eg. social media): they look permanent, simply because they're so popular, even when they never are really permanent. At some point, they're going to fall into disuse, and from then on, documentation will slowly fade away.
This is why it's so important to work on permanently documenting these file formats today, while the 'documentation rot' is still fairly limited. It's why things like this exist: http://fileformats.archiveteam.org/wiki/Main_Page
> I'm starting to think the fear we had 20 years ago about formats going obsolete is largely overblown
I don't think it's overblown. I think we recognized an upcoming problem and some of us made efforts to mitigate it. We still couldn't save everything, but like extinction, is difficult to prove.
Atm I have about 40TB of raw storage, of which 30TB are available and 18TB used for data. Most of that data is hoarded data. Stuff like the entire image archive of the apollo missions or all public domain research papers.
I find it very important that people keep this stuff around, the internet forgets so easily.
There is a fairly large archive under the torrent crossref-pre-1909-scholarly-works, which contains pre-1909 works and is public domain. I didn't grab that one but it should be easy to find with any search engine.
I can relate,too; but, I'm not sure I can access some of it. While some is on floppies and I can read them -- some have failed -- stuff that needs my Zip Drives or Bernoulli drives? I'm not sure either drive has survived the last few moves. (I know the Windows 98 computer runs; but, I'm not sure about the one with Windows 3.11.) Maybe that should be my project for 2019.
15 comments
[ 3.5 ms ] story [ 44.0 ms ] threadConcerning hubs: There are hublists, which include open hubs; but don't except quality and safety there. Most great hubs from the past went off the grid at least a decade ago. To go there, you need an invite. You also need to obey to draconic rules, like 24/7 uptime or 6 TB min high quality share.
I know libraries and museums are full of oddball things like wire recorders and wax cylinders that are used as examples, but I'm just not sure that's applicable to most (not all) digital files now. I just can't imagine there will ever come a day where we'll say "It's time to convert these 1 billion PNGs we have saved to the latest greatest format or we'll never be able to use them". Hopefully I'll be alive in 30 years to see if I'm wrong :-)
Digital data formats have the same problem as large and popular websites (eg. social media): they look permanent, simply because they're so popular, even when they never are really permanent. At some point, they're going to fall into disuse, and from then on, documentation will slowly fade away.
This is why it's so important to work on permanently documenting these file formats today, while the 'documentation rot' is still fairly limited. It's why things like this exist: http://fileformats.archiveteam.org/wiki/Main_Page
I don't think it's overblown. I think we recognized an upcoming problem and some of us made efforts to mitigate it. We still couldn't save everything, but like extinction, is difficult to prove.
I find it very important that people keep this stuff around, the internet forgets so easily.
You mean all of them? How many are there?
There is also lots on archive.org!