HN Explorer

Ask HN: How do you organize all your personal data?

9 points by unicornporn ↗ HN
I've decided to shape up 4 TBs of mess and I want your insights! What data organization strategies have you settled for? What strategies failed? This is a big topic that few seem to discuss.

I have saved webpages (in MAFF file format), eBooks, word documents, wget mirrors of sites, audio recorded with my phone, etc etc. The only thing that's really in shape is my Lightroom library and music library.

I'm mostly after the grand scheme, the basic structure of your files (ie. the top level and perhaps the directories following these).

Some seem to start by organize by file type (directories for Audio, Video etc) and some seem to organize by source at the top level. Source could be "Me", "Internet", "My Company". The funny cat pictures would go in a folder under the "Internet" directory.

You could illustrate your file system layouts by using code, like this:

  Audio/
        Music/
        Speech/
  Documents/
            eBooks/
            Web pages/
I use OS X, but I guess these strategies should apply to any OS.

6 comments

[ 3.2 ms ] story [ 18.7 ms ] thread
stangeek
I would say it very much depends on which computer you're trying to organise.

For my work drive, everything is sorted by year then project, and each project starts with a timestamp (= start date), so: 2014/ 2014_01_Project A/ 2014_03_Project B/ etc. and on top of this there's an admin folder to sort contracts etc.

For my personal computer, I tend to organise things by file type (apart from personal projects which are organised as per above), so one folder for videos / images / audio / documents, and then I tag the file accordingly (using the Mac OS X tagging feature).

Hope this helps!

daeghnao
I have Projects, Details, Media and Mirrors. Mirrors are computers (whole drives), archives (mainly of retro stuff) and websites (usually my own). Media is divided into Books, Audio, Video, Music, Pictures and so on. Projects I divide up into academic, coding, creative, games, there are a few others. Details is where I keep admin reminders, dotfiles for various systems, common admin address book, stuff like that. I tend to have Details on all the systems and keep it small, then specific bits of the rest when I use them. Well, in theory, I am also in midst of organising it all.
Chevalier
I've given up on sorting my enormous hoard of duplicate photos. At some point, Dropbox/GDrive/OneDrive/etc. will offer a deduplication feature and I'll move them on there. Local programs like Visipics or Duplicate Annihilator just can't do the job on my underpowered hardware. At the moment, I just keep my past couple years' worth of photos on GDrive for viewing through G+ Photos.

Ebooks are far and away best sorted with Calibre. It's not as intuitive as I'd like (does anyone know how to combine multiple formats under one title, or the key command to search your books?), but it's the only viable way to maintain your library.

OneNote is PERFECT for personal notes and projects. Project management is best outsourced to Trello, in my experience. I'm a fanatic of both.

For non-personal data, I've forced my wife to keep pictures of purses and whatever on Pinterest rather than on her local HDD (where she would never view them again). Likewise, articles I want to read are best saved to Pocket, where viewing, archiving, and deleting are dead simple. And of course, Kindle is a great place to buy and sync commercial ebooks.

My whole life is moving online, and I fully expect everything I have will be stored and presented through online services in five years or less.

Kribbly
Have you ever tried something more advanced than VisiPics and Duplicate Annihilator? I've found that Duplicate Photo Cleaner is reliable for finding similar photos and Easy Duplicate Finder works really well for the rest of the duplicates. They are not free, but I think these programs work better than the free ones like VisiPics, Auslogics, Duplicate Annihilator and the rest.
MalcolmDiggs
I keep a working directory of everything that's relevant to my day-to-day right now. That folder is chaos but it's easy to keep mental track of it because it's small. Once something is no longer relevant (that project ends, I've shared that photo, I've cooked that recipe, I've listened to that album etc) then stuff goes into the archive.

Within the archive I group everything by year. Then within the year I'll subdivide by project/event and then by document type.

So the archive looks something like:

  2014
    pics_from_my_phone
    project_a
        .doc
        .pdf
        .ppt
    project_b
    downloaded_music
        album_a
            track_1.mp3
        album_b
    ...

  2013
    pics_from_my_phone
    project_c
    ...

  2012
    project_d
    project_e
    ...
Having suffered from catastrophic data loss in the past I usually store the archive redundantly on different physical drives and cloud services, and sync the backups every few days.