Ask HN: How do you organize your files

If its for e-books only, you can try adobe digital editions or calibre. You can tag and create collections with search functionality on most formats.

Calibre may be a little rough looking but it's very powerful and it's what I use.

Edit: Also you might want to make a small title edit s/files/ebooks unless you are inquiring about other types of files as well.

Beside the usual `Images`, `Videos`, `code` directory, the single most important directory on my system is `~/flash` (as in : flash memory). This is where my browser downloads files and where I create "daily" files, which I quickly remove.

This is a directory that can be emptied at any moment without the fear of losing anything important, and which help me keeping the rest of my fs clean. Basically `/tmp` for user.

Most of my files stay in the download folder. If I think I will need them at a later stage against I upload them to my Google Drive. Google is quite good at searching stuff - for me that also works for personal files. I have probably 100 e-books that are on my reading list and will never get read by me...

I have my files pseudo-organized, meaning I kind of try to keep them where they should be logically, but since this varies a lot - they're not really organized. The thing is - I use "everything" a free instant file search tool from voidtools. It is blazingly fast, just start typing and it finds files while you type. It uses the ntfs file system (windows only, sorry everyone else) existing index to perform instant searches, it is hands down the ultimate most fast file search tool I have ever encountered - files literally are found while you type their names, without waiting for even a milli second.

So, no organization (the ocd part of me hates this) but i always find my files in an instant, no matter where i left them.

recoll has worked great for a document index.

https://www.lesbonscomptes.com/recoll/

I also recommend calibre for e-books, but I never got to the "document store" stage that I think some people have.

Roughly this scheme:

~/dev for any personal project work

~/$COMPANY for any professional work I do for $COMPANY

~/teaching for teaching stuff

~/research for academic research (it's a big mess unfortunately)

~/icl for school related projects (where "icl" is Imperial College London)

For my PDFs I use Mendeley to organize them and have them available everywhere along with my annotations.

I store my books in iBooks and on Google Drive in a scheme roughly like: /books/$topic/$subtopic

Usually organizing your files is usually just commitment, move files off ~/Downloads as soon as you can :-)

Dump everything on desktop or downloads folder then use Void Tools Everything to find what I need.

Home directory is served over NFS (at work). Layout is as follows:

  phireal@pc ~$ ls -1
  Box/       - work nextcloud
  Cloud/     - personal nextcloud
  Code/      - source code I'm working on
  Data@      - data sources (I'm a scientist)
  Desktop/   - ...
  Documents/ - anything I've written (presentations, papers, reports)
  Local@     - symlink to my internal spinning hard drive and SSD
  Maildir/   - mutt Mail directory
  Models/    - I do hydrodynamic modelling, so this is where all that lives
  Remote/    - sshfs mounts, mostly
  Scratch/   - space for stuff I don't need to keep
  Software/  - installed software (models, utilities etc.)

At home, my main storage looks like:

  phireal@server store$ ls -1
  archive     - archived backups of old machines
  audiobooks  - audio books
  bin         - scripts, binaries, programs I've written/used
  books       - eBooks
  docs        - docs (personal, mostly)
  films       - films
  kids        - kids films
  misc        - mostly old images I keep but for no particular reason
  music       - music
  pictures    - photos, organised YYYY/MM-$month/YYYY-MM-DD
  radio       - podcasts and BBC radio episodes
  src         - source code for things I use
  tmp         - stuff that can be deleted and probably should
  tv_shows    - TV episodes, organised show/series #
  urbackup    - UrBackup storage directory
  web         - backups of websites
  work        - stuff related to work (software, data, outputs etc.)

~/work - everything related to job

~/github - just cloned repos

~/fork - everything forked

~/pdf - all science papers

One external raid (mirrored) that holds information only necessary for when I'm working at my desk. Within that drive I have an archive folder with past files that are rarely/ever needed. The folder structure is labeled broadly such as "documents" "media" and more specific folders within. For the file level I usually put a date at the beginning of the name going from largest to smallest (2017-6-21_filename). For sensitive documents; I put in encrypted DMG files using the same organization structure.

As for all "working" documents, they're local to my machine under a documents or project folder. The documents folder is synced to all my devices and looks the same everywhere with a similar organization structure as my external drive. My projects folder is only local to my machine, which is a portable, and contains all the documents needed for that project.

TL;DR Shallow folder structure with dates at the beginning of files essentially.

One thing I do that I've found to be pretty helpful is to prefix files/directories with a number or date, for sorting. Some things are naturally ordered by date, for example events. So I might have a directory "my-company/archive", where each item is named "20170621_some-event".

Other things are better sorted by category or topic. For tools or programming languages I'm researching I might have a directory with items "01_some-language", "02_setup", "10_type-system", "20_ecosystem", etc.

My home folder is it.

  .
  ├── Desktop
  ├── Downloads
  ├── Google Drive // My defacto Documents folder
  │   ├── legal
  │   ├── library // ebooks and anything else I read
  │   ├── ...
  ├── Downloads
  ├── Sandbox //  all my repositories or software projects go here
  ├── Porn // useful when I was a teen, now just contains a text file with lyrics to "Never Gonna Give You Up"

I backup my home folder via Time Machine. I haven't used Windows in years but when I did, I used to do something similar. Always kept a separate partition for games, and software because those could be reinstalled easily, personal data was always kept in my User folder.

I try not to over think it, just:

  ~/$MAJOR_TOPIC
  |
  |--- ./$MORE_SPECIFIC
      |
      |--- ./$MORE_SPECIFIC
          |
          |--- ./general-file.type
      |
      | ./general-file.type
  |
  |--- ./$MORE_SPECIFIC
  |
  |--- ./general-file.type

etc

As you find yourself collecting more general files under a directory that can be logically grouped, create a new directory and move them to it.

Also keep all your directories in the same naming convention (idk maybe I'm just OCD)

I mostly work with Golang so usually all work related stuff will be in my GOPATH in ~/code/go/src/github.com/company-name/.

Non Golang code will go to ~/code, sometimes ~/code/company-name but I also have couple of ad hoc codebases spread around in different places on my filesystem.

So it is a bit disorganized. However last few years I have rarely ever needed to cd outside of ~/code/go.

Some legacy codebases I worked on (and still need to contribute to from time to time) can be in most random places as it took some effort and time to configure local environment of some of these beasts to be working properly (and they depend on stuff like Apache vhosts) so I am too afraid to move those to ~/code as I might break my local environment.

~/Dropbox/dev-media/books

~/Dropbox/dev-media/slides

~/Dropbox/dev-media/video

When reading for pleasure I typically read paper, try to limit the screen time if possible.

- ebooks: I don't love Calibre, but it's the only game in town.

- music: Musicbrainz Picard to get the metadata right. I've been favoring RPis running mpd as a front-end to my music lately.

- movies/TV: MediaElch + Kodi

I don't have a good solution for managing pictures and personal videos that doesn't involve handing all of it to some awful, spying "cloud" service. Frankly most of this stuff is sitting in Dropbox (last few years worth) or, for older files, in a bunch of scattered "files/old_desktop_hd_3_backup/desktop/photos"-type directories waiting for my wife and I to go through them and do something with them. Which is increasingly less likely to happen—sometimes I think the natural limitations of physical media were a kind of blessing, since one was liberated from the possibility of recording and retaining so much. Without some kind of automatic facial recognition and tagging—and saving of the results in some future-proof way, ideally in the photos/videos themselves—this project is likely doomed.

My primary unresolved problem is finding some sort of way to preserve integrity and provide multi-site backup that doesn't waste a ton of my time+money on set-up and maintenance. When private networks finally land in IPFS I might look at that, though I think I'll have to add a lot of tooling on top to make things automatic and allow additions/modifications without constant manual intervention, especially to collections (adding one thing at a time, all separately, comes with its own problems, like having to enumerate all of those hashes when you want something to access a category of things, like, say, all your pictures). Probably I'll have to add an out-of-band indexing system of some sort, likely over HTTP for simplicity/accessibility. For now I'm just embedding a hash (CRC32 for length reasons and because I mostly need to protect against bit-rot, not deliberate tampering) at the end of filenames, which is, shockingly, still the best cross-platform way to assert a content's identity, and synchronizing backups with rsync—ZFS is great and all but doesn't preserve useful hash info if a copy of a file is on a non-ZFS filesystem, plus I need basically zero of its features aside from periodically checking file integrity.

-clients - For client specific work

   -Project1

-devel - For development/research

   -Language/technology

       -specific research case

And I built my own bookmarking tool for references/citations.

symlinks for ~/Downloads and ~/Documents into ~/Dropbox is my only interesting upgrade. Across the varying different devices I have different things selectively synced. Large media files are the only things that don't live in dropbox in some way or another. It's pretty convenient for mobile access (everything accessible from web/mobile). I've done some worrying about sensitive documents and such, but most of it is also present in my email, so I think I lost that battle already. It also means there's very little downside to wiping my HD entirely if I want to try a different OS (which I used to do frequently, but ended up settling on vanilla ubuntu).

If you're particularly asking about reference material that you take notes about and would like to search and retrieve and produce reports on, Zotero might work for you. I have many years of research notes on it - it's a hyper-bookmarking tool that can keep snapshots of web pages, keep PDFs and other "attachments" within saved articles, lets you tag and organize them along with search capabilities.

Outside of that scope, my files reside randomly somewhere in the ~/Documents folder (I use a mac) and I rely on spotlight to find the item I need. It's not super great but is workable often enough.

It's not a silly question!

edit: I've been trying to find a multi-disk solution and haven't had much success with an easy enough to use tool. I use git-annex for this and it helps to some extent. I've also tried Camlistore, which is promising, but has a long way to go.

    documents/projects/projectname/whateverlayoutthetoolsdemand

with each project under Git. Layouts for Go, Rust, ROS, and KiCAD are forced by the tools. Python isn't as picky.

Web sites are

    sitename/
        info - login data for site, domains, etc.
        site - what gets pushed to the server
        work - other stuff not pushed to server

with each site under version control.

Zim wiki

Another option is to have a look at a tag-based filesystem instead of hierarchical ones to organize everything semantically. I'm using Tagsistant (there're other options) for a couple of months now and I'm almost happy. More satisfied with the idea itself and the potentiality.

I made an automatic document tagger and categorizer. It collects any docs or HTML pages saved to Dropbox, dropped into a Telegram channel, saved with zotero, Slack, Mattermost, private webdav, etc, cleans the docs, pulls the text, performs topic modeling, along with a bunch of other NLP stuff, then renames all docs into something meaningful, sorts docs into a custom directory structure where folder names match the topics discovered, tags docs with relevant keywords, and visually maps the documents as an interactive graph. Full text search for each doc via solr. HTML docs are converted to clean text PDFs after ads are removed. This 'knowledge base' is contained in a single ECMS, external accounts for data input are configured from a single yaml file. There's also a web scraper that takes crawl templates as json files and uploads data into the CMS as files to be parsed with the rest of the docs. The idea is to be able to save whatever you are reading right now with one click whether you are on your mobile or desktop, or if you are collaborating in a group, and have a single repository where all the organizing is done actively 24/7 with ML.

Currently reconstructing the entire thing to production spec, as an AWS AMI, perhaps later polished into a personal knowledge base saas where the cleaned and sorted content is public accessible with REST/cmis api.

This project has single handedly eaten almost a third of my life.