I remember I deleted a built-in directory like "Documents", and it lost its custom document icon, even when I created a "Documents" directory anew. No amount of googling / asking forums and IRC could bring my custom icon back. :)
If you delete one folder the file $HOME/.config/user-dirs.dirs will be updated sooner or later, and point to $HOME/ and the icon will not be restored by recreating any folder.
Having come up with a light structure and kept it for 7 years now, mine is:
media,
life,
edu,
data,
dev,
pix
The most heavily structured is /edu, everything else is a shitshow but I generally know where to find stuff. There are a few others that are dead. I don't bother putting dotfiles into this, but I probably should. I try to minimize directories with the same first letter for faster tab completion.
I bought the book "Building a Second Brain"[^1] by Tiago Forte and its recommended PARA methods is well-conceived, directly usable and tried and trusted for me. There is even a follow-up book from him about PARA only [^2]. I recommend to check it out (you'll also find many resources in the internet without the need to buy a book - although I highly recommend it and it would be an appreciation for the author)
I'm a big fan of PARA method, but I only use it in my note-taking app. I tried to apply it to filesystem, but it didn't work for me. So I went with his philosophy of keeping it simple and what works for you.
This is so individual; his solution is not useful for me, and my solution won't be useful for anyone else.
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
The only thing that matters to me is portability when I switch systems, and aware of work vs personal environments
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
I have found it surprizingly difficult to categorize my files in a way that is elegant and intuitive. I've been meaning to look into FUSE for a tagging-based system instead of the current hierarchical one, although I suspect that the difficulty of categorizing things is unlikely to go away even with tags.
I am genuinely surprised tagging never got more traction in the native FS space. I use MacOS's tagging pretty extensively and Dropbox at least keeps them in sync, a lot of other services lose them.
I've tried using tags, but the effort isn't worth it for me. Between search, occasional housekeeping, and a decent folder structure I have everything I need without being coupled to Apple.
The problem I have is that the optimal categorization seems to change depending on my current job and life situation. Tagging requires a whole new level of discipline ...
I always like the idea of tags, but whenever I get into a proper tagging system with thousands of entries I always end up having to regularly "refactor" my tags, which is fun exactly once and then it's just another goddamn chore.
The underlying problem is that the way your brain works doesn't really map well into categories. Text searching sometimes works, but has the same problem.
Source: Nearly two decades ago we worked on this problem and read a bunch of research. We ended up making an app that let you input multiple facets to narrow things down, with the basic view being a calendar: https://nemo-docs.com/
We didn't have anyone marketing it, so it never really took off. The original plan was to build a serverless/P2P storage system underneath so sharing would be seamless and not require messing with servers, but never got that far.
> The underlying problem is that the way your brain works doesn't really map well into categories. Text searching sometimes works, but has the same problem.
Oh, I see. You put files/objects/data on a timeline. I wish the semantic desktop had brought us that. There's a firefox extension that conveniently displays your history in a calendar form and it's really useful.
The “user-dirs.dir” setting is something I didn’t know was configurable. I thought I was always stuck with ~/Desktop, ~/Documents, …
Have slowly been using nixos as my primary driver (still in an emulated vm though). One thing I love here is the declarative nature. Lately have been experimenting with various desktop environments. I tried gnome and “it works” but have been fascinated with the idea of “ricing” my setup.
I once accidentally did an `rm -rf ~/` on a machine without backups and it removed everything alphabetically up until `~/Coding` (everything else being unimportant). So I fixed that by setting up backups and renaming it `~/Zoding` lol. Haven't been foolish enough to make that same mistake again, though.
Yes, that's why due to yet another totally unrelated quirk of history you
> have all the basic hidden stuff which ... such as .config
> but I only leave those dotfiles in place which work identically across the different systems I use
or you can use a better solution like chezmoi config manager and have templates to allow you to store changes even for those files while retaining the history of changes
But the most important piece of advice is missing: get an Everything-like utility (fsearch is a worse alternative on Linux) and bind it to an easy shortcut, that'd be very often the fastest way to find any file and not have to remember all those folder hierarchies (not that they aren't useful)
The only things in my home directory (note: I use Windows, but the convention still applies) are stuff that demand to be in there, like savegame files and software like Discord which demand to reside in %appdata% or else.
My stuff are found literally everywhere but, usually in their own drives and partitions.
Same. I’ve relinquished control over all the OS-provided directories to Adobe and Microsoft and go and whatever else assumes I want its crap in those locations.
Above someone jokes about just putting everything in Desktop, but honestly, it’s what realistically winds up happening. I can’t be bothered to manage files myself, really. Mostly, the apps I use know where their things are, I have a rudimentary system for archiving finished work, and the rest I can handle with search.
I’m just not in any way geared to be a file clerk or self librarian. Fortunately I have a machine that is very good at those things
I have something similar with ~/Stuff. ~/Stuff contains a folder per month, and ~/CurrentStuff links to the current folder. From there I have a simple shell-function `mkstuff` to create a folder ~/CurrentStuff/$DAY-"$1".
Naturally, it's a huge mess, but it helps if a colleague is like "Hey, didn't we have an issue back in August with the XYZ?". `ls ~/Stuff/2023-{07,08,09}` tends to find things again.
Besides that, I mostly have Projects/ (git projects I change) and Repos/ (git projects I look at) on my work laptop.
just like septem is 7, novem is 9 and decem is 10.
so the parent comment is saying that while at one point it may have been possible to derive the name from the number of the month, that is no longer the case because names and numbers do not match up and therefor the name would not be undevigintiber. thought it might be septendecimber ;-)
To alleviate the issue with most applications dumping their files in your home directory, i found xdg-ninja[0] to be helpful.
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
I had an experience where I attempted to install a new Mac from a Time Machine backup and my Mac couldn't see anything on the Time Machine backup. Called Apple support and found out about a rare bug where the install process can sometimes initialize the Time Machine backup instead of installing from the backup.
Luckily I had backblaze set up but it took a loooong time to get a few hundred gig restored.
Now I have Time Machine, BackBlaze and iCloud backups. Every now and then I dump everything to a .tgz on S3
I prefer to either LUKS-encrypt the volumes used by ZFS, or use ecryptfs at the user level. ZFS ‘encryption’ has some nasty gotchas, such as not encrypting snapshot names (which just boggles the mind …).
I add ~/tmp (cleaned out on every boot), and use ~/.config/user-dirs.dirs to assign good lowercase names to everything.
I wish every OS had a space that is only and truly my space. No predefined directories, no config files that I did not put there myself, and no app should ever create files or directories there.
I settled a very long time ago to organize all my own data into projects under /proj and an alias `cdp` for `cd /proj`. That was what decades ago my colleagues were doing, and it has worked well for me ever since.
I appreciate the[1] home dir as a place for config files mostly and despise it for having become the system's junkyard otherwise.
[1] I'm hesitant to say 'my' because since everyone seems to put all their junk there, it does not feel like it is mine at all.
I would say that ~/Documents is pretty good here. The only trouble I have had is it being the default save location of some apps. But I haven't had something yet write there without any explicit action.
I hate having a polluted home directory, especially when an application thinks it should get a non hidden directory in my home!
The one that upsets me the most is the default directory for go modules `~/go`. This frustrates me so much. I refused to install any go apps or use it for development for years. I've unfortunately had to give in, and it can at least be overridden by setting `GOPATH`, but it is a terrible, terrible default.
Same thing, wish that from the start unix had some standardized "applications put everything they want here" home directory that's separate from the "here user chooses to put their own files" home directory
At the start, unix commands didn't need or use configuration files. The world really was that simple, for a while. Now we have even simple desktop apps, that I have not changed a single setting in, insisting to write a whole directory tree of whatever data in my home.
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
The worst offenders are CLI tools written by people on Macs that do not respect XDG, because it's not a thing over there. So every tool gets to pollute your dotfiles with its own stupid directory. .rustup, .mix, .npm, .yarn, etc.
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
>But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
In theory, all applications following XDG would also solve the problem, as it has only 1 configuration directory. (Although I admit some apps use the data directory which brings it up to 2)
"I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in."
You seem to be mixing together two concepts here:
1. The files created by the installer, which are handled by the package manager. I can consult my package manager for files created by a specific package: pacman -Ql package_name.
2. The files created after installation (user preferences, plugins). The program should follow the XDG specification.
And those should be avoided as much as possible because they're always problematic and break randomly. How could they not break, when the package manager doesn't even know they exist or what they depend on?
> XDG is not a thing on Windows either afaik. Is a convention on one operating system something other operating systems should be faulted for not following?
GP is saying they should follow XDG on Linux not do it anywhere else!
Though if you're a CLI app on OS X, maybe consider it (especially if XDG_CONFIG_HOME _is_ defined), seems more common than the native OS X location for this type of app.
I guess the ire towards macOS-created apps comes from the fact that you can write macOS things that work on Linux without doing the “proper” things for the platform.
Windows is SO different that you have to do real porting work.
When something requires a minimal amount of porting effort to do 100% perfectly, or zero effort to do 99% it seems a lot of apps go for that 99%.
For my part writing cross platform apps I never actually realized there was a standard for arranging dotfiles so I wouldn’t have even realized I was leaving something out. XDG was one of those things I was vaguely aware existed and figured had something to do with the house of cards that is Linux desktop environments.
Weirdly, enough, golang is one of the only programming languages that actually has built-in support for a cross-OS config dir location: [os.UserConfigDir()][1].
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
function userConfigDir() {
switch (process.platform) {
case 'darwin':
return `${os.homedir()}/Library/Application Support`;
case 'win32':
if (process.env['APPDATA']) {
return process.env['APPDATA'];
} else {
throw new Error('%APPDATA% is not set correctly');
}
case 'aix':
case 'freebsd':
case 'openbsd':
case 'sunos':
case 'linux':
return process.env['XDG_CONFIG_HOME'] || `${os.homedir()}/.config`;
default:
throw new Error(`The platform ${process.platform} is currently unsupported.`);
}
}
IMO, the GOPATH itself is a terrible design. It makes you mix together things from unrelated projects in the same directory structure, instead of each project having its own independent directory structure, like on every other programming language. The GOPATH design goes so much against the way I organize my projects, that it's the main reason I have never got interested into learning Go.
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
Can you expand on this a bit please? What does $host mean? Why would I need that for a purely local project that is only created for my own use?
And what about grouping projects? E.g. "personal", "work", etc. And where in the structure are languages? Is that the overarching directory?
In Go parlance, it would be the remote host where the repository is hosted, e.g. github.com, dev.azure.com, golang.org, etc.
> Why would I need that for a purely local project that is only created for my own use?
If nobody else is using your purely local project, and you're sure nobody will ever use it until the end of time, then I guess you could just use "~/src/$HOSTNAME/$USERNAME/$PROJECTNAME". Otherwise, it would be wise to setup a remote repository ahead of time.
Go has a strong opinion that, in this day and age of distributed computing, projects should be online-first, so they can be easily used as dependencies. One of the nice consequences of this opinion is that Go dependencies can just be specified in the import statement - e.g. using grpc dependency is just:
import "google.golang.org/grpc"
No need for pom.xml, requirements.txt, cmake.txt, or any other kind of dependency configuration. It just works (unless it doesn't, like with private repositories, in which case it requires some exotic configurations in ~/.gitconfig or ~/.netrc, but that's a whole other can of worms - for most public repositories I've used it works flawlessly).
> And what about grouping projects? E.g. "personal", "work", etc.
Assuming you only use one repository hosting service and have one username, all your personal projects would be under "~/src/$PERSONAL_HOSTING_SERVICE/$USERNAME/", and your work would be under "~/src/$WORK_HOSTING_SERVICE/$WORK_ENTITY/" or something like this.
> And where in the structure are languages?
It isn't. That's either a bug or a feature. If it's a bug, you could just do the whole thing by language, e.g. "~/src/go/", "~/src/java/", etc.
> No need for pom.xml, requirements.txt, cmake.txt, or any other kind of dependency configuration. It just works.
...until the project decides to switch to another hosting provider. Which has happened more than once in the past; it used to be common to host projects in Sourceforge, for a while Google Code was common, now many projects are on GitHub, and it won't surprise me at all when another forge becomes the popular one. Individually, projects might switch between being self-hosted (in their own domain name) and hosted on a shared forge (using the forge's domain name).
IMO, it's a bad design. It forces the project's repository's location to become the project's official "name", that is, it mixes up location and naming. It's better to have an indirection layer to map the project name to the project location, like most other languages do.
Go already supports an indirection layer, commonly known as vanity URLs. It works by making a request to a domain owned by the project and parsing a meta tag in the response that points to the actual repository location. Of course, the problem is that few projects bother to set this up.
> ...until the project decides to switch to another hosting provider
Or decides their github username looks better with an upper case letter (https://github.com/sirupsen/logrus/issues/570). Or for people who use their real name as their github name, updating their username after marriage, divorce, gender transition or whatever.
The "owner" could be multiple directory levels depending the hosting service. Gitlab lets you have arbitrary sub levels. The owner of the files also isn't necessarily related to the owner of the repo on Github.
This hasn't been the case since go modules exist, if I understand your issue correctly: https://go.dev/blog/using-go-modules. You can additionally vendor each project's dependencies
In a situation like this, when I have control over /etc/passwd and can create accounts as I please, then I might have a single account, let's say /home/amturing, then (as a subdirectory) another account /home/amturing/amtgo.
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
The `~` path (and `cd` with no arguments, to get there) are also part of the prime real estate. It's as if these polluters think you'll enjoy accessing their stuff via `~/<tab>` which might or might not be true.
Polluters probably assume you'll never access anything at all except through dedicated apps, and bookmarks. In my case it's mostly true but it's still icky and ugly to see a crowded home dir.
Yeah, but that is still one application. You might come fr enough with CDPATH and configuring .inputrc to do something smart with ^m and the %-character (for the root of your document folder).
If it will be enough, I don't know. It might be enough for some, but it is still something which can be done without getting the maintainers of openssh to agree with the idea of a clean ~.
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
I follow the above advice. My personal stuff is in $HOME/$MY_FIRSTNAME.
I can organize this directory any way I want to, e.g. by project, time, purpose, whatever. It’s my stuff.
The stuff in $HOME is then more “the computer’s stuff.” (E.g., on MacOS, Photos, Desktop, Downloads - that’s where the computer dumps that stuff.) The other place is where I put things.
This turns out to work quite well. It allows me to be unconcerned when some application wants to put something in $HOME.
I too follow this advice. I am the only user on my system. I would put all my stuff directly into /home if that were an option which was unlikely to have huge gotchas at some point.
To me its a sane choice on my own single-user machine. I used to create a top level `/data` directory to put my stuff into (before Android was widespread :) ). Nowdays, my data resides in `/data/main`. This scheme has served me well for more than 10a and through various OS migrations (Windows -> Ubuntu -> Debian).
I don't want to have all this `.cache` and similar stuff to take up my expensive backup storage, nor do I want to waste time cleaning up after all the applications that create/expect files in `$HOME` or think this a sensible place to put executable files. I don't want arbitrary applications (heard of tracker?) to start indexing my files and waste CPU on that etc.
Also I don't believe in /home being portable across multiple Linux distributions because e.g. different application versions might require config files to be migrated. Hence I opt for `/data` that can safely be shared across multiple systems because it contains actual data and not some OS-related state files.
On multi-user systems, the `/data` approach doesn't scale. There, I prefer to do just as boomboomsubban writes: I create a subdirectory in $HOME (usually, I call it `wd`, the "working directory") and treat that like my local `/data/main`. For large/temporary files (e.g. application downloads etc.) I add a `$HOME/large` directory to keep the `wd` reasonably small in such cases such that I can again include it in backups without having to worry about the size.
It's not sarcasm. It's a pro tip which makes your life easier.
Also $HOME is a system directory, and it belongs to your operating system (.bashrc and such), also to your DE, also about every program and standard that I am aware of claims it for itself to use (freedesktop and such).
> In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
No. You got it backwards. If you put your stuff in a folder (not in the $HOME folder, but practically anywhere else), then your stuff will be in 1 place, there won't be any name collision with the system, it won't be polluted, and such.
Putting your stuff in $HOME is exactly what you want to avoid.
I wish more developers would seek out and embrace libraries for saving things in sensible locations. Python has one called “platformdirs” that takes the guesswork out of where to save things automatically for the user’s platform. I have no doubt all the other major environments have something similar.
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
I have given up on the idea of a clean home dir a long time ago. I have "networked" directories (pCloud,Dropbox) in my home for everything I care for, and those are organized "perfectly".
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
each year I make a year folder. And each project has a month + day prefix.
Sometimes I want long term projects to pop up on top, so I prefix them with 0000 (or only make the day 00).
It is simple, works on any OS. Although on Linux I do have some helper scripts.
And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
A lot of filesystems don't store file creation dates. Posix only requires last modification date (which for directories, is the last time a file/subdirectory was added, removed, or renamed), last access date, and last status change date.
My experience with metadata is that a lot of applications don't care about its integrity but will never mess with the filename unless that is the purpose of the application.
Demo a sync program and find that all the creation dates have changed to today!
> My experience with metadata is that a lot of applications don't care about its integrity [...]
This has led me to formally maintaining significant metadata in the filename, for certain kinds of files; everything else gets lost or diverges. For instance, a book file might be named `Title [a=Jim Smith; isbn=9781234567890].pdf`.
No, I'm serious. There's no portable filesystem metadata, and while a few kinds of media files have facilities for internal metadata (e.g. EXIF and ID3), most don't. A file name is the only non-content that is easy to keep associated with any kind of file.
The syntax is definitely an imperfect compromise, but most of the things I use this for have titles that frequently contain spaces (books, papers, music, etc.) so quoting would be necessary anyway, and I made myself a Python package to help with escaping and do other utility tasks like normalizing ISBNs.
that's a good idea, especially if you don't want to keep a database to manage your library. In the piracy side of things we just make sure to name files something that can be used to look it up in the database and get metadata from that.
Although I have to warn you that ISBNs are not a perfect guid and they sometimes get reissued. Good enough if you aren't hoarding the world I guess.
I do something similar for relevant folders, where I organize by time.
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
The advantage of 041924 is that, when changing directories from the terminal, you can type 0419 and tab the rest of the way (99% of the time the only completion is 24). It's more efficient. If I prepended the year, I'd have to type out 240419 every time.
Using it with zoxide[1] allows me to run "z weba" and get taken to:
~/Projects/client_org/amazing-webapp
I also use a couple of helper functions in my zshrc while working in the Scratch directories:
# Marky Mark and the Function Bunch
function gcd {
git clone $1 && cd "$(basename "$_" .git)"
}
function showpath {
echo $PATH | sed -e $'s/:/\\\n/g'
}
if [[ $OSTYPE =~ ^darwin.* ]] then
function brewup {
brew update && brew upgrade && brew cleanup
}
fi
function mc {
command mkdir $1 && cd $1
}
function mcd {
local TODAYDATE
TODAYDATE=$(date +%F)
command mkdir ${TODAYDATE} && cd ${TODAYDATE}
}
function mce {
command mkdir ${EPOCHSECONDS} && cd ${EPOCHSECONDS}
}
EPOCHSECONDS comes from the zsh/datetime module[2]
I think it's a nice idea, but don't like the media structure into family and so on. Looks like he's going to have a ton of problems with duplicates and then edited duplicates down the line, which can easily be mixed up, and then you lose your edits and so on.
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
Media-specific content-addressed stores (ideally as overlays which refer to but don't change your directory structure or file organization) seem to be the way to go with this.
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
Depends a bit on the use case. For personal photos, mobile photos and the like, I use Google Photo as well. But it just isn't suitable for professional storage or sorting IMHO, tho the face recognition is certainly neat. The best I've came across was FotoStation, but it's many years since I worked with those sorta things.
You may want to try TagSpaces https://www.tagspaces.org/ or TMSU https://tmsu.org/ which provide mechanisms for managing tags of arbitrary files (not only EXIF or ID3 ones).
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.
I personally would rather type "ThisIsSomeOtherProject" than "this-is-some-other-project" as advocated on the page. Approximately the same number of keypresses but fewer characters to read.
171 comments
[ 5.1 ms ] story [ 245 ms ] thread(Linux Mint, Cinnamon)
If you delete one folder the file $HOME/.config/user-dirs.dirs will be updated sooner or later, and point to $HOME/ and the icon will not be restored by recreating any folder.
Just change the values in there to whatever.
The most heavily structured is /edu, everything else is a shitshow but I generally know where to find stuff. There are a few others that are dead. I don't bother putting dotfiles into this, but I probably should. I try to minimize directories with the same first letter for faster tab completion.
[^1]: https://www.buildingasecondbrain.com/book
[^2]: https://www.buildingasecondbrain.com/para
My home directory is nearly empty, because all the files I work with are in OwnCloud (so the real question is: what is the directory structure in OwnCloud). Local Git repositories are on a completely separate partition.
Since KeepassXC now handles SSH keys, the keys from .ssh are now in the Keepass-file, which is on OwnCloud. That was a huge simplification, and now there is nothing in my home directory that I really care much about.
I need to be able to login and run a single command to sync to have the appropriate set of files for whatever /linux/work/personal/desktop/server I am on
For example, I never want certain environment variables to be exported to work systems, like my home vault endpoint or certain tokens.
I recently switched to home-manager from the NixOS project and it looks very promising. The Nix language is complicated but the abstraction to define all these different types of environments is exactly what I need, and git branches manage the work/personal split of file content
Tagging data is such a niche feature. But when it's useful, it's very useful.
Source: Nearly two decades ago we worked on this problem and read a bunch of research. We ended up making an app that let you input multiple facets to narrow things down, with the basic view being a calendar: https://nemo-docs.com/
We didn't have anyone marketing it, so it never really took off. The original plan was to build a serverless/P2P storage system underneath so sharing would be seamless and not require messing with servers, but never got that far.
What did you find that works ?
Do you have any articles about that ?
Have slowly been using nixos as my primary driver (still in an emulated vm though). One thing I love here is the declarative nature. Lately have been experimenting with various desktop environments. I tried gnome and “it works” but have been fascinated with the idea of “ricing” my setup.
Currently using a Wayland compositor called hyprland to achieve something like this: https://github.com/outfoxxed/hy3?tab=readme-ov-file#demo
Community has experienced some drama though with fdo/x/wlroots. Always willing to make a change up if a better alt comes up though.
Desktop, Dev, Downloads, Documents, Dropbox etc.
I thought about taking action about this, but as the author points about, many applications are quite opinionated on the matter.
Yes, that's why due to yet another totally unrelated quirk of history you
> have all the basic hidden stuff which ... such as .config
> but I only leave those dotfiles in place which work identically across the different systems I use
or you can use a better solution like chezmoi config manager and have templates to allow you to store changes even for those files while retaining the history of changes
But the most important piece of advice is missing: get an Everything-like utility (fsearch is a worse alternative on Linux) and bind it to an easy shortcut, that'd be very often the fastest way to find any file and not have to remember all those folder hierarchies (not that they aren't useful)
My stuff are found literally everywhere but, usually in their own drives and partitions.
Above someone jokes about just putting everything in Desktop, but honestly, it’s what realistically winds up happening. I can’t be bothered to manage files myself, really. Mostly, the apps I use know where their things are, I have a rudimentary system for archiving finished work, and the rest I can handle with search.
I’m just not in any way geared to be a file clerk or self librarian. Fortunately I have a machine that is very good at those things
so i open my shell, it drops me into ~/t/, which is symlinked to ~/work/YYYY/DD/MM.
my GTK file picker is also setup to automatically give me the last 3 days on the bookmark bar.
Time is not the _best_ heuristic, but it's not bad.
screenshots from scrot all get dumped into ~/screenshots/ by date.
the only real 'folders' i keep around are ~/src/, ~/bin/ and ~/documents/, with the last being like, tax documents.
Naturally, it's a huge mess, but it helps if a colleague is like "Hey, didn't we have an issue back in August with the XYZ?". `ls ~/Stuff/2023-{07,08,09}` tends to find things again.
Besides that, I mostly have Projects/ (git projects I change) and Repos/ (git projects I look at) on my work laptop.
Please confirm you are not a monster and that you meant /YYYY/MM/DD.
just like septem is 7, novem is 9 and decem is 10.
so the parent comment is saying that while at one point it may have been possible to derive the name from the number of the month, that is no longer the case because names and numbers do not match up and therefor the name would not be undevigintiber. thought it might be septendecimber ;-)
but then the year 2022 would be far in the future as the current baha'i year is 181.
that date in the baha'i calendar would be march 4th 3866 in the gregorian calendar
In short, it scans all your programs and determines if you can configure it to respect the xdg standards. It doesn't work for everything, but most applications do seem to have such an option.
[0] https://github.com/b3nj5m1n/xdg-ninja
I had an experience where I attempted to install a new Mac from a Time Machine backup and my Mac couldn't see anything on the Time Machine backup. Called Apple support and found out about a rare bug where the install process can sometimes initialize the Time Machine backup instead of installing from the backup.
Luckily I had backblaze set up but it took a loooong time to get a few hundred gig restored.
Now I have Time Machine, BackBlaze and iCloud backups. Every now and then I dump everything to a .tgz on S3
I add ~/tmp (cleaned out on every boot), and use ~/.config/user-dirs.dirs to assign good lowercase names to everything.
And of course all code lives in ~/src.
I settled a very long time ago to organize all my own data into projects under /proj and an alias `cdp` for `cd /proj`. That was what decades ago my colleagues were doing, and it has worked well for me ever since.
I appreciate the[1] home dir as a place for config files mostly and despise it for having become the system's junkyard otherwise.
[1] I'm hesitant to say 'my' because since everyone seems to put all their junk there, it does not feel like it is mine at all.
The homedirs belong on the OS drives and are managed by the OS. Anything you want to keep safe goes on the data volume where backups run.
It’s also very nice to just disconnect the data drives before any upgrades. You can’t corrupt a filesystem that isn’t even present to the system
[0] https://github.com/andsens/homeshick
The one that upsets me the most is the default directory for go modules `~/go`. This frustrates me so much. I refused to install any go apps or use it for development for years. I've unfortunately had to give in, and it can at least be overridden by setting `GOPATH`, but it is a terrible, terrible default.
Or absolutely moronic things like ~/.config/VSCod*/*Cache* -- really, cache in config?
But polluting your home directory like ~/go, without even the decency to hide it, is extremely rude and offensive.
Counterpoint:
I actually like having visible directories, versus having to figure out where in /usr/share or /usr/local/ or ~/.local or /var an installer chose to sneak their files in.
I got used to it and in HOME I put some of my manually installed tools like `AMD_AOCL` for the AMD AOCL libraries, `Android/android-studio` for Android Studio, `intel` for IPP/OpenAPI, etc.
Don't want it anymore? Easy rm -rf, no need to go digging for where it could be hidden.
1. The files created by the installer, which are handled by the package manager. I can consult my package manager for files created by a specific package: pacman -Ql package_name.
2. The files created after installation (user preferences, plugins). The program should follow the XDG specification.
Some install methods are outside of your package manager, but try to touch /usr or /opt or /var, like NVIDIA sh scripts and the likes.
1. External install scripts that put stuff in system directories are sloppy and we don't like them
2. External install scripts that put stuff clearly-named non-hidden directory in your $HOME are better than the above
GP is saying they should follow XDG on Linux not do it anywhere else!
Windows is SO different that you have to do real porting work.
When something requires a minimal amount of porting effort to do 100% perfectly, or zero effort to do 99% it seems a lot of apps go for that 99%.
For my part writing cross platform apps I never actually realized there was a standard for arranging dotfiles so I wouldn’t have even realized I was leaving something out. XDG was one of those things I was vaguely aware existed and figured had something to do with the house of cards that is Linux desktop environments.
I don't really ever program in golang, but whenever I write a Node.JS/Python tool that does need a user-global config file, I just write my own implementation of it:
[1]: https://pkg.go.dev/os#UserConfigDirEnvironment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData)
> pollute
gah, reminds me of my pet peeve.
Insert a drive, it gets .Trashes ._Trashes .DS_Store .fseventsd .Spotlight-V100 and other nonsense everywhere.
And if you are ever trying to recover a failing drive, do not mount it on a mac.
Perhaps this design makes more sense for those who prefer to keep several unrelated projects together in a single monorepo, but that's not my preference.
~/src/$host/$owner/$repo
organization structure makes a ton of sense for every project and as long as you organize all of your languages into this one tree, everything just works.
In Go parlance, it would be the remote host where the repository is hosted, e.g. github.com, dev.azure.com, golang.org, etc.
> Why would I need that for a purely local project that is only created for my own use?
If nobody else is using your purely local project, and you're sure nobody will ever use it until the end of time, then I guess you could just use "~/src/$HOSTNAME/$USERNAME/$PROJECTNAME". Otherwise, it would be wise to setup a remote repository ahead of time.
Go has a strong opinion that, in this day and age of distributed computing, projects should be online-first, so they can be easily used as dependencies. One of the nice consequences of this opinion is that Go dependencies can just be specified in the import statement - e.g. using grpc dependency is just:
No need for pom.xml, requirements.txt, cmake.txt, or any other kind of dependency configuration. It just works (unless it doesn't, like with private repositories, in which case it requires some exotic configurations in ~/.gitconfig or ~/.netrc, but that's a whole other can of worms - for most public repositories I've used it works flawlessly).> And what about grouping projects? E.g. "personal", "work", etc.
Assuming you only use one repository hosting service and have one username, all your personal projects would be under "~/src/$PERSONAL_HOSTING_SERVICE/$USERNAME/", and your work would be under "~/src/$WORK_HOSTING_SERVICE/$WORK_ENTITY/" or something like this.
> And where in the structure are languages?
It isn't. That's either a bug or a feature. If it's a bug, you could just do the whole thing by language, e.g. "~/src/go/", "~/src/java/", etc.
...until the project decides to switch to another hosting provider. Which has happened more than once in the past; it used to be common to host projects in Sourceforge, for a while Google Code was common, now many projects are on GitHub, and it won't surprise me at all when another forge becomes the popular one. Individually, projects might switch between being self-hosted (in their own domain name) and hosted on a shared forge (using the forge's domain name).
IMO, it's a bad design. It forces the project's repository's location to become the project's official "name", that is, it mixes up location and naming. It's better to have an indirection layer to map the project name to the project location, like most other languages do.
Who takes care of the indirection layer when the upstream decides to switch to another hosting provider?
What do you do if they fail to update their name::location mapping, and the language doesn't provide a way to do it yourself?
At least in Go, when that happens we can just add a `replace` statement in the go.mod file:
[0] https://go.dev/ref/mod#go-mod-file-replace
Or decides their github username looks better with an upper case letter (https://github.com/sirupsen/logrus/issues/570). Or for people who use their real name as their github name, updating their username after marriage, divorce, gender transition or whatever.
Unruly apps that benefit from UID/GID regimentation can be constrained in this way. The whole app assuming ~amtgo is also helpful.
EDIT: I also remembered this useful advice on prefixing private shell scripts in your path with a comma:
https://news.ycombinator.com/item?id=31846902
If it will be enough, I don't know. It might be enough for some, but it is still something which can be done without getting the maintainers of openssh to agree with the idea of a clean ~.
$HOME is the one directory which belongs to the user. In some cases, it might even be encrypted with a user-owned key. I can’t imagine being comfortable putting my files anywhere _outside_ of the home directory. That feels like going back to the DOS / Win3.x days where hard drives were the wild west.
I can organize this directory any way I want to, e.g. by project, time, purpose, whatever. It’s my stuff.
The stuff in $HOME is then more “the computer’s stuff.” (E.g., on MacOS, Photos, Desktop, Downloads - that’s where the computer dumps that stuff.) The other place is where I put things.
This turns out to work quite well. It allows me to be unconcerned when some application wants to put something in $HOME.
I don't want to have all this `.cache` and similar stuff to take up my expensive backup storage, nor do I want to waste time cleaning up after all the applications that create/expect files in `$HOME` or think this a sensible place to put executable files. I don't want arbitrary applications (heard of tracker?) to start indexing my files and waste CPU on that etc.
Also I don't believe in /home being portable across multiple Linux distributions because e.g. different application versions might require config files to be migrated. Hence I opt for `/data` that can safely be shared across multiple systems because it contains actual data and not some OS-related state files.
On multi-user systems, the `/data` approach doesn't scale. There, I prefer to do just as boomboomsubban writes: I create a subdirectory in $HOME (usually, I call it `wd`, the "working directory") and treat that like my local `/data/main`. For large/temporary files (e.g. application downloads etc.) I add a `$HOME/large` directory to keep the `wd` reasonably small in such cases such that I can again include it in backups without having to worry about the size.
YMMV
Also $HOME is a system directory, and it belongs to your operating system (.bashrc and such), also to your DE, also about every program and standard that I am aware of claims it for itself to use (freedesktop and such).
No. You got it backwards. If you put your stuff in a folder (not in the $HOME folder, but practically anywhere else), then your stuff will be in 1 place, there won't be any name collision with the system, it won't be polluted, and such.
Putting your stuff in $HOME is exactly what you want to avoid.
If you ask me, this should all be built-in, but even then (because on many platforms, it is), people can’t help rolling their own.
Standards XKCD I guess.
Additionally I symlink .vimrc and .gitconfig to a git repos. Then everything else in my home dir can be total garbage, it no longer matters. The big plus is that if a machine dies I can be back in business in minutes on another one.
The `.config` folder is a big headache when trying to be strategic about backups due to apps putting gigs of session data there.
"session data" is not "config", consarnit! An app's "config" shouldn't be gigs large. Grumble grumble, grouse grouse, and so on. :^)
Thinking about multiple storage locations is effort and most users don't care about this too much anyway
It is simple, works on any OS. Although on Linux I do have some helper scripts. And it is very easy to quickly make a directory and move files from downloads into this directory. Keeping the nesting only one level deep helps for the discoverability. (versus the YYYY/MM/DD pattern which uses an extra month level)
Demo a sync program and find that all the creation dates have changed to today!
This has led me to formally maintaining significant metadata in the filename, for certain kinds of files; everything else gets lost or diverges. For instance, a book file might be named `Title [a=Jim Smith; isbn=9781234567890].pdf`.
Maybe not psyched about the shell-interpreted brackets, semicolons, and spaces. But the idea is interesting!
The syntax is definitely an imperfect compromise, but most of the things I use this for have titles that frequently contain spaces (books, papers, music, etc.) so quoting would be necessary anyway, and I made myself a Python package to help with escaping and do other utility tasks like normalizing ISBNs.
Although I have to warn you that ISBNs are not a perfect guid and they sometimes get reissued. Good enough if you aren't hoarding the world I guess.
I have a top level dir where I usually have what I'm working on at the moment - think "this day" or "this week".
When it's time to archive it, outside that time window, I created a folder, using this format, for example for today:
041924
I move all the files I created on that day into there.
As a person with a typical human lifespan, I see no need to use a "2024" string - I don't think I'll live to see 2100, and anything before 2000 isn't relevant. "24" is just fine.
And that's it. Time keeps rolling on, so the number of 'archive' folders keeps increasing, but they're small in size and easy to locate.
This works really well when you're creating not-that-unique standard files every day, btw, which is my use case.
I can accept YY instead of YYYY, but surely YYMMDD is preferable due to sort order.
[1] https://github.com/ajeetdsouza/zoxide
[2] https://zsh.sourceforge.io/Doc/Release/Zsh-Modules.html#The-...
Pictures are better sorted with EXIF keywords. So, metadata stored in the picture itself, perhaps even in the MIME type. Thus, if a picture is family related, just tag it #family, or #personx and so on. This is why I store them in the same folder on a per-date basis. The rest is keywords edited with programs such as Adobe Bridge and so on.
As for document file name structure, text files and the like, I've used both `Date then Descrpition.txt` or `Keyword Title or Description and then Date.txt` with the date obviously being an ISO date such that `YYYY-MM-DD-hhmm` with `-hhmm` being optional, also for sorting reasons. Sometimes I like sorting on "topics" i.e. keywords or titles, but other times, such as in logs, I like the date first because it's more essential to know when you logged something, and not necessarily the topic.
You should think the date is superfluous since it's also stored in the system. However, my experience that when you move around a file, the date eventually changes, and certainly if you make a mistake along the way. Meanwhile a filename date doesn't change. Also it helps with list sorting where applicable.
I think photoprism and photostructure don't really care about your directory structure or organization, but paperless(-ngx, or whatever is the most current iteration) is "notorious" for being opinionated about organization / not wanting to respect your organization.
I used camlistore/perkeep for photos for a while, but google photos has an absolutely killer feature of knowing who is in every picture, even accounting for age: although two of my boys who are 8 years apart look really similar at similar ages, it knows who is who. I don't know if it uses facial analysis or photo metadata or what, but it has never mistaken one for the other. I don't have a reasonable way of getting that tag information out of google photos (even though I'm paying for the service).
It might be time to revisit this, though.. I don't recall whether photostructure / photoprism try to do facial recognition, but even if they don't already, I bet they'll soon be roughly on par with google photos (or good enough that I can stop depending on google for this).
That's documents and photos / videos, what about music? For better or for worse, It's been at least a ~decade since I tried keeping my own music collection as files. Are there things these days like paperless / photoprism, a library system for music that has deeper integration with the content than just "files on disk"?
I've also been on the lookout for an AI assisted duplicate finder. Like, it'll compare pictures and then rate similarity between pictures on some scale, by appearance, colour or topic, perhaps with an auto-tagging feature as well. Ofc it should also compare dates, file sizes and the usual MIME type or hash based similarities as well.
Capital names for GUI things, lowercase names for CLI things. I'd prefer ~/documents, etc, but GUI people insist on capitalizing, so whatever. Very rarely I have to mix those, so it's not a big problem.
~/dotfiles is my dotfiles directory with git. I create ~/.zshrc -> dotfiles/zshrc, etc. I don't use any software to manage that, I just create symlinks. In the past I've used ~/.dotfiles but I think that making it visible makes more sense.
~/projects is my projects directory. ~/projects/test is throwaway test projects to check things out, etc. ~/projects/my is my personal projects. ~/projects/company is projects for company I'm working for. I'm kind of freelancer and sometimes work for different companies, so separation is necessary.
~/tmp is my throwaway directory for everything. I have shell function mkcdtmp which creates ~/tmp/240419 (current date) and cds into it. It's a wonderful way. I rarely clean it, because I prefer to buy big discs and keep trash around, somewhat organized. If I need something from yesterday or past month, I know where to find it. That's the most important thing I did for organizing my temp work which is plenty. Of course I can create ~/tmp/whatever if needed, it's all trash stuff.
Well, that's all, I guess. I don't use ~/Desktop. I rarely use ~/Documents, still need to find a way to organize it. Also I dump my short notes and stuff to my github repo which builds a personal website. I tried various notes software but ordinary website with markdown turned out the best way for me.
I never was able to organize my work in a very structured way, it's always piles of trash moving around which eventually turn into something usable, so instead of fighting myself I decided to make that trash organized.
Essentially my computer is disposable. Everything in ~/projects is in git. Everything in ~/tmp is not very important and more like a cache or discarded work. I'm trying to organize things in a way so restoring it from the clean state wouldn't take much time. I often reinstall OS from the scratch and I often switch between operating systems and laptops, that suits me best.