It feels unsafe to store dot files in source control as the article mentions. Is this a common practice? They could contain sensitive information and I fear people might be tempted to push to GitHub.
What about storing such files in a password manager/database?
Anecdotal, I encounter it quite often at work among 'newer' developers, who have been doing this ever since Github private repos went free! That's why my question. Thanks for the answers everyone, it does come down to diligence and knowing what you're doing.
Just don't push to GitHub. :) You can also pick and choose what you want in the repos with this method. It's a very useful technique for tracking and rolling back changes to dotfiles that you care about. You can even make backups of the directory which can help you bootstrap new machines really quick.
In a similar vein, I also have a git repo containing my todo list, journal, etc. and review and commit the changes once a week. Very helpful for catching accidental deletions/modifications. I have never been tempted to push these repos.
There are plenty of tools to manage secrets in VCS, so that's always an option. As an aside, most programs now have a method to move sensitive information outside of the configuration file, so you can minimize what ends up in there.
You simply don't push anything that has sensitive information in it. You pick and choose. Stow makes this easy, because the only things being pushed are files you explicitly move to a specific directory.
Real world example:
- Let's say I want to version control ~/.ssh/config and ~/.ssh/authorized_keys
- I move these two files to ~/.dotfiles/ssh
- ~/.dotfiles is version controlled, so I push the new files to gitlab. I'm only pushing two harmless files: my ssh configs, and authorized keys. Git, stow, and ~/.dotfiles don't see anything else.
- After pushing, I run "stow ssh" inside ~/.dotfiles. This symlinks these two files to ~/.ssh
I don't worry about pushing anything sensitive, because it's basically impossible for me to accidentally move something to ~/.dotfiles and then run 'git push.'
… and lose change history, handle merge conflicts by yourself (or lose edits made on other systems if you ever forget to re-sync), and all the other features of a proper VCS.
These comments are like that famous one where dropbox was announced on HN and someone replied to it with an rsync script “that is essentially the same”.
Hah, I recently had to setup a cronjob that rsync's my org-mode files to iCloud. (iCloud didn't work with symlinks, but I needed to somehow sync my org-mode files to my phone.) Anyway, my org directory happens to be a git repo, and I rsync the whole repo including .git, so that's technically something you can do. :)
It's pretty much the same as creating a git repository in /, but doesn't mess with your normal git usage (so if you run `git status` in `/home/foo/src/my-awesome-project` and git repository has not been created there yet, git will fail with "not a git repository" and won't show you the contents of your whole filesystem tree).
I used this initially (before running into the link above), but it still requires using a 'global' git repository which had its share of issues (some build scripts didn't like it, for example).
Doesn't matter in my case, there are no secrets there (so I use a public git forge). If this doesn't work for you, you can just push to any old server via ssh:
This is what I do, it has worked great for me. The only annoying thing is I can’t get tab completing to work in my shell (ZSH). Something about the alias I think.
I've been doing this for a few years now and definitely agree! I use .home.git as my .git-aside name and also named the access alias 'homegit'.
I didn't know about `git config --local status.showUntrackedFiles no` so I just put '/*' into .home.git/info/exclude, and then of course tracked that in itself by adding '!/.home.git/info/exclude' in there too.
It's been amazingly useful when acquiring a new laptop, or wanting to quickly set up a comfortable environment on remote machines.
Shameless self-plug: I could never get along with storing the whole content of the dotfiles when I casually just added a few lines I cared about, so I developed my own tool called confible. You can specify to just append a few lines or add the whole config and it can run commands (e.g. installing the specific tool together with its config). You can find it at https://github.com/sj14/confible
That’s pretty cool. The dumbed down version of that for me is sourcing different zshrc files depending on the machine I’m using…which obviously wouldn’t work for anything else.
I started using fossil for this very thing. I was using git, however I like that fossil allows you to put the repository where ever you want on the file system. I don't have to worry about alias's, using .gitignore, or setting it it up where it doesn't show untracked files.
As far as fossil is concerned, once you close the repo, it's out of site of mind. I still use git for most everything else. Only my dot files and my personal obsidian vaults are backed up to fossil.
I have a small digital ocean droplet, lowest tier serving mine. I need to probably do a write up of the steps soon before I forget. I'ts its own web server so you have a lot of options. SCGI, HTTPS, SSH, Socket Listener. I have mine behind reverse proxy sand boxed with systemd's DynamicUsers feature.
> I was using git, however I like that fossil allows you to put the repository where ever you want on the file system. I don't have to worry about alias's, using .gitignore, or setting it it up where it doesn't show untracked files.
Git also lets you put the repository wherever you want on the filesystem, via worktrees or via the `--separate` link. And I fail to see the relationship with aliases or ignore files (the last item I guess you're talking about fossil not showing untracked files by default?)
Sure (Sorry was mobile earlier, normally my grammar isn't that bad.)
Fossil doesn't show much unless you perform a open or a checkout against the repo file. Generally, it only shows the location of the config database ($HOME or $XDG) exmaples below. The added benefit is that you can do this open/checkout anywhere on the system. For instance, if I'm making changes in my /etc, I can commit a file from there without having to move the file. Another nice feature is they have an unversioned [1] mechanism as well, which allows you to commit files that you want to save but don't care about the history.
Ran from my $HOME
-------------------------
$ fossil info
config-db: /home/$USERNAME/.config/fossil.db
fossil: /home/$USERNAME/.local/bin/fossil
version: 2.19 [1e131febd3] 2022-07-21 16:10:55 UTC
$ fossil status
current directory is not within an open checkout
--------------------------
I still prefer git, and all my career and personal projects reside there. But every tool has its place, and I think that even though fossil doesn't fit my other SCM needs. What I'm using it for is perfect for fossils design.
Used several times to back up some critical configurations (3rd/4th paranoia backup right before maintenance) on a few servers because the commit/open from anywhere. It doesn't pollute the file system as it's a single file.
With the untracked files: It's a little different. The git way you usually use do the dot file backup is with an alias `--untracked-files=no` so you don't see them unless you override it. At least most of the examples I followed the processes I've experimented with over the years recommended.
With fossil you see them when you consciously issue a fossil open in order to do a commit. I'd rather see the untracked files, when I'm ready to commit then to not see them at all and miss something. Out of site. out of mind is something I personally struggle with.
It's 2022 and I still use this program since I read that article so many years ago.
It's handy but sometimes a bit of manual preparation must be done on a new clean machine. For example if you want to stow only some files under ~/.config/program/ and not the whole program/ directory, you must first run an mkdir and then run stow to put the appropriate symlinks in place.
I use this too but every other time I come back to it I have to really read through the docs to figure stuff out. That’s just me not writing very useful notes though -_-
To reiterate, the --no-folding flag prevents creation of symlinked directories, and it really should be the default behavior.
I don't blame you for not knowing about this flag. Here's its description in the manpage:
> Disable folding of newly stowed directories when stowing, and refolding of newly foldable directories when unstowing.
This is confusing because it's a circular definition. It doesn't explain what "folding" means. It should just read, "prevents creation of symlinked directories" or something.
Problem is, this still makes sense in some cases but doesn't in others, and stow, as it is, doesn't care about the distinction.
For example, on a machine where ~/.config/autostart/ doesn't still exist, I do not want that either ~/.config or ~/.config/autostart are created as symlinks to my stowed package; otherwise any newly installed file, from unrelated apps, would end up in my package! I just want this path to exist as a proper dir, and only contain whatever symlinks are needed from stow. Thus an `mkdir -p` is still needed before running stow.
On the other hand, I do want that ~/.config/wireshark/ is a symlink to the stow repository. So no mkdir in this case.
Regardless, having a script that adds extra per-package pre-process/post-process logic before/after running stow is still needed, anyway.
Been doing this for about a decade, it works nicely. Def recommend stows `--no-folding` flag (which prevents it from symlinking directories, just files)
I wound up writing my own stow replacement in Ruby, that moved the files rather than symlinked them. It does pattern-match the filenames with a list, anything on the list gets symlinked. It can also copy home-rolled binaries into /usr/local/bin. I suppose I could use both stow and etckeeper but a simple script can keep everything synced to my precise liking.
After trying a bunch of tools I settled on git-tracking $HOME with '*' on my .gitignore (ignores everything, you have to add new files by --force). Is this better? symlinks seem like a smell.
I didn't know about the --force flag. You can also whitelist files in .gitignore with !filename.[0]
Is there a reason to avoid symlinks? I've been using this setup for a few years now, (using an artisanal, home-rolled script before I discovered stow) and it's worked superbly.
Track you ~/.config (or whatever your XDG_CONFIG_HOME is) directory in git, and create symlinks for those programs that don't look there, optionally by script. Many programs already use that directory to store their configs. The Arch Linux folks are nagging every upstream into supporting it[1]. Also it requires no extra dependency aside git.
I think i say this everytime a dotfiles manager comes up.
This sounds like a good idea, I'd save all the `ln` commands the install.sh of my dotfiles repo. But do you just .gitignore all stuff in your .config you're not interested in?
I take the opposite approach in that I keep the default config file locations and have .gitignore set to "*", and just force-add files as needed. Also have GIT_DIR and GIT_WORK_TREE=$HOME
There's also `homesick`[1], which is a Ruby dotfile manager. If you don't feel like managing a Ruby distro and want something more portable (and `homesick` looks to be a stale project anyway), you can use `homeshick`[2] which is a Bash port that's still being maintained. (I use `homeshick`)
The last time I dug into this, `homeshick` was had more features and fit my needs better than `stow`.
Alternatively, check out YADM[3], "Yet Another Dotfile Manager", which I'm probably switching to once I get some time.
I've seen this post on hackernews a few times now (as dang diligently pointed out).
However, typically they don't mention that more recent versions of Stow actually have specific functionality for dotfiles, which is support for the `dot-` prefix.
Check the man page for your version of stow. If the string "dot-" doesn't appear anywhere, you have an old version of stow.
Also interesting to me was the fact that the `make` manual has information on how `stow` is actually one of the recommended workflows in makefiles (in the context of setting a DESTDIR); but I've never seen anyone actually do that in practice.
60 comments
[ 2.4 ms ] story [ 99.3 ms ] threadWhat about storing such files in a password manager/database?
Nothing can really protect someone from this level of insanity.
In a similar vein, I also have a git repo containing my todo list, journal, etc. and review and commit the changes once a week. Very helpful for catching accidental deletions/modifications. I have never been tempted to push these repos.
git-remote-gcrypt: https://github.com/spwhitton/git-remote-gcrypt
git-crypt: https://github.com/AGWA/git-crypt
git-secret: https://github.com/sobolevn/git-secret
However, most people who self-host their Git repository are fine with just transport-level encryption (TLS).
Real world example:
- Let's say I want to version control ~/.ssh/config and ~/.ssh/authorized_keys
- I move these two files to ~/.dotfiles/ssh
- ~/.dotfiles is version controlled, so I push the new files to gitlab. I'm only pushing two harmless files: my ssh configs, and authorized keys. Git, stow, and ~/.dotfiles don't see anything else.
- After pushing, I run "stow ssh" inside ~/.dotfiles. This symlinks these two files to ~/.ssh
I don't worry about pushing anything sensitive, because it's basically impossible for me to accidentally move something to ~/.dotfiles and then run 'git push.'
What’s it called when people do that?
Dismissiveness?
Being an HNer?
https://www.atlassian.com/git/tutorials/dotfiles
tl;dr: it puts the .git directory aside and lets you use it to store any files on your filesystem.
All the usual commands work, except you use something like `config` or `dofiles` instead of `git`:
It's pretty much the same as creating a git repository in /, but doesn't mess with your normal git usage (so if you run `git status` in `/home/foo/src/my-awesome-project` and git repository has not been created there yet, git will fail with "not a git repository" and won't show you the contents of your whole filesystem tree).Is this a problem if you gitignore * (whitelisting)?
I didn't know about `git config --local status.showUntrackedFiles no` so I just put '/*' into .home.git/info/exclude, and then of course tracked that in itself by adding '!/.home.git/info/exclude' in there too.
It's been amazingly useful when acquiring a new laptop, or wanting to quickly set up a comfortable environment on remote machines.
Interestingly, I got it from a Stackoverflow answer (https://stackoverflow.com/a/27336975/8002) and not the atlassian tutorial you listed.
As far as fossil is concerned, once you close the repo, it's out of site of mind. I still use git for most everything else. Only my dot files and my personal obsidian vaults are backed up to fossil.
Otherwise, you can always use chisel [0]
[0]: https://chiselapp.com
> I was using git, however I like that fossil allows you to put the repository where ever you want on the file system. I don't have to worry about alias's, using .gitignore, or setting it it up where it doesn't show untracked files.
Git also lets you put the repository wherever you want on the filesystem, via worktrees or via the `--separate` link. And I fail to see the relationship with aliases or ignore files (the last item I guess you're talking about fossil not showing untracked files by default?)
Fossil doesn't show much unless you perform a open or a checkout against the repo file. Generally, it only shows the location of the config database ($HOME or $XDG) exmaples below. The added benefit is that you can do this open/checkout anywhere on the system. For instance, if I'm making changes in my /etc, I can commit a file from there without having to move the file. Another nice feature is they have an unversioned [1] mechanism as well, which allows you to commit files that you want to save but don't care about the history.
I still prefer git, and all my career and personal projects reside there. But every tool has its place, and I think that even though fossil doesn't fit my other SCM needs. What I'm using it for is perfect for fossils design.Used several times to back up some critical configurations (3rd/4th paranoia backup right before maintenance) on a few servers because the commit/open from anywhere. It doesn't pollute the file system as it's a single file.
With the untracked files: It's a little different. The git way you usually use do the dot file backup is with an alias `--untracked-files=no` so you don't see them unless you override it. At least most of the examples I followed the processes I've experimented with over the years recommended.
With fossil you see them when you consciously issue a fossil open in order to do a commit. I'd rather see the untracked files, when I'm ready to commit then to not see them at all and miss something. Out of site. out of mind is something I personally struggle with.
[1] https://fossil-scm.org/home/help/uv
* Edited for formatting.
It's handy but sometimes a bit of manual preparation must be done on a new clean machine. For example if you want to stow only some files under ~/.config/program/ and not the whole program/ directory, you must first run an mkdir and then run stow to put the appropriate symlinks in place.
I don't blame you for not knowing about this flag. Here's its description in the manpage:
> Disable folding of newly stowed directories when stowing, and refolding of newly foldable directories when unstowing.
This is confusing because it's a circular definition. It doesn't explain what "folding" means. It should just read, "prevents creation of symlinked directories" or something.
For example, on a machine where ~/.config/autostart/ doesn't still exist, I do not want that either ~/.config or ~/.config/autostart are created as symlinks to my stowed package; otherwise any newly installed file, from unrelated apps, would end up in my package! I just want this path to exist as a proper dir, and only contain whatever symlinks are needed from stow. Thus an `mkdir -p` is still needed before running stow.
On the other hand, I do want that ~/.config/wireshark/ is a symlink to the stow repository. So no mkdir in this case.
Regardless, having a script that adds extra per-package pre-process/post-process logic before/after running stow is still needed, anyway.
obligatory vanity link: http://noriceno.life
Infrequently, even on binary files (smirk).
There, I have said it, and taken the load off my conscience...
Using GNU Stow to manage your dotfiles (2012) - https://news.ycombinator.com/item?id=25549462 - Dec 2020 (113 comments)
Using GNU Stow to manage dotfiles (2013) - https://news.ycombinator.com/item?id=15196141 - Sept 2017 (24 comments)
Using GNU Stow to manage your dotfiles - https://news.ycombinator.com/item?id=8487840 - Oct 2014 (68 comments)
Using GNU Stow to manage your dotfiles - https://news.ycombinator.com/item?id=6331485 - Sept 2013 (69 comments)
Is there a reason to avoid symlinks? I've been using this setup for a few years now, (using an artisanal, home-rolled script before I discovered stow) and it's worked superbly.
[0]: https://monkeyfacts.io/git-whitelist-directory-subfiles/
And I agree, symlinks seem like a smell.
I keep my dotfiles in a private git repo, and in that repo is a shell script that creates symlinks (or creates folders with symlinks inside).
Paths are hard-coded in the shell script, but that actually reduces my cognitive load.
https://gist.github.com/clktmr/7343895cc0dcc5a80a6c3d39d22ca...
This will also backup and restore any files it might override.
I think i say this everytime a dotfiles manager comes up.
[1] https://wiki.archlinux.org/title/XDG_Base_Directory
The last time I dug into this, `homeshick` was had more features and fit my needs better than `stow`.
Alternatively, check out YADM[3], "Yet Another Dotfile Manager", which I'm probably switching to once I get some time.
[1] https://github.com/technicalpickles/homesick
[2] https://github.com/andsens/homeshick
[3] https://yadm.io/
a magic word used in the Infocom/Sorcerer games to record an incantation in your spell book.
https://en.wikipedia.org/wiki/Sorcerer_(video_game)
However, typically they don't mention that more recent versions of Stow actually have specific functionality for dotfiles, which is support for the `dot-` prefix.
Check the man page for your version of stow. If the string "dot-" doesn't appear anywhere, you have an old version of stow.
Also interesting to me was the fact that the `make` manual has information on how `stow` is actually one of the recommended workflows in makefiles (in the context of setting a DESTDIR); but I've never seen anyone actually do that in practice.