Syncthing is really nice. It does a thing, and does it well. Once it's set up on the devices that share some stuff it just works.
Another combination that works really well for me is the pass Password store¹ and Syncthing to share some passwords with another user.
I have pass set up to have a subdirectory shared with my partner (by registering both our GPG-keys in a .gpg-id file in that directory), and that subdirectory is synched by Syncthing to our devices.
So now we can store shared accounts there, and have them available in our respective password managers.
No, you can use as many keys as you like. By default it uses the local user's key, but you can specify a list of keys in any directory part of pass' store by adding a .gpg-id text file with one key fingerprint per line.
In particular I use this so that I can have one gpg key for my iphone (which is secured with a long passcode + face lock) and one for my laptop (where I use a gpg key that's baked into a yubikey). I keep it in sync with a git repo (I started with a free tier CodeCommit back before github allowed unlimited free private repos).
I agree about Syncthing, I've been using for various folders between various computers and it always works really well. I even have it sync my programs directory. It's fast, very light on resources and syncs very well.
The amount of times I have managed to slightly misconfigure Syncthing and as a result it _deletes all my files_ is incredible.
Last time, I configured a Receive only folder in device Y, but forgot to mark it as "shared to device Y" in device X's settings.
Net result: directory erased in device Y.
Sure, I still had the copy in device X, but that was gigabytes that needed copying around, a couple hours of wasted time. It's just mind boggling that Syncthing would happily go around deleting thousands of files without warning specially with a minor config error .
This is why I use Git for this (running MGit on my Android phone). It's really not a huge hassle to do this manually once a day or so (maybe more if I'm doing some out of band stuff).
When doing this, I was always wondering how to name my commits, when I was in a good mood I wrote good and meaningful commit messages, but when it was not the case, I just slap "aaa". In any case, nothing prevents you to use both, syncthing for a more reactive sync and git as a kind of backup.
For "running" files I've rarely found commit messages to be all that useful and instead just like to diff head versus last week or something.
I have a git alias setup called "git autopush" that adds all, commits with a generic "[autopush]" message, and pushes to the upstream remote. I use this for my repos of "running" files like org files etc. I've modified it recently to require a config param to explicitly enable it because I'm afraid of typing it in work repos :)
Ah yes, the typical problem of Open Source applications, especially ones without a company behind it.
The implementation of the actual sync is pretty good (I never had a problem with it).
But, the UI is not intuitive at all. Its hard to setup a new sync, and if you're not careful you can set it up in a way that deletes all your files without any warning.
If anything, it is the UI of Synctying that allows me (sometimes) to try and protect some directories.
When the same thing happens with Dropbox, all my computers automatically lose all files because there is no way to force a one directional sync to one specific directory.
> The amount of times I have managed to slightly misconfigure Syncthing and as a result it _deletes all my files_ is incredible.
That only happened to me once (when I was initially setting up Syncthing), but even once is too often for something like this.
The incredible part for me is that there was no clear indication in the UI that I was doing something destructive, I only realized that Syncthing was destroying my files because I was manually keeping an eye on it through my file browser. Surely deleting local files should require the user to first confirm using large warnings and red buttons?!
This did happen for me one time in the past, which turned me off Syncthing then. Since that time though, I've tried again and never had this happen. I assumed it was an edge case of some sort that they fixed, but you're saying you've had this experience recently? (As in the last year or two?)
I double that. Some years ago I almost lost the entire synced state after adding a new device that hadn't synced anything yet. For some reason syncthing decided that the new device has the most recent version of the data and happily started deleting everything on the other peers. Thankfully I had a backup, and not all peers were connected. After that I concluded that syncthing is fundamentally broken.
I sync my org files using a low-tech solution: I have a .dir-locals file that injects a post-save hook that automatically commits changes to git and pushes them to the remote. It's stupid, but it works, and I never have to think about it.
I've been using Emacs org-mode + Syncthing + Orgzly (org-mode on Android) for 2 years now, and it's my favorite notes, todo-list and reminders setup so far.
On top of that, my org folder is under version control, very useful if you somehow manage to lose all your devices (very unlikely but... the Tao of backup you know http://www.taobackup.com/)
Unfortunately, in my experience using Syncthing while doing cross-platform development (coding on a Mac, compiling and testing on a Windows machine, both machines connected to the same Ethernet switch and sitting right next to each other), syncing with Syncthing is far from instant.
If you just want a cheap Markdown editor with instant syncing, I recommend self-hosted CodiMD. Of course that’s not an option if you actually want the full power of org-mode.
Not sure if it has anything to do with file count. I sync a couple thousand files in a directory with close to a million files (aggressively ignored most), although I was talking about changing one file at a time and having the change show up on the other machine maybe 15 to 20 seconds later — or longer, can’t recall exactly. Scanning is very slow too (this is on a fast SSD).
Hmm, that's definitely not typical for me, it only takes a second or two for files to show up. It should be using inotify to be notified of changes, but I don't know if there's a difference on OS X (I'm on Linux).
My experience has been on macOS and Windows, slow both ways. I haven’t used it on Linux.
On macOS FSEvents or kqueue can be used, I’m not sure which one is used. They are definitely using a notification mechanism though, otherwise changes won’t even happen in 15s.
Have you checked "Monitor filesystem for changes" in the config? It polls if you don't do that, and I think it doesn't get checked by default, but I'm not sure.
> There are two methods how Syncthing detects changes: By regular full scans and by notifications received from the filesystem (“watcher”). By default the watcher is enabled and full scans are done once per hour.
I personally would only use the main one unless I had a specific reason to need the fork. I've never had any problems with Syncthing from F-Droid (and syncthing lite for the wimpy cheap android tablets)
The fork works way better as an app. The original just wraps the whole web interface as a single screen inside the app, so you have to trod through that web interface to do any setup.
On top of that, afaik the original is rather sparsely developed and released.
It is indeed, but within the context of this overall discussion, note that BeOrg doesn’t support SyncThing.
There isn’t an alternative for that particular user case without dual-syncing with another sync provider. Best I can tell, there are zero iOS apps of any kind with SyncThing support.
You can (and I have) configure SyncThing to track and sync changes within the iCloud app directory corresponding to BeOrg. It definitely feels janky and I worry about file corruption.
I am a total emacs noob given my relatively shallow exploitation of it, and my absolute ignorance of lisp. This is absolutely the first time I've provided an emacs answer someone found useful in a technical forum.
I still had many problems with writes not making it out of syncthing before the laptop/phone would go to sleep - thus creating lots of conflicts to resolve all the time.
It was a frustrating experience, because basically ~once a day I'd have to manually fix diffs on each side.
I've been using the same combo for a few years. You can run a Syncthing image off Docker Hub directly e.g. using compute engine and optionally persist config and data in volumes, making for a pretty easy setup.
I love org mode (with evil for vim goodness), but the one problem I really wish was easier is image support. Also sometimes a few missed key strokes can really screw up a document! My normal mistake is hitting "J" instead of "j" (vim command for combining the next two lines vs just navigating down one line).
I am always interested in seeing what else is out there for organizing, glad to see org mode is still popular!
I've tried Org+Evil a couple of times and really liked it, but missing key strokes always make me go back to Vim+Markdown. If I mistype something in Vim it's trivial to undo/redo to get where I need to be, but I never managed to get my head around the Emacs undo behaviour and that just scared me away.
Thanks for the explanation, that makes sense. Good to know that states A and B are still accessible, I remember thinking they'd disappeared forever. Next time I try emacs (no doubt there'll be a next time!) I'll bear this in mind.
The tree in vim is weird and hidden. Basically there is an operation to go from a state to the state from which it was derived (“up the tree”) and an operation to go forward/backward in time (of state creation), which is basically navigation in changenum space. Ie if you make some changes:
A -> B -> C -> D
Undo:
A -> B
Make some other changes:
A -> B -> E
Then undo takes you up the ABE chain but if you use g- and g+, you navigate through the list ABCDE
Those are the operations, yeah! They went a long way for me. I think there's also a command or a third-party addon to see the undo states, in the undo-tree spirit—but surprisingly I've never needed it.
>I love org mode (with evil for vim goodness), but the one problem I really wish was easier is image support.
Yes, inline images are an important part of rich documents. I'm not sure what you're looking for in terms of images, but I've written this function which allows you to paste images directly from the clipboard into documents:
In case anyone is interested in learning more about Org-mode, there are a lot of wonderful tutorials out there. I've found the trick is to start small, usually just with some tasks, and let it grow from there as your needs change.
I've been a happy Dropbox + org-mode user across multiple devices for some time. I tend to divide up my org files across different subjects and then tie them into org-agenda. I also archive my org items on a regular basis to clean up certain files.
What's the benefit of Syncthing? I know nothing about it.
I took the time to configure Dropbox from the command line on my headless Linode, and while it's not foolproof -- which is at this point probably on me, since I don't have it running as a proper init or whatever -- it's so far Good Enough, and as such I'm not super convinced going to a secondary sync tool is useful to me.
(I mean, since I already rely on Dropbox to manage sync of a broader set of files between 3 other computers + my iOS devices.)
I use it for everything that I don't need to share with other people, because it scales 'for free' with the size of my data (my music library, for instance), and because I can use it across many different devices but selectively sync different 'types' of things to different devices (e.g., a large photo collection doesn't need to be on my mobile phone).
Similar to my note setup with one change -- Vimwiki + Syncthing. I'm also using(abusing) git to daily auto-commit everything, so I could delete everything without worry, and browse history with tooling I'm already familiar with.
This setup works quite well across all the devices I use (desktop, laptop, phone). With a caveat that phone setup is mostly read-only (apart from ticking the grocery list on occasion) due to it being just a folder with text files there.
I'm actually curious to hear from other vim to emacs org-mode converts since the article didn't elaborate on the cons and pros of the switch.
How significantly is org-mode better than vimwiki and is it worth exploring for someone who's been baking in vim ecosystem for a decade?
Thanks, I just watched the video, and it did intrigue me enough to install spacemacs to start playing around with it.
Having "real" spreadsheet like tables within my wiki does look appealing. I do in fact already have a bunch of small spreadsheets doing some very basic stuff, and it does make more sense for them to be within the wiki itself.
I'm a refugee from the Great Google Tasks switch to a tiny UX and switching off their Canvas (full screen) mode. Had to migrate thousands of tasks. I used https://tasks-backup.appspot.com/ but that has shut down. Don't know of an easy alternative.
I now use a combination of Orgzly, Dropbox, Tasker and Emacs to keep my desktop and mobile notes in sync and it works great.
The benefits are: plaintext/markdown (org-mode), automatic backup with restore (dropbox), open source or free apart from I think a couple of pounds for Tasker and of course whatever you are paying for dropbox. Plus, unlike Syncthing, only 1 version of a to-do list file at any one time, so less conflicts. Wider benefit of to-do list customisation limited only by your imagination and patience, via emacs lisp.
My various To-do lists (split according to the Eisenhower Matrix) are saved in a dropbox folder.
My Orgzly app is set to read from that dropbox folder: Settings > Sync > Repositories > Connect to dropbox
My Tasker is set up as Tasker > Tasks > New '+' > Enter Name > New '+' > Code > Run Shell
Copy and paste into Command field:
am start-foreground-service --user 0 -a
com.orgzly.intent.action.SYNC_START
com.orgzly/com.orgzly.android.sync.SyncService
You can also set a helpful notification via Alert > Notify > Title
Then in Profiles I set the task to run
- Every hour
- Whenever Orgzly is opened or closed
- Whenever the phone display is switched on or off
Takes about 2 seconds to sync, whenever triggered.
One problem with my device (Huawei) is the very aggressive battery optimisation which simply can't be turned off via their settings. You can fix this by going in via ADB as follows:
... It's crazy that in 2020 I have to set all this up for a smooth experience. What I really, really want is an org-mode compatible (I'd live with regular markdown) web app with offline access and auto-sync, accessible from my mobile device and desktop browser and customisable or just any sane keyboard shortcuts. If free or open-source in Vue.js or React + Firebase even better.
I am syncing between Orgzly and Emacs by using an automated git script via systemd in laptop and crond inside Termux in phone, along with a git forge for upstream. This is the only setup that worked for me (I don't have a personal server to keep syncing with syncthing and Orgzly WebDav experience was frustrating. )
Thanks for posting about this. Sharing urls/files/text/etc to termux is something I never thought about doing. I usually end up sharing urls to the gmail app and just sending it to my todo@ e-mail to go through later. I like your idea of sharing to a script in termux better though.
Is your script available somewhere? Is it just a simple "echo org-stuff >> capture.org"?
I’m syncing my org files with Nextcloud, with an S3 bucket configured as a backend [1].
I can even sync to my org files and agenda to my phone configuring beorg to sync to Nextcloud via WebDAV. I’m still pretty new to emacs and org but this is very close to what I would consider an ideal note / task management solution. It’s all plain text (even in S3) and S3 is configured with server side encryption and versioning enabled.
I highly recommend Doom Emacs [2] and Zaiste Programming’s DoomCasts playlist on YouTube [3] to those just getting started.
77 comments
[ 2.5 ms ] story [ 105 ms ] threadAnother combination that works really well for me is the pass Password store¹ and Syncthing to share some passwords with another user.
I have pass set up to have a subdirectory shared with my partner (by registering both our GPG-keys in a .gpg-id file in that directory), and that subdirectory is synched by Syncthing to our devices.
So now we can store shared accounts there, and have them available in our respective password managers.
1: https://www.passwordstore.org/
> Multiple GPG keys can be specified...
I can't recommend it enough.
Last time, I configured a Receive only folder in device Y, but forgot to mark it as "shared to device Y" in device X's settings.
Net result: directory erased in device Y.
Sure, I still had the copy in device X, but that was gigabytes that needed copying around, a couple hours of wasted time. It's just mind boggling that Syncthing would happily go around deleting thousands of files without warning specially with a minor config error .
I have a git alias setup called "git autopush" that adds all, commits with a generic "[autopush]" message, and pushes to the upstream remote. I use this for my repos of "running" files like org files etc. I've modified it recently to require a config param to explicitly enable it because I'm afraid of typing it in work repos :)
The implementation of the actual sync is pretty good (I never had a problem with it).
But, the UI is not intuitive at all. Its hard to setup a new sync, and if you're not careful you can set it up in a way that deletes all your files without any warning.
Software UI is hard, that's it.
If anything, it is the UI of Synctying that allows me (sometimes) to try and protect some directories.
When the same thing happens with Dropbox, all my computers automatically lose all files because there is no way to force a one directional sync to one specific directory.
That only happened to me once (when I was initially setting up Syncthing), but even once is too often for something like this.
The incredible part for me is that there was no clear indication in the UI that I was doing something destructive, I only realized that Syncthing was destroying my files because I was manually keeping an eye on it through my file browser. Surely deleting local files should require the user to first confirm using large warnings and red buttons?!
I do take regular backups to Blu Ray optical disks 'just in case' but have yet to have need of them.
I sync my org files using a low-tech solution: I have a .dir-locals file that injects a post-save hook that automatically commits changes to git and pushes them to the remote. It's stupid, but it works, and I never have to think about it.
If you just want a cheap Markdown editor with instant syncing, I recommend self-hosted CodiMD. Of course that’s not an option if you actually want the full power of org-mode.
On macOS FSEvents or kqueue can be used, I’m not sure which one is used. They are definitely using a notification mechanism though, otherwise changes won’t even happen in 15s.
According to https://docs.syncthing.net/users/syncing.html
> There are two methods how Syncthing detects changes: By regular full scans and by notifications received from the filesystem (“watcher”). By default the watcher is enabled and full scans are done once per hour.
----------
[1] https://play.google.com/store/apps/details?id=com.nutomic.sy...
[2] https://play.google.com/store/apps/details?id=com.github.cat...
EDIT: Fork looks really good too, I've installed it to check.
On top of that, afaik the original is rather sparsely developed and released.
https://beorgapp.com/
https://beorgapp.com/
There isn’t an alternative for that particular user case without dual-syncing with another sync provider. Best I can tell, there are zero iOS apps of any kind with SyncThing support.
You can (and I have) configure SyncThing to track and sync changes within the iCloud app directory corresponding to BeOrg. It definitely feels janky and I worry about file corruption.
If you add Orgzly to the mix for mobile access, you also want to configure Orgzly to auto-save whenever you open it or make changes.
With these two adjustments this becomes a pretty good setup.
(global-auto-revert-mode 1)
(and thanks to the GP as well, of course!)
I'm gonna need a minute.
(heh)
(setq auto-revert-interval 1)
Found in https://www.gnu.org/software/emacs/manual/html_node/emacs/Re...
It was a frustrating experience, because basically ~once a day I'd have to manually fix diffs on each side.
I am always interested in seeing what else is out there for organizing, glad to see org mode is still popular!
1. You have a linked list of states with your current state:
2. Pressing undo moves you to earlier states. Once: Twice: Doing anything that isn’t undoing again will add your undoings to the history: So now when you undo you in some sense redo what you had preciously undone, then undo it again, then undo further.The advantage is that you never lose history by undoing. The disadvantage is that it can be confusing.
Undo in region complicates this.
Undo-tree's navigation interface isn't a pinnacle of intuitiveness, however.
Those are the operations, yeah! They went a long way for me. I think there's also a command or a third-party addon to see the undo states, in the undo-tree spirit—but surprisingly I've never needed it.
Yes, inline images are an important part of rich documents. I'm not sure what you're looking for in terms of images, but I've written this function which allows you to paste images directly from the clipboard into documents:
https://paste.debian.net/plain/1144961
Some great guides:
https://orgmode.org/worg/org-tutorials/org4beginners.html
https://orgmode.org/orgguide.pdf
What's the benefit of Syncthing? I know nothing about it.
I took the time to configure Dropbox from the command line on my headless Linode, and while it's not foolproof -- which is at this point probably on me, since I don't have it running as a proper init or whatever -- it's so far Good Enough, and as such I'm not super convinced going to a secondary sync tool is useful to me.
(I mean, since I already rely on Dropbox to manage sync of a broader set of files between 3 other computers + my iOS devices.)
Not needing a service/server is probably the main benefit, compared to stuff like Dropbox or Nextcloud.
I'm actually curious to hear from other vim to emacs org-mode converts since the article didn't elaborate on the cons and pros of the switch. How significantly is org-mode better than vimwiki and is it worth exploring for someone who's been baking in vim ecosystem for a decade?
https://www.youtube.com/watch?v=JWD1Fpdd4Pc
Having "real" spreadsheet like tables within my wiki does look appealing. I do in fact already have a bunch of small spreadsheets doing some very basic stuff, and it does make more sense for them to be within the wiki itself.
I now use a combination of Orgzly, Dropbox, Tasker and Emacs to keep my desktop and mobile notes in sync and it works great.
The benefits are: plaintext/markdown (org-mode), automatic backup with restore (dropbox), open source or free apart from I think a couple of pounds for Tasker and of course whatever you are paying for dropbox. Plus, unlike Syncthing, only 1 version of a to-do list file at any one time, so less conflicts. Wider benefit of to-do list customisation limited only by your imagination and patience, via emacs lisp.
My various To-do lists (split according to the Eisenhower Matrix) are saved in a dropbox folder.
My Orgzly app is set to read from that dropbox folder: Settings > Sync > Repositories > Connect to dropbox
My Tasker is set up as Tasker > Tasks > New '+' > Enter Name > New '+' > Code > Run Shell
Copy and paste into Command field:
am start-foreground-service --user 0 -a
com.orgzly.intent.action.SYNC_START
com.orgzly/com.orgzly.android.sync.SyncService
You can also set a helpful notification via Alert > Notify > Title
Then in Profiles I set the task to run
- Every hour
- Whenever Orgzly is opened or closed
- Whenever the phone display is switched on or off
Takes about 2 seconds to sync, whenever triggered.
One problem with my device (Huawei) is the very aggressive battery optimisation which simply can't be turned off via their settings. You can fix this by going in via ADB as follows:
adb shell pm uninstall --user 0 com.huawei.powergenie (thanks to https://dontkillmyapp.com/huawei)
You may need to use ./adb (thanks to https://stackoverflow.com/questions/7609270/not-able-to-acce...)
and Settings > System > Dev options > dev options switch on (thanks https://www.xda-developers.com/install-adb-windows-macos-lin... )
... It's crazy that in 2020 I have to set all this up for a smooth experience. What I really, really want is an org-mode compatible (I'd live with regular markdown) web app with offline access and auto-sync, accessible from my mobile device and desktop browser and customisable or just any sane keyboard shortcuts. If free or open-source in Vue.js or React + Firebase even better.
Is your script available somewhere? Is it just a simple "echo org-stuff >> capture.org"?
I highly recommend Doom Emacs [2] and Zaiste Programming’s DoomCasts playlist on YouTube [3] to those just getting started.
[1] https://docs.nextcloud.com/server/15/admin_manual/configurat...
[2] https://github.com/hlissner/doom-emacs
[3] https://www.youtube.com/playlist?list=PLhXZp00uXBk4np17N39Wv...