569 comments

[ 2.5 ms ] story [ 314 ms ] thread
This article is a background of why I built my own PKMS. I've also written another article with a step-by-step on how I built it here https://amberwilliams.io/blogs/the-last-note-system
Can I ask how you created the image at the top of the article? I really like it.
I noticed it too (not necessarily in a bad way) - I'd put it at 99% probability it was generated using OpenAI's gpt-image-1 model.
Good article but as a heavy user of Obsidian (and previously Evernote), I would offer some counterpoints:

> After some mental gymnastics weighing if I should continue with Obsidian, I found solace when asking myself "Can I see myself using this in 20 years?". I couldn't. The thought of cyclically migrating notes from one PKMS to another every 5 years, as I had done from Evernote to Notion to Obsidian, made me feel tired.

In point of fact this is actually an argument IN FAVOR of Obsidian. While the editor might be proprietary - the notes themselves are just standard markdown. If somehow all the copies of Obsidian magically disappeared off the earth tomorrow, I could easily switch over to Emacs org mode, VS Code, or literally anything else.

> Obsidian was a great tool for me personally for a long time. But I felt frustrated when I wanted to access my notes on my phone while on-the-go and saw that I had to pay for this feature.

Again, a little bit odd considering that the author is technically savvy enough to write an entire PKMS but didn't seem to consider that you can just check your markdown notes into a git repository and sync with the native android/iOS Obsidian app on a mobile device. All my notes sync up to Gitea hosted on my VPS and it works relatively seamlessly.

I'm glad the author had fun. Personally, I'm very happy with Obsidian and the plugin architecture has made it easy for me to extend it where necessary.

> The thought of cyclically migrating notes from one PKMS to another every 5 years, as I had done from Evernote to Notion to Obsidian, made me feel tired.

I had a very similar thought process about 15 years ago, and went on a quest to write my own notes system - after trying out a lot of ideas and giving up, I washed up in emacs and gave org-mode a try. It's actually good enough, and I can grep through my notes easiy, and sync them with git.

Thanks for the feedback! Agreed Git can be used to sync your notes. Its a great solution for those comfortable putting their notes into a Git repo like Github. I wasn't comfortable with that however.

Currently vetting a way to sync my database files with my markdown files on my laptop, so it functions similar to Obsidian. I enjoy Vim too much to work constrained to Directus' markdown editor!

What about git makes you uncomfortable?

I saw that you didn’t want to use a 3rd party provider, but why not stick a git repo on your VPS (which you are trusting with your data today) and use that to coordinate syncs between your client devices?

Made a comment in the thread explaining this https://news.ycombinator.com/item?id=44023090

I expect my PKMS to evolve and wouldn't rule out a self-hosted Git server if I find it's a better option long term.

> wouldn't rule out a self-hosted Git server

I don't think you really get it. Git is distributed. There's no need for "a git server". You already have a machine on which you host the SQL database, you can just use that as yet another git remote.

Thanks for the reply. I do agree with sibling comment from tasuki that I think you’re missing the simpler solution of plain git repos to solve “owning your own data in a future-proof manner”.

If you’re not trying to coordinate work among multiple people, and aren’t trying to enforce a single source of truth with code, you don’t _need_ “git server” software. You just need a git repository (folder & file structure) in a location that you consider to be your source of truth.

I’m not trying to convince you to change it now, especially if you’re happy with what you have, but I would suggest reading some (or all) of https://git-scm.com/book/en/v2

I think the first ~4 subsections of chapter 4 cover what I & tasuki were suggesting could be sufficient for you. If you’re the type of engineer to read through the code of your data storage layer, I think you’d find Chapter 10 (Git Internals) interesting, because it can demystify git. I enjoyed the whole book.

As with any engineering project, I see lots of questions about your choices, and I applaud you for sticking around. I would make very different decisions than you, based on your stated priorities, but that’s okay.

You only really need SSH access on a box to use it as a git remote - no server needed.

I learnt this quite late and was not obvious to me so hope it's helpful for you too.

Git is decentralised. You can sync between laptop and phone directly, no third party server required.
To be clear, GitHub is centralized, but Git is not. You can sync between laptop and phone directly with Git -- no third party server required.
How do you use git on a phone? I haven't found an easy way to pull to my phone and open Obsidian files.
The odd part here is why take it to 100%+ when you can just build a plugin on Obsidian rather than re-building the whole thing? Seems a bit extreme.
In 20 years will that plugin work? I doubt it.
This is why I didn't like Obsidian, half the plugins I tried didn't work despite them being in the top 20 downloaded ones. Meanwhile I'll use like 15 year old emacs plugins that haven't been updated in like 5 years and they'll work fine (I think org-diary or something along those lines was what I tried).
You can’t even compile stuff from 20 years ago without some extensive archeological efforts. I doubt this is your largest problem by then.
???

I have dozens of projects from 20 years ago that I can compile today.

With respect, I doubt it. Have you tried pulling out a 20 year old tarball and compiling it, without modification, on a modern distro?
I recently unearthed something that I thought was 20 years old when someone asked me about it. I checked and it was only 14 years old based on mtime (thought I suspect I started the project nearly 20 years ago). Another project I unearthed for a different reason was only 13 years old by mtime (again, it was started before that). I must concede that I haven't actually recently compiled and used anything that was untouched for 20 years.

I should note that the first program I wrote that was actually used for a purpose (it calculates energy based on an internal stopwatch and then typing in values from a volt and ammeter, for a science project in 1992) still works in qb64 today.

The second program I wrote that was actually used for a purpose assumes a parallel-port printer on DOS that uses a fairly old version of PCL, and was written in 16-bit C, so probably won't work today.

A lot of these things can be made to work. That isn't being contested. But if you take a random piece of code of the internet from 20 years ago, it very likely won't compile out of the box on a modern system.

For example, I just took the oldest version of openssl I could find with a quick search (2015, so only 10 years old), and it fails to compile on my Mac. It detects macOS/darwin, and then proceeds to compile for 32-bit Intel, which obviously doesn't work. OpenSSL has fallbacks for straight C implementation to support platforms that haven't been customized, but their build scripts assume that macOS = Intel.

Ok sure, changing the whole freaking CPU architecture will bork a build script. So to prove a point I just downloaded v2.6.11 of the Linux kernel (released in 2005), unpacked (this time on Ubuntu 24.04 on real Intel), and did a `make menuconfig && make`. Of course I don't expect a 20 year old kernel to run on modern hardware, but could I compile it? No, I could not: modern GCC forces PIC by default, which parts of the Linux kernel do not support. I was able to fix that by editing the makefile to pass `-fno-pic` in CFLAGS. Then I get hit with another error due to "multiple definitions" of functions that are declared slightly differently. Turns out old GCC didn't warn about this, but modern GCC handles these declarations differently. This is after pages upon pages of warnings, btw, with only a few source files compiled so far.

I gave up. This is what is meant by archeology required: for anything nontrivial you often have to build the environment in which the code was originally compiled in order to get it to compile again.

I have lots of stuff that's 20 years old that still builds ... C, Perl, shell scripts ...
A shell script is something very different from a markdown editing app with plugins, file synchronisation, multi-platform support, and many more moving parts. And even a 20 year old shell script is probably going to fare pretty poorly.

Do you remember what the most common processor was in 2005? A Pentium 4, or a Celeron maybe. That was when 64 bit operating systems just became a thing. I’d really like to see you getting a version of, say. OpenSSL from 2005 to compile on modern hardware…

"A shell script is something very different"

So what? Maybe read the thread that you're responding to.

"And even a 20 year old shell script is probably going to fare pretty poorly."

You can't just say "my sweeping assertion is probably right". I have shell scripts that are more than 20 years old that still run. And I have C89 programs that still compile and run.

"I’d really like to see you getting a version of, say. OpenSSL from 2005 to compile on modern hardware…"

This is a ridiculous disingenuous strawman. I can't get my FORTRAN II programs that I wrote in 1965 to run, but that has nothing to do with the original claim that I responded to, which was a sweeping generalization that a SINGLE counterexample refutes.

Over and out.

Directus is not eternal either. They are OSS, but you can't support it yourself forever. For a such a long run this looks like a controversial choice for me.
your ai will straight up write you the plugin if it hadnt already done that seamlessly when you requested it render your file.
Some people just enjoy the process, and you'll always learn something new
It's not just git. You have the plugins available for S3, couchdb, FTP, MongoDB, cloud drives, rsync, syncthing, and probably every other storage/protocol in the world. And they're all available for free in obsidian.
No one said anything about GitHub… git is perfectly fine for this use case and 100% private.
Common ways to sync Obsidian are through cloud tools (Google Drive, OneDrive, etc.), SyncThing Fork or Git.

I'd recommend you to look into SyncThing Fork or a similar tool if you never want your notes to leave your own server.

I wrote about ways to sync Obsidian here: https://bryanhogan.com/blog/how-to-sync-obsidian

If you need multiplayer sync, I've been working on a plugin that makes Obsidian real-time multiplayer (like google docs) called Relay [0]. A lot of our users use it in combination with SyncThing to keep it entirely free for up to 3 users (we also offer a paid plan with attachment storage and more seats).

[0] https://relay.md

> In point of fact this is actually an argument IN FAVOR of Obsidian. While the editor might be proprietary - the notes themselves are just standard markdown. If somehow all the copies of Obsidian magically disappeared off the earth tomorrow, I could easily switch over to Emacs org mode, VS Code, or literally anything else.

100% this. The reason I started using Obsidian in the first place is that it's built on the exact directory structure and file formats that I was already using to manage my writing and notes, and if Obsidian goes away for some reason, that won't change.

Big obsidian fan, but I will say: notes being “just markdown” is not entirely true depending on how you use obsidian. If you are a plug-in heavy user, and those plugins introduce new syntax and lots of JavaScript functionality, you are accumulating a bespoke custom syntax that only works on your copy of obsidian with your set of plugins. Obsidian and those plugins are still free and are a huge benefit, but just something to keep in mind regarding data hygiene and longevity.
True, but the format is still text. In a "catastrophe", you can always just a) ignore these, or b) write custom code to process them (e.g. port the plugin to VSCode or whatever).

Still far better than a proprietary format.

A proprietary format with an export function allows you the same inconvenience of having to write code for processing.
No. You might not be able to load the program to get to the export button. They might paywall it away. Etc.
No, there is no paywall or etc. This is not an imaginary anything goes area, but simple note taking where you have local client with locally synced data which can always export, so this risk doesn't exist
Not true for a variety of reasons:

1. You're relying on the external service to continue providing the export functionality, or else doing regular backups.

2. The format of the exports might be proprietary, so it might be orders of magnitude more difficult to parse.

3. The export might not contain all the data.

4. Even if the export isn't to a proprietary format, it might be to a format that's much harder to parse than Markdown. Markdown is not only a standard, it's fairly readable even without any parsing, as opposed to, say, exporting in HTML. Losing some functionality (often minor, depending on what you use Obsidian for) is better than losing more or all functionality.

1. No, the data is already local, the app is already local, you're not relying on anything.

2. Or it might be orders of magnitude easier to parse vs replicating all the plugins functionality. You're just arbitrarily making the alternative worse

3. That's again something you've made up that's not a generic feature of alternative proprietary format

4. It might also be export to markdown. Again, unless you make up artificial barriers

But you can also do it the other way, for example, anything non-trivial like some large table with in-cell formatting won't be readable in your primitive plain text-based proprietary format, so it will be much worse that the unreadable Excel xml or its binary alternative, but that would still be a much preferable export format since no, you're not going to develop a new spreadsheet parser that some obsidian plugin uses to make sense of it

> it's fairly readable even without any parsing, as opposed to, say, exporting in HTML.

that's true for primitive formatting needs, but in this case there are tools that can convert html to markdown that would easily do that

> 1. No, the data is already local, the app is already local, you're not relying on anything.

That's not necessarily true. Some apps keep only cached copies of the data and the rest on the cloud. Sometimes the local files are in a binary format that is unreadable without the export functionality, and newer releases of certain apps remove the export functionality.

> 2. Or it might be orders of magnitude easier to parse vs replicating all the plugins functionality. You're just arbitrarily making the alternative worse

Obviously this depends on the exact app.

But I don't think you can credibly claim that a textual format like markdown isn't easier to parse than... well, almost any other format.

> 3. That's again something you've made up that's not a generic feature of alternative proprietary format

I didn't make it up. It depends on the alternative app you're talking about. Some export full data including all metadata, some don't include all metadata, etc.

My point is that if all the data is actually just markdown files on your computer, there is no question of whether you have all the data.

> 4. It might also be export to markdown. Again, unless you make up artificial barriers

Once again, depends on the specific app. I was a long-term user of Evernote, and still have a subscription. I just checked, and it looks like you can export to a format called "enex", or to a single html page, or to pdf. That's awesome! And the chance that you won't be able to use this in another app is next to nothing, since everyone works to be able to import Evernote.

It's still a tradeoff between the extra functionality you get from Evernote, vs. the simplicity of the "export" files you have. In Obsidian, there's no separate export, the files are stored in simple-to-read Markdown. But you get less functionality.

It's a tradeoff. I'm not saying one is better than the other. But pretending there isn't a tradeoff is quite simply wrong.

> But you can also do it the other way, for example, anything non-trivial like some large table with in-cell formatting won't be readable in your primitive plain text-based proprietary format, so it will be much worse that the unreadable Excel xml or its binary alternative, but that would still be a much preferable export format since no, you're not going to develop a new spreadsheet parser that some obsidian plugin uses to make sense of it

Yes. I wouldn't use Obsidian to do anything that would require a spreadsheet. I'd simply use Excel, since it's a billion times better at it.

I'm certainly not against using the right tool for the job, nor am I against proprietary formats in general.

1. It true since the argument about formats. You can limit storage of open format to the cloud as well.

> and newer releases of certain apps remove the export functionality.

Then you'd just use the old release with the export functionality intact. You can also make up stuff like "Obsidian can release an app that deletes/encrypts all local files, retaining only the cloud copy, and start charging for it without having any export functionality"

> But I don't think you can credibly claim that a textual format like markdown isn't easier to parse than... well, almost any other format.

This isn't markdown, but markdown + dozens of extensions, so it's very easy to claim that it's much harder to write custom parsers for dozens of formats rather than use an existing parser for some more elaborate format that doesn't need those extensions.

> the files are stored in simple-to-read Markdown

they aren't, they're stored in an undefined format depending on which extensions you use. Part of it is markdown (which is not simple to read in the non-primitive case of richly formatted docs)

> there's no separate export

That's not a benefit! It means that you can't move outside of the Obsidian ecosystem because there is no well-defined format that you could use another app with! So it's (practically) even worse than Evernote since that one is already widely supported, though theoretically it's the same.

> But pretending there isn't a tradeoff is quite simply wrong.

Yet you've failed to identify it, turns out it all "depends on the specific app"! Fine, compare apps, but the general argument was about text-based proprietary format with a chance of data loss if the ecosystem dies (or a chance of requiring a lot of effort to convert), and a generic proprietary format that can be exported into a text-based format... with the same risks!

I don't really know what we're disagreeing on.

> This isn't markdown, but markdown + dozens of extensions,

Yes, if the way you use Obsidian includes dozens of extensions that each use a proprietary format, then it's similar to just using Evernote in many ways.

If you're mostly using plain markdown with only a few custom formats, then it's still easier.

If today, right now, Obsidian stopped working, I could literally open my Obsidian folder in VSCode and still be able to do 90% of the things that I do in Obsidian.

If today, right now, Evernote stopped working, it would take some effort to find a working version, export the files, convert them to markdown or whatever, etc.

I just don't know how you can claim that Obsidian is more effort to use outside of Obsidian than something proprietary.

> If today, right now, Evernote stopped working, it would take some effort to find a working version, export the files, convert them to markdown or whatever, etc.

No, at your accepted level of the loss of functionality that would be trivial.

  - Launch the app you already have, export
  - Launch another app, import. Could be Obsidian. Here is their guide. https://help.obsidian.md/import/evernote
  - Open results in VSCode and ignore the 10% lost in conversion
> I just don't know how you can claim that Obsidian is more effort to use outside of Obsidian than something proprietary.

Because at every step you trivialize one option and complicate the other. While they're generically equivalent. All the same things apply...

> If you're mostly using plain

If you're mostly using plain notes in Evernote, then your conversion to the same plain markdown will be trivial, so using another plain markdown isn't easier

Very much this. I cannot even fully agree with "plug-in heavy" remark: I mean, how heavy must it be, to be considered "plug-in heavy"? I consciously tried to limit plugin usage. But it really gets pretty wild soon. I was relatively lean for maybe the first 6 months, but when some patterns of how I use it become clear enough, it becomes pretty evident how inefficient many super-common situations are and how I can fix them just by installing a plugin.

Fast-forward a year, and all your vault structure implicitly relies on the quirks of Obsidian search behavior, the markdown you write is extremely obsidian-flavored markdown, and you don't even remember how to write LaTeX without LaTeX-Suite shortcuts.

I've been using Obsidian for years now and besides some experiments use zero plugins. What inefficient patterns are you running into?
I've thought about this and I think Templater and Dataview are the two plugins I'd miss if Obsidian was sold to a VC tomorrow and enshittified.

And I'm pretty sure both will be forked and modified to run independently of Obsidian within a week of the theoretical enshittification.

If you're willing to reimplement them in your own obsidian-like editor anyway, I don't quite see the difference

I wouldn't so I keep to markdown and minimize plugins where aplicable, if I need to run for the hills, I don't expect to lose much

> Again, a little bit odd considering that the author is technically savvy enough to write an entire PKMS but didn't seem to consider that you can just check your markdown notes into a git repository and sync with the native android/iOS Obsidian app on a mobile device.

Even simpler, I have mine in a Dropbox folder. Felt very strange for _this_ to be the straw that broke the camel's back for the author.

Nonetheless, very glad for them that they enjoyed and learned from the experience of building a replacement!

Yeah, syncing text files across devices is a problem that has little to do with obsidian or whichever editor/renderer one uses. As long as one keeps things relatively simple with plugin-related syntax flavours, editors are interchangeable.
Even simpler, just pay for the feature
Yeah agreed - does anyone really care for $4/month?
Fun fact: Dropbox doesn’t support emoji in file names ( or at least, didn’t last time I checked. )

Deal breaker for me - adding iconography to file and folder names can be a natural, zesty enterprise.

And here I am still coming to terms with using spaces in filenames.
(comment deleted)
I'm willing to pass on this one, IStillPascalCaseMyFolders (And camelCaseMyFiles.txt)
Still doesn't. I recently moved my vault to Dropbox and had to rename a bunch of filenames
Yeah, I have my obsidian vault in Dropbox and synced to my phone and back with Dropsync on android. The obsidian mobile app Just Works™ with this. It was a one time setup. Of course there's no fancy conflict resolution going on, but it's unlikely I'm editing in two places at once so it's not needed.
> Again, a little bit odd considering that the *author is technically savvy enough to write an entire PKMS*

I’m pretty sure author just wanted to build PKMS. These types of “oooh, will it be there in 20 years” are standard OCD/procrastination.

Again, a little bit odd considering that the author is technically savvy enough to write an entire PKMS but didn't seem to consider that you can just check your markdown notes into a git repository and sync with the native android/iOS Obsidian app on a mobile device.

Also, Obsidian supports free iCloud sync if you are a Mac and iOS user. I know that's only a subset of users, but a nice option to get Obsidian to sync on the phone if you are in the Apple ecosystem.

Also, they have a cheaper Sync plan now that is $4 per month.

I can't really be bothered that Sync would cost up to 1000 in ten years. If you use Obsidian daily, it had an immensive value and it's cheaper than most services out there.

To add to your last point - 10 years is a very long time frame. Any recurring cost grows to eye-watering levels if the time interval is huge enough. In 10 years, a lot can happen in the space and you are not locked in with obsidian - if the next better thing comes along in 3 years, you can easily migrate.
iCloud sync is too unreliable and opaque for me.
Yes I found it wasn't reliable. Perhaps it has improved, but Obsidian Sync has been excellent. I'm happy to support the developer as I'd like Obsidian to stick around.
It has been reliable for me with Obsidian. I still prefer Obsidian Sync because it has good integrated version history and shared vaults.

Also agree with the sibling poster - I like subscribing to support them on a regular basis (and of course Catalyst).

Obsidian Sync is great (it helps that I got in on early-bird pricing...but it really is reliable).

In addition, the Git plugin, coupled with Working Copy on my iPhone and iPad, also work extremely well, with a bit of added overhead. Though iCloud works reasonably well for just about everything else, I haven't used it for syncing Obsidian data.

Syncthing... It can do it too

handles syncing my pictures too

I can second Syncthing. I just use a simple markdown editor with Syncthing on Windows, Android, and a Linux.
Wasn't the syncthing android app deprecated relatively recently?
Syncthing-fork (of the android app, not syncthing itself) is a straight improvement.
Yes, I use this and it is pretty solid. I can recommend it
It was if you install it from google's app store, but it's still fully functional for now if you install it from F-droid.
Even setting up your own couchdb + livesync should be a small task if one can consider to 'write a whole app' to replace obsidian

edit note, if you want to build something :) please expand livesync or a new plugin to allow easily sharing of self hosted obsidian notes :) All the ones I tried use some 3rd party hosting which I don't like even if its encrypted.

Yeah the article was painful to read. $1000 in a decade? Its a decade, who cares. If you make 100k a year its 0.01% of your salary towards something you found worthwhile enough to use for ten years straight

the cheap sync plan is a scam though, it doesn't do enough to be even worth bothering with iirc

To add to your point, the phrasing of "$1K in a decade" seems convoluted to intentionally try to over-emphasize the cost. A much more fair/normal way of phrasing that cost would be $100/year, or $9/month, which is very little for someone who has the time and energy to write their own and gets enough value out of it that they find that worthwhile.

The tradeoff doesn't make sense otherwise. If you're poor, then you wouldn't have the luxury of the available time to write your own PKM - the opportunity cost is huge, it's massively preferential to use an existing free solution that is "good enough" (Obsidian without sync or using your own sync solution, Dendron, Roam, Org). If you're not poor, then $9/month is a rounding error, especially compared to the value provided.

I'm migrating what's in Evernote to Gmail because the "upgrade to premium" popups are too tricky and the value of Evernote just isn't worth the subscription unless you're relying heavily on it, which I can't because of the popups.
(comment deleted)
Concerning, maybe. Definitely not surprising. One’s technical ability to do something has, if anything, a negative correlation with their ability to value and manage their own time. The author’s justification is absolutely ridiculous, hands down. I simply pray that they’re never in charge of deploying another human’s time effectively.
Totally agree, I personally have obsidian set up on multiple devices, and they all automatically sync to my local Synology NAS.
> In point of fact this is actually an argument IN FAVOR of Obsidian. While the editor might be proprietary - the notes themselves are just standard markdown. If somehow all the copies of Obsidian magically disappeared off the earth tomorrow, I could easily switch over to Emacs org mode, VS Code, or literally anything else.

Not really.. This problem runs far deeper than most are willing to see. First, Obsidian is using a personalized flavour of markdown, and seconds, for many heavy features it's leaning strongly on plugins which are prone to break or even die. Obsidian has a vibrant plugin-community, which also seems to die really fast. This becomes even more critical by plugins dying from changes in Obsidian itself. So while Obsidian is in theory a nice open app, it's longevity-aspect is really awful. I already had many features and plugins dying in the last years, and who know how much more will break in the next 20 years. Simply switching to another text-editor will not do, because they won't offer the missing features. So at best, you are just not losing your data, but you still won't have the tooling to use them.

Someone creating their own system, where they have full control over everything, even if they will have to sacrifice some benefit in the short run, just makes sense in a bigger picture.

Don’t use plug ins
then you lose a large amount of functionality and value. That's the point they were making.
You may lost a large amount of functionality and value. If you never become dependent on plugins, as is true of some Obsidian users, you lose nothing. Obsidian has considerable functionality and value right out of the box, at least as much as many other Markdown editors, and costs nothing.
But you do have full control, if you want it. Nothing stops you from altering plugins or making your own.

The plugins sit in a local directory. Very easy to modify.

> Nothing stops you from altering plugins or making your own.

You still have no control over obsidian itself. Any change can and will break plugins. So you either settle with one version for the next decade, or you have to maintain them. This is just the normal dependency-hell that every project has, where you have to compromise with external dependency and their whims. Just that neither plugins nor obsidian (to some degree) are the level of professional software-projects in that regard.

And let's not talk about changing Obsidian on fundamental levels. You have even less control on how it works on everything which is not accessible by plugins.

Genuinely curious, which plugins do you mean for example when you say that many heavy features are leaning on them?

I could see the dataview plugin as an example (even though I don't use that one personally) but most other plugins seem like they just add more convenient ways to do something that would be still pretty simple to do manually. (Templates for example).

> Genuinely curious, which plugins do you mean for example when you say that many heavy features are leaning on them?

Depends on what you are doing. But the whole task & project-management-corner is constantly moving. Everything which modified the editor and preview was also regular breaking in the last years. For example, there were some plugins adding banners at the top of documents, or background-images or some icons. Or plugins modifying the yaml-area. They were all breaking multiple times when Obsidian was switching to the new live-preview-editor, then on changing frontmatter to properties, and on some other occasion IIRC. Usually after some months a new plugin appears, or someone forks the old one and fixes it. But as a user, it's pretty annoying to constantly have something breaking outside your control and getting stripped of features you want/need for various reasons.

Obsidian is useful, but far from being stable long-term yet. It's still very young.

> I could see the dataview plugin as an example

Yes, dataview was also very unstable the first 2 years or so, switching code and concepts, breaking old code along the line. It seems to be stable now, as the focus is on datacore.

> but most other plugins seem like they just add more convenient ways to do something that would be still pretty simple to do manually. (Templates for example).

Does it matter what a plugin is doing? If it breaks, it's a loss, whether it's crucial or just annoying.

Plugins aren't required to use Obsidian. I have just a few, none of which make consequential modifications to my files. I've never had a need to use anything like Dataview, for instance.

To me, the "dialects" of Markdown have never made any difference in using it, even between applications. I think the only non-standard (for whatever "standard" means in Markdown) markup I use in Obsidian is "%%" to indicate comments. That's certainly something easy enough to search for should it be necessary in a text editor.

Just use Joplin, it’s open source and syncing to many cloud providers you already probably have is free.
I think of the sync paid tier as analogous to a patreon membership, combined with paying someone a tiny amount to manage my data for me. The fact that it's all markdown makes me confident I could take my files and go play elsewhere at any time, but I enjoy knowing my money helps keep Obsidian going.
Yeah, I don't even do that. I just copy the files over if/when I want to. (Do people really not know how to use a basic filesystem to copy files these days?)

But mostly I don't. My work notes are on my work laptop and my personal notes are on my PC. I might copy them onto a mobile device if I'm traveling, but I might not bother. Mobile devices don't have the good keyboard and large screen to really be useful for stuff like that. But I have copied them over before just in case I wanted to find something in them.

There is also LogSeq which is open-source and very similar to Obsidian. I use it with Syncthing to sync notes across devices. I would rather see more effort put there, to improve table management, for example...
The main thing being complained about here is that you have to pay for device sync. But instead of setting ups FOSS alternative like with a-shell and git you decided to… checks notes… build a less featureful obsidian without getting all the benefits of the obsidian ecosystem?

I’m all for doing projects like this as an intellectual exercise. It’s just that the motivation behind doing so in the article is a bit more “huh?”

Yep fair point. I'm doing the project in chucks and writing about it. his written part notably unlocks the ability to use my phone. Currently vetting a way to sync my database files with my markdown files on my laptop as I enjoy using Vim.

Funny enough I had downloaded a-shell and experimented with it and going git based. But ultimately didn't want my notes stored through Github. If that way works for you, cool!

I do agree with the author and others that I also wouldn't feel comfortable storing personal notes on Github. As I mentioned in a previous comment - you can use "git" without Github by hosting an instance of the open-source Gitea service.

https://github.com/go-gitea/gitea

Thanks for sharing. I'll have a play with Gitea

My concern with this approach would be I've read through Directus' codebase and can understand it. With a self-hosted Git server like this I'd be worried if shit hit the fan and corrupted my Git files or stopped being maintained I'd be a duck out of water

It's also worth noting that Gitea forked a while back. The community fork is Forgejo.

https://forgejo.org/

And if you really just want a simple hosting system, https://tangled.sh is really easy to set up. It uses atproto (network underlying bluesky) as their identity provider and for tracking issues, PRs, comments, etc. Their "knot server" is basically just a little self-hosted go node that manages git repos. The project is fairly small atm and it's pretty much all in go so it's not too hard to skim through if you want to see how it works under the hood (or if you are afraid of needing to be able to keep it maintained long term).

> if shit hit the fan and corrupted my Git files or stopped being maintained I'd be a duck out of water

You should have the same concern with anything you're hosting yourself, and you should have 3-2-1 backups to mitigate that concern. Gitea just uses regular Git repositories under the hood last i checked, and Git is an extremely mature system; I'd expect 20+ year old repositories to work fine as long as the data are kept physically intact.

You don't need Gitea (nor Forgejo, nor GitLab, ...). You just need `git` installed on whatever server you already have. And just use that as one of the git remotes.
Good call out. Most of the time people actually want one of the "wrapper" softwares because they want the collaboration components and they aren't old greybeards that want to ship around mailserv lists. But that use case doesn't apply here. There is probably no one to collaborate with in this use case, so barebones git is probably totally fine.

Though I'd say, if you're going through all of the effort to self host something, git or otherwise, doing the "wrapper" is probably worthwhile anyway due to the convenience if you ever do need those features. It's not that much harder to do that than host a vanilla git server.

> The thought of cyclically migrating notes from one PKMS to another every 5 years, as I had done from Evernote to Notion to Obsidian, made me feel tired.

git or syncthing

Doesn’t HAVE to be GitHub, git is git is git. You could host your own git server like a self hosted Gitlab if privacy is a concern (which is a totally valid concern! I share the same concern, I don’t necessarily want all of my inner brain workings available to GitHub). You could probably also figure out some clever way to encrypt the files too, I bet there’s a plugin for that. Then you could use anything you want without that worry
You have a good point. I don't have experience hosting Git servers personally. Is it easy to run and maintain? I'll have a try on my VPS if it is.
In a single user scenario where you don't care about a web interface (and its associated additional features) for your repository you can literally use any server that is accessible to you via ssh and has git installed as a git remote for your repository.
Heck, you don't even need to run something like GitLab. Owing to Git's design as a distributed version control system, a "Git server" isn't even really a separate piece of software—it's the same software being used in a different way. Details @ https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-... , but you basically just need to `git init` somewhere on your VPS that you can later ssh to and add as a remote a la `git remote add origin username@yourvpsserver:/srv/git/project.git`
The only limitation here is that (on iOS at least) the git plugin on mobile cannot do ssh, only https.

(This is at least 1+ years old info, might have changed)

If you're not too wedded to git, have a look at fossil, perhaps. Its a dvcs just like git, but is itself distributed as a single static binary. This binary, in addition to allowing you to init & interact with fossil repo's, also comes with an integrated bug tracker, gui (for logs, etc), wiki, even a web forum with threaded discussions (so kind of a replacement for a mailing list).

And a builtin admin interface to govern permissions on the above. Plus, its its own server. All of this ... is builtin to that single binary.

It has a few of its own quirks as a vcs, but its pretty much equivalent to git, both functionally & conceptually (if you know git, you'll know fossil in 5 minutes)

So, if you want a dvcs for syncing, and don't want to setup git for external access, try fossil?

I appreciate that you are taking the time not only to do the work but to document your experience and share it.
Thanks. More devs should write about what they're building. Its the hardest part for me
I really don't want to critisize OP since building stuff for yourself is always a good mentality. But lets be realistic, 1000$ over 10 years is nothing.

It will always cost more if you consider your own time for maintenance long term. Obsidian is one of the most consumer friendly business for note taking out of there, they are not VC so the Evernote comparison is unwarranted IMO.

FWIW, I’d be more concerned about the implications of the company having my notes in lieu of the pure cost perspective. But the thing is, you can avoid that entirely too by implementing your own sync
And implementing local sync for obsidian is just running on docker container
FWIW unless they are outright lying this is a choice, one of the choices when setting up a vault is E2E that you have to enter whenever setting up a new sync, but they are really clear that if you lose this password you are at the whims of your own backups.

They do also publish the “verify the encryption steps” for this.

Of course, depending on your threat model this could be insufficient, but then you probably wouldn’t trust obsidian in the first place.

I think the author point still stands though: Obsidian won’t probably be here in 20 years.
I’m a time traveler from 2046 and I hate to break it to you but it’s still running strong.

Couldn’t avoid the computation panic of 2038 but it got by

Neither solution is guaranteed to stick around for 20 years.

As we’ve seen before, it takes one VC investment to change a source available license into something not so friendly and forks are never guaranteed.

For me that's one of the great points about obsidian's choice of all notes being Markdown.

Even if Obsidian vanished tomorrow and the application became unmaintainable, I'd still have all my notes in a text based format.

I wish all markdown editors just had their markdown files in a simple folder like Obsidian does.

I wanted to like Bear, which advertises that it uses markdown. But when I went looking for the files, they were locked away in a database. This was many years ago, so if this has changed, I’d be happy to hear it.

I’d love to be able to easily jump between apps, which markdown should allow in theory, but in practice view apps allow for. I don’t find using a text editor to be ideal here as a solution, as I want my notes to look like notes and hide away the syntax when the cursor isn’t on the syntax. Obsidian handles this well, most text editors do not.

Why not? It’s got a huge user base, a massive open source plug-in ecosystem and a sensible revenue model. It’s probably one of the note apps that has the largest community around it outside of Notion, which is heavily VC influenced and is more of a do everything app
We'll see. I think there is a chance that it will.
But Op's program have even less probability to be here in 20 years
How?
Because it takes time and energy to develop and maintain a software
Because Obsidian exists, is finished, has a community, and is in use?
Can I have those 1000$ if you think that is nothing?
Sure, but only in installments over the next 10 years and in exchange I need you to provide a sync service for my notes.
> 1000$ over 10 years is nothing

It's a non-trivial amount of money to a lot of people (myself included). I spend way more than that on Free Software, but I'm not throwing money to a proprietary program if I can choose.

> But lets be realistic, 1000$ over 10 years is nothing

Where is the limit ?

While $100/year maybe doesn't sound like much, it's hardly the only subscription service you have, and they all add up, from your mail provider, office suite, cloud storage, streaming services, phone bills, internet service, etc.

Personally I find $100/year to edit notes on my phone to be a bit much, but then again, I just use iOS Notes.

I am so fed up with everything turning into subscriptions, that I've just completely stopped buying things that are subscription based.

I understand developers need to make a living, but simply throwing a subscription on top of it won't convince me to buy your product. You convince me by making a compelling product, and by continuously updating it, adding new features, which will convince me to buy another version.

Personally, I just set myself a budget of $100/month for subscriptions. I was going to drive myself crazy judging every one all the time, so I decided as long as I’m under this threshold I’m not going to stress.

I track the ones I have so I can compare the cost, looking at either daily, monthly, or yearly cost. Sorting by price, I can look them over to judge if one of them seems unusually expensive for what it is, and regularly review to see if there are and I’m not using and need to be cancelled.

My most expensive is the could backup for my NAS. $8/month is about what I pay for Proton, which offers a lot more than just note syncing. So $8 for notes does seem like a lot. Looking at Obsidian’s pricing page[0], the $8/month is for publishing… hosting a website with your Obsidian data. Just syncing is only $4, and there are many free ways to do it. That part of the article felt like the author was trying to justify writing their own tool due to cost. That doesn’t feel justified, and they were stretching… but the good thing is there doesn’t need to be any financial justification at all. Just make your own tool for the sake of making your own tool. That’s good enough.

[0] https://obsidian.md/pricing

I think that no matter that they do you will always find people complaining. That is humanity's favorite sport. Even when it's free and open source people still complain.

Making software for individual consumers is freaking hard. My own perspective as a founder shifted from "this is a viable method to build a sustainable business" to "let's use it as the base for B2B sales, but it is not viable".

Obsidian is not free and not open source
Yeah I just wanted to chime in here on what i see as a world view problem that might need to be updated with the times. I see your point about everything being subscription based.

There are definitely software out there that do not deserve to be subscription based. But there are some developers that I think should be supported on a regular basis. Especially obsidian because they develop one product and continually work on it to make it better. IMHO they deserve the money.

On side note I wish there was a way for it to be open source, and the team's reasons for not supporting open source seems a little iffy. They could still make money of of Obsidain Sync or other features that does not need to be part of a Open Source Release. Commercialism of a project this important worries me because people that depend on it can be easy side-lined if the team decides to sell out. Look what happened to the Atom editor. Microsoft brought it then killed it. I know with Obsidian you can walk away from it and thats good, but I always worry about commercial domination of a market causing limited choice.

I know your point was about subscriptions, and I guess I am saying im more supportive of that model, based on the times we are in, for developers that have genuine passion for a project that they want to continue developing.

Heck I know am realizing I need to support open source projects with regular donations because I want them to thrive long term

> There are definitely software out there that do not deserve to be subscription based

My personal experience with software switching to a subscription model is that often that means you will now only get "bugfixes", and new features are usually few and far apart.

There's no longer any incentive to produce major versions with new breaking features, and instead it just turns into a maintenance product, if you even get that.

An Example could be Sublime Text, that while not traditionally subscription based, the license expires after 3-4 years, and needs to be renewed, so it's just a 3-4 year subscription. It was released in May 2021, and receives 1-2 updates yearly, and every update since 2021 has been "fixes" or "improvements". Nothing new has been added for 4 years.

Another example could be Arq Backup. Version 7 was released in February 2021, and while the changelog does have some "new feature" entries, those are mostly just "added the possibility to backup to X service". It does however see much more frequent updates than Sublime Text.

Don't get me wrong, bugfixes and improvements are great, but I expect new features as well. I expect the product to be moving forward, keeping up with current "best practice" standards, and not just turn into a money pit for the developer, just pushing out the obligatory "yeah, we fixed a few bugs" releases.

There are of course also various projects that do things "well enough" that new major releases are not required, like NextDNS. NextDNS works well, and is priced cheap enough to rival the electricity consumption price of a Raspberry Pi running at home. They don't have big releases, are mostly doing maitenance releases, but for the cost, and function of the service that is OK.

And no, not all software falls into this category, and there are plenty of great software that i pay for, which is actively maintained.

Obsidian falls into the new features category of not just updates, hence why I think they are the exception to the rule about the laziness of developers around the subscription model. But I do understand where you are coming from. I just don't think everything above reflects the entire industry.
even better: Obsidian is only $480 over 10 years!
> Since my PKMS is hosted online to manage notes across devices, I have multiple layers of security to ensure my notes are kept private. {Screenshot of a login form}

The biggest life hack I can recommend for a self hoster is to set up a VPN on your local network and then just never expose your services on the public internet unless you're specifically trying to serve people outside your own household.

Before I did this I was constantly worried about the security implications of each app I thought about installing or creating. Now it's not even worth setting up auth on a lot of simple services I build because if someone is able to hit their endpoints I'm already in deep trouble for many other reasons.

Good point. I also use my PKMS as a CMS for my blog. Might just split out the services and go this route.
The downside is that if you’re on a two-week vacation and your home network/server goes down on day two, there’s probably nothing you can do until you get home. If it’s hosted online, you can count on that 99.99…% uptime and SSH access no matter what.
Ssh exposed on a non-standard port, with root disabled, using key-based auth should be pretty non-controversial.

The security through obscurity (non-standard port, no root) are both kinda silly but why not.

That said, with awesome services like TailScale, it's pretty hard to get locked out of your network. TailScale is so so good at "just working".

Fail2ban or rate-limiting SSH into a block table are useful layers to have as well.
> The security through obscurity (non-standard port, no root) are both kinda silly but why not

I think these are decent controls when layered with others. The effectiveness differs depending on your threat models, of course, but at the very least it helps reduce the noise seen from most automated scans reducing the effort involved in monitoring your assets.

Another option is port knocking. Super easy to set up and with 4 knocks it provides 64 bits of randomness.
Disabling root provides more than security-through-obscurity if your sudo config requires a password to elevate: it essentially means you need both your SSH private key and your password to gain root.
I think what they meant is that if it's hosted online / home-network, only allow access to all services through a VPN. Wireguard is relatively easy to setup, and you can configure all your services to only be available through wireguard.

Ever since ssh almost got backdoor-ed, the only thing "exposed" on my servers is Wireguard, which is UDP based and therefore harder to know if it's running. SSH also goes over wireguard.

> ... Wireguard, which is UDP based and therefore harder to know if it's running.

Isn't it basically impossible to know if it's running unless you have an authorize key? I thought it didn't respond at all unless you ping a valid entry key off it.

Well, there is packet inspection. My ISP / Government will know.
These solutions are composable. Just run it on a VPS over a VPN.
That’s a good point.
But what if your home hypervisor goes down?
Nothing. You probably though I said "VPN over a VPS", which would just be the same as a VPN, which isn't composing the solutions.
You're suggesting to use a VPS in a thread that's specifically about self-hosting on a home network, and about there being cons to that compared to hosting on a cloud/VPS platform.

So you're sort of driving parallel to the existing commentary.

The parent comment to mine did that.
If you trust someone in your neighborhood/family, you can ask them to power cycle your host and water your plants
Although not perfect, I added a couple features to help ensure uptime:

* LAN components are on a UPS, helps keep continuity between power blips and breaker flips

* Dynamic DNS, cron runs a script 4x per day to ensure a DNS name points to my IP, even if issued a new one by the ISP

* Rebooting everything occasionally to ensure the network and services come back up on their own and I didn't make a mistake with some config that loads at boot, etc.

I run Tailscale on an Apple TV which is on a UPS. The thing uses very little power so the UPS lasts a long, long time.
Do you have to hack the Apple TV to run tailscale on it or how does that work?
There's an official tvOS app and you just have to configure the Apple TV to be an exit node. It doesn't require any special setup otherwise.
You can also enjoy your vacation instead :)
Tailscale is actually great for this if you configure an exit node on a device in your home.
It doesn't need to be an exit node unless you want to direct all traffic through that node, like a traditional VPN.

The beauty of Tailscale is that it will only direct traffic to a node that is meant for that node, everything else goes via the internet.

So you've managed to unlearn the last decade of security learnings in regards to zero-trust and similar concepts?
You need to understand your own risk tolerance and, more importantly, effort/resource threshold. Zero-trust is great if you have the resources to put to it, and companies should do it. But individuals trying to manage multiple companies worth of services, alone, on their own network? There's going to be corners cut.
That's a comically uncharitable extrapolation on what was said.
self hosting is entirely different than enterprise security practices. You're a little out of touch with reality if you don't realize this.
I'm not running a business, I'm running a home. The threat models are totally different and I adjust my security posture accordingly.

Besides, I don't bother with auth for simple services, not stuff that actually hosts data. If someone unauthorized is inside my network they're not going to be interested in using my TTS/STT service or in finding out the last barcode I scanned or in using my tiny consumer GPU to generate tokens on an LLM—there are way worse things they could be doing at that point than fiddling with the many tiny services I have set up.

Also: I couldn't set up so many silly, inconsequential services if I didn't have a VPN. With my setup, every new idea I have can be a quick service on my network accessible by me anywhere in the world. If I had to expose each of these things to the internet I wouldn't bother running them at all lest they have an exploit that ends up being an entrypoint into my network.

I'm being serious - please educate us, how do you think that we can do better in a homelab setting? How do you apply zero-trust principles in homelab environment with reasonable effort and without relying on 3rd party services?
+1. I have Wireguard set up on all my mobile devices and configured to automatically start when connecting to any wifi that isn't mine, so I can take my devices anywhere and I'm still on my home LAN. It works seamlessly and flawlessly.

I self-host a lot of services, and without Wireguard (or equivalent), remote access just wouldn't be realistic.

On Android devices? If so: how? :)
I don't use it with wireguard, but Zerotier works just fine on Android.
As does tailscale!
This is my recommendation for anyone that wants full control.

I use Tailscale, purely out of laziness and a willingness to trust them today, but I'd move to head scale if either of those caveats changed.

My only issue with wireguard/tailscale is that somehow my work IT has managed to make it unusable on our wifi. It's really annoying. I'm pretty sure it's a misconfiguration on their end but they don't have any reason to care.

OpenVPN does seem to always work everywhere (presumably because outside contractors and support personnel use OpenVPN when onsite so it's a squeaky wheel that matters) so I've moved to that instead. Beyond that I can't figure out what the hell is the problem and the way IT works, they have no reason to fix it. I did get them to somewhat work on it by reporting Google VPN as randomly failing, but they just fixed Google VPN and nothing more than that. So anyway wireguard is great until you encounter bullshit corporate firewalls.

How did you automate it to start on networks which are not yours?

This is like the only piece of the puzzle for me.

For the Tailscale app on Apple platforms it’s called VPN On Demand. Here's how you can configure it.

https://tailscale.com/kb/1291/ios-vpn-on-demand

Does it still eat your battery? It used to, and it was so annoying I moved to Nebula
Tailscale is built on wireguard but it is not just wireguard. I want a wireguard only solution.
Got it. The key there would be to find a WireGuard app that supports the "VPN on Demand" feature of iOS. It should be called out on their app page in the App Store, if not in their documentation.
Set the AllowedIPs wireguard setting (and/or the route, if you can set that separately) to one larger than your home network (i.e., if your home network is 192.168.1.0/24, use 192.168.0.0/23). Then, block wireguard packets from the internal network on your router. Then the tunnel will always be running; it just won't be used when you're at home because there's a more specific route
But this will also break whenever there's any more specific route, no?
All my regular mobile devices are Apple, so whoever said that has it right. I also have a Linux laptop but NetworkManager support for Wireguard was broken last I checked, and at the time I didn't care enough to set it up by hand.
I have wg into home and wg into an aws instance. All my mobile and laptops can reach them.
welcome to the world when not TailScale, not a private WireGuard, not aL2TP, OpenVPN works. No, SSTP doesn't too. HTTPS works. Even on the devices you don't or can't control.
You can also set up DNS records pointing to your home server's VPN IP, which, with Tailscale, I've found to be pretty static and then a reverse proxy on your home server. So I have my home network apps running on app1.my-domiain.com app2.my-domain.com, app3.my-domain.com etc, which only work when I'm connected to the VPN.

The downsides are that I need to be connected to the VPN at home to use the domain and I currently don't have SSL set up on the domains, so browsers complain when I connect to them. The second problem I could fix, but I'm not sure if there's a solution for the first.

You can fix them both in one. In your local network you host a local DNS, in my case I’m using pihole. It has records which point to the local IP of a reverse proxy. With this setup you can have SSL for your domain names on your local network.

To make it then work outside your local network, in tailscale settings you use “split dns” to set your DNS to be the IP of your pihole in the tailnet for your domain. Now when you try hit your local domains you should receive the same local IPs that you do at home. Then in the tailscale route settings of your machine hosting the reverse proxy you make it advertise the subnet of those local IPs. Now when you receive the local IPs your devices using the tailscale VPN should go to your home server with SSL and no external DNS.

Hope that’s somewhat clear enough

There is a solution for the first. I have setup my home server torun Tailscale _and_ be a router to 192.168.2.x network (you can set this up in the Tailscale UI). I have server.mydomain.com to resolve to 192.168.2.x address and this way I can access it from the outside via Tailscale and from inside without the need to turn on Tailscale. I have https setup via DNS-01 challenge as well and updated automatically.
The internet is not a friendly place, so a VPN is a great idea.

With modern tools like Wireguard, you can even set it up relatively easy, either as Wireguard alone or as Tailscale (or ZeroTier, though that's not Wireguard).

Wireguard (and Tailscale) allows you to setup the tunnel so that only local (RFC1918 ie) traffic is routed over it, meaning it won't eat up your battery like when just routing all traffic over it.

I have Wireguard setup like that. It enables on any Wi-Fi network that isn't mine, as well as cellular, and the battery impact is less than 2% over a day.

Your "life hack" is not a good advice. There's plenty of well-written explanations for why perimeter based security doesn't work. What is strange is that you've started in the right place: by being "constantly worried about security implications of each app". Unfortunately it's annoying and time consuming, but that's the right way to keep your data private. And if that's too much hassle, it means it's worth it to pay others to do it.

When I'm thinking about a hypothetical situation when I need to save the world by hacking into a hypothetical villain, my best hope will be him using your approach to security.

Could you provide an example of one of the well written explanations?
just go and look at any of the vendors selling “zero trust” solutions. They all have white papers available about how a) perimeter security is “dead” and b) how their specific flavor of zero trust is the One True zero trust and the only thing you can trust to protect your data.

You will without exception need to provide an email address to access these white papers, so their inside sales team can ensure you fully understand the importance of trusting their zero trust, and not trusting anyone else’s.

I’m not kidding - even a little.

It's sad that "zero trust" has become almost the opposite of what it originally meant. Now you can have the same insecure RDP server with admin/admin login but at least it's protected by a VPN that only a few tens of thousands of people have access to.
Defense in depth though: against a strong threat actor, perimeter based security won't prevent you from getting pwnd, but it will likely take them longer than if everything is on the public internet, and a private network will make weak threat actors ignore you, so it is still valuable.
> There's plenty of well-written explanations for why perimeter based security doesn't work

It certainly helps when your attack surface consists of numerous web apps of unknown quality.

Drive-by RCEs (e.g. log4j) then suddenly become much less of a headache when none of it is reachable by the world at large.

Exactly how you do that, whether via an authenticating reverse proxy or VPN doesn’t really matter.

Any serious approach to security begins with a reasonable and clearly defined threat model, and my threat model for my home network doesn't currently include a team of superheros targeting my file backups in an effort to save the world. But I'll definitely keep your advice in mind when I do decide to start executing on my evil plots.

For now my threat model consists of script kiddies and abusive corporations. Self-hosting gets me away from the corporations and keeping my stuff off of the public internet keeps me away from script kiddies.

(comment deleted)
Even better, set up Tailscale.

It's far easier to set up, is much more reliable (e.g. when devices are behind firewalls), and uses direct (encrypted) connections when possible.

You can get it to do what you want with just a few clicks. Things like exposing a IoT VLAN on your Tailnet or setting up an exit node to tunnel all internet traffic through your home are super easy. You can even share specific devices with friends, which is super useful. If you have anything particularly sensitive (e.g. a notes app that you wouldn't want your children / partner to have access to), you can limit access to specific users / devices on the TS side, without bothering with implementing auth.

I think there's even a way to look up the user and device based on their IP, which is one way to add painless authentication to your apps. There are reverse proxies that do it and inject the info as HTTP headers.

If you aren't comfortable with trusting them with control over your network, you can always host your own Headscale server.

> If you aren't comfortable with trusting them with control over your network

Wrt the possibility of Tailscale being compromised, there's the in-beta tailnet lock feature:

> Tailnet lock lets you verify that no node is added to your tailnet without being signed by trusted nodes in your tailnet. When tailnet lock is enabled, even if Tailscale infrastructure is malicious or hacked, attackers can't send or receive traffic in your tailnet. [1]

[1] https://tailscale.com/kb/1226/tailnet-lock

The pricing page suggests this is only for the "enterprise" plan.
Not sure which page you're referencing, but the linked page states it's available for Personal (free) as well:

> Tailnet lock is available for the Personal, Personal Plus, and Enterprise plans.

It is definitely on personal as I use it myself.
Thanks for the tip!

I've had the Device approval setting on, and wished there were more robust lock features, but not enough to want to run my own coordinator. So Tailnet lock seems like a good security upgrade.

Or, for those who are paranoid about relying on a company, setting up headscake is relatively quick and painless too - currently using it to sync between devices across multiple cities.
Should I be paranoid? I never tried Tailscale, and the idea of trusting 3rd party with managing access to my network does give me chills. But IDK, honestly, maybe it's silly? Is it in all honesty less likely that I'll fuck things up setting my own Headscale server, than that Tailscale™ will (consciously or otherwise) fuck me up?
Tailscale has made all of their client source code available for anyone to view so if you want to confirm that you’re not sending unencrypted data or keys through their servers you’re more than free to do so.

https://github.com/tailscale/tailscale

I think there is some merit to setting up wireguard (e.g. you want more devices than what Tailscale offers for free, or their servers become unreliable for some reason)

But people who push the “scarey boogeyman will look at your data” with Tailscale are either technically illiterate or overly-paranoid.

I don't think the hesitation comes from the boogeyman you speak of, but just relying on someone else other than yourself.

This has further implications than just security. Also, whoever gets into self hosting already wants to not rely on others for the most part. If you didn't go into this for the tinkering aspect, I guess Tailscale is completely ok for most, perhaps even ideal.

Nevertheless, none of this has anything to do with the achievements or not of Tailscale. People saying that your own VPN is ok have a point.

afaik if you set up your own headscale server, i don't think there is a device count limit (?)
If you got yourself into self hosting, you might as well go fully independent. You have already taken care of the most complicated part anyways.
What makes Tailscale more secure, or more reliable, than just a direct Wireguard tunnel?

Tailscale's complexity and features make sense when you have 200 nodes, or maybe 20 nodes at least. When you have 3-5 nodes, I think it's overkill, and a bunch of extra dependencies which may fail, and lock you out of your private nodes when you need it most.

NAT busting, and no key management. What extra dependencies does Tailscale have?
Well, the dependency on Tailscale's servers, for one. You're getting that NAT-busting because Tailscale is running servers to handle that for you, and you're getting around key management by having them manage your keys and overlay their own auth layer for you.
You can always run Headscale: https://tailscale.com/opensource#encouraging-headscale

> Headscale is a re-implemented version of the Tailscale coordination server, developed independently and completely separate from Tailscale. Headscale is a project that complements Tailscale — with its own independent community of users and developers. Tailscale does not set Headscale’s product direction or manage the community, and neither prohibits nor requires employees from contributing to Headscale.

I had a Headscale server running for a few years with no hiccups. Setup was easy, it's not too resource intensive, and you can use the normal Tailscale client.
So Tailscale has "extra dependencies" on Tailscale. Gotcha.
Tailscale is an extra dependency vs using WireGuard directly, which is what was being discussed.
The benefit of Tailscale is that it gives you “lots” of wireguard tunnels that work through NAT with near zero configuration and a central admin interface.

I use a personal plan and have multiple nodes. Desktop, laptop, tablet, phones, docker containers just for me and a couple of raspberry Pis on my families home networks.

Only once have I been “locked out” of a node and that was due to an expired key.

Sure, for just connecting one node to another with a known IP and accessible port it’s overkill, but for anything more complex it an awful lot of awesome for very little effort.

I second this, and want to highlight that only recently I learned about this similar project called netbird. I tried it, and it looks and works very similarly from the first glance. However, Magic DNS like feature did not work for me, for some reason. Maybe I need to do something, e.g. to enable it. More likely there are others, similar projects, but I’m not aware of them. By this point I stayed with Tailscale, but in any case, I see such project as of tremendous help for a self-hoster.
For the most part I like Tailscale, but there's weirdness on Android with their app + Private DNS.
This is also my big frustration. I've set it to an Adguard instance that's also on tailscale and the app keeps getting into a faulty state. I've been looking into hosting the Adguard instance on the open web and securing it another way but bit short on time lately.
Unraid added native Tailscale support in 7.0, now you can just add an "app" (a docker container) and tick a checkbox and it'll appear in your tailnet directly.
Tailscale is a VPN, how is it "better" than using a VPN?
Tailscale is more than just a VPN. It has a number of features and capabilities which make it more like a private overlay network that just seems to work wherever and whenever. Some of those features; WireGuard (the VPN bit), NAT punching, automatic key distribution, ACLs, split or full tunnel routing to internet resources, SSO. Just to name a few.
I do this too yes. It's a great extra level of security though probably not too smart to rely on as the only one.

It's true you're already screwed if they get past the VPN and I also don't set up Auth for stuff that doesn't have any private details. But don't count on just the VPN for sensitive stuff. Like password manager.

Yeah, definitely don't do turn off auth for things that matter, if only because I'm not perfectly disciplined at keeping my guests off my main network. I'm talking about tiny services that do one thing—ollama, tts/stt, and similar. Stuff where the worst case scenario is someone using up some processor cycles, not data leaks.
This is what Tailscale is great at.
Any idea on how to segment a VPN between friends? I have a few critical servers on my VPN I only want one or two devices to access, but no other devices. Mostly so that in case a rogue device enters or a friend gets hacked, there's no access to my sensitive services.
You can do this on tailscale with their ACL. It is super flexible for nailing down exactly which users/devices can talk to what on your network.
In my case I simply create two wireguard tunnels (one called "vpn", the other called "guest") and use firewall rules to block all traffic from the "guest" tunnel to all service except the one that should be "public" (in my case a minecraft server).

I think you could technically do it with a single tunnel by using firewall rules that refer to the IP address of the single peer but it's less convenient.

NOTE: I also added a dedicated dnsmasq instance only for the "guest" tunnel so that they have DNS working and can use hostnames instead of IP address.

This setup is trivial with both OpenWrt and OpnSense, but it should be doable also with manual setup

I built a home server project that manages many of the rough edges around deployment of self-hosted apps. It includes DDNS, router+firewall, headscale, automatic setup of apps, and automatic deployment of subdomains and TLS certs for each app.

Apps are blocked to the public by default but accessible using a Tailscale client.

It's built on top of NixOS and completely configurable through a single module.

Still in heavy development but I've replaced an entire rack in my closet with an Intel NUC.

https://homefree.host

fwiw, on mac/ios you can put your obsidian vault inside icloud directory and have a “free” cross-device sync feature.
Works if you are in the Apple ecosystem entirely. I have read there are difficulties if you want to sync to a non Apple device under this approach
This is one of the reasons I use a git repo along with iCloud. Anyway, using iCloud across 4+ Apple devices has not been a problem in general.
Any notes on how you achieve this? I use a-shell on my iOS devices but it is... sometimes flaky
Hell, you don't even need obsidian. Just create a bash function

    notes()
    {
    if [ ! -z "$1" ]; then
        mkdir -m 00750 -p /Users/User/iCloud/Documents/notes
        Now=$(date '+%B %d %Y %H:%M')
        echo -en "\n$Now\t$@\n" >> /Users/User/iCloud/Documents/notes/notes.txt
    else
        echo "${Now}"
        cat /Users/User/iCloud/Documents/notes/notes.txt 2>/dev/null
        fi
    }
While we're sharing simple note taking functions, here's mine that I used for a few years. :)

    function nod { mg +-1 "/home/user/notes/$(date "+%Y-%m-%d").txt"; }
* 'nod' stands for "notes of (the) day" and was quick to type.

* 'mg' is micro emacs, my first shell-based editor thanks to OpenBSD. The '+-1' syntax means "open at end of file" so I could easily append.

I had the bright idea of symlinking $HOME/.local to an iCloud directory once. About a week later it got completely deleted. No way to restore, or any indication of what happened. Luckily I had a backup with another provider, but I will never trust iCloud again for anything that’s not on the golden path (e.g. photos)
What blows my mind is Apple doesn’t include an editor for text files!
Interesting that storing images is not something solved yet.

If you watch the animated gif, he is still using a third party service to store that graph.

I also think people to tend to like Markdown mostly because it’s plain text. The added benefits of that preview view is minimal. Like my gut feeling Markdown is popular 90% because of it’s in an accepted way to do plaintext and only 10% for the added formatting.

Obsidian can automatically ingest files and store them on disk while giving links to it. My personal vault contains many kinds of files living in the "Attachments" folder.

> Markdown is popular 90% because of it’s in an accepted way to do plaintext and only 10% for the added formatting.

For me Markdown allows me to write and format text at the speed of thought. Added bonus is that it's readable with "less xyz.md" or anything which can render text.

Yep. Obsidian strikes a good compromise of automatically copying attachments into a relative subfolder for the note and then linking them in the MD file:

  lotr-recipes
  lotr-recipes/manflesh.md
  lotr-recipes/media/manflesh_1.png
  lotr-recipes/media/manflesh_2.png
Also makes it trivial to run a note through a static site generator and publish online.
I believe tiddlywiki stores PNGs as base64 strings, so image is always there.

Yes, the file can grow large with many images, but it's a single file containing everything... even scripting!

Yes, it embeds all attachments as base64.

TiddlyWiki is great until you want to add a structure to your Wiki. I was using it like mad, then I found out that linking pages took more time then writing notes, and I pulled the trigger and moved to Obsidian.

If you run the node.js server version it can handle images properly, as separate files. That also gives you the practical ability to use many large images and videos.
With plaintext, it's very trivial to add a script that put images in some location and build the link to that.

Markdown is great because you can easily add structure while typing compared to other format which have a more extensive markup format. I prefer org-mode because what Markdown can do, but also more extensive capabilities if you need so, but there's not a lot of editors for it especially on mobile.

> "The added benefits of that preview view is minimal."

In relative terms you may be right... but subjectively, having grown accustomed to Obsidian's live view in editor mode, I'd have a hard time giving it up.

> But I felt frustrated when I wanted to access my notes on my phone while on-the-go and saw that I had to pay for this feature. Obsidian charges $8 a month to access the same notes across multiple devices.

Errm, no? Obsidian sync is optional. I pay for it to support them, but my main vaults are all synced by iCloud, which was auto set-up by Obisidan during initial setup on my iPhone.

On the Android side, any service which can sync files can work, I assume.

Note: Yes, I use Obsidian on my phone without sync, all the time, and it syncs.

Not practical if you want to also sync with non-Apple devices.
I think any app which can sync in the background can automatically sync things? Dropbox, Moebius, PCloud, Google Drive?

My Office vault lives in a separate cloud service, and it works?

My understanding is that on iOS your only non-paid choice is iCloud, and iCloud doesn’t reliably sync to non-Apple systems. To clarify, the use case here is that you have an iPhone but also non-Apple systems, which is a fairly common scenario.
TBH that sounds like a better reason to get something other than Apple systems instead of changing all your apps to work around Apples bizarre limitations.
To clarify, by “on iOS your only non-paid choice is iCloud” I meant for the Obsidian app specifically. Other apps do provide the option to sync with non-Apple cloud storage, without payment.

There are multitudes of pros and cons regarding choosing an iPhone. The restrictions of the Obsidian app is only a single one of those. Choosing an Android phone has drawbacks of its own.

Obsidian lets you pick any folder you want when setting up the notebook, so you could put it in any of your synced folders right?

Or does the iOS version of Obsidian do things differently?

The iOS version asks you whether you want to store your vault in iCloud or not only.

However, if you don't store your vault in iCloud, it creates an Obsidian folder inside the area which can be accessed by Files app (as I just checked), which means, any application having files integration can access and sync that folder.

Even if you store your vault in iCloud, it's still accessible by any app which offer files integration [0].

[0]: https://news.ycombinator.com/item?id=43972056

iCloud is not the only alternative to Obsidian Sync on iOS. There are also several Obsidian plugins that allow you to sync to a variety of services.
Then use the git-sync plugin, or many of the others mentioned here.
OP's main arguments to build their own PKMS are: - cost (feature or maintenance) - migration because it won't exist in the future

But their solution is to depend on directus, which can lead to the exact same issues. To my eyes, they just added an extra step...

The guy just created some bullshit excuses in order to advertise his new product.
I wrote my own CLI tool for notes a few months ago (https://github.com/ollien/quicknotes). A web interface with proper rendering is something I thought about, but didn't pursue because I just know my UI skills aren't up to the task. Directus is a really interesting compromise!
You can do it so easily with AI coding tools. I'm not a frontend dev, but now I can whip up something decent looking in 10 minutes.
Always possible, just not something I really want to do :)
I used git to sync a work related repo, but now use remotely-save with WebDAV (nextcloud, with base set to /Notes). No cost for sync and still access to the ecosystem of Obsidian.
I just don’t have any of these worries with Obsidian. I pay for it because it’s great software and needs support. The sync is amazing, totally solid. The data is wherever you want that data to be. It’s just MD files. You can adapt the tool to be whatever you want from a PKM system - massively complex, with some kind of dataview hell, or just some files in a hierarchy. You can use plugins or not use plugins. You can build your own. There’s no lock in. “Migration” isn’t really a thing - it’s some files in a folder system. It’s as future proof as it can be.

I mean go nuts and roll your own if you want, but really, what’s not to like?

Kudos to the author for scratching their own itch.

People in a similar position might be interested in Joplin, which is indeed FOSS, and has lots of sync options. I personally use SyncThing, which keeps things free, but you can also use a number of other free cloud providers. You can choose to encrypt your notes to protect your privacy.

Self hosted Joplin with encryption has been more than enough for me. It was pretty easy pulling all my apple notes into joplin as md files. Building my own like this just seems excessive
If you need a little bit more than Joplin (which is a great notes taking app, but not so much of a PKMS), give Trilium¹ a try! (I used Joplin for a little while, and it's a good tool, and it's fine not to chase for more if you don't need it, but if you do, not much beats Trilium, IMO)

¹: https://github.com/TriliumNext/Notes/

> Obsidian was a great tool for me personally for a long time. But I felt frustrated when I wanted to access my notes on my phone while on-the-go and saw that I had to pay for this feature.

I'm using Syncthing [0] to sync my vault between devices. On my main PC, Syncthing runs constantly in the background. Say, if I made a change, and want to send those changes to my phone, I open the application on my phone and let it fetch the changes. It's not perfectly smooth, like Obsidian's own integration, but I prefer this instead of setting a Git repository. Also, the files don't stay in a remote server.

[0]: https://syncthing.net

I've been meaning to switch over to syncthing. I currently use insync for google drive syncing on Linux and it's basically instant and constant. I can make an edit on one machine and in the time it takes me to grab my laptop, it's been synced. That said, using google drive which I don't want to do anymore.
too bad Syncthing is no longer officially maintaining andoird app https://forum.syncthing.net/t/discontinuing-syncthing-androi...
It’s $4 a month to sync Obsidian notes, for anyone wondering.
It's a good price but still feels wasteful if you also run/pay for nextcloud or similar.
There are plugins allowing you to sync via other means (for free). I don't know how the author fails to realise/mention this. I've been using Remotely Save with WebDAV or years without issue.

And the notes are all just markdown files. If the obsidian software were to disappear you have all your notes. It's fine someone wanted to spend a load of time writing their own software but none of the reasons presented in this piece make sense.

It's not wasteful to support great software.
For me it was the It Just Works-iness of it combined with a handy way to support the project cheaply.

As long as I'm paying for it, I'm the client and not the product.

I tried some of the third party stuff, iCloud, Dropbox, etc and with all of them I either lost data because of notes not being in sync or had to manually fix stuff. $4/month and zero issues was well worth it for a tool I literally use every day.

This was news to me, so this must have changed recently, as I have been billed more than that ever since I signed up.

I looked at my account, and I am charged $10 but it seems they automatically moved me to a "Plus" plan that has more storage. So no complaints from me really. Either that or the $4 plan is new. [1]

The $4 only comes with 1GB of storage. I would recommend the $10 for 50GB if you use images in your notes.

[1] https://news.ycombinator.com/item?id=37251708

I think your comment is very disingenuous. And not just because it's another subscription. That plan does not work if you got more than one vault, and if you use Obsidian you will probably have more than one vault.

https://obsidian.md/sync

It's a time/cost tradeoff and for me personally $4 has been fine for out-of-the-box syncing between clients for the last ~2 years of using just the one vault. It's now $8 for 10 vaults (I only recently added my second), which is still a relatively insignificant amount considering I spend more than that on toilet paper.
I use Obsidian and don't have more than one vault. What a weird thing to claim.
> if you use Obsidian you will probably have more than one vault.

Why would anyone ever want to use more than one vault? I just use different folders. The only reason I can think of would be if you are using Obsidian for work where you aren't allowed to use unapproved services.

Just me perhaps, but I have three. Personal, Work, and Shared. But only personal and shared are synced as work stuff needs to stay on the work PC.
I use Obsidian as a distant secondary notetaking app.

I use one vault per video game. I don’t need to clutter up a vault just so I can link from a page on Art combos in Xenoblade games to a page detailing all the random things I need to pick up around Hyrule to upgrade this or that piece of armor.

I don't think you know what the word "disingenuous" means, and I advise you not to makes such personal attacks in any case.
Syncthing is a lifesaver, it's such a useful tool!

There are also several Obsidian community plugins for sync, I use Remotely Save via WebDAV.

Or you setup a Couchdb and the self hosted live sync plugin. Although the data will reside on a remote server in this case.
Apple allowing iCloud directories to be permanently downloaded fixed this for me.
Yep. I use iCloud for my Obsidian vault, set to always be downloaded. I haven’t had an issue, and doesn’t cost me anything (beyond what I’m already paying for iCloud due to Photo Library).
I've also been using Obsidian a lot. I recommend the SyncThing Fork over SyncThing.

I myself currently use Google Drive with DriveSync on Android to sync my notes, which works great. Other cloud providers also work well.

I wrote a comparison of different tools to sync here: https://bryanhogan.com/blog/how-to-sync-obsidian

SyncThing has some gotchas when it comes to sync, for example I've had files reappear when I edit them on one device and move them on another. It also seems to create "sync-conflict" files when there is no conflict at all (no modification on one side). I'd also quite like to be able to see the history of my notes sometimes...

I also had this nice surprise literally today, all my synced folders stopped working and had to be re-added: https://github.com/Catfriend1/syncthing-android/issues/1430

But overall it has worked well, I am using it between 4 devices.

Same. I admit I've running into a few syncing headaches over the years, but given the cost of $0 and the fact it's open source, I recommend it too.

I previously rolled my own notes system and I find Obsidian plus Syncthing is better. Plugins are a big deal.

I did this and hooo does Apple make it difficult to sync files without iCloud. I felt like it was too janky for my liking, and paid for obsidian sync, but I felt like it was really silly how an official app could do it no problem, but you can’t get generic functionality to work with iOS.
> Obsidian charges $8 a month to access the same notes across multiple devices.

It's $4 actually, for the normal plan that works perfectly well for most use cases. It's also end to end encrypted, which is great. And it's not just about syncing for me, it's about a backup solution for the notes.

> I started to have concerns about the longevity of the plugins and app itself. Some of you may remember when Evernote aggressively limited free users to 50 notes, many users migrated their notes elsewhere. I was one of those users.

The great thing about Obsidian (in comparison to Evernote), is that everything is just a plain text markdown file on disk. You can open those files in any app. If Obsidian goes away someday, all your notes can continue to be edited in any plain text editor. Sometimes I open notes in VS Code, because there are certain things I just prefer writing there.

I tend to start a note with frontmatter in Obsidian, drop into vim to do the writing, then back into Obsidian to clean things up. Wish there were a cleaner way to do it, but it works for me.
Emacs chads just keep winning.
Younger folk and beginners keep ignoring Emacs (and Lisp in general), without the slightest attempt to even understand what kind of philosophy makes it appealing.

The profound difference lies in ontological fungibility – Emacs isn’t software you use, but cognitive clay that becomes an extension of your mind’s operating system. Where any specialized app is inevitably doomed to constrain you to some kind of constructed imagination of what note-taking/knowledge work should be, Emacs+Org erases the distinction between a tool and thought through radical philosophical pillars.

1. The Medium is the Message Paradox

Emacs rejects the app paradigm's fundamental axiom. Instead of being a "notes app" or "writing app", it's a meta-medium where:

- Your notes can spontaneously become a calendar event → spreadsheet formula → email draft → code compiler

- The act of writing is programming your environment (Org markup becomes executable functions)

- Tools aren't discrete entities but fluid expressions of your current mental state (e.g., I can run a shell command piping it to grep and then pipe the results into a text buffer)

1. Agency Through Textual Primordial Soup

By rooting everything in plain text + programmable buffers, you're working with the substrate of computation itself. Unlike database-driven apps that entomb your ideas in rigid schemas:

- Every thought remains perpetually protean – a TODO item can morph into a API documentation generator through markup alone

- You manipulate knowledge at the level of semantics (headings, tags, properties) rather than fighting GUI metaphors

- The friction between "taking notes" and "building systems" disappears – your journal entries are the configuration files of your life (I manage all my dotfiles — for Linux, Mac, home and work machines via Org-mode)

1. Compounding Selfhood

Specialized apps optimize for atomic efficiency; Emacs thrives on continuous identity investment. Each macro you write, each Org capture template, each minor mode becomes:

- A cognitive microhabitat that evolves with your thinking patterns

- Permanent infrastructure that pays compound interest (my 2010 Org config still works, while Evernote of 2010 is abandonware)

- A mirror of your epistemology – the keybindings/hierarchies are your neural pathways externalized

This creates an irreducible satisfaction: you're not just using tools but cultivating a personal universe where every interaction leaves permanent fertile ground for future growth. The specialized app user lives in rented apartments; the Emacs devotee walks through an ever-expanding mansion whose rooms rearrange themselves to their thoughts.

(Long time Emacs user, abandoned since VSCode became a thing because it hurt my wrists so much.)

Do you think what you're talking about is hard to demo?

"- The act of writing is programming your environment (Org markup becomes executable functions)

- Tools aren't discrete entities but fluid expressions of your current mental state (e.g., I can run a shell command piping it to grep and then pipe the results into a text buffer)"

I haven't seen an impressive demo of this kind of stuff tbh.

No, it's not hard to demo, and I've been thinking about making some vids, but it's just difficult for me for multiple reasons. Besides, the whole topic feels too grandiose to cover easily and make it satisfying for every level of expertise — newbies and seasoned veterans.

I'm a regular dweller of https://www.meetup.com/emacsatx. We meet every first Wednesday of the month - if timezone permits it, come talk to us if you have specific questions. I will promise you though to make an effort to produce some demos and publish them.

Very impressed by Rainer König's Org-Mode tutorial series on YouTube. He manages to keep most episodes under 15min. As you progress the knowledge keeps building slowly. Highly effective. His paid courses are likely even better. During the 1st series He only touched init.el a few times. Opting instead for using Customize functionality. He used the default theme, if you can all it that. Toolbar on and menu bar and he used them during the sessions.

System Crafters tends to mostly do live streams for more a couple of hours. You learn a lot watching them. But many people don't have the patience for that.

Rainer is indeed great and he also published an Udemy course on the topic. I've never tried it, by the time he released it I already was well-versed in Org-mode.

He does a good job of showing some interesting features of Org-mode, yet he doesn't provide any insights of how cross-referencing things in a real workplace scenario may look like, how does Org-mode/Emacs help you to communicate with your colleagues. How it allows you to investigate failing http endpoint or document some investigation where you need to dig into a sql db. How Org integrates with other packages, like Anki, Pomodoro or Noter (for pdf annotations).

Wilson's vids also great, but that too, doesn't work as "marketing material", it's mostly "preaching to the choir" and unlikely to bring attention outside of the Emacs-world, still, those videos are very good.

The point I'm trying to make — there are great videos and tutorials for specific things — Org-mode, text-editing features of Emacs, version control, etc. There aren't many vids to demo "the point of Emacs holistically", with the emphasis on "Always bet on text" https://graydon2.dreamwidth.org/193447.html, I'm just gonna throw a random quote from that great article: "Text can convey ideas with a precisely controlled level of ambiguity and precision, implied context and elaborated content, unmatched by anything else."

That is the greatest selling point of Emacs. That's what actually makes a huge difference. When it comes to manipulating text - fetching, processing, searching through, dicing, slicing, summarizing, converting, publishing, etc. — Emacs truly is the unmatched king of that prose.

Did you use evil mode? I am basically just running Spacemacs and I don't often need to use ctrl and option most of the time by leveraging the modal editing.
People definitely sleep on emacs, which is a shame, but I do get it. I switched to emacs fulltime for coding for my work a few years ago, and really had it tailored to my preferences but drift (that took just slightly too long for me to keep up with while on the job) combined with AI features in other editors led me to land on Zed for code, which saddened me but is probably the best fit for coding.

However, Org-Mode is seriously a killer feature that is well worth it on its own. I use emacs to handle all my finances, notes with org-roam, blogging with ox-hugo, dotfiles in a massive org-tangle document, etc. I couldn't find anything remotely to compare it to, and have failed attempts at switching. There are tons of ways of interacting with Emacs that are just lightyears ahead of anything else. Literate programming and tangle are a godsend for tons of different use cases for me as well. I even keep running notes in the base of my codebase so I can hop to various files and capture locations in the relevant files as links in notes.

The only pain point for these tools for me is trying to use them as part of a commandline integration. I really wish org-mode and org-roam's functions were callable outside of emacs, e.g. for automated document generation from org-mode. One of the most painful things I've had to setup is an org-roam-server integration that updates and deploys org-roam-server when the git repo containing its files is updated. Only way to do that seemingly is to run emacs in a very strange way inside of a docker image. Intuitively it feels like org-roam-server should be able to build itself into a static website for deployment. Might seem like a nitpick, but it's really not -- that interface is a huge part of why I use it and not being able to host it without horribly hacky workarounds seems like a massive weakness.

Which is why I've settled on neovim; I've written my code to be as modular as possible, so I can pull it in an CLI program if needs be.
Neovim is a text editor. You can't compare Emacs with just a text editor. Just like you can't compare Emacs with a browser, music player, note-taking app, email client, etc. Emacs, first and foremost, is a Lisp REPL, with an embedded text editor. Without deeply understanding and respecting this aspect it is difficult to appreciate what Emacs could be, what you can do with it.

Imagine this — one morning I was watching my colleague showing me a bunch of things over a Zoom video, and I didn't feel comfortable interrupting him with "wait, don't scroll away, I didn't get that, allow me to read it", "hold on, I'm taking notes", etc. I sacrificed my lunch break to write a small Elisp command that runs tesseract to OCR the image in the clipboard. Now, I don't even have to ask my colleagues for explanations or tell them to share the url they are currently browsing — I can just grab it with a couple of keystrokes to make a note. And that's just one, single, isolated example. It cost me twenty minutes of my time. Twenty minutes of investment that paid itself many times over already. And I have tons of similar examples.

And this is why I stay away from eMacs.
"Staying away" implies understanding the perils or benefits, I don't think you ever fully understood either. My point is not to bash on your choice of an editor — I use Neovim myself (it serves me well occasionally).

You "stay away" from Emacs most likely because you don't know any better. Do you use your editor to read and annotate pdfs? Or watch videos? Or manage the library of your ebooks? Or track your expenses? Or control project management like Jira? Or keep your knowledge base and note-taking? Or interact with LLMs? Or explore APIs like Postman? Or keep your spaced repetition flash cards like Anki? Or use it for chat over platforms like Telegram and Slack? Or find and read RFCs and manpages? Or to perform web-search, search through your browser history, Wikipedia, Youtube? Do you have etymology lookup, thesaurus, dictionaries, translation? Or to order pizza? Or measure distances between coordinates on a map? Automate things based on solar calendar or moon phases? Manage all your configs, aka dotfiles? List, browse and code review Pull Requests, etc., etc.

Now tell me, what a sane person ever exposed to all this usefulness would ever reject it? Only those who never had patience to reach it, or those who are oblivious to the possibilities.

I only know my perspective:

I get the appeal of Emacs but this is exactly where it falls short for me. It’s brittle, monolithic, and too controlling (restricting).

Emacs owns the entire platform. If you want to use an external tool, you have to wrap it in Emacs abstractions. That means dealing with buffers, subprocess plumbing, or writing Elisp bindings. You’re not really integrating; you’re translating everything into “Emacs-speak.”

Meanwhile, tools outside Emacs—like in a Unix shell—are composable by default. I can swap fzf for peco, tesseract for ocrmypdf, or even Neovim for Helix without rewriting my entire environment. That’s real modularity. With Emacs, everything is wired together internally, so small changes can break big things.

And yeah, I can technically replicate workflows inside Emacs, but often it’s reinventing worse versions of tools I already have—eshell instead of zsh, eww instead of a browser, magit instead of plain git in a terminal. That makes it feel more like a self-contained OS than a part of the system.

I’d rather keep my editor focused and compose my environment around it—not inside it. Plus, I rather do certain things in Lua, Python, shell, etc. where I can call it and run it through awk, sed, rg/grep, and so on. I can port that to any other platform and adapt it quickly rather than try to rewrite Lisp.

So... to answer your points specifically --

Do you use your editor to read and annotate pdfs? -- Sure. Using my PDF editor.

Or watch videos? -- I think that's a little easier in shell (mpv, vlc) or directly in a browser.

Or manage the library of your ebooks? -- Calibre

Or track your expenses? -- I have a tool for that.

Or control project management like Jira? -- Jira or their respective tool(s).

Or keep your knowledge base and note-taking? -- Neovim+Lua

Or interact with LLMs? -- There's a shell script for that, browser, or the ChatGPT application.

Or explore APIs like Postman? -- curl, wget, ... or Postman itself.

Or keep your spaced repetition flash cards like Anki? -- Inside Anki. I have a simple shell tool that goes through them as well.

Or use it for chat over platforms like Telegram and Slack? -- In their respective apps?

Or find and read RFCs and manpages? -- My browser, or the shell.

Or to perform web-search, search through your browser history, Wikipedia, Youtube? -- I think it's called a browser. Or the shell.

Do you have etymology lookup, thesaurus, dictionaries, translation? -- Browser. Or the shell.

Or to order pizza? -- Browser. Or the shell.

Or measure distances between coordinates on a map? -- You got me there.

Automate things based on solar calendar or moon phases? -- You got me here, too. I don't have a need for this.

Manage all your configs, aka dotfiles? -- stow, git

List, browse and code review Pull Requests, -- browser, git

etc. -- etc.

etc. -- etc.

Now tell me, what a sane person ever exposed to all this usefulness would ever reject it? -- I feel like I answered this.

Only those who never had patience to reach it, or those who are oblivious to the possibilities. -- I'm aware.

Thanks, this was an entertaining 10 minutes to write.

You perfectly just illustrated my original point of "without the slightest attempt to even understand what kind of philosophy makes it appealing..."

Saying "I can just use my pdf editor" is on the same level of "integration" as pulling another laptop with a pdf next to the one you have already. What I'm talking about is something like ITEE (Integrated Text Editing Environment), and you simply just don't get it. Emacs allows you to get closer to the plain text as possible. And the value of the plain text proposition is enormous. When every bit of information is reduced to text — it allows you to manipulate information more easily. In Emacs, everything becomes text — the list of directories and files? You can freely edit them — using all the editor features you have — multiple cursors, search-find-replace, macros, whatever. Extracting a bunch of URLs from a web page, or a PDF? Easy. Finding a specific URL from that list and retrieving a description for it? Piece-of-cake. Your browser history — it's just text. Various search engines? They are just a middle-man and you talk to them in text. Version-control interactions? All happen in text. etc. etc.

Of course, browsers allow you to do a ton of interesting things, and btw. it's not using Emacs 'instead', but rather having a choice — e.g., I do use zsh, but some things I do in Eshell in a much more productive way.

Browsers, shells, specific apps, scripting engines, etc. I have no problem using them when I see fit, but it really seems like you have zero idea how actually awesome it is to be able to not only perform spell-checking, but also consult the Merriam-Webster thesaurus, get the definitions for specific words (from another service), translate entire sentences and paragraphs (e.g. via google translate), parley with a bunch of LLM models, perform web search and etymology lookup, consult your own notes and knowledge-base, check through your browser history, etc. etc., and all that "on point" — in the same context and place where you're composing a piece of text. Like a comment you're replying on Hacker News. The mental model between approaches is just vastly different. It puts you into the "flow state" described by renowned psychologist Mihaly Csikszentmihalyi. And btw., If you think I had to leave my text editor even for a second, to lookup his name to mention it here — you still don't get it.

> I really wish org-mode and org-roam's functions were callable outside of emacs

What are you even talking about? It's like wishing for java programs to be able to execute without JVM, or Erlang-code without BEAM. While that's theoretically possible to a certain degree (using Graal and Lumen), it's not easily achievable.

Org-mode is written in Elisp, of course it needs Emacs to run. You just need Emacs executable and you can absolutely run scripts in batch mode. I have done it many times, running tests on GitHub Actions, etc.

"What are you even talking about?"

Not understanding something is not a virtue, not something to act smug and superior about.

"It's like wishing for java programs to be able to execute without JVM, or Erlang-code without BEAM."

No, it's not like that.

"Org-mode is written in Elisp, of course it needs Emacs to run."

There are (incomplete of course) apps outside of emacs and elisp that process org-mode files. And even if there weren't, it makes sense to wish that there were. In the mind of a clear and competent thinker, the mere fact that org-mode needs emacs to run is not a reason not to wish it weren't.

> Not understanding something is not a virtue, not something to act smug and superior about.

I'm sorry that I sounded like acting smug, I'm honestly just trying to understand what you're wishing for. An external (non-elisp) org-mode parser or something? I don't get your last paragraph in your latest response, like at all.

I thought I was pretty clear? It would be nice to be able to call functions from org-mode and its related packages without having Emacs installed, or with some minimal subset of emacs meant for the commandline which caters to the needs of people running headless commands. It would solve a whole ton of issues for me. In particular, org-roam-ui is amazing. To run it, I have to open emacs and then... have emacs launch it into a browser for me. But the hosted interface won't export to anything that can be statically (or dynamically, really) hosted, so I can never access this awesome note-viewing interface from the web, even though to do it locally I already have to use a browser?

I have a project for hosting org-roam-ui as a website that took a ton of finagling to figure out and is extremely brittle. I had to finally load a custom .el script to prepare emacs to even try to evaluate the 'org-roam-ui' command from the commandline. If emacs prioritized headless / cli operations, this wouldn't have been such a nightmare, but it absolutely was. Just finding the right ports and re-assigning them to get out of each others way was a huge learning curve even for a seasoned emacs-er.

> It's like wishing for java programs to be able to execute without JVM, or Erlang-code without BEAM

No, it's not. It's wishing for basic functionality offered by all other markup languages of which I'm aware. That includes latex and everything supported by pandoc. Org-mode can do all sorts of crazy rendering tasks from its files, and it is worse than pulling teeth to get that functionality into a build pipeline of any kind.

Love this ! But I disagree to the thought that youngsters don't use emacs. It is amazing to actually see a number of people who are very young learning about emacs and loving it. The community behind emacs was always niche but it is amazing that in every generation there are people who resonate with its ethos and mental models.
All excellent and true points. However, it is my personal opinion that first impressions matter. When a newbie installs Gnu Emacs the default theme and overall appearance of the GUI is shockingly dated and the defaults are bizarre compared to 'modern' UX. That's what happens when you create software 40 years before there were any UX standards. Heck there was no mouse and no arrow keys on most keyboards at the time. Full screen editing was brand spanking new. The vi editor came after Emacs because UNIX came after Emacs.

Most new young potential Emacs users are students and new developers being raised on VS Code or JetBrains IDEs. Which out of the box appear modern to their young eyes. So when they see vanilla Gnu Emacs the reaction may be, "Okay Boomer, whateva".

I would argue that the reason Spacemacs and Doom resulted in such massive growth of the Emacs user base had less to do with evil-mode and more to do with attractive theming and applying reasonable default settings that match how those modern UX IDE's work. Not to mention the myriad of YouTube videos about them.

I fail to comprehend why the old school graybeards refuse to update the default appearance of Emacs. Of course the out of box experience should be mostly spartan but it could look a whole lot prettier. I would like to see Emacs devs adopt Prot's standard light and dark themes, they need to replace the traditional default theme and be distributed with Emacs. The default theme can hardly be called a theme, it likely predates theming in Emacs.

I don't know anyone who sticks with the out of box appearance of Emacs.

I think we are going to need some curated video demonstrations including slow-motion and key presses being shown. Real world tasks being demonstrated. Including eshell and REPL, etc.

I too have been toying with creating some of these demos. We need to create focused compelling videos that are rather short. Showing off just how great Emacs is overall. Seeing is believing.

So many junior devs never saw Emacs and only maybe they heard something about it but never took a deeper look. It is always some rogue graybeard who shows off in front of junior dev and that dev needs to pick their jaw off the floor. similar occurs with Neovim.

We also should be targeting non-developers such as writers and researchers and students. I know some people who use Emacs to manage their Dungeons & Dragons games. Both as a dungeon master and as a player. Heck I use org to take notes while I am playing a complex video game.

We should espouse the wonders of Plain Text and how you'll never be locked into a proprietary file format. You can use it with Git for revision history and tracking changes. The power of Elisp enables non-developers.

We need to start a sort of grassroots marketing campaign to spread the word. In a nice way. i.e. don't bash the competition. Just show what Emacs can do in short highly focused videos. Emacs can speak for itself.

> my personal opinion that first impressions matter.

It certainly often does, but there also examples where maybe it's not the biggest issue? git, vim, LaTeX, ffmpeg — they all find their users despite being so demonstrably bad with "the first impression".

Besides, I feel here you're talking about Emacs-the-text-editor, while what I'm rather babbling about Emacs-the-lisp-machine. You see, my point is, people talk about Emacs-the-text-editor all the time and keep arguing if it's "modern enough" or "outdated", "archaic" or "useless". The same way it can be rationalized as Emacs-the-web-browser, Emacs-the-email-client, Emacs-the-IDE, Emacs-the-version-control-system, etc. Of course, anyone who learned how to operate a decent email client perhaps wouldn't ever see a point of suddenly switching to Emacs-the-email-client, unless they could look at it from my POV — a Lisper's point of view. Because the email client aspect of it has been thoroughly imbued with Lisp, it makes it extremely appealing terrain for me — I can with relative ease erect my creations on top of it, limited maybe only by the boundaries of my imagination, rather than technical limitations.

If only it was possible to convince an entire generation of programmers of unthinkable elegancy and pragmatism of Lisp, wouldn't that be great? Imagine a world where "every programmer knows some Lisp" is the norm, rather than the current assumption that "every programmer is familiar with Javascript, SQL, and Bash" – which, in reality, is almost universally not the case.

That is the essence of my plea — it isn't so much about Emacs itself, but about the widespread ignorance of coders who dismiss Lisp as an idea. If that weren't the case, we wouldn't even be here, discussing the reasons why Emacs is so unpopular.

Young people and beginners don't even know these things exist, and if they do happen to know they exist they have no reason to look into them. It's absurd to attack people for not "attempting to understand" something that they don't know enough about to have reason to "attempt" it. There are all sorts of technologies that I haven't "attempted to understand" (including many emacs packages) because they aren't on my radar or I haven't been made aware of their value. This lack of an attempt is not some sort of flaw.
I'm not trying to attack or blame anyone, but rather share my empirical observations. "Young people and beginners don't even know these things exist" is also along the same lines of thought.

> This lack of an attempt is not some sort of flaw."

It could be, but perhaps on the other end here — maybe the evangelism and elucidation are at fault? Seasoned adepts often forget "the beginner's journey" and can't break through the "communication disconnect". It's hard to explain the "what" and "how" without effectively conveying the "why", especially when the other side is not interested in hearing the "why".

I'm just sad that a lot of very talented folk either choose to ignore or deliberately dismiss some great ideas, building their assumptions based on some shorts they watch on YouTube or TikTok.

The lack of attempt may not be a flaw, but maybe "the lack of attention" is our generational flaw? Maybe while I'm focused on Emacs and Lisp, I'm failing to see a bigger picture, where tons of other pragmatic ideas get dumped overboard because they don't align with the prevailing narrative?

I think it is way simpler than that. IMHO, for many people young or old the value proposition of emacs simple is no longer there and/or worth the huge learning curve. For most things people would use something like emacs the alternatives are simple good enough and some cases better. By better I mean mostly more accessible, less complex and with a better ecosystem. One tool that does all, also simply does not resonate with everyone nor do many people feel the need to make their almost their lifestyle (not saying you do, I'm speaking in general to the Emacs runs my life folks). Another thing to consider often is that when working in a group or team at work for example it can be the easiest path forward to use the tool the company officially uses and your teammates do. If your team uses for example slack to communicate why go through the hassle of setting up some most likely cumbersome workaround to target it via emacs instead of just using the slack.
My team does use Slack app and I use it too. The point is not to do "everything in Emacs" because you love that so much. The point is that when plain text is the main medium (and for majority of tasks we deal with daily, it is), then using text-oriented tools can give you certain edge.

When communicating over Slack my colleagues would send me a ticket number in Jira or a link to some code snippet on GH. When I have to share the same (or similar) stuff — I can easily provide the exact link to the ticket, with its description, I can quickly retrieve some relevant data from that ticket, if I'm adding some notes. When I'm sharing a link to a code snippet, I can easily retrieve the fully-qualified namespace and the name of the function, so the url is not some arbitrary incantation that my teammates have to guess about before clicking on it — it contains the exact piece of information they may like to see even before opening it.

When someone shares 'some incantation' with me, I don't even hesitate — I delegate the task to Emacs, and it is smart enough to recognize that a piece of plain text like "XYZ-146" is a Jira ticket, that "RFC-6364" is a document, that a github url is a link to a PR — with a single keystroke I can browse the jira ticket, read RFC document, explore the changes in the PR. I can invoke an LLM on the spot to summarize the points of the RFC document, to give me some insight to add a comment to the PR review.

All that without a hassle, all that done with simple keystrokes, all in one place. I can easily copy relevant information into my notes, I can grab stuff from my notes and share with my colleagues, easily converting Org-mode markup into Markdown without even blinking. The flow is there. And that flow also contributes to the general well-being of my teammates. I can assure you, they are happier with me using Emacs, even though most of them don't even know anything about it.

> the value proposition of emacs simple is no longer there

My main point from the get-go was that most people don’t even realize what a ‘value proposition’ there is.

All fair. I suppose my point is that I think people do actually have a decent sense of what value proposition there with emacs but simply don't agree that its strengths are worth the steep climb and or a the much better that their current tool needs replacing.
Emacs can be a curse too. Once you get into it, it's really hard to use anything else..
Does anyone know of a tool that can e2e + collaboration (or in other words: notion but e2e)?
Maybe Anytype, or self hosted appflowy
Anytype. Local-first, p2p sync with e2ee. It uses the notion model (everything is an object, you can create "databases" with queries of objects). It doesn't store markdown directly I believe, but it can export into markdown perfectly.

I recently moved to anytype from logseq as it hit for me all the right notes for personal stuff, shared family stuff and work stuff.

I wanted something like notion but faster and most importantly, private.

We're building https://thymer.com/ to do this. Real-time collaboration, local-first + end-to-end-encrypted (and optionally self-hosted).
Cool. May I ask how you implement the collaborative e2e? Do you encrypt the content multiple times - once for every user?
The workspace data is encrypted with a (symmetric) workspace key. Each user gets hteir own encrypted copy of that key, using the user's public key. By wrapping the data-encryption key for each user the content is only encrypted once but each authorized user can decrypt it.
Ah ok. I guess that means when a user leaves the team, you have to re-encrypt the codebase with a new symetric key (as the user would still know the old symetric key)?
Depending on the threat model you want to protect against, yes. For example, although a revoked user doesn't have direct access to data anymore, they could technically collude with us to get future encrypted messages and then decrypt them. At the moment rekeying is only possible by exporting a workspace, changing its key (with a command line tool), and then reimporting it.
I didn't want to sidetrack you with this. I just wanted to know if I understood correctly. I believe this is nothing you need to worry about at this time. Thanks.
I've built a couple for myself so far; the most recent is in zig (sqlite extension that treats markdown files / frontmatter as virtual tables) and it's lasted me. I plan to rewrite it soon to adapt to how I've been using it :)

https://github.com/kunalb/termdex

Reads like a mix of valid concerns and a plug for Directus, which is sort of fishy.

Either way, like many others, I use SyncThing to sync my vault, and routinely edit it with vim, so Obsidian is just one comfortable shell that can (relatively easily) be replaced.

I'm not saying AI is going to replace programmers, but it's been almost a century already and we still don't have a decent note taking app or even a todo list app and that's like the first app you learn to make. Maybe humanity just kind of sucks at this whole application development thing.

I use cherrytree currently, by the way.

There are hundreds of decent note taking apps and todo apps. The issue is that almost no one has the exact same needs or workflow for either of those things. I’ve given up on suggesting those things to people and sticking with “whichever one you use is the best”. So the best you’ll ever do is a decent one //for you// but it’s possible that definition may not work for anyone else.
I ditched for [silverbullet](https://silverbullet.md). MIT licensed, markdown editor with embedded lua scripting. It's a PWA app that works offline and syncs well.
Thanks for mentioning this, it looks exactly what I want; a markdown web app that stores the files using the filesystem on the backend!
Very impressive!
Silverbullet is aptly named but I'd actually wait a month or two because it's going through the transition to V2 which shuffles a few things around.

Other than that it's exactly what it says on the tin, a hacker's digital notebook. While it would take a bit of rolling up your sleeves to make it look as nice as Logseq or Obsidian it's actually closer in form and function to something like org-mode for Emacs (though not quite hitting the mark regarding linking sadly).

v2 is here already as far as I know, I mean I'm happily using it.
It's got some rough edges. I want to sit down and build it so I can add an id to the TOC so I can CSS style it to be floating and submit it upstream.

I'm also having issues with integrating it with Authentik's header proxy auth, keeps directing me to a note with the outpost path as the name. The only guide is for authelia.

It does, it's not perfect, but nothing ever is.

I haven't tried putting it behind Authentik or Authelia. They make it known in the Authelia guide what it is they care about being always exposed, vs password protected. Hopefully you get it figured out and you can update the docs so the next person doesn't have the same headache!

I just put it behind a <uuid>.mydomain.com with a domain TLS cert and use the built-in auth.

The wildcard TLS cert keeps the <uuid> from being public in the cert log. The only way you know the URL is if you have access to my DNS queries or have a MITM setup. Plus you still have to know my password.

Good Enough for me.

If I cared a bit more I'd put it behind Tailscale/Nebula/etc instead of having it publicly accessible. Maybe next time I'm bored I'll do that.

The Maya Angelou quote is a very poor choice. I don't know if the author realizes the absurdity of putting the civil rights movement in parallel with her "PKM journey"
Does it really matter what context the quote was initially said in, if it generalizes to be applicable to many other things?

For context, this is the quote that is "absurd":

> You can’t really know where you are going until you know where you have been

I feel like that'd be fine in a lot of different contexts.