The author’s journey is probably just starting. I had this exact mindset about 10 years ago. Long story short: distributed systems are hard. A linear log of changes is an absolute lie, but it is a lie easy to believe in.
I feel like a broken record here: it is not an engineering problem.
Local-first and decentralized apps haven't become popular because SaaS has a vastly superior economic model, and more money means more to be invested in both polish (UI/UX) and marketing.
All the technical challenges of decentralized or local-first apps are solvable. They are no harder than the technical challenges of doing cloud at scale. If there was money in it, those problems would be solved at least as well.
Cloud SaaS is both unbreakable DRM (you don't even give the user the code, sometimes not even their data) and an impossible to evade subscription model. That's why it's the dominant model for software delivery, at least 90% of the time. The billing system is the tail that wags the dog.
There are some types of apps that have intrinsic benefits to being in the cloud, but they're the minority. These are apps that require huge data sets, large amounts of burstable compute, or that integrate tightly with real world services to the point that they're really just front-ends for something IRL. Even for these, it would be possible to have only certain parts of them live in the cloud.
Now that people are used to having someone in a data center do their backing up and distributing for them, they don't want to that work themselves again, privacy be damned.
I always start with a local first application. And then find that I want to access List X, Design Y, Scribble Z on the road and on my phone. Resulting in a frenzied search for a mobile / desktop alternative that includes syncing. Then finding a solution that requires a subscription and then falling back into fiddling with Google Drive, or something in the Google workspace.
I guess I should bring my devices back to exactly 1 device. Or just take a subscription on one service.
IMO, offline read-only is enough of a compromise. How many times are users truly offline, AND want to be able to edit at that time (and deal with the potential conflicts, which by the nature of the operation, won't have a good UX)?
Local-first was the first kind of app. Way up into the 2000s, you'd use your local excel/word/etc, and the sync mechanism was calling your file annual_accounts_final_v3_amend_v5_final(3).xls
But also nowadays you want to have information from other computers. Everything from shared calendars to the weather, or a social media entry. There's so much more you can do with internet access, you need to be able to access remote data.
There's no easy way to keep sync, either. Look at CAP theorem. You can decide which leg you can do without, but you can't solve the distributed computing "problem". Best is just be aware of what tradeoff you're making.
If they're building this on top of sqlite, probably worth considering adopting https://rqlite.io/ that has already done a lot of work for a clustered sqlite
Technical reasons are honestly overblown - it all boils down to one, business, reason - control.
When you do serverside stuff you control everything. What users can do, and cannot do.
This lets you both reduce support costs as it is easier to resolve issues even by ad-hoc db query, and more importantly - it lets you retroactively lock more and more useful features behind paywall. This is basically The DRM for your software with extra bonus - you don't even have to compete with previous version of your own software!
i want my local programs back, but without regulatory change it will never happen.
I've only started playing with local first recently to learn the Service Worker API and IndexedDB, and I'm looking forward to learning about CRDTs more. Here's a little todo app that I built:
I spent some time learning about PouchDB a little while ago, it seems to be a nice solution at least for a No SQL approach. Although I still need some more practical experience to understand the security model, because it feels weird to just sync a database that was updated on some web page, really you want to ensure there is validation and treat it as an untrusted source to some extent. Still not sure of the best way to deal with that without implementing most of the application as server side validation scripts, or maybe that is just the way to do it.
The real answer to the question isn't technical. Local-first apps haven't become popular because companies recognize that their value comes from controlling your data.
In a talk a few years ago [1], Martin Kleppman (one of the authors of the paper that introduced the term "local-first") included this line:
> If it doesn't work if the app developer goes out of business and shuts down the servers, it's not local-first.
That is obviously not something most companies want! If the app works without the company, why are you even paying them? It's much more lucrative to make a company indispensable, where it's very painful to customers if the company goes away (i.e. they stop giving the company money).
I believe the lack of popularity is more of an economics problem. There are established business models for SaaS apps or freemium with ads. But, the business model for local-first apps is not as lucrative. Those who like the local-first model value features like: data-sovereignty, end-to-end encryption, offline usage, etc. These properties make existing business models hard-to-impossible to apply.
My current thinking is that the only way we get substantial local-first software is if it's built by a passionate open-source community.
I'm building a file-over-app local-first app call Relay [0] (it makes Obsidian real-time collaborative) and I agree with you.
We have a business model that I think is kind of novel (I am biased) -- we split our service into a "global identity layer"/control plane and "Relay Servers" which are open source and self-hostable. Our Obsidian Plugin is also open source.
So while we have a SaaS, we encourage our users to self-host on private networks (eg. tailscale) so that we are totally unable to see their documents and attachments. We don't require any network connection between the Relay Server and our service.
Similar to tailscale, the global identity layer provides value because people want SSO and straightforward permissions management (which are a pain to self-host), but running the Relay Server is dead simple.
So far we are getting some traction with businesses who want a best-in-class writing experience (Obsidian), google-docs-like collaboration, but local-first. This is of particular interest to companies in AI or AI safety (let's not send our docs to our competitors...), or for compliance/security reasons.
The use case I always think of is the developer experience for regular hobbyist and workaday devs writing their apps with local-first sync.
Apple comes close with CloudKit, in that it takes the backend service and makes it generic, basically making it an OS platform API, backed by Apple's own cloud. Basically cloud and app decoupled. But, the fundamental issue remains, in that it's proprietary and only available on Apple devices.
An open source Firebase/CloudKit-like storage API that requires no cloud service, works by p2p sync, with awesome DX that is friendly to regular developers, would be the holy grail for this one.
Dealing with eventually consistent data models is not so unusual these days, even for devs working on traditional cloud SAAS systems, since clouds are distributed systems themselves.
I would be very happy to see such a thing built on top of Iroh (a p2p network layer, with all the NAT hole punching, tunnelling and addressing solved for you) for example, with great mobile-first support. https://github.com/n0-computer/iroh
I think that the database layer is the wrong layer for reconciliation of change sets.
The main problem with any sync system that allows extensive offline use is in communicating how the reconciliation happens so users don't get frustrated or confused. When all reconciliation happens as a black box your app won't be able to do a good job at that.
I wonder about the categories of apps for which offline first with potentially infinitely delayed sync provides a better experience and how large those really are.
It seems like most of those are apps where I'm creating or working on something by myself and then sharing it later. The online part is almost the nice-to-have. A lot of other apps are either near-real-time-to-real-time communication where I want sending to succeed or fail pretty much immediately and queueing a message for hours and delivering it later only creates confusion. Or the app is mostly for consuming and interacting with content from elsewhere (be that an endless stream of content a la most "social media", news, video, etc. or be it content like banking apps and things) and I really mostly care about the latest information if the information is really that important at all. The cases in those apps where I interact, I also want immediate confirmation of success or failure because it's really important or not important at all.
What are the cases where offline-first is really essential? Maybe things that update, but referencing older material can be really useful or important (which does get back to messaging and email in particular, but other than something that's designed to be async like email, queueing actions when offline is still just nice-to-have in the best cases).
Otherwise the utility of CRDTs, OT, et al. is mostly collaborative editing tools that still need to be mostly online for the best experience.
you are missing one key aspect: high latency. i am in a rural area with bad internet access. especially bad latency. every request to hackernews for example takes seconds. even if it doesn't make sense for hackernews to be offline. updating new messages, posting replies, upvotes, downvotes. etc, would all work better if they happened in the background, so that i don't have to wait for them.
that's why local first also makes sense for server bound applications. it's not offline first, but by moving online activities into the background, all my interaction with the site would be snappy and smooth, and i could read and comment and move on to the next action without waiting for the browser to reload the page
Apps that let you manipulate artifacts are often local. MS Office, Photoshop, Blender, CAD tools.
But it turns out that actually, humans rarely work or live alone, and you can't facilitate communication through a local-first app.
It’s a hard balance, local first a lot of times feel local only. I really like the model on Home Assistant/Nabu Casa. Local first and you can pay a subscription to cloud access.
156 comments
[ 3.0 ms ] story [ 102 ms ] threadhttps://en.wikipedia.org/wiki/HCL_Notes
The future? I thought all apps were like this before this web2.0 thing ruined it.
Everything is hard if you don't know how to do it.
I.e., most people don’t care.
Local-first is optimal for creative and productivity apps. (Conversely, non-local-first are terrible for these.)
But most people are neither creative nor optimally productive (or care to be).
If you think this is only a problem for distributed systems, I have bad news for you.
Local-first and decentralized apps haven't become popular because SaaS has a vastly superior economic model, and more money means more to be invested in both polish (UI/UX) and marketing.
All the technical challenges of decentralized or local-first apps are solvable. They are no harder than the technical challenges of doing cloud at scale. If there was money in it, those problems would be solved at least as well.
Cloud SaaS is both unbreakable DRM (you don't even give the user the code, sometimes not even their data) and an impossible to evade subscription model. That's why it's the dominant model for software delivery, at least 90% of the time. The billing system is the tail that wags the dog.
There are some types of apps that have intrinsic benefits to being in the cloud, but they're the minority. These are apps that require huge data sets, large amounts of burstable compute, or that integrate tightly with real world services to the point that they're really just front-ends for something IRL. Even for these, it would be possible to have only certain parts of them live in the cloud.
Now that people are used to having someone in a data center do their backing up and distributing for them, they don't want to that work themselves again, privacy be damned.
I guess I should bring my devices back to exactly 1 device. Or just take a subscription on one service.
But also nowadays you want to have information from other computers. Everything from shared calendars to the weather, or a social media entry. There's so much more you can do with internet access, you need to be able to access remote data.
There's no easy way to keep sync, either. Look at CAP theorem. You can decide which leg you can do without, but you can't solve the distributed computing "problem". Best is just be aware of what tradeoff you're making.
When you do serverside stuff you control everything. What users can do, and cannot do.
This lets you both reduce support costs as it is easier to resolve issues even by ad-hoc db query, and more importantly - it lets you retroactively lock more and more useful features behind paywall. This is basically The DRM for your software with extra bonus - you don't even have to compete with previous version of your own software!
i want my local programs back, but without regulatory change it will never happen.
[1] GitHub: https://github.com/hasanhaja/tasks-app/ [2] Deployed site: https://tasks.hasanhaja.com/
In a talk a few years ago [1], Martin Kleppman (one of the authors of the paper that introduced the term "local-first") included this line:
> If it doesn't work if the app developer goes out of business and shuts down the servers, it's not local-first.
That is obviously not something most companies want! If the app works without the company, why are you even paying them? It's much more lucrative to make a company indispensable, where it's very painful to customers if the company goes away (i.e. they stop giving the company money).
[1] https://speakerdeck.com/ept/the-past-present-and-future-of-l...
My current thinking is that the only way we get substantial local-first software is if it's built by a passionate open-source community.
We have a business model that I think is kind of novel (I am biased) -- we split our service into a "global identity layer"/control plane and "Relay Servers" which are open source and self-hostable. Our Obsidian Plugin is also open source.
So while we have a SaaS, we encourage our users to self-host on private networks (eg. tailscale) so that we are totally unable to see their documents and attachments. We don't require any network connection between the Relay Server and our service.
Similar to tailscale, the global identity layer provides value because people want SSO and straightforward permissions management (which are a pain to self-host), but running the Relay Server is dead simple.
So far we are getting some traction with businesses who want a best-in-class writing experience (Obsidian), google-docs-like collaboration, but local-first. This is of particular interest to companies in AI or AI safety (let's not send our docs to our competitors...), or for compliance/security reasons.
[0] https://relay.md
Apple comes close with CloudKit, in that it takes the backend service and makes it generic, basically making it an OS platform API, backed by Apple's own cloud. Basically cloud and app decoupled. But, the fundamental issue remains, in that it's proprietary and only available on Apple devices.
An open source Firebase/CloudKit-like storage API that requires no cloud service, works by p2p sync, with awesome DX that is friendly to regular developers, would be the holy grail for this one.
Dealing with eventually consistent data models is not so unusual these days, even for devs working on traditional cloud SAAS systems, since clouds are distributed systems themselves.
I would be very happy to see such a thing built on top of Iroh (a p2p network layer, with all the NAT hole punching, tunnelling and addressing solved for you) for example, with great mobile-first support. https://github.com/n0-computer/iroh
The main problem with any sync system that allows extensive offline use is in communicating how the reconciliation happens so users don't get frustrated or confused. When all reconciliation happens as a black box your app won't be able to do a good job at that.
It seems like most of those are apps where I'm creating or working on something by myself and then sharing it later. The online part is almost the nice-to-have. A lot of other apps are either near-real-time-to-real-time communication where I want sending to succeed or fail pretty much immediately and queueing a message for hours and delivering it later only creates confusion. Or the app is mostly for consuming and interacting with content from elsewhere (be that an endless stream of content a la most "social media", news, video, etc. or be it content like banking apps and things) and I really mostly care about the latest information if the information is really that important at all. The cases in those apps where I interact, I also want immediate confirmation of success or failure because it's really important or not important at all.
What are the cases where offline-first is really essential? Maybe things that update, but referencing older material can be really useful or important (which does get back to messaging and email in particular, but other than something that's designed to be async like email, queueing actions when offline is still just nice-to-have in the best cases).
Otherwise the utility of CRDTs, OT, et al. is mostly collaborative editing tools that still need to be mostly online for the best experience.
that's why local first also makes sense for server bound applications. it's not offline first, but by moving online activities into the background, all my interaction with the site would be snappy and smooth, and i could read and comment and move on to the next action without waiting for the browser to reload the page
Sync problems are harder than just 'use a CRDT'.
What counts as 'consistent' depends on the domain and the exact thing that is being modelled.