How is this approach better than using react-query to persist storage which periodically sync the local storage and the server storage? Perhaps I am missing something.
Yes, you’re missing a lot of things. Like how to update that data when you are offline and have everything sync around when others have made updates to the same data in the meanwhile. Among many other concerns and use cases.
ElectricSQL and TanStack DB are great, but I wonder why they focus so much on local first for the web over other platforms, as in, I see mobile being the primary local first use case since you may not always have internet. In contrast, typically if you're using a web browser to any capacity, you'll have internet.
Also the former technologies are local first in theory but without conflict resolution they can break down easily. This has been from my experience making mobile apps that need to be local first, which led me to using CRDTs for that use case.
I think this is a fascinating and deep question, that I ponder often.
I don't feel like I know all the answers, but as the creator of Replicache and Zero here is why I feel a pull to the web and not mobile:
- All the normal reasons the web is great – short feedback loop, no gatekeepers, etc. I just prefer to build for the web.
- The web is where desktop/productivity software happens. I want productivity software that is instant. The web has many, many advantages and is the undisputed home of desktop software now, but ever since we went to the web the interaction performance has tanked. The reason is because all software (including desktop) is client/server now and the latency shows up all over the place. I want to fix that, in particular.
- These systems require deep smarts on the client – they are essentially distributed databases, and they need to run that engine client-side. So there is the question of what language to implement in. You would think that C++/Rust -> WASM would be obvious but there are really significant downsides that pull people to doing more and more in the client's native language. So you often feel like you need to choose one of those native languages to start with. JS has the most reach. It's most at home on the desktop web, but it also reaches mobile via RN.
- For the same reason as prev, the complex productivity apps that are often targeted by sync engines are often themselves written using mainly web tech on mobile. Because they are complex client-side systems and they need to pick a single impl language.
Mobile has really strong offline-primitives compared to the web.
But the web is primarily where a lot of productivity and collaboration happens; it’s also a more adversarial environment. Syncing state between tabs; dealing with storage eviction. That’s why local first is mostly web based.
> For the uninitiated, Linear is a project management tool that feels impossibly fast. Click an issue, it opens instantly. Update a status and watch in a second browser, it updates almost as fast as the source. No loading states, no page refreshes - just instant, interactions.
How garbage the web has become for a low-latency click action being qualified as "impossibly fast". This is ridiculous.
I'm also building a local first editor and rolling my own CRDTs. There are enormous challenges to make it work. For example the storage size issue mentioned in the blog, I end up using with yjs' approach which only increase the clock for upsertion, and for deletion remove the content and only remain deleted item ids which can be efficiently compressed since most ids are continuous.
Local-First & Sync-Engines are the future. Here's a great filterable datatable overview of the local-first framework landscape:
https://www.localfirst.fm/landscape
My favorite so far is Triplit.dev (which can also be combined with TanStack DB); 2 more I like to explore are PowerSync and NextGraph. Also, the recent LocalFirst Conf has some great videos, currently watching the NextGraph one (https://www.youtube.com/watch?v=gaadDmZWIzE).
I developed an open-source task management software based on CRDT with a local-first approach. The motivation was that I primarily manage personal tasks without needing collaboration features, and tools like Linear are overly complex for my use case.
This architecture offers several advantages:
1. Data is stored locally, resulting in extremely fast software response times
2. Supports convenient full database export and import
3. Server-side logic is lightweight, requiring minimal performance overhead and development complexity, with all business logic implemented on the client
4. Simplified feature development, requiring only local logic operations
There are also some limitations:
1. Only suitable for text data storage; object storage services are recommended for images and large files
2. Synchronization-related code requires extra caution in development, as bugs could have serious consequences
3. Implementing collaborative features with end-to-end encryption is relatively complex
The technical architecture is designed as follows:
1. Built on the Loro CRDT open-source library, allowing me to focus on business logic development
2. Data processing flow:
User operations trigger CRDT model updates, which export JSON state to update the UI. Simultaneously, data is written to the local database and synchronized with the server.
3. The local storage layer is abstracted through three unified interfaces (list, save, read), using platform-appropriate storage solutions: IndexedDB for browsers, file system for Electron desktop, and Capacitor Filesystem for iOS and Android.
4. Implemented end-to-end encryption and incremental synchronization. Before syncing, the system calculates differences based on server and client versions, encrypts data using AES before uploading. The server maintains a base version with its content and incremental patches between versions. When accumulated patches reach a certain size, the system uploads an encrypted full database as the new base version, keeping subsequent patches lightweight.
I've been very impressed by Jazz -- it enables great DX (you're mostly writing sync, imperative code) and great UX (everything feels instant, you can work offline, etc).
Main problems I have are related to distribution and longevity -- as the article mentions, it only grows in data (which is not a big deal if most clients don't have to see that), and another thing I think is more important is that it's lacking good solutions for public indexes that change very often (you can in theory have a public readable list of ids). However, I recently spoke with Anselm, who said these things have solutions in the works.
All in all local-first benefits often come with a lot of costs that are not critical to most use cases (such as the need for much more state). But if Jazz figures out the main weaknesses it has compared to traditional central server solutions, it's basically a very good replacement for something like Firebase's Firestore in just about every regard.
Local first is amazing. I have been building a local first application for Invoicing since 2020 called Upcount https://www.upcount.app/.
First I used PouchDB which is also awesome https://pouchdb.com/ but now switched to SQLite and Turso https://turso.tech/ which seems to fit my needs much better.
Local first is super interesting and absolutely needed - I think most of the bugs I run into with web apps have to do with sync, exacerbated by poor internet connectivity. The local properties don't interest me as much as request ordering and explicit transactions. You aren't guaranteed that requests resolve in order, and thus can result in a lot of inconsistencies. These local-first sync abstractions are a bit like bringing a bazooka to a water gun fight - it would be interesting to see some halfway approaches to this problem.
It's starting to feel to me that a lot of tech is just converging on other platforms solutions. This for example sounds incredibly similar to how a mobile app works (on the surface). Of course it goes the other way too, with mobile tech taking declarative UIs from the Web.
"Works like a charm" / "not listed" does not really do it justice, its much worse. All of the mentioned "solutions" will inevitably lose data on conflicts one way or another and i am not aware of anything from that school of thought, that has full control over conflict resolution as couchdb / pouchdb does. Apparently vibe coding crdt folks do not value data safety over some more developer ergonomics. It is an tradeoff to make for your own hobby projects if you are honest about it but i don't understand how this is just completely ignored in every single one of these posts.
I’ve been working on a small browser app that is local first and have been trying to figure out how to pair it with static hosting. It feels like this should be possible but so far the tooling all seems stuck in the mindset of having a server somewhere.
My use case is scoring live events that may or may not have Internet connection. So normal usage is a single person but sometimes it would be nice to allow for multi person scoring without relying on centralized infrastructure.
I really like electric approach and it has been on my radar for a long time, because it just leaves the writing complexity to you and the API.
Most of the solutions with 2 way sync I see work great in simple rest and hobby "Todo app" projects. Start adding permissions and evolving business logic, migrations, growing product and such, and I can't see how they can hold up for very long.
Electric gives you the sync for reads with their "views", but all writes still happen normally through your existing api / rest / rpc. That also makes it a really nice tool to adopt in existing projects.
47 comments
[ 3.4 ms ] story [ 63.1 ms ] threadAlso the former technologies are local first in theory but without conflict resolution they can break down easily. This has been from my experience making mobile apps that need to be local first, which led me to using CRDTs for that use case.
I don't feel like I know all the answers, but as the creator of Replicache and Zero here is why I feel a pull to the web and not mobile:
- All the normal reasons the web is great – short feedback loop, no gatekeepers, etc. I just prefer to build for the web.
- The web is where desktop/productivity software happens. I want productivity software that is instant. The web has many, many advantages and is the undisputed home of desktop software now, but ever since we went to the web the interaction performance has tanked. The reason is because all software (including desktop) is client/server now and the latency shows up all over the place. I want to fix that, in particular.
- These systems require deep smarts on the client – they are essentially distributed databases, and they need to run that engine client-side. So there is the question of what language to implement in. You would think that C++/Rust -> WASM would be obvious but there are really significant downsides that pull people to doing more and more in the client's native language. So you often feel like you need to choose one of those native languages to start with. JS has the most reach. It's most at home on the desktop web, but it also reaches mobile via RN.
- For the same reason as prev, the complex productivity apps that are often targeted by sync engines are often themselves written using mainly web tech on mobile. Because they are complex client-side systems and they need to pick a single impl language.
But the web is primarily where a lot of productivity and collaboration happens; it’s also a more adversarial environment. Syncing state between tabs; dealing with storage eviction. That’s why local first is mostly web based.
How garbage the web has become for a low-latency click action being qualified as "impossibly fast". This is ridiculous.
My favorite so far is Triplit.dev (which can also be combined with TanStack DB); 2 more I like to explore are PowerSync and NextGraph. Also, the recent LocalFirst Conf has some great videos, currently watching the NextGraph one (https://www.youtube.com/watch?v=gaadDmZWIzE).
This architecture offers several advantages:
1. Data is stored locally, resulting in extremely fast software response times 2. Supports convenient full database export and import 3. Server-side logic is lightweight, requiring minimal performance overhead and development complexity, with all business logic implemented on the client 4. Simplified feature development, requiring only local logic operations
There are also some limitations:
1. Only suitable for text data storage; object storage services are recommended for images and large files 2. Synchronization-related code requires extra caution in development, as bugs could have serious consequences 3. Implementing collaborative features with end-to-end encryption is relatively complex
The technical architecture is designed as follows:
1. Built on the Loro CRDT open-source library, allowing me to focus on business logic development
2. Data processing flow: User operations trigger CRDT model updates, which export JSON state to update the UI. Simultaneously, data is written to the local database and synchronized with the server.
3. The local storage layer is abstracted through three unified interfaces (list, save, read), using platform-appropriate storage solutions: IndexedDB for browsers, file system for Electron desktop, and Capacitor Filesystem for iOS and Android.
4. Implemented end-to-end encryption and incremental synchronization. Before syncing, the system calculates differences based on server and client versions, encrypts data using AES before uploading. The server maintains a base version with its content and incremental patches between versions. When accumulated patches reach a certain size, the system uploads an encrypted full database as the new base version, keeping subsequent patches lightweight.
If you're interested in this project, please visit https://github.com/hamsterbase/tasks
Main problems I have are related to distribution and longevity -- as the article mentions, it only grows in data (which is not a big deal if most clients don't have to see that), and another thing I think is more important is that it's lacking good solutions for public indexes that change very often (you can in theory have a public readable list of ids). However, I recently spoke with Anselm, who said these things have solutions in the works.
All in all local-first benefits often come with a lot of costs that are not critical to most use cases (such as the need for much more state). But if Jazz figures out the main weaknesses it has compared to traditional central server solutions, it's basically a very good replacement for something like Firebase's Firestore in just about every regard.
First I used PouchDB which is also awesome https://pouchdb.com/ but now switched to SQLite and Turso https://turso.tech/ which seems to fit my needs much better.
For instance one closes an something and another aborts the same thing.
My use case is scoring live events that may or may not have Internet connection. So normal usage is a single person but sometimes it would be nice to allow for multi person scoring without relying on centralized infrastructure.
Described here https://blog-doe.pages.dev/p/my-front-end-state-management-a...
I've already made improvements to that approach. decoupling of backend and front end actually feels like you're reducing complexity.
Most of the solutions with 2 way sync I see work great in simple rest and hobby "Todo app" projects. Start adding permissions and evolving business logic, migrations, growing product and such, and I can't see how they can hold up for very long.
Electric gives you the sync for reads with their "views", but all writes still happen normally through your existing api / rest / rpc. That also makes it a really nice tool to adopt in existing projects.