Show HN: Triplit – Open-source syncing database that runs on server and client (github.com)
(npm|bun|yarn) create triplit-app
We finally decided to build the system we always wanted. Triplit enables your app to work offline and sync in real-time over websockets with an enjoyable developer experience.
Triplit lets you (1) define your schema in Typescript and simply push to the server without writing migration files; (2) write queries that automatically update in real-time to both remote changes from the server and optimistic local mutations on the client—with complete Typescript types; (3) run the whole stack locally without having to run a bunch of Docker containers.
One interesting challenge of building a system like this is enabling partial replication and incremental query evaluation. In order to make loading times as fast as possible, Triplit will only fetch the minimal required data from the server to fulfill a specific query and then send granular updates to that client. This differs from other systems which either sync all of a user’s data (too slow for web apps) or repeatedly fetch the query to simulate a subscription (which bogs down your database and network bandwidth).
If you’re familiar with the complexity of cache-invalidation and syncing, you’ll know that Triplit is operating firmly in the distributed systems space. We did a lot of research and settled on a local first approach that uses a fairly simple CRDT (conflict-free replicated data type) that allows each client to work offline and guarantees that they will converge to a consistent state when syncing. It works by treating each attribute of an entity as a last writer wins register. Compared to more complex strategies, this approach ends up being faster and doesn’t require additional logic to handle conflicting edits between concurrent writers. It’s similar to the strategy Figma uses for their collaborative editor.
You can add Triplit to an existing project by installing the client NPM package. You may self-host the Triplit Server or pay us to manage an instance for you. One cool part is that whether you choose to self-host or deploy on Triplit Cloud, you can still use our Dashboard to configure your database or interactively manage your data in the Triplit Console, a spreadsheet-like GUI.
In the future, we plan to add APIs for authentication, file uploads, and presence to create a Supabase/Firebase-like experience.
You can get started by going to https://triplit.dev or find us on Github https://github.com/aspen-cloud/triplit. Thanks for checking us out and we are looking forward to your feedback in the comments!
105 comments
[ 3.9 ms ] story [ 230 ms ] threadIt's actually running two clients and the server inside the browser window so even though it's simulating real-world, it's the actual code that powers Triplit.
However, when it comes to collaborative text you're absolutely right and support is on our roadmap. A researcher in this space has already implemented his collaborative sequence library on top of Triplit with the Quill text editor and you can check it out here: https://github.com/mweidner037/list-positions-demos/tree/mas...
But also I’m getting old, and I had the same feeling when RethinkDB came out. Do you guys have any thoughts on how your system compares to what they were doing and what eventually happened to them?
In terms of scaling to a million-ops per day we are probably not quite there yet but one advantage to having the server be the authority is that, in the future, the Triplit server could track the timestamps that each client last synced at and progressively prune the history similar to how Postgres handles VACUUM'ing dead tuples.
I'd recommend reading this: https://drewdevault.com/2020/07/27/Anti-AGPL-propaganda.html
AGPL is based on GPL, not LGPL, and like the GPL linking would be considered creating a derivative work which should then be made available as per the [A]GPL licence.
This won't stop me licensing my work using AGPL. If commercial interests don't want to use it then I'm fine with that¹. If they do want to use my stuff but don't want to use anything covered by AGPL then they can always contact me to try negotiate a different arrangement.
--------
[0] IANAL, hence “might be”. Whether loose coupling counts the same as linking is a grey area IIRC. This potential confusion is why some avoid GPL/AGPL, if only to be on the safe side from their PoV.
[1] Give the choice between “give us what we want with no come back and we'll use what we want and give nothing back” and “don't give us what we want, and we'll go away and leave you alone”, I'll take option two. I get nothing either way.²
[2] As the linked article says: “If you don’t use the AGPL[³], they’re just going to take your software and give nothing back. If you do use the AGPL, they’re just going to develop a solution in-house. There’s no outcome where Google pays you.”
[3] Here they are comparing AGPL to less restrictive licenses such as MIT.
This seems less fitting if it's true that what you're "giving" is some React bindings, and what they have to "give back" is their entire frontend application.
They could also try negotiate an alternate arrangement, some authors may be willing to do that (assuming they have relevant permission from other contributors if it isn't a solo project, of course).
The example you give seems to be the unreasonable one IMO: “our thing is bigger, so we expect to use your smaller thing how we want”.
Either way your interpretation is not our intention so I'll make sure that's clarified.
The license [1] says this in section 5:
> 5. Conveying Modified Source Versions.
> You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:
> [...]
> c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
If you intended it to be non-viral (weak copyleft) and based on distributing modified versions of your files, the Mozilla Public License [2] may be a better fit for the frontend libraries. Then people could use them unmodified in software with any license, but they would have to release source code of modified versions of those files. This would not do what you want for the server software though, as the "release source code" only applies when distributing software (which is not happening if it's staying on the server).
I am not an expert, and it is not clear to me whether the frontend and backend constitute a single work according to the license. This is important to investigate. If they did count as a single work, rather than two separate works communicating, then the AGPL from the server software would apply to the client software and licensing the frontend libraries MPL wouldn't work. This says something about that in the other direction: https://www.gnu.org/licenses/gpl-faq.html#AGPLv3ServerAsUser
[0] https://en.wikipedia.org/wiki/Copyleft#Strong_and_weak_copyl...
[1] https://www.gnu.org/licenses/agpl-3.0.html#license-text
[2] https://en.wikipedia.org/wiki/Mozilla_Public_License
This always comes up with copyleft licenses. It's rarely 100% clear what's allowed, except maybe the LGPL where the intent of the license is obvious. Everything else is some amount of a viral license, and it's safest to just avoid it.
It's like saying that Oracle owns my product because I use MySQL.
The difference would be if your code used a library that was GPL/AGPL. For server-side software, under the GPL, mere users wouldn't be entitled to the code, while under the AGPL they would be.
This also assumes that library was used in a way that counts as derivative work, which isn't always the case.
That's something a court would have to decide.
Whenever the AGPL comes up, people start arguing that in their opinion it doesn't cover certain things and therefore criticism of the license is fear mongering, but this misses the point that even if there is room to argue for that position, no company is going to want to touch AGPL software when the situation is ambiguous, since that creates a lot of risk, so that ambiguity is itself a fatal flaw of the AGPL license.
> Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software.
The AGPL doesn't really require this though. It requires the source code to be distributed to users, it does not require anyone to contribute those modifications back.
Is my understanding incorrect?
I guess there's the argument that transpiled, minified javascript program is a compiled binary? I'd love to know whether these things have been tested in various courts.
That's the context where LGPL is used most often, but the license itself is more general than that. In particular 4/d/0 in [1].
AFAIK, LGPL is about replaceability. If you distribute a compiled program (yes, a minified JS program should qualify) then you need to distribute the source of the LGPL part and document a way how the user can replace that LGPL part in your program with their own modified copy.
[1] https://www.gnu.org/licenses/lgpl-3.0.en.html
People are trying to have their cake and eat it too with open-source DBs (and it's mostly DBs). Historically, OSS was either for anyone for anything, or it was a viral license trying to build a GNU ecosystem. The DBs playing licensing games have been using OSS as a growth hack, they doing a bait-and-switch once they're popular.
At least starting with the AGPL up-front is honest, but considering the client-server nature of this DB, it makes it a non-starter for me.
I agree with you w/r/t "Why build your own query builder" though since (AFAIK) you can't touch the underlying SQL. I'm making heavy usage of FTS5 in my app, and it doesn't look like they support that.
And, to be fair, feathers/rethink also have the weaknesses you list:
- Why build your own query builder
- This also isn't really something you could drop into an existing project very easily
Personally, I've been pretty happy with https://github.com/vlcn-io/cr-sqlite/ which is just a sqlite extension that can run in the browser.
Regarding compatibility with our databases, we do have some internal experiments of running a Triplit Server in front of existing databases as almost like a syncing cache. Still very experimental but maybe you'd be more into that use case.
1. https://github.com/aspen-cloud/tuple-database
What's the reason for the fork?
If they don't they're not a good match at all.
0. https://tanstack.com/query/latest/docs/framework/react/quick...
Having a complete query engine on the client makes this a lot easier to deal with. It's why most popular messaging apps will use SQLite on the client to render the UI and build their own sync engine to keep it up to date with the backend.
Do you have a specific use case in mind?
Overall triplit has been really great, both as a frontend dx and also their support - whenever we find an issue or have a feature it gets handled very quickly by the team which is awesome!
As soon as they have an answer for HA deployments we will be moving more critical data there instead of Postgres
So with all that, my question is, is there a reason you guys went with more of a full language level solution rather than being more agnostic to the client and server? It seems harder to support other languages and frameworks other than JS based ones in the future, but I suppose the market for that is already big enough. ElectricSQL etc also have SDKs for TypeScript as well as Flutter and others so they are similar to your solution but it seems like they can support more clients and servers in the future just by building SDKs for them.
Another question, looks like you eventually want to compete with Supabase but they are already experimenting with database level syncing and CRDTs in Postgres [2] and might catch up with your solution, any thoughts on that?
[0] https://www.youtube.com/playlist?list=PLTbD2QA-VMnXFsLbuPGz1...
[1] https://localfirstweb.dev/
[2] https://news.ycombinator.com/item?id=33931971
Re: Flutter (and other native support) it's something we've thought a lot about (Flutter specifically comes up often) but we decided to focus on pure-Typescript to start because it's a big enough market (I'm personally a big believer in the future of PWA's) and we know we can make the best experience by focusing on it. I do think that we'll eventually make something more platform agnostic but not sure when that will happen.
Re: ElectricSQL & Supabase: both teams are obviously very talented and thoughtful and, I believe, will continue growing in the SQL space which is the most fundamental difference between our approaches. I think Triplit can make the best experience for developers by avoiding SQL and think there's undoubtedly room for both philosophies.
I'm looking through the docs for more info on this line:
> The server supports different storage adapters, such as SQLite
What are the other storage adapters? I may just be blind, so if so please forgive me!
- Good sane query language (not SQL) - Great typescript support - Offline support - React native support
The cherries on top is that it's open source and self-hostable.
Congrats on the launch and thanks for the awesome product! I've been using Triplit in one of my projects [1], and it do work as expected. In my self-promotion on Reddit [2], I posted about Triplit as well:
> I think Triplit is a nice database and works as expected. It's data model fits well with what I have in mind (more decentralized/P2P instead of having a single centralized database as the source of truth), but there are 2 areas I find lacking:
> - Server side/self-hosted: Triplit server requires a token for authentication. Triplit's documentation has a section about how to start the server without explicitly showing how to generate the token, which is mildly inconvenient. Therefore, on self-hosting the server, I opted for using their CLI command dev since the command has the token generation that I needed. I know it is not a good security practice as when the command is used as a system service, the token will be logged in plain text, but I have a bigger problem when someone can access that anyway.
> - Query language: I find their custom query DSL not as expressive as a full-fledged query language (lacking UNIQUE and COUNT like SQL is on the top of my mind). You'll have to do a bit of data aggregating yourself.
Recently, I found Evolu [3], which is quite similar to your project in terms of scope and functionalities as well. From a quick skim of their documentation, I think the differences are:
- Triplit have `.subscribe()`, while Evolu don't
- Evolu's querying is more familiar/advanced (typed SQL via Kysely)
- In the browser, Evolu seems to use SQLite on top of OPFS, while Triplit uses IndexedDB
I think there are more intricacies on the way Triplit differs from Evolu, so can you enlighten me on that?
Really appreciate your comment! Thanks!
- [1]: https://github.com/thanhnguyen2187/cryptaa
- [2]: https://www.reddit.com/r/sveltejs/comments/1dndpj8/cryptaa_a...
- [3]: https://www.evolu.dev/docs
on self-hosting: We're cleaning up the docs on self-hosting to make the configuration clearer, thanks for point that out.
on querying: Yeah we don't have aggregations yet but it's on our roadmap. Don't want to over promise but I think we can make something awesome here by leverage our incremental querying engine. Like consider a data dashboard that needs to be updated every hour; in a traditional system (postgres, mongo, etc) you would need to rerun the query from scratch each time. Our plan is to create something closer to what Materialize does and just process the new data so it's much more efficient and can just update continuously rather than every hour.
re Evolu: I haven't actually gotten a change to try it out but there might be in someone in our Discord[1] that has that could compare/contrast
1. https://triplit.dev/discord
My context here is having worked in the past on a mobile health app, and recalling all the pain we had around this problem.
Also it's important to have a table that is used to coordinate latest version. If you make a breaking change have the client that is behind to ask the user to upgrade. You can also tie this into how long you keep the dual write/read around with a min version supported across all clients.
However, overtime this can naturally lead to a mess of a schema definition that has a lot of confusing names. We haven't released a solution to fix this yet but we're working on a few things that should make this less painful. For background on the various approaches, the Cambria doc is an amazing resource: https://www.inkandswitch.com/cambria/
I have the same questions about Supabase and Firestore so it seems like I'm missing something.
In enterprise this is obviously the opposite way around. It's actually super frustrating to watch discussions that ignore this, it's a really big problem on tech twitter etc especially, where people advocate for certain stacks or ways or working or whatever that make it clear as day they've never had to build a business system, they've just been building CRUDs, so they're incapable of understanding why experienced devs who have disagree with them.
The client can fire requests to Supabase, but Supabase runs additional queries on the back-end that determine whether an incoming request is allowed or not.
A simple example would be that a client is only allowed to read, write or update a row if the value in the UserID column is the same as the requesting authenticated user's.
I'd like to mention the Meteor.js framework (https://www.meteor.com/) too, which is in a bit of a transitioning phase right now to Meteor version 3, but is a really amazing full-stack app building solution I and many others have been working with for ~10 years.
It has a lot of batteries included, build system, pluggable frontend frameworks, lots of libraries, based on node.js, meaning it's pretty much compatible with the Node.js environment.
It's based on a really pretty simple syncing strategy for years:
It's original client side data provisioning layer is based on having
a) MongoDB on the Server and b) a javascript-native implementation of a subset of MongoDB in the Client.
Using a publish / subscribe mechanism the client can subscribe to the subset of data relevant to his current view, eg. dependent on the current user & view.
Updating is theoretically possible by using a syncing mechanism via writes into the client database and an elaborate allow/deny mechanism, but in practice most people use the following simple workflow:
Meteor also provides a method-call-mechanism by which a client can call the server to do (or fetch) stuff. So that's basically RPC in a very simple but powerful format.
These methods also allow for "client side simulations", optimistic updates to the client side database, with the changes getting rolled back / updated once the server part of the method has done it's updating of the database.
So the workflow for working & updating data in the DB looks like this:
- Data is "canonical" on the server DB
- Client subscribes for the necessary data for its client side cache
- When the user triggers an action, the client calls a method on the server, eg: "likePost(post_id)"
- The client syncs its database after the method has completed to make sure its optimistic update lead to the same results as the server call did on the server.- The client updates its UI always reactively as soon as the data in its local db has changed.
All of this is very performant as long as you keep it cool and subscribe only what's actually needed for the current view,
Oh, and all of this has fine grained reactivity, on the client. The whole solution is really powerful, while deceptively simple.
Having the same API on both client and server allows for "isometric" code, meaning code which runs on both the client and the server, so you don't have to have two different versions of helper code, which is really cool too.
Meteor.js is pretty much a bit like "the old PHP experience" to me: As a full Stack developer I can write powerful apps with only one codebase which is shared in parts between client and server.
Link to Pub/Sub docs: https://docs.meteor.com/api/pubsub Link to the Method docs: https://docs.meteor.com/api/methods