Show HN: InstantDB – A Modern Firebase (github.com)
Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can make it difficult to build delightful software. Could it be better?
We were senior and staff engineers at Facebook and Airbnb and had been thinking about this problem for years. In 2021, Stopa wrote an essay talking about how these schleps are actually database problems in disguise [1]. In 2022, Stopa wrote another essay sketching out a solution with a Firebase-like database with support for relations [2]. In the last two years we got the backing of James Tamplin (CEO of Firebase), became a team of 5 engineers, pushed almost ~2k commits, and today became open source.
Making a chat app in Instant is as simple as
function Chat() {
// 1. Read
const { isLoading, error, data } = useQuery({
messages: {},
});
// 2. Write
const addMessage = (message) => {
transact(tx.messages[id()].update(message));
}
// 3. Render!
return <UI data={data} onAdd={addMessage} />
}
Under the hood we save data to postgres as triples and wrote a datalog engine for fetching data [3]. We don’t expect you to write datalog queries so we wrote a graphql-like query language that doesn’t require any build step.
Taking inspiration from Asana’s WorldStore and Figma’s LiveGraph, we tail postgres’ WAL to detect novelty and use last-write-win semantics to handle conflicts [4][5]. We also handle websocket connections and persist data to IndexDB on web and AsyncStorage for React Native, giving you multiplayer and offline mode for free.
This is the kind of infrastructure Linear uses to power their sync and build better features faster [6]. Instant gives you this infrastructure so you can focus on what’s important: building a great UX for your users, and doing it quickly. We have auth, permissions, and a dashboard with a suite tools for you to explore and manage your data. We also support ephemeral capabilities like presence (e.g. sharing cursors) and broadcast (e.g. live reactions) [7][8].
We have a free hosted solution where we don’t pause projects, we don’t limit the number of active applications, and we have no restrictions for commercial use. We can do this because our architecture doesn’t require spinning up a separate servers for each app. When you’re ready to grow, we have paid plans that scale with you. And of course you can self host both the backend and the dashboard tools on your own.
Give us a spin today at https://instantdb.com/tutorial and see our code at https://github.com/instantdb/instant
We love feedback :)
[1] https://www.instantdb.com/essays/db_browser
[2] https://www.instantdb.com/essays/next_firebase
[3] https://www.i...
305 comments
[ 4.9 ms ] story [ 282 ms ] threadPocketbase (at a very high level) takes sqlite and layers on a go ORM and a relatively loose (on the scale from typical rest/http on one end to graphql on the other) autogenerated REST api, a grammar for expressing authz constraints, websocket updates and an embedded js engine for registering handlers for the various crud model and webserver events.
It's _very_ well designed, but I think the approach taken here with datalog and triples on top of postgres is worth a close look as the trade-offs will be very different, though it does seem like a lot of layers
With Instant, you get a relational Firebase.
[1] https://www.instantdb.com/essays/next_firebase#firebase
I realise that this is probably a symptom of "holding it wrong" and not embracing denormalization, but it was always present and also horrifically inefficient w.r.t to firebases pricing due to the amplifying effect it had on reads - none of those projects would've been practical to take to market without reworking that significantly.
Really what I wanted was a relational database that also did the fun/flashy real time updates without the heavy lifting. At face value it sounds like you're offering exactly what I wanted so I look forward to giving it a try next time!
I'm not sure about how things grow from here in terms of larger aggregates and more complex queries though so am slightly worried I'm painting myself into a corner. Do you have any guides or pointers here? Or key areas people shouldn't use your db?
My gym uses griptonite, but it's so slow I feel like it'd be quicker overall to create my own app for logging things. I decided to use instantdb as a backend database and it's working nicely so far.
My gym uses griptonite which is fine except there's an ad for their premium thing every time I open it and it's extremely slow. Also either due to the gym or griptonite not all the climbs get added, and not right away. It's frustrating hitting a new grade and then not being able to record it, as then I've got a log saying I have maxed out somewhere I haven't.
The main realisation I had was that griptonite tags are IDs, I don't actually need any griptonite data or to get the gym to sign up for anything to start tracking my own climbs. I can just scan the tags as they are. I'm also making it require a photo of the climb as I'm tired of trying to talk with my wife about which ones we were working on "the pink, next to the green that's orange? With the weird hold? Oh the round one?" and would like just a picture to point at.
I'm planning to do a few things like add live customisable leaderboards, which should be easy with instantdb, mostly for fun. Maybe I'll throw on a few affiliate things on top, otherwise I expect this should be lightweight enough it won't really cost me anything to run.
Also my background is AI stuff and I'd like to try things like using SAM to pull out the specific climb from a messy image, classify holds to say whether climbs look "crimpy" or "juggy", find similar climbs by image (recommend next ones to try) etc.
I'd love to see a smart Hold that integrates with an app where the gym cann associate holds/grips in a DB inventory - and pull them out and assign them to a boulder/wall & route. The smart holds have simple pressure sensors for knowing when they are gripped, and for how long. Advanced ones measure force/weight.
Just walk in scan the boulder's code to slurp in all the holds to the local table and then you have them all ping back to the app with every touch. and map your route and exactly telemetry applied using route. So it shows you the wall and the grips you hit as you went up and how much energy and time spend on each node.
Report climbing timings and accomplishemnets on the Gym leaderboard... have challenges - subscribe to a wall/route/buolder and set a threashold to show you when anyone beats your time etc etc
Strava for the actual climb. (The gym could then have a db of all the positions fastener locations in the facility, and then simply assign a hold to a given slot - each hold has a json of its Spec-Sheet for hold type, rating, syle/whatever metrics climbing geeks geek out on, I mean get a grip folks)
Anyway - it would allow for the local climbing app to them maintain a db history of every route and grip and difficulty you've actually touched. Rate the hold-types, routes, wall, boulder - and have that realtime update across al clients subscribed to a wall or gym. A gym can post new routes and have your app subscribers just get the new table for that route and what grips and their profiles each have.)
Then (the not eco friendly version) is epoxy a BLE tag to a routes holds in the wild and you can have the app point at the climb and ping all the BLEs and learn the route - and have ClimbGps - and it can run it locally with a dynamic table of the BLEs it can sense as checkpoints as you pass them (obviously only timing data could be saved unless you do RSSI/something to get vertical and horizontal pathing by triangulating ...
(built these features in previous varying capacities for different things - but climbing would be fun to do it with as well.
I am sure there are already smart grips out there... but this rappelled into my thoughts like Tom Cruise as soon as I read your post.
https://old.reddit.com/r/Spline3D/
Can see a simple route design widget in spline that can be a nifty ux for a climbing thing that uses the app ID.
Can you clone an Instant AppID?
I know I am just thinking out loud - but its pretty easy to envision how to use these neato tools and services that have been cropping up recently... Its inspiring.
https://www.youtube.com/watch?v=4pTy1TB-2z8
And we did spatial RFID for warehouse (cannabis) tracking with Z-Slot level:
https://www.youtube.com/watch?v=FXkNY0vTATA
https://www.youtube.com/watch?v=c7qvfm6vhF0
But the thing I was focusing on WRT Instant would be the ability to "Clone an AppIS" -- where when you fork an AppID, it dupes the tables into your personal account...
I havent fleshed out the thoughts on this too much because I am not sure if you can easily do that with Instant, as its a hosted thing....
(YC Rejected these) :-)
> larger aggregates and more complex queries
Currently Instant supports nested queries, pagination, IN, AND, and OR. We have an internal implementation for COUNT [1], but need to update permissions for aggregates.
We're always hacking away on blocker features. If we can't get to it in time and it blocks your app, you can reach to the admin SDK for an escape hatch [2]
[1] The 'admin-only' count starts here: https://github.com/jsventures/instant/blob/main/server/src/i...
[2] https://www.instantdb.com/docs/backend
Btw I guess you created a new repo for the public release, so https://github.com/instantdb/instant/blob/main/server/src/in... is the link for others.
Great to see the open source release, congrats! : )
Good luck, Joe, Stopa and team!
It was jarring to find out that indexes are required for every combination of filters your app applies, but then you quickly realize that Firebase solves a particular problem and you're attempted to shoehorn into a problem-space better solved by something like Supabase.
It's not too dissimilar to DynamoDB vs RDB.
Seems the biggest problem is that Firebase doesn't have relations. How can you query that which does not exist?
I'm guessing what they really want is SQL? Once upon a time when I was stuck on a Firebase project I built a SQL (subset) engine for Firebase to gain that myself, so I expect that is it.
But I can’t.
Grabbing all the docs in the db into my controller and filtering down that array is what Firebase makes me do instead of writing queries.
Why not? You're asking a question, of sorts, and getting an answer from the result. That's the literal definition of querying.
> Grabbing all the docs in the db into my controller and filtering down that array is what Firebase makes me do instead of writing queries.
A query language is just an abstraction. One you can have in your code. At some point you still need to "grab all the docs into a controller and filter them down", though. You could push that step into the Firebase service, but it would still have to do the same thing you're doing. There is no magic.
Better would be to provide something indexable so that you don't have to go through all the docs, but you can't index that which does not exist.
It's really the definition of an managed database/datastore.
Do you see InstantDB as a drop in replacement ?
To be honest I don't want to have to worry about my backend. I want a place to effectively drop JSON docs and retract them later.
This is more than enough for a hobbyist project, though I imagine at scale things get might not work as well.
If you wanted to store documents, you could write:
```
useQuery({docs: {}}) // get documents
transact(tx.docs[docId].update({someKey: someValue}); // update keys in a doc
transact(tx.docs[docId].delete()) // delete the doc
```
2 questions.
How hard is it to swap our firebase for instant? I've had an amazing time with firebase, but I sorta want to switch to using a completely local solution.
I have a small lyric video generator, and while I don't care about my own songs potentially leaking, I would never want to take responsibility for something else's data. I basically use firebase for the lyrics afterwards I transcribe them .
Second, do you offer your own auth or just integrate with other solutions.
> How hard is it to swap our firebase for instant? I've had an amazing time with firebase, but I sorta want to switch to using a completely local solution.
It should be relatively straightforward to switch. If you have any questions, you could always reach to us on Discord [1]
The only caveat though: Instant is like Firebase; it is not a completely local solution. If you are worried about exposing some data over the internet, I would store the same kind of stuff you were thinking about with Firebase.
> Second, do you offer your own auth or just integrate with other solutions.
We offer our own auth. You have magic code emails and Google Sign in out of the box. We also expose auth functions in admin SDK, in case you want to create a custom solution. [2]
[1] https://discord.com/invite/VU53p7uQcE [2] https://www.instantdb.com/docs/auth
Thank you for your help. I'll definitely look into this for my next project
What does this mean exactly? If you host your own it is still not local?
Made me feel quite old that Firebase is no longer "modern" though...
One thing I discovered in the process is that the word water comes to english all the way from Proto Indo European. The word hydro, however, comes from ancient greek, which comes from the same PIE word for water.
From where I am, it is simply telly, because why not bastardise it some more while we are at it.
Sounds like someone made up the name to just sound better than trifecta. It's marketing speak.
Also, as the link says, it has been used to mean more than four. And other languages use their own equivalent of "quadfecta" instead.
Plus, I knew exactly what "quadfecta" meant, but would have no idea about "superfecta".
In the docs[1]:
> Instant uses a declarative syntax for querying. It's like GraphQL without the configuration.
Would you be interested in elaborating more about this decision/design?
[1] https://www.instantdb.com/docs/instaql
Our initial intuition was to expose a language like SQL in the frontend.
We decided against this approach for 3 reasons:
1. Adding SQL would mean we would have to bundle SQLite, which would add a few hundred kilobytes to a bundle
2. SQL itself has a large spec, and would be difficult to make reactive
3. What's worst: most of the time on the frontend you want to make tree-like queries (users -> posts -> comments). Writing queries that like that is relatively difficult in SQL [1]
We wanted a language that felt intuitive on the frontend. We ended up gravitating towards something like GraphQL. But then, why not use GraphQL itself? Mainly because it's a separate syntax from javascript.
We wanted to use data structures instead of strings when writing apps. Datastructures let you manipulate and build new queries.
For example, if you are making a table with filters, you could manipulate the query to include the filters. [2]
So we thought: what if you could express GraphQL as javascript objects?
``` { users: { posts: { comments: { } } } ```
This made frontend queries intuitive, and you can 'generate' these objects programatically.
For more info about this, we wrote an essay about the initial design journey here: https://www.instantdb.com/essays/next_firebase
[1] We wrote the language choice here: https://www.instantdb.com/essays/next_firebase#language
[2] We programatically generate queries for the Instant Explorer itself: https://github.com/instantdb/instant/blob/main/client/www/li...
I think this looks like, a backend-end-in-box type of product? So that you just have to focus on front end mostly?
Could be cool for early stage projects.
I appreciate that you're thinking about relational data and about permissions. I've seen a bunch of sync engine projects that don't have a good story for those things.
imo, the more that you can make the ORM feel like ActiveRecord, the better.
can you elaborate more on how you achieve this
For example, given a query like "fetch user where id = 1", there could be a topic like "users:id:1" [1]
When a WAL record comes in, we find queries by their topics, and invalidate them. This triggers a refresh.
This is inspired by Figma's LiveGraph [2], which in turn is inspired by Asana's Luna [3]. The essays cover the idea in more detail, but you can also start diving into the code, at invalidator.clj [4]
[1] This is an example. Inside Instant, we encode them as a 'datalog' pattern. Something like `[:ea "1" :users/id]` [2] https://blog.asana.com/2020/09/worldstore-distributed-cachin... [3] https://www.figma.com/blog/livegraph-real-time-data-fetching... [4] https://github.com/instantdb/instant/blob/main/server/src/in...
1. first we would use the WAL records to invalidate queries that could be affected (with optimizations for fast matching) and requery the data
2. then we used the info from the WAL record to update the query in-memory without asking the DB for the new result, it worked for majority of the queries that can be reliably modeled outside of the DB
3. I believe after I left the team reverted to the re-query approach, as managing a system that replicates the DB behavior was not something they were excited to maintain, and as the DB layer got scaled out, extra DB queries were less of a downside
I'll reach out to you on twitter; would love to learn more about your experience
You can learn more about the rules language in the permission docs: https://www.instantdb.com/docs/permissions
[1] https://github.com/google/cel-java
[1] https://www.instantdb.com/essays/next_firebase#the-missing-c...
[1] https://www.instantdb.com/essays/next_firebase#supabase-hasu...
Interestingly the team at electricsql are now rewriting their solution because it didn’t scale and was too complex https://next.electric-sql.com/about
> 1. the usual "vendor locked in". Is there a recommended escape hatch?
Instant is completely open source. We have no private repos, so in the event that you want to run the system yourself, you can fork it.
> 2. any big clients on this yet or at what scale do you expect people to start rolling their in house product
We have startups in production using us today. We would love to learn more about your use case. You can reach out to us directly at founders@instantdb.com
It was so fast I was able to build basic collision physics of letter tiles and have their positions sync to multiple clients at once. What a shame to be killed by Google.
I haven't had a need for real time databasing since, but this is inspiring me to build another collaborative app.
One of the things I find quite nice about firebase is the quite powerful separation between the logic of data retrieval / update and the enforcement of access policy -- if you understand it you can build the prototype on a happy path with barely any authorization enforcement and then add it later and have quite complete confidence that you aren't leaking data between users or allowing them to change something they shouldn't be able to. Although you do need to keep the way this system works in mind as you build and I have found that developers often don't really grasp the shape of these mechanisms at first
From what I can tell -- the instant system is different in that the permission logic is evaluated on the results of queries -- vs firebase which enforces whether the query is safe to run prior to it even being executed ...
We built a permission system on top of Google's CEL [1]. Every object returned in a query is filtered by a 'view' rule. Similarly, every modification of an object goes through a 'create/update/delete' rule.
The docs: https://www.instantdb.com/docs/permissions
The experience is similar to Firebase in three ways:
1. Both languages are based on CEL 2. There's a distinct separation between data retrieval and access policy 3. You can start on a happy path when developing, and lock down later.
AFAIK, Firebase Realtime can be more efficient, as it can tell if a permission check has passed statically. I am not sure if Firestore works this way. We wanted to be more dynamic, to support more nuanced rules down the road (stuff like 'check this http endpoint if an object has permissions'). We took inspiration Facebook's 'EntPrivacy' rules in this respect.
Is that efficient for queries that return many rows but each user only has access to a few?
Is there a specific reason to not use something like postgresql RLS that would do the filtering within the database where indexes can help?
Not being able to leverage permission rules to optimize queries (predicate pushdown) seems like too big a compromise to me. It would be too easy to hit pathological cases, and the workaround would probably be something akin to replicating the permission logic in every query. Is there any plans to improve this?
Firestore's rules are also able to prove before the query runs if the query will only return data that the user has access to according to the rules. That's a pretty important property that "rules aren't filters" because it prevents bad actors from DDOSing your system. My former colleague wrote about this: https://medium.com/firebase-developers/what-does-it-mean-tha...
I’ve built a prototype of a full stack using Flask + SocketIO + SQLite as well as an iOS client to prove the concept to the VPs.
How well do you think this can scale? Any plans to make native SDKs for iOS and Android?
> How well do you think this can scale.
We have startups in production using Instant. It could hiccup if you get a big spike of traffic, but all of us will be on deck to fix it.
> Any plans to make native SDKs for iOS and Android?
It's definitely on the roadmap, but we want to get the Javascript / React Native experience really right first.
> Currently we have SDKs for Javascript, React, and React Native.
[1] https://www.instantdb.com/essays/next_firebase#the-missing-c...
Congrats team!
[1] https://mitchellh.com/writing/zig-and-swiftui
Right now because our client SDKs are in JS, we can share much of the logic between React Native, React, and vanilla JS.
If we were to add native client SDKs, we'd have to duplicate the logic across different languages. One alternative path could be to write some shared code in a core language.
This idea is not set in stone at all though, I just enjoyed Mitchell's experience report.