After reading the landing page, I have no idea what I would use this for or what a tenant means. How do I manage my schema? How do I do migrations? Is this thing running all the time? What's the tech stack underneath?
Nile CEO here. Thanks for the feedback. Let me try to explain it a bit.
Modern SaaS applications are multi-tenant. We’re the first database that virtualizes tenants into the database. A tenant is primarily a company, an organization, or a workspace in your product that contains a group of users. This enables seamless tenant isolation, per-tenant backups, and placement on multi-tenant or dedicated infrastructure, anywhere on the planet. You can do all this with the experience of a single Postgres! You don’t have to manage multiple databases, build complex permissions for isolation, or write buggy scripts to read specific tenant data from backups. On top of the tenant model, we provide opt-in user management capabilities, customer-specific vector embeddings, and instant tenant admin dashboards.
You manage schemas like how you manage with standard Postgres. Your favorite tooling should work. When you make the schema change, Nile pushes the schema to all the tenants. For the user, it is like interfacing with a single Postgres instance.
We have built a Rust based layer internally to manage tenants, route them, shard them if needed and have multiple tenants talk to each other. Happy to chat more. My email is ram@thenile.dev
This is true in a sense “most modern SaaS apps have at least two unrelated users or user groups” (forgive me please if I’m wrong here, but that’s how I loosely understand “multi-tenant”), but it’s not exactly clear why this is important.
I strongly suspect that the majority of modern SaaSes don’t do any database-level tenant isolation or per-tenant backups. The smaller the company the less it’s likely. Row-level security is a very rare sight to behold, even in regulated areas (certifications like HIPAA compliance don’t really require this). The best they do if they’re really global (I’m sure most are only global in “we accept users from anywhere”, but not so much under the hood - even though cloudy sales pitches love to paint a different picture, most SaaSes I’ve seen were one or two-region deployments) is try to move/shard data closer to the the user’s region.
I think one concrete example (as simple as possible) would help a lot.
I think it makes more sense if you think of it in terms of "B2B" SaaS applications. Think selling a subscription to a medium-sized company, all in one geographical location, where data is shared between all users of that organization, and very carefully not shared outside of that group. For a more concrete example, think Salesforce.
I'm not sure that I would generalize that to all modern SaaS applications though.
Nile CEO here. Here are a few use cases that can help you. We believe SaaS is 95% of new companies today and it is very broadly applicable. Happy to chat further if interested
1. GitHub helps a group of developers manage and deploy their code. Each Github organization is a tenant, and the developers within those organizations are users.
2. Salesforce or Hubspot helps sales reps manage their leads. Each company is a tenant, and the users are the sales reps.
3. Ring is a home security company that provides alarm services to different households. Each household is a tenant, and people living in a house using their service are users.
4. Toast is a platform to help build software for restaurants. The restaurants are tenants, and the employees of the restaurants are the users.
> 1. GitHub helps a group of developers manage and deploy their code. Each Github organization is a tenant, and the developers within those organizations are users.
Nope, Github users exist outside of organizations in a global namespace, and can be invited to be members of multiple organizations.
Yes, that is correct. In Nile, users are shared tables. They can belong to multiple tenants. In the case of the Github example, the organization is a tenant and users will be a shared table. Nile provides this out of the box. Will fix the usecase example to be more clear.
We don't have per-tenant schema right now, but we have started designing multi-version experience and it doesn't seem too far from here.
Nile is different from DB per tenant in a single PG cluster in that it is much lower overhead per tenant. Since we offer a service, it will translate to lower cost.
The DB-per-tenant gets painful after the first few 100 tenants, but some SaaS have thousands of tenants, each with very low activity. Our model makes inactive tenants nearly free.
In addition, we also provide the "unified view", which lets you query all your tenants as if they were in the same DB. So you can connect as an admin and "select * from my_table" will give you the results for all tenants. Can be pretty handy.
What are you thinking will happen when a SaaS grows like Salesforce, with a very large number of tenants, where some of the tenants have very large databases? What will Nile do when the database just can't fit in a single Postgres cluster?
Edit: Never mind, I see from your web site that moving tenants between databases is one of the main things you try to address.
It is a Postgres, so schema and migrations work exactly as they would elsewhere.
Tenants are your customers. It is normal for SaaS to have either a schema per tenant or one schema and tenant_id column in most tables. Nile provides you with both experiences at the same time.
At first pass, I'm impressed with the multi region/tenant built-in support. I take it that changing a region is as changing the table row key. I wonder though about this strategy strictly place user's data as an entry into the data store... what if my users travel a lot between regions? Is the best strategy that upon user login to update their region in all their related tables?
If they travel a lot, you may want the data in one location (although it still depends, moving data is often faster than flying to a different region).
Tenant placement is especially great if their location has data residency requirements. We discovered that companies with many EU customers need this.
Nile CEO here. Nile provides tenant virtualization. Each tenant has a group of users within them. Typically, you would choose one location for a tenant. This will be based on your customer location. This would mostly stay the same. The users in a company could travel, but the choice of region is usually decided by where the company is located. Yes, you would be able to change the location of a tenant and the data will be moved to a new region but this usually happens if there is a major decision about location in a company.
> This would mostly stay the same. The users in a company could travel, but the choice of region is usually decided by where the company is located.
For most of my apps, I have very little idea on their region aside from their login IP origin, and we get a high level of multi-region travels (like conference organizers). I'd encourage you to transform the notion of a tenant transfers from a 'major decision' to 'assume users will/should switch to closest tenant login'. It looks like your platform already allows this move-tenant-on-login approach, but I'd love to see more docs and more tests around that use-case.
How interesting. Does that mean the same is true for deployment_mode? As in, assuming 'customer 2' was originally made 'dedicated', is this how we'd move them back to shared deployment mode?
update tenants set deployment_mode = 'shared' where name = 'customer 2';
Does that mean some downtime or broken connections for that tenant while data is shuffled around?
Cool product if it works as promised with one caveat - I have saas app on django. I would happily migrate it from current setup to something that promises Tenant separation out of the box. But I'm not dropping the framework anytime soon. How will that work together?
Nile CEO here. We work out of the box with Django. We are working on providing examples and tutorials for Django soon.We definitely want to play well with the ecosystem.
Ooh, yes! I need to get a Django and other Python examples going.
With Flask it is fairly straightforward - you add the tenant configuration to the DB connection when you pick it up to serve a request. The Java and NodeJS examples do something similar if you need to see the idea.
I have no need for the product currently but website's really nice with the black-white color schema; I rarely see a website that actually feels as nice. Can't check currently on phone but this is probably nextjs underneath, right?
This looks like an interesting take on serverless Postgres! Is this type of multi tenant separation enough for the kinds of users who care about it? I’d imagine a lot of the concerns around multi-tenant would be related to potential application layer bugs that can co-mingle data.
Data isolation, recovery, backups are a big problem in the world of SaaS, especially at huge scale. Companies end up hiring a dedicated infra team to handle such issues.
Makes sense, but if you're doing multi-tenancy, why not deploy a postgres database for each user so co-mingling is not even possible? That would make on-prem deployments easier to migrate to as well.
Nile lets you do this without incurring the operational complexity of managing multiple databases. We let you place the virtual tenant DB either on one multitenant Postgres DB or a dedicated Postgres. So, you can actually create a dedicated one for each of your tenant in Nile but interact with them like a single Postgres instance. We take care of schema updates across them, client-side routing and metadata management
> We let you place the virtual tenant DB either on one multitenant Postgres DB or a dedicated Postgres.
Sounds promising, but by "a dedicated Postgres." do you mean a different server or just a different db on the same server? I'm trying to get at how the cost would change between the two solutions. Right now we can run N databases on the same Postgres server for roughly the same cost as running a single multitenant db. Switching to "server per customer" seems like it would be cost prohibitive.
We are going to support a database in the same server initially but could also provide dedicated server in the future. The nice thing about Nile's design is that we can provide different placements for different cost profiles with the developer experience of a single Postgres instance.
Nile CEO here. Yes, this is docs, examples and our website right now. Our background is in open source for the last 15 years. We helped build Apache Kafka and grow the community to a very successful one. We believe open source should be done well and hope to consider it once we have the bandwidth to provide DIY ways to run Nile.
Nile CEO here. We do offer serverless Postgres but that is where the similarity ends.
We’re the first database that virtualizes tenants into the database. This enables seamless tenant isolation, per-tenant backups, and placement on multi-tenant or dedicated infrastructure anywhere on the planet. You can do all this with the experience of a single Postgres.
Here are some examples that can help you to understand the type of use cases. Most applications are multi-tenant and this is what we help with.
1. GitHub helps a group of developers manage and deploy their code. Each Github organization is a tenant, and the developers within those organizations are users.
2. Salesforce or Hubspot helps sales reps manage their leads. Each company is a tenant, and the users are the sales reps.
3. Ring is a home security company that provides alarm services to different households. Each household is a tenant, and people living in a house using their service are users.
4. Toast is a platform to help build software for restaurants. The restaurants are tenants, and the employees of the restaurants are the users.
Don't take this the wrong way, because I think serverless postgres is a major achievement. But would you say multitenant means use Nile, and for everything else, use neon?
If you are building a static website (usually your website) or building a social networking product like Facebook, I would not recommend Nile. We recommend Nile for any usecase where you are serving multiple groups of users. In our experience, 95% of applications in the market today seem to fall into this category.
Nile CEO here. Yes, you can store embeddings per tenant and search for it. You can read more about vector embeddings and how we support it here https://www.thenile.dev/docs/ai-embeddings
Nile CEO here. Nile abstracts away compute machines and provides an experience where you can easily scale up or down based on the load you push.
Specifically, Nile lets you do the following
- Create a database and start querying it. Nile takes care of adding capacity to tenants as workload increases.
- Pay for what you use. You will only pay for what you use. We have plans for pricing where you can pay based on usage per tenant. This will ensure your business value is aligned with the cost of your database. For example, a customer on your free tier may not be an active user and you would not pay for them in Nile.
- We have built multitenancy into Postgres and a gateway layer that routes. This helps us to scale to zero with instant availability when you want to scale back up.
- You can create even a million tenants if you have that many customers.
- We have built connection pooling into Nile. This helps to provide limitless connections as you grow
I'm still trying to understand the scaling story better. When we say serverless it mentions automatically scaling when it detects some sense of resource pressure. If I have a "hot tenant-database", does that mean this shard will be scaled automatically without impact to existing queries? Or would there be some "blip". I suppose it's unavoidable in edge cases but curious about the regular ones as well.
It's an incredibly cool CX you have here with the automated query routing/tenancy story though, looking forward to what happens in this space.
I learned this probably later than I should have, but "serverless" now has two meanings.
The first meaning I was familiar with was the idea that there was no long-running process operating a service. This was like a lambda, executing upon a trigger and then stopping.
The second meaning I learned about more recently was the idea of "serverless" being an abstraction layer, where you don't think about what server the service is deployed on (specifically, you get to stop caring about the resources allocated to that server), even though it is deployed as a continuous, long-running process.
Serious follow-up question: So the difference between 'managed' and 'serverless' databases is that for the former I know some level of infrastructure (CPU, RAM, size, etc.) whereas for the latter it's completely hidden and priced in?
I was also only aware of your first definition of serverless.
Serverless isn't just hidden and priced in - it is supposed to scale automatically with your use.
AWS Lambda lets you think about "invocations" and not CPU and you trust that the CPU will show up when needed.
At Nile we hope to let you focus on "queries per sec" and even "queries per second per tenant" while we scale the infrastructure as needed for you.
Incidentally, our product has a dashboard that shows you exactly these metrics :)
But in reality those limits don't disappear, they just get buried deeper, hidden in the docs somewhere.
AWS Lambda has a default concurrency limit of 1k and each function has a configurable memory limit.
Sure, those limits allow for a lot more elasticity and flexible pricing but the limitations haven't disappeared.
Also, there are now some 'serverless' hosting providers that do fixed monthly pricing (fermyon.com/pricing). Isn't this taking the best part of serverless ad absurdum or is there something I'm missing?
Lastly, I hope I don't sound argumentative. I work for a company that has the described problem of multi-tenant SaaS and I love Postgres, so I am fully supportive of your mission!
This is brilliant. Founders are top notch. They truly understand control plane development needed for true regional services. Cant wait to see more of this!
neon and supabase are both the up and comers in the postgres world i think. the real incumbents are the AWS RDSes of the world and their azure and gcp equivalents.
Would it be fair to compare this to things like Azure SQL Server Hyperscale architecture? My first impression of the docs indicates this is a similar idea but for Postgres.
They don’t seem to get into their architecture under the hood. The novel aspect seems to be that they have tenants as a first class member of the database
Azure SQL Server Hyperscale architecture gives you plenty of scale but does not do anything to help with managing tenants.
I will explain what Nile tries to do. Happy to follow up.
The most foundational element in SaaS is a tenant. It makes a lot of sense to build this core concept into the database. Imagine having a lot of virtual tenant databases that can be co-located on one physical Postgres (multi-tenant) for better cost, and some of them can be placed on a dedicated database for better isolation. The virtual tenant DBs can be located anywhere on the planet for low latency or compliance. The client can route to the right tenant seamlessly without routing logic in the application.
Isolating tenants into their own virtual DBs is great, but you will also want to be able to share data across tenants where it makes sense. Backups should be available for each tenant, and it should be possible to restore them instantaneously. Schema changes should be applied seamlessly across all the tenant DBs, and it should also be possible to do staged rollouts for different tenant tiers. While supporting all this, all the standard SQL capabilities should work across the tenants for admin operations. All the standard Postgres tooling should work. You want the experience of a single Postgres.
This is what Nile tries to achieve. A model that gives you fine-grained control over your tenants. You create, scale, locate, pay all by tenants.
Having never worked on a multi-tenant SaaS app, is this how multi-tenancy is typically implemented (a per-tenant-database)? Is there a certain scale at which this becomes the ideal pattern? If so, has anyone made the shift from a single-database approach to a per-tenant-database approach?
I'd say it's probably not the typical implementation. Mainly because it's a pain to implement and manage.
I guess that's why this was created though. It's definitely an intriguing tool.
I did shift in past from single database to sharded via citus. The actual migration was smooth, but 6+ months of work went into ensuring compatibility.
There's ways to nearly guarantee compatibility without actually using a sharded approach, like enforcing all tables have a uniform shard key.
Sometimes it is. Often it's done in the database itself, but then you have to be very careful you never have a query that returns data that belongs to someone else.
Notion does something similar, sharding postgres grouped by their tenants. It takes a huge effort and preparation to get it right, both at the application and infra level. Link: https://www.notion.so/blog/sharding-postgres-at-notion
Nile CEO here. There are many approaches to doing multitenancy.
- You can create one physical DB per tenant
- You can place multiple tenants on the same DB
Both approaches have pros and cons. This is exactly what Nile is solving. We want users to not worry about all the operational complexity of picking a choice. You can choose any approach in Nile.
In Nile, the tenant DB is a virtual concept. You can choose to place it on a multitenant DB along with all the other tenants or choose to place it on a dedicated DB. Nile provides a single experience irrespective of your choice and takes care of all the operational complexity. We also go one step further and also let you place a tenant in any location worldwide but still have one Postgres experience.
> is this how multi-tenancy is typically implemented (a per-tenant-database)?
There's a few different approaches and deciding which to use can be tough. When I was adapting a formerly single tenant Django application to a multi-tenant solution, I went for a "db-per-customer" (with many databases on a single db server) for a few main reasons:
1) Allows you to backup and restore databases on a per customer basis.
2) Helps with IT reviews if you can say "your data exists in its own DB and is not co-mingled with other customers data"
3) Implementation seemed like it was going to be simpler, especially for adapting an existing single tenant application to a multi-tenant one.
4) Makes it easy to migrate existing single tenant-db's to the multi-tenant app.
The main drawback for us I would say is that it makes managing database migrations/schema changes a bit more challenging since you now need to update N db's every time there's a schema change. This was not super difficult though.
Overall I am very happy we decided to go with a multi-db multi-tenancy solution.
Same setup for us, as this is the only option for B2B/enterprise products you have contractual obligations to delete all tenant data at the end of the term.
Additional benefits: easy to support customer-managed encryption keys and can give their BI people direct access.
Yeah, worked in projects with both of those methodologies. Combining tenants in one database is good for cross tenant aggregation but causes people to reach for...
> brittle permission logic at the application level or complex, hard-to-debug, row-level security policies in databases like Postgres
Separating them into separate databases has its own problems with connection pooling, resource management, and query speed. It's also a pain when you have to combine the data. Then you go down the rabbit hole of ETL and SaaS products with a "scratch" interface that promises to speed things up through (yet another) customer DSL and new naming convention for everything you already know.
A better option is separating tenants by schema. Think about how you switch between schemas in Postgres. Queries look something like this:
> set search_path to public, tenantA;
> select * from somewhere as smw inner join elsewhere lsw on smw.id = lsw.somewhere_id;
The somewhere table is in the public schema, elsewhere is in the tenantA schema.
In Rails, there are a couple gems that come to mind for multi-tenancy. acts_as_tenant works exactly like this, by swapping the tenant schema in ActiveRecord (the Rails ORM layer). The ros-apartment gem is another (formerly just 'apartment'). I ran across another gem called 'roomer' too but it is outdated. It works the same way though. You could even roll your own if you insert the logic at the right layer of whatever stack you're using.
Set the tenant at the request level and the ORM auto switches the schema for you. Match a subdomain to the tenant and away you go. You got a SaaS application. edit: And, it'll pass a security audit because of the separation. And you can query across LOB tenant data in a BI front end if you can live without real-time data and build some materialized views; if Redshift and Snowflake are just too expensive.
Using schema separation you get the best of both worlds; smaller data in tenant specific tables with the ability to cross tenant query because they're in the same database. And yes, I'm well aware foreign servers are a thing in Postgres but with sufficiently large data you have to spend more time tuning them for fetch size and indexes to get decent performance. Forget the fact that sometimes you have to implement dynamic SQL using SQL which is a nightmare of repetitious quotation marks with any complex data structure. Foreign servers amplify the escaped quotes. My stomach turns remembering having to write that code...
Is there a npm package for something like this? React/node is not my daily driver.
We don't have an official example yet, but I expect it will work.
So far every JS framework that uses https://node-postgres.com works great and so no reason to think Drizzle wouldn't.
Nile CEO here. I agree, and I am with you. I can vouch that this is not done with any malicious intent. We were not ready for HN yet but here we are. Our goal was to work with initial users to really polish the product before we open it up to everyone. We understand this would discourage some to not join and we hope to get them back once we are available for general availability.
"Grow for years" only really works if your SaaS also exists in years. I can't imagine deciding to run a production DB with a custom RLS thing going on without being able to directly port that off-provider in a worst case scenario. Supabase sort of gets around this by being open source, but even then depending on their custom setup gives me a little anxiety. Not sure about other managed/"serverless" Postgres.
Nile CEO here. Nile will promise to make the experience of moving out of Nile really easy. This is one our core product principles. We want users to stay for the value we provide.
Nile user permissions are optional. You can decide to use any third-party provider for it. We are in the design phase for permissions at this point. We would love to get your feedback to understand more about your concerns
https://github.com/orgs/niledatabase/discussions/159
This doesn't necessarily mean that any user could pick up and switch to a different PG instance, as such a young provider maybe look at having some docs for migrating off-platform as a sort of assurance. Maintaining those docs might also help you prevent adding or changing things that would introduce lock-in, it should be a consideration for every feature or API change.
Agree. We will prioritize how to migrate from Nile. At the Postgres level we don't want any lock-ins. We would have APIs that simply SaaS workflows that users may choose to use.
This is why we don’t run a fork of Postgres, and we lean heavily into extensions rather than customizations. As the docs mention: this forces us compete on experience.
(Speaking here to your point about supabase, not to detract from Nile, which looks very cool)
I use Supabase at work for this reason, just not sure I'm _completely_ prepared to pick up and move to a non-Supabase PG if I really had to, which is a little unsettling for long term reliability. This is probably a me problem, you guys are doing great work on avoiding lock-in.
I recommend complimenting the supabase tooling with “external” tooling - pgadmin, direct Postgres libs, etc. Even if you don’t migrate (which we hope you never do) you’ll definitely become more seasoned Postgres developer and you should find that we’re not doing anything too “magical”. Everything you learn with supabase should be transferable
Nile CEO here. You can place tenants on multitenant DB or a dedicated Postgres DB. Placing the tenants on multitenant DB is the default. This has no cold start time since the instance/container is shared. If you choose to place a tenant on a dedicated tenant DB, you would have cold time of a few seconds. We are building this as we speak and will have more numbers to share soon. However, if you have chosen to place a tenant on a dedicated DB, these typically would be critical customers who are actively using the DB and may never want the instance to go down. We are exploring provisioned compute tokens for this purpose.
What cold start times have you experienced? I work at Neon, so I'm more than happy to pass along your experience if you want to share it with me. You can email me at tristan at neon.tech. Or you can reply here.
Very cool. This is Sai, ex-Citus and currently PeerDB. You are promising many features incl. region aware tenant placement, tenant level metrics, easy tenant isolation etc. which would really help multi-tenant SaaS customers. Back during my days at Citus, we spent many cycles to a build a database for SaaS. We built some (not all) the features you mentioned and saw customers really finding value.
Congrats on the launch. I would love to try out Nile in the future!
Currently, we support only AWS but do want to support other cloud providers. Technically, you can even have one DB that spans across cloud providers with your tenants, but we have not yet seen many use cases for multi-cloud.
W.r.t replicating data across tenants, could you give an example use case for it?
Let's say I'm building a CRM and I want to have an "industry" or "cities" table that is the same across all tenants. I want a copy of cities in each tenant for referential integrity, but I also need to update it periodically across all tenants.
Great question. Nile supports shared tables for this purpose. Shared tables are regular tables that can be shared across all the tenants in another table. We have example use cases and how this works in Nile here https://www.thenile.dev/docs/tenant-sharing. Let me know if this works for you.
We do plan to support a bunch of extensions soon. We do support pg_vector at this point. Would love to know if you have any favorite ones. Also, could you share the use case for listen/notify? We currently do not support it but have plans for something that gives you the same functionality.
The tenants concept is really cool and something I'm fighting with right now. I actually went single-tenant from day 1 (not exactly on purpose) and it's a been a pain to find a DB solution that will work well for me.
I absolutely love PlanetScale but having 1 database per client means paying for 1 plan per client. I'm B2B so I have way fewer "clients" but my load is incredibly spiky for each client (75% of the time they have near-0 usage, 20% of the time they have a tiny amount of usage, and the 5% they use a bunch). As it stands right now their heavy load never overlaps so I've been mulling over a move to an RDS cluster where I can run multiple databases on the same hardware.
I'm not super interested in migrating to Postgres (though I think my app would move relatively easily, I don't use anything super MySQL-specific, it's just what I know) but I'll be watching this project as it moves forward. I'd really like to not manage my own DB, it's not where I'm strong and I'd rather focus on other things.
Sorry to hear your problems. We would be a great fit for your use case. We are serverless and support spiky workloads. You can map your clients to tenants in Nile. Let us know if anything changes on your side in the future. I would love to connect.
I've been interested to see what Nile will be since the Twitter pre-announcement, and: uff.
The tenant capabilities are nice to have, but even when building a tenant-focused SaaS not really a big draw. The examples shown even come down to "you can achieve 80% of this by just adding a tenant column everywhere". When comparing this to e.g. Supabase, their autogenerated REST/GraphQL APIs solve a lot more significant pain points, especially when starting out.
Sorry to hear that. We understand each one will have a different set of problems. From our experience and talking to others, it is usually not as simple as adding a tenant_id column. You typically care about security from day one and want to be really careful how you isolate customers. You would also want to add user and organization management easily to the product. We have made these steps really easy without having to spend time writing complex permission logic and potentially buggy code. We also are able to provide query insights by tenant since the database understands tenant boundaries. There are also other things that I have mentioned in the blog. Having said that, if our users want autogenerated APIs, we would be more than happy to invest in it. We did have plans for it but in our conversations most did not bring it up. Hopefully, we can convert you to love Nile as we grow.
> You would also want to add user and organization management easily to the product.
That's usually something that I'd outsource as much as possible to Auth0 (yikes, considering the last days), as building out the UIs and state handling for all user auth and management flows is a huge chore. I think recently Clerk has set a new standard on how easy that can be.
I'd be surprised if a database product could compete in a meaningful way in that area.
I'm looking forward to see what Nile evolves to! (IIRC Supabase also didn't look that enticing when it first appeared on the map). In general I'm really happy to see databases as such an active space of innovation again.
I wonder how this compares to e.g. Citus with some triggers and RLS on top.
As this does regional accesses of shared and modifiable data, does Nile implement or guarantee snapshot consistency for cross-shard queries? Citus doesn't have this yet, iirc.
From the docs, I assume Nile does something like the following:
- "smart partitioning" of tables on tenant_id when tenant_id is detected on table creation. These partitions are either region-local when the tenant is located in that region, or foreign tables. (Alternatively, non-local tenant data is replicated with e.g. logical replication/LR, or triggers)
- global tables are replicated across all regions, again presumably with LR
- RLS on the smart partitioned tables, which applies the tenant filter when nile.tenant_id is set.
Do you have any docs where I can read up on this in more detail?
All in all this looks like a real neat project. Congrats on launching.
116 comments
[ 2.0 ms ] story [ 116 ms ] threadNice surprise to see our launch here. Happy to answer any questions...
Modern SaaS applications are multi-tenant. We’re the first database that virtualizes tenants into the database. A tenant is primarily a company, an organization, or a workspace in your product that contains a group of users. This enables seamless tenant isolation, per-tenant backups, and placement on multi-tenant or dedicated infrastructure, anywhere on the planet. You can do all this with the experience of a single Postgres! You don’t have to manage multiple databases, build complex permissions for isolation, or write buggy scripts to read specific tenant data from backups. On top of the tenant model, we provide opt-in user management capabilities, customer-specific vector embeddings, and instant tenant admin dashboards.
You manage schemas like how you manage with standard Postgres. Your favorite tooling should work. When you make the schema change, Nile pushes the schema to all the tenants. For the user, it is like interfacing with a single Postgres instance.
We have built a Rust based layer internally to manage tenants, route them, shard them if needed and have multiple tenants talk to each other. Happy to chat more. My email is ram@thenile.dev
This is true in a sense “most modern SaaS apps have at least two unrelated users or user groups” (forgive me please if I’m wrong here, but that’s how I loosely understand “multi-tenant”), but it’s not exactly clear why this is important.
I strongly suspect that the majority of modern SaaSes don’t do any database-level tenant isolation or per-tenant backups. The smaller the company the less it’s likely. Row-level security is a very rare sight to behold, even in regulated areas (certifications like HIPAA compliance don’t really require this). The best they do if they’re really global (I’m sure most are only global in “we accept users from anywhere”, but not so much under the hood - even though cloudy sales pitches love to paint a different picture, most SaaSes I’ve seen were one or two-region deployments) is try to move/shard data closer to the the user’s region.
I think one concrete example (as simple as possible) would help a lot.
I'm not sure that I would generalize that to all modern SaaS applications though.
1. GitHub helps a group of developers manage and deploy their code. Each Github organization is a tenant, and the developers within those organizations are users. 2. Salesforce or Hubspot helps sales reps manage their leads. Each company is a tenant, and the users are the sales reps. 3. Ring is a home security company that provides alarm services to different households. Each household is a tenant, and people living in a house using their service are users. 4. Toast is a platform to help build software for restaurants. The restaurants are tenants, and the employees of the restaurants are the users.
Nope, Github users exist outside of organizations in a global namespace, and can be invited to be members of multiple organizations.
Also, how is Nile different from creating a database (or several databases) for each tenant within a single Postgres cluster?
Nile is different from DB per tenant in a single PG cluster in that it is much lower overhead per tenant. Since we offer a service, it will translate to lower cost.
The DB-per-tenant gets painful after the first few 100 tenants, but some SaaS have thousands of tenants, each with very low activity. Our model makes inactive tenants nearly free.
In addition, we also provide the "unified view", which lets you query all your tenants as if they were in the same DB. So you can connect as an admin and "select * from my_table" will give you the results for all tenants. Can be pretty handy.
What are you thinking will happen when a SaaS grows like Salesforce, with a very large number of tenants, where some of the tenants have very large databases? What will Nile do when the database just can't fit in a single Postgres cluster?
Edit: Never mind, I see from your web site that moving tenants between databases is one of the main things you try to address.
Tenants are your customers. It is normal for SaaS to have either a schema per tenant or one schema and tenant_id column in most tables. Nile provides you with both experiences at the same time.
If they travel a lot, you may want the data in one location (although it still depends, moving data is often faster than flying to a different region).
Tenant placement is especially great if their location has data residency requirements. We discovered that companies with many EU customers need this.
For most of my apps, I have very little idea on their region aside from their login IP origin, and we get a high level of multi-region travels (like conference organizers). I'd encourage you to transform the notion of a tenant transfers from a 'major decision' to 'assume users will/should switch to closest tenant login'. It looks like your platform already allows this move-tenant-on-login approach, but I'd love to see more docs and more tests around that use-case.
The architecture includes proxy layer that handles routing and maintains the connections, so we believe broken connections are avoidable.
With Flask it is fairly straightforward - you add the tenant configuration to the DB connection when you pick it up to serve a request. The Java and NodeJS examples do something similar if you need to see the idea.
This is the approach we took and I think it helps with IT review docs.
Sounds promising, but by "a dedicated Postgres." do you mean a different server or just a different db on the same server? I'm trying to get at how the cost would change between the two solutions. Right now we can run N databases on the same Postgres server for roughly the same cost as running a single multitenant db. Switching to "server per customer" seems like it would be cost prohibitive.
LICENSE
Apache-2.0
https://github.com/niledatabase/niledatabase/blob/main/LICEN...
Edited:
This is the docs repo.
We’re the first database that virtualizes tenants into the database. This enables seamless tenant isolation, per-tenant backups, and placement on multi-tenant or dedicated infrastructure anywhere on the planet. You can do all this with the experience of a single Postgres.
Here are some examples that can help you to understand the type of use cases. Most applications are multi-tenant and this is what we help with.
1. GitHub helps a group of developers manage and deploy their code. Each Github organization is a tenant, and the developers within those organizations are users. 2. Salesforce or Hubspot helps sales reps manage their leads. Each company is a tenant, and the users are the sales reps. 3. Ring is a home security company that provides alarm services to different households. Each household is a tenant, and people living in a house using their service are users. 4. Toast is a platform to help build software for restaurants. The restaurants are tenants, and the employees of the restaurants are the users.
Happy to chat more. My email is ram@thenile.dev
If you are building multi tenant saas you should totally look at them!!
Specifically, Nile lets you do the following - Create a database and start querying it. Nile takes care of adding capacity to tenants as workload increases. - Pay for what you use. You will only pay for what you use. We have plans for pricing where you can pay based on usage per tenant. This will ensure your business value is aligned with the cost of your database. For example, a customer on your free tier may not be an active user and you would not pay for them in Nile. - We have built multitenancy into Postgres and a gateway layer that routes. This helps us to scale to zero with instant availability when you want to scale back up.
- You can create even a million tenants if you have that many customers. - We have built connection pooling into Nile. This helps to provide limitless connections as you grow
I'm still trying to understand the scaling story better. When we say serverless it mentions automatically scaling when it detects some sense of resource pressure. If I have a "hot tenant-database", does that mean this shard will be scaled automatically without impact to existing queries? Or would there be some "blip". I suppose it's unavoidable in edge cases but curious about the regular ones as well.
It's an incredibly cool CX you have here with the automated query routing/tenancy story though, looking forward to what happens in this space.
The first meaning I was familiar with was the idea that there was no long-running process operating a service. This was like a lambda, executing upon a trigger and then stopping.
The second meaning I learned about more recently was the idea of "serverless" being an abstraction layer, where you don't think about what server the service is deployed on (specifically, you get to stop caring about the resources allocated to that server), even though it is deployed as a continuous, long-running process.
I was also only aware of your first definition of serverless.
AWS Lambda lets you think about "invocations" and not CPU and you trust that the CPU will show up when needed. At Nile we hope to let you focus on "queries per sec" and even "queries per second per tenant" while we scale the infrastructure as needed for you.
Incidentally, our product has a dashboard that shows you exactly these metrics :)
AWS Lambda has a default concurrency limit of 1k and each function has a configurable memory limit.
Sure, those limits allow for a lot more elasticity and flexible pricing but the limitations haven't disappeared.
Also, there are now some 'serverless' hosting providers that do fixed monthly pricing (fermyon.com/pricing). Isn't this taking the best part of serverless ad absurdum or is there something I'm missing?
Lastly, I hope I don't sound argumentative. I work for a company that has the described problem of multi-tenant SaaS and I love Postgres, so I am fully supportive of your mission!
I will explain what Nile tries to do. Happy to follow up.
The most foundational element in SaaS is a tenant. It makes a lot of sense to build this core concept into the database. Imagine having a lot of virtual tenant databases that can be co-located on one physical Postgres (multi-tenant) for better cost, and some of them can be placed on a dedicated database for better isolation. The virtual tenant DBs can be located anywhere on the planet for low latency or compliance. The client can route to the right tenant seamlessly without routing logic in the application.
Isolating tenants into their own virtual DBs is great, but you will also want to be able to share data across tenants where it makes sense. Backups should be available for each tenant, and it should be possible to restore them instantaneously. Schema changes should be applied seamlessly across all the tenant DBs, and it should also be possible to do staged rollouts for different tenant tiers. While supporting all this, all the standard SQL capabilities should work across the tenants for admin operations. All the standard Postgres tooling should work. You want the experience of a single Postgres.
This is what Nile tries to achieve. A model that gives you fine-grained control over your tenants. You create, scale, locate, pay all by tenants.
I guess that's why this was created though. It's definitely an intriguing tool.
I did shift in past from single database to sharded via citus. The actual migration was smooth, but 6+ months of work went into ensuring compatibility.
There's ways to nearly guarantee compatibility without actually using a sharded approach, like enforcing all tables have a uniform shard key.
Curious if the sharding strategy that you were shifting to was company-based, as is implemented in this case?
Both approaches have pros and cons. This is exactly what Nile is solving. We want users to not worry about all the operational complexity of picking a choice. You can choose any approach in Nile.
In Nile, the tenant DB is a virtual concept. You can choose to place it on a multitenant DB along with all the other tenants or choose to place it on a dedicated DB. Nile provides a single experience irrespective of your choice and takes care of all the operational complexity. We also go one step further and also let you place a tenant in any location worldwide but still have one Postgres experience.
You can read more about it here https://www.thenile.dev/docs/tenant-management https://www.thenile.dev/docs/tenant-placement
Would love the feedback
There's a few different approaches and deciding which to use can be tough. When I was adapting a formerly single tenant Django application to a multi-tenant solution, I went for a "db-per-customer" (with many databases on a single db server) for a few main reasons:
1) Allows you to backup and restore databases on a per customer basis.
2) Helps with IT reviews if you can say "your data exists in its own DB and is not co-mingled with other customers data"
3) Implementation seemed like it was going to be simpler, especially for adapting an existing single tenant application to a multi-tenant one.
4) Makes it easy to migrate existing single tenant-db's to the multi-tenant app.
The main drawback for us I would say is that it makes managing database migrations/schema changes a bit more challenging since you now need to update N db's every time there's a schema change. This was not super difficult though.
Overall I am very happy we decided to go with a multi-db multi-tenancy solution.
Additional benefits: easy to support customer-managed encryption keys and can give their BI people direct access.
> brittle permission logic at the application level or complex, hard-to-debug, row-level security policies in databases like Postgres
Separating them into separate databases has its own problems with connection pooling, resource management, and query speed. It's also a pain when you have to combine the data. Then you go down the rabbit hole of ETL and SaaS products with a "scratch" interface that promises to speed things up through (yet another) customer DSL and new naming convention for everything you already know.
A better option is separating tenants by schema. Think about how you switch between schemas in Postgres. Queries look something like this:
> set search_path to public, tenantA; > select * from somewhere as smw inner join elsewhere lsw on smw.id = lsw.somewhere_id;
The somewhere table is in the public schema, elsewhere is in the tenantA schema.
In Rails, there are a couple gems that come to mind for multi-tenancy. acts_as_tenant works exactly like this, by swapping the tenant schema in ActiveRecord (the Rails ORM layer). The ros-apartment gem is another (formerly just 'apartment'). I ran across another gem called 'roomer' too but it is outdated. It works the same way though. You could even roll your own if you insert the logic at the right layer of whatever stack you're using.
Set the tenant at the request level and the ORM auto switches the schema for you. Match a subdomain to the tenant and away you go. You got a SaaS application. edit: And, it'll pass a security audit because of the separation. And you can query across LOB tenant data in a BI front end if you can live without real-time data and build some materialized views; if Redshift and Snowflake are just too expensive.
Using schema separation you get the best of both worlds; smaller data in tenant specific tables with the ability to cross tenant query because they're in the same database. And yes, I'm well aware foreign servers are a thing in Postgres but with sufficiently large data you have to spend more time tuning them for fetch size and indexes to get decent performance. Forget the fact that sometimes you have to implement dynamic SQL using SQL which is a nightmare of repetitious quotation marks with any complex data structure. Foreign servers amplify the escaped quotes. My stomach turns remembering having to write that code...
Is there a npm package for something like this? React/node is not my daily driver.
We build our JS SDK by wrapping Knex (nice query builder, not quite an ORM) and injecting our logic to the connection process. You could do the same.
The issue is that there are at least 5 popular JS ORMs, so we don't get the same solve-it-once that we get in the Rails ecosystem.
I guess this gets a pass since it's not a "Show HN" but that junk drives me crazy
Nile user permissions are optional. You can decide to use any third-party provider for it. We are in the design phase for permissions at this point. We would love to get your feedback to understand more about your concerns https://github.com/orgs/niledatabase/discussions/159
https://supabase.com/docs/guides/getting-started/architectur...
This is why we don’t run a fork of Postgres, and we lean heavily into extensions rather than customizations. As the docs mention: this forces us compete on experience.
(Speaking here to your point about supabase, not to detract from Nile, which looks very cool)
Congrats on the launch. I would love to try out Nile in the future!
W.r.t replicating data across tenants, could you give an example use case for it?
I use listen/notify and triggers as some sort of pubsub.
I absolutely love PlanetScale but having 1 database per client means paying for 1 plan per client. I'm B2B so I have way fewer "clients" but my load is incredibly spiky for each client (75% of the time they have near-0 usage, 20% of the time they have a tiny amount of usage, and the 5% they use a bunch). As it stands right now their heavy load never overlaps so I've been mulling over a move to an RDS cluster where I can run multiple databases on the same hardware.
I'm not super interested in migrating to Postgres (though I think my app would move relatively easily, I don't use anything super MySQL-specific, it's just what I know) but I'll be watching this project as it moves forward. I'd really like to not manage my own DB, it's not where I'm strong and I'd rather focus on other things.
The tenant capabilities are nice to have, but even when building a tenant-focused SaaS not really a big draw. The examples shown even come down to "you can achieve 80% of this by just adding a tenant column everywhere". When comparing this to e.g. Supabase, their autogenerated REST/GraphQL APIs solve a lot more significant pain points, especially when starting out.
That's usually something that I'd outsource as much as possible to Auth0 (yikes, considering the last days), as building out the UIs and state handling for all user auth and management flows is a huge chore. I think recently Clerk has set a new standard on how easy that can be. I'd be surprised if a database product could compete in a meaningful way in that area.
I'm looking forward to see what Nile evolves to! (IIRC Supabase also didn't look that enticing when it first appeared on the map). In general I'm really happy to see databases as such an active space of innovation again.
As this does regional accesses of shared and modifiable data, does Nile implement or guarantee snapshot consistency for cross-shard queries? Citus doesn't have this yet, iirc.
From the docs, I assume Nile does something like the following:
- "smart partitioning" of tables on tenant_id when tenant_id is detected on table creation. These partitions are either region-local when the tenant is located in that region, or foreign tables. (Alternatively, non-local tenant data is replicated with e.g. logical replication/LR, or triggers)
- global tables are replicated across all regions, again presumably with LR
- RLS on the smart partitioned tables, which applies the tenant filter when nile.tenant_id is set.
Do you have any docs where I can read up on this in more detail?
All in all this looks like a real neat project. Congrats on launching.