Prisma is an offshoot of graphql.cool, which was a "graphql API as a service". It seems to me that was a potentially good and viable business. However they stopped graphql cool and released Prisma, where there's no business model (who's gonna pay for an ORM?). I'm still left wondering why they did that?
Whether it's a viable business is to be seen though.
I've been a happy user of Prisma for a while, and so far we've not been running into severe limitations nor need for their additional services. Who knows what the future will bring though.
IMO foreign key constraints are bad; they make migrations and maintenance a nightmare. It's better to have application-level processes in place to ensure data consistency. I don't agree that "no foreign key support" is a good reason to not use a particular solution. Pricing concerns are valid though.
That is a strange take tbh. For me, ensuring data integrity at the database level gives me a lot of peace of mind. Migrations are actually safer and easier because the database will complain if something's wrong, and at the application level I can do an opportunistic insert and if the FK is not valid, I'll get an error, which is really nice if I don't need the related record at all so I don't need to fetch it first.
With foreign keys, I can't use https://github.com/github/gh-ost for zero downtime migrations. Instant deal breaker, because schema changes are inevitable given business evolution.
It sounds like gh-ost is supposed to offer more control over the process but I question how many companies need that level of sophistication in DB migrations.
On postgres at least you can split creating the foreign key, and validating it into separate steps and ensure you have zero downtime Migrations. No idea about MySQL though, so maybe it's different there.
True, but going from there to "foreign keys are bad" is still quite a jump. There are other options available for migrations as well, but I understand that it can become a trade-off at scale, especially if availability and uptime are more important than data integrity. But as I said, it's a trade-off and I suppose in most situations you won't have noticeable downtime due to migrations or you can even avoid downtime altogether without ditching foreign keys.
Then don't use it. A migration tool that has no foreign key support is not a good migration tool. And it's even a worse reason to tell other people FK's are bad.
GitHub famously has a hard time managing their database and it's a source of frequent outages. Though, to be fair, they're also working with a massive volume of traffic.
There are other options for zero downtime schema changes in MySQL and MariaDB.
You can use the database's built-in support for instant DDL (algorithm=instant) for adding and dropping columns. And then use Percona's pt-online-schema-change for all other cases, since it supports foreign keys.
Or if you're using a physical replication setup -- for example AWS Aurora without any traditional binlog replicas -- you can get away with using the database's built-in "online DDL" (algorithm=inplace, lock=none) for quite a few other cases. Historically the main problem with that method is that it causes replication lag, but that's a non-issue if you're not using logical replication in the first place. So then you only need to fall back to pt-osc for the rarer ALTER cases which don't support online DDL.
MariaDB 10.8+ also has an option called binlog_alter_two_phase, which can allow you to use online DDL without causing much replication lag.
As tables scale you may need to partition them. FKs also introduce a performance it on writes and DDL. In some DDL cases a table rewrite may be required, such as if FKs cascade.
Maybe it's just me, but if I'm doing any sort of migration like this, the first thing I do is create a small PoC that emulates the full process to iron out any weird, obvious edge cases and ergonomic issues.
In this case, hitting the Lambda size limit would have been the first red flag that triggered a re-evaluation of the whole approach.
Understanding the limits of the stack (Prisma + PlanetScale) would have been obvious from a few simple toy examples.
Seems like this team dove headfirst without even building a simple toy to determine if the choice was actually viable.
I am amazed at the number of engineers I've worked with who jump straight into the full implementation. I've done it myself on a few occasions thinking "how hard can this be".
Building the "toy" first is great.
In my experience, about one third of the time the toy is all you need, you can stop there, and what you were going to build fully would be over-engineering. About one third of the time, building the toy tells you you're taking the unworkable approach as you mentioned. And the other third of the time you can extend the toy.
I have a directory on my drive called `sandbox` where I basically throw together small toys for anything with some unknown complexity. Anything from how an ORM might model a specific type of relationship (throw together a basic replica with a Docker compose file) to replicating the deployment model (poking AWS Copilot, for example) to testing out some tooling flow (e.g. local build process change).
The main thing with a toy model is speed. You can build/deploy/test with a smaller scope and progressively scale it up some reasonable scale of the full thing (whatever you're testing for) and you can iterate your testing faster. But many times, the key issues show up quite early in the process of grokking the toy model.
I too have this "sandbox" directory. I use it for what you say. But also for troubleshooting, bugreports or a stackoverflow question.
Just throw up a new project, hack around in it for an hour, and most often the problem/bug in my original code becomes apparent because of the isolation. I'll easily write four such sandbox projects per week.
> In my experience, about one third of the time the toy is all you need, you can stop there, and what you were going to build fully would be over-engineering.
On the flip side, the cases where the toy ends up being promoted to production service end up being riddles with technical debt, missing features, and buggy behavior that jeopardizes the whole project, also happen.
Survivorship bias is also a major problem. It's easy to presume that the winning bet you took is the right path.
Nothing wrong with that if the toy meets all of the functional and non-functional requirements.
If you find offense to this, the easiest way to mitigate is with process and practice: sandbox code goes into a dedicated "Sandbox" mono-repo and if it's suitable for production, you rebuild it appropriately in a production repo.
Exactly. That's why I don't build the toy anymore: Too many broken promises of "Yes, we won't put it into production until it's ready", and then my team is left maintaining a system in production that had no business of ever being in production.
> Further inspecting, we discovered that your code never makes the actual DB call. Your Prisma code performs a GraphQL network request (?) to the Prisma Rust query engine, which then translates your request into actual DB calls.
I haven't used Prisma past fiddling with it a bit and this is extremely surprising to me. I guess it stems from Prisma's history but to me it's really off-putting and I don't think I'd use it in production. Not that I've got too much against graphql, but shipping a SQL ORM with this in between feels unnecessary imo.
> Every new insert via Prisma opened a database-level transaction (?).
Surely there must be some way to control this? Can a seasoned Prisma veteran here chime in?
They have a number of tools to consolidate queries into a single transaction (createMany in this case), or, you can control transactions directly using their $transaction api.
TypeORM is great and all - but it has its own quirks too and it's not a silver bullet.
In their case they would also maybe run into some issues as TypeORM has some initialization cost (building a model in memory when it first connects to the DB etc), some object mapping costs and other complexities (particularly around efficient query generation).
While there is nothing stopping it from working in a Lambda this would increase initialization time and cost which both matter since this is for a GraphQL API. I think the solution they've selected (Kysely) is probably a better fit.
> Weird pricing: PlanetScale prices you on row-reads and row-writes per month. What is weird is that row-reads is something that nobody controls. It is the SQL query planner which determines the query plan, which results in how many rows you’re reading internally. Remember that row-reads are not row returns. You can write a wrong query that returns 0 rows but can still do a full table scan of 1M rows and PlanetScale will charge you for it.
Followed by:
> AWS Aurora Serverless v2 Postgres
I hope they don't get an unpleasant shock when they see the IO pricing on their aurora bill (it's per block read/write rather than per row, but that's proportional in the random access case, and you're also beholden to the query planner, so basically the same).
Row-reads sounds like a perfectly acceptable metric to use for billing. It's basically changing based on disk io. So what if the result of your query is 0, the server has to do a bunch of work to figure out the answer is 0.
I would modify this to "Don't use Prisma if you're using serverless"
With actual servers and prisma running close to the underlying database, it should be much better.
Regarding using joins, that's not necessarily the best choice either. When using simple flat joins naively, you get a lot of repetition for the more toplevel nodes of the join tree, which eventually adds up to a lot of network traffic and allocation. (TypeORM tends to have this issue, for example)
This is probably okay, although I do wonder what happens after you hit the compute limits of your database server (i.e. scenarios when you don't use things like planetscale). In those cases (again, non-serverless), Prisma's choice might work better as long as it runs close to the DB. It will still likely be slower than the fancy JSON join, but probably need fewer database resources (depending on how its done).
Also, you need to specifically create a transaction to encompass your Prisma calls to limit them to a transaction. This is the default for PostgreSQL as well.
If this is not done carefully, it can also be slow in some situations, even with the right indeces in place. Lets say we have the following slightly contrieved case:
We want to get one organization with all its users. Lets imagine our org has 100 users, each of which has 100 posts on average, each of which has 100 comments on average. In a naive, flat join, we'll be repeating the organization description column 1 million times, and each post's content about 100 times on average.
This can still be problematic without joins, as sending 1 million comment IDs in a query to the DB isn't going to work, so even the multiple queries version will need to do better than that
Still not a reason to not leverage the database. Use something like Postgres' ARRAY_AGG; reduces the duplicates, faster and better than rolling your own JOINs in your app.
The drizzle example I linked to is a realistic example into what would actually be involved in using JSON aggregation in the datbase in PG. Its painful to write manually.
I’ve been tempted to ditch django for typescript in the backend but the sql and migrations story looks a bit … confusing and underwhelming? I feel like typeorm should cover 99% of cases, due to its feature list and age (7 ish years…ancient!), but then you see people often recommending other projects (eg pg_typed). Do people just string things together? Make use of Liquibase?
People needing to query huge amounts of data discovering technology that's been developed over the past 45 years for querying huge amounts of data efficiently; news at 11.
Seems all these shops starting out with MongoDB or the like always need to do a huge, complicated migration to a /proper/ optimized data store, when they could have started with a sane design in the first place...
> People needing to query huge amounts of data discovering technology that's been developed over the past 45 years for querying huge amounts of data efficiently; news at 11
I think a lot of folks are quick to discount just how scalable a database like Postgres or SQL Server can be (not to mention battle tested). Maybe its because of a lack of DBMS experience as we've move more and more from owning database servers to consuming *-as-a-service. Maybe it's because everyone thinks that they'll have Facebook scale traffic and concerns so they immediately gravitate to technologies designed to solve problems at a different order of magnitude.
The problem is when people pick Mongodb thinking they can use it like a SQL database. I've been using Mongodb since v1.6. You have to use it differently then a SQL database. The way you design your schema and code needs to change also. Its just a different way of thinking about a database. When you use Mongodb you treat the db like a dumb black box. You design in data integrity into your code. One of the big reasons you use Mongodb is because you don't need an ORM. Your POCOs are the tables. People that say they lost data while using Mongodb. Were using Mongodb wrong.
> Seems all these shops starting out with MongoDB or the like always need to do a huge complicated migration
First thing to note, I couldn't find any reasoning for the change in the original post. This makes the speculating on it rather pointless, but clearly you are able to fill in the details quite easily somehow.
As a counter point. Last four businesses I've worked at all ran on MongoDB. More happenstance than something I've looked for.
They are all relatively small businesses and all ran fine, the biggest was ingesting 1-10GB/day of new data from their IOT fleet.
So this perspective that everything mongo is fundamentally broken and plain bad is pretty foreign to my real world experience.
These shops all liked what they were doing and there was zero talk of scaling issues or changing out the database.
Wow just one look at this issue was all I needed to see. https://github.com/prisma/prisma/discussions/12715
“Please open an issue if something really is unusable or breaks in any way so we have these concrete cases recorded. In our knowledge Prisma works just fine and returns the data that was requested, just using a different method.”
I found MikroORM [0] to be quite reasonable if you're in the TS ecosystem already. It was also easy to do custom, raw queries, and really just felt like it wasn't in the way.
> Weird pricing: PlanetScale prices you on row-reads and row-writes per month. What is weird is that row-reads is something that nobody controls. It is the SQL query planner which determines the query plan, which results in how many rows you’re reading internally. Remember that row-reads are not row returns. You can write a wrong query that returns 0 rows but can still do a full table scan of 1M rows and PlanetScale will charge you for it.
This is expected pricing. The pricing is based on the load on the db, not based on results.
Usually there are two ways working with this:
DB supports showing estimated query plan , which explains how query would be ran without running it. Allowing to capture any missing indexes or filtered indexes.
DB allows only querying against indexes (this is usually done by no-sql dbs which have flexibility to deny bad queries).
I kind of want to say "y-yeah" regarding Prisma, because all this information (especially the fact that it creates a transaction every time and downloads those engines) is either available in the documentation or visible upon install/build - you can actually make it point to specific executables instead of downloading each time using environment variables but again - it's in the docs.
I picked Prisma because I'm not actually a backend developer and the tradeoffs were okay for my use case.
I wish performance was better, but a few indexes here and there went a long way.
The only surprising bit for me was peak memory usage - it comes in bursts and can get to hundreds of MB when doing even a simple query.
The queries themselves are also kind of out there in terms of complexity but so far I haven't found a reason to rewrite them in raw SQL, since that isn't where our performance bottlenecks are at the moment.
The lack of rollbacks is one legitimate pain point and I wish they addressed it.
I added prisma to a server I have and I was surprised it was running out of memory when using "prisma migrate deploy" (only migrations, not actually just running prisma client). The server was configured to use only 512mb of RAM though. It seems prisma migrate needs quite a bit of RAM.
From the article there is no detail of what problem they were solving, just "we used tech stack A, we ran into issues, so we went with tech stack B". So I have no idea whether MongoDB was a good choice to begin with, or not, or whether the problem needed a relational database or not. Or for that matter GraphQL or AWS Lambda.
I do get the feeling they went YOLO and built their solution on an over-engineered approach with a stack they were unfamiliar with, and when they hit problems, they YOLO'd into another solution without properly evaluating why, which feels like a team of junior developers run amok.
They also seemed to not really understand relational databases, and why foreign keys are kind of important when you are choosing a relation database implementation (yes you can do without them, but it's best to understand them first).
With that said I was planning to use prisma and went and looked at issues on GitHub. They’re egregious … and I will not be using prisma
The library also looks like it was written in a yolo way as well.
My guess is the graphql background has permanently stunted it, as the project manager keeps wanting examples for performance issues instead of realizing the entire fact joins are missing to begin with is idiotic and absurd.
So my guess is they’re piecemeal fixing one off issues instead of fixing the problem at its root, probably because prisma code is not designed with that in mind from the beginning
One thing that keeps coming up is that SQL equals low productivity. I don't think this is true. I think the culprit is that most developers are using to heavily abstracting SQL using ORMs like Prisma that hides the database and SQL logic.
Since building a SQL generator (https://aihelperbot.com) as a side project, I have become much more proficient in SQL and even though I am also locked into Prisma, I use the `queryRaw` all the time to execute raw SQL queries. You can understand the code without knowing Prisma API. It is more performant. For more complex SQL queries, I use the SQL generator for initial suggestions and adapt if needed.
For the next projects I build I want to use the minimal Postgres client (https://github.com/brianc/node-postgres) combined with a lightweight migration library.
I dunno, 90% of my queries are super simple and prisma does the job pretty well. In my relatively small prisma project I have a grand total of 2 raw queries.
Also prisma raw query builder and migrations are really good.
I don't see all the hate, you can mix and match both approaches just fine.
Indeed. A query builder is only really useful for dynamic queries and in practice they are few. I would argue that for performance and complexity reasons dynamic queries in real-world applications should be avoided.
I'm always surprised when I see this opinion. I think I might be writing fundamentally different kinds of applications because I need dynamic queries all the time. I often need several different kinds of filters for an endpoint that can be combined arbitrarily. Any UI that allows users to filter stuff by various criteria needs some kind of dynamic query building in the end.
Warning - opinions ahead. I've use a bunch of ORMs (in various languages) over the last two decades. Imho, one stands distinctly above the others - Microsoft's Entity Framework (EF). EF does something fundamentally different, while the rest are all some version of Hibernate with a different API or Programming Language.
EF converts your code into SQL with the help of the compiler, and is not purely a runtime affair. This is what makes it relatively more usable.
To illustrate, if JS/TS were to approach the problem in a similar way, you'd do this:
I was saying that ORMs should make use of parsers and modern language features (such as first class functions) to offer a unified interface to collections in memory and collections in the db. Essentially treating the table as a list on which you can use operators you're already familiar with.
By doing so, your core API is timeless. Because you didn't invent an API.
.NET and EF Core is the platform that most teams probably want but .NET and C# can't quite shake it's pre-Core image.
It always strikes me as odd that teams can embrace GitHub, TypeScript, and VS Code and then react negatively to C# and .NET being Microsoft platforms.
EF Core is perhaps the most powerful, most mature ORM at this point. .NET Web APIs are incredibly productive and performant with really good DX. C# isn't that far off from TypeScript syntactically and structurally.
I feel like teams that are "graduating" from TypeScript on Node probably want C# on .NET but end up exploring Go or Rust instead.
Rebranding would have helped in sooo many ways, even for the devs that moved forward from .NET Framework.
For starters, it would have at least made search results less confusing and more contextual. There were years of confusion between .NET Framework, .NET Standard, .NET Core, and .NET 5. From a basic SEO perspective, rebranding would have been huge.
The thing is, if you're comfortable with TypeScript, it's really more or less just a small step to C#; the Venn diagram of the most commonly used keywords between TS and C# has a high degree of overlap.
"Too many keywords" is the weirdest objection to a programming language versus actually using the language to build something practical.
Edit: I'm actually curious as to which keywords you personally found the most challenging to grok or what tipped the scales of "this is too complicated".
I've been burned by EF more times than I'd like, the most recent (it has been a few years), was the change in default to single queries, instead of split queries. Which absolutely destroyed our performance, until we figured we should turn single query off.
EF Core expression trees are nice, however, the change tracking behind the scenes are an absolute menace. I've gotten errors more times than I'd like, because EF Core had loaded a key in its store, and another query got the same key, resulting in an Exception because both touched the same record in the change tracker. Even if it was two separate requests.
I've become a purist sort of, using sqlc for golang, dapper for c# and sqlx for rust.
ORMs are 'fine' for simple crud apps, but once you go a bit deep in the sql, it becomes impossible to reason about. Telling an engineer that they can just run `explain analyze` on their query, and them having no idea how to actually reproduce their sql is not great.
This is also just my opinion, I've been in the same place, I used to love EF, but now I dread having to debug that stuff =D
Last time it was sending a non pointer to gorm in golang, panicing the application :,)
It looks like one poor choice leading to another poor choice. They probably chose Prisma because it may have helped them with GraphQL, whereas they most likely don't need any GraphQL at all.
Then they are screwing around with AWS Lambdas and package size, but what is the advantage of the Lambdas if they don't make your deployment easier? Just have a ECS and run whatever you want for better price/performance ratio and much higher performance ceiling.
I don't get which of their tools are improving developer productivity or software quality. It looks like every tool stands in their way. Why won't they just learn SQL and use it? What's is so low productivity about it? It's the highest level data manipulation language there is. Now they are bringing Kysely for supposed type safety. Yeah, good luck debugging, optimizing or changing the queries in any way. You'll have to convert from SQL to TypeScript and back whenever you'd want to develop the query in your SQL console.
What I see is a bunch of poor choices. Shiny tools standing in the way of getting stuff done.
Your last sentence nails it. They’ve been sold a bunch of tools by tech influencers. Told that if you don’t use X you’re doing it wrong. They believed the hype.
It’s hard to remember that, at this point, most tech influencers are shitposting or are full of shit.
This article has no depth and seems to me author has made some obvious mistakes in this migration while being totally oblivious. More red flags when the word insane is used, 3 times.
This is a very strange comment section. I can't in good faith recommend Entity Framework to anyone who doesn't want to torpedo their project with ORM problems.
Additionally, this article is really poorly written.
> Last week, we completed a migration that switched our underlying database from MongoDB to Postgres.
Okay cool, but why? MongoDB is a very capable and fast database.
If you did any research on Prisma rather than diving in head-first, you'd realize this is a core part of why Prisma exists.
> we discovered that at a low level, Prisma was fetching data from both tables and then combining the result in its “Rust” engine. This was a path for an absolute trash performance.
Can you confirm this is actually the case? Can you show some benchmarks re: this claim? Or are you just assuming this is the case? We're using Prisma in production and it is blazing fast.
I’m using Prisma and PlanetScale and while I’ve been happy with PS, Prisma hasn’t been the best to work with. Especially in a serverless environment the engine size is painful. I’m able to work around it and I have a good amount of headroom still but I’d rather have that space back. For anything other than basic CRUD on a single entity or a few easy relationships I feel the need to write raw SQL. The DSL for Prisma’s ORM (schema is fine) is annoying and I feel like I’m fighting with it more often than not. Even once I have it working I’m not confident that it’s efficient at all, it worries me that the code “looks” simple (again it’s not easy to write in the first place) but isn’t.
I started with Aurora and DataApi and switched to Prisma and PlanetScale. I have every intention of staying with PlanetScale but I’ve considered rewriting my codebase to not use Prisma. All the raw queries would be easy enough and I think I’d be happier with a type safe query _builder_ instead of a full ORM but I could be wrong. I’m almost glad Prisma’s aggregate helpers are such trash since it made me just write my own queries instead of standing on my head to make Prisma’s work.
One thing I won’t miss is scouring my codebase for .prisma folders that might contain an outdated schema file that the engine is somehow picking up. I even have an npm script that I run to find and delete all but the main one I know is supposed to be there.
103 comments
[ 2.7 ms ] story [ 162 ms ] threadWhether it's a viable business is to be seen though.
I've been a happy user of Prisma for a while, and so far we've not been running into severe limitations nor need for their additional services. Who knows what the future will bring though.
I think it amounts to "use views to decouple access to the table with a fixed interface" and "use triggers for migrating data between tables"
You can use the database's built-in support for instant DDL (algorithm=instant) for adding and dropping columns. And then use Percona's pt-online-schema-change for all other cases, since it supports foreign keys.
Or if you're using a physical replication setup -- for example AWS Aurora without any traditional binlog replicas -- you can get away with using the database's built-in "online DDL" (algorithm=inplace, lock=none) for quite a few other cases. Historically the main problem with that method is that it causes replication lag, but that's a non-issue if you're not using logical replication in the first place. So then you only need to fall back to pt-osc for the rarer ALTER cases which don't support online DDL.
MariaDB 10.8+ also has an option called binlog_alter_two_phase, which can allow you to use online DDL without causing much replication lag.
Your database is the last line of defense for data integrity.
Good luck doing that correctly and with good performance, especially in concurrent and multi-client system.
https://stackoverflow.com/questions/20842756/sql-indirect-fo...
In this case, hitting the Lambda size limit would have been the first red flag that triggered a re-evaluation of the whole approach.
Understanding the limits of the stack (Prisma + PlanetScale) would have been obvious from a few simple toy examples.
Seems like this team dove headfirst without even building a simple toy to determine if the choice was actually viable.
Building the "toy" first is great.
In my experience, about one third of the time the toy is all you need, you can stop there, and what you were going to build fully would be over-engineering. About one third of the time, building the toy tells you you're taking the unworkable approach as you mentioned. And the other third of the time you can extend the toy.
The main thing with a toy model is speed. You can build/deploy/test with a smaller scope and progressively scale it up some reasonable scale of the full thing (whatever you're testing for) and you can iterate your testing faster. But many times, the key issues show up quite early in the process of grokking the toy model.
Just throw up a new project, hack around in it for an hour, and most often the problem/bug in my original code becomes apparent because of the isolation. I'll easily write four such sandbox projects per week.
On the flip side, the cases where the toy ends up being promoted to production service end up being riddles with technical debt, missing features, and buggy behavior that jeopardizes the whole project, also happen.
Survivorship bias is also a major problem. It's easy to presume that the winning bet you took is the right path.
In my experience, about two-thirds of the time management sees the toy and ShipsIt thinking that's all you need.
If you find offense to this, the easiest way to mitigate is with process and practice: sandbox code goes into a dedicated "Sandbox" mono-repo and if it's suitable for production, you rebuild it appropriately in a production repo.
You get TS support, tooling, and a very thin layer on top of SQL so little performance hit.
And a lot of mysql bugs are reported in GitHub and waiting for fixes.
I haven't used Prisma past fiddling with it a bit and this is extremely surprising to me. I guess it stems from Prisma's history but to me it's really off-putting and I don't think I'd use it in production. Not that I've got too much against graphql, but shipping a SQL ORM with this in between feels unnecessary imo.
> Every new insert via Prisma opened a database-level transaction (?).
Surely there must be some way to control this? Can a seasoned Prisma veteran here chime in?
In their case they would also maybe run into some issues as TypeORM has some initialization cost (building a model in memory when it first connects to the DB etc), some object mapping costs and other complexities (particularly around efficient query generation).
While there is nothing stopping it from working in a Lambda this would increase initialization time and cost which both matter since this is for a GraphQL API. I think the solution they've selected (Kysely) is probably a better fit.
Followed by:
> AWS Aurora Serverless v2 Postgres
I hope they don't get an unpleasant shock when they see the IO pricing on their aurora bill (it's per block read/write rather than per row, but that's proportional in the random access case, and you're also beholden to the query planner, so basically the same).
https://aws.amazon.com/about-aws/whats-new/2023/05/amazon-au...
With actual servers and prisma running close to the underlying database, it should be much better.
Regarding using joins, that's not necessarily the best choice either. When using simple flat joins naively, you get a lot of repetition for the more toplevel nodes of the join tree, which eventually adds up to a lot of network traffic and allocation. (TypeORM tends to have this issue, for example)
What you really want are joins that build the JSON on the server. In MSSQL this can be as simple as `FOR JSON`. In Postgres it can get... a bit more involved - Drizzle can do it: https://github.com/drizzle-team/drizzle-orm/releases#:~:text...
This is probably okay, although I do wonder what happens after you hit the compute limits of your database server (i.e. scenarios when you don't use things like planetscale). In those cases (again, non-serverless), Prisma's choice might work better as long as it runs close to the DB. It will still likely be slower than the fancy JSON join, but probably need fewer database resources (depending on how its done).
Prisma has some support for joins with relations: https://www.prisma.io/docs/concepts/components/prisma-schema...
Also, you need to specifically create a transaction to encompass your Prisma calls to limit them to a transaction. This is the default for PostgreSQL as well.
organizaiton(id, name, description) -> users(id, name) -> posts(id, content) -> comments(id, userId, content)
We want to get one organization with all its users. Lets imagine our org has 100 users, each of which has 100 posts on average, each of which has 100 comments on average. In a naive, flat join, we'll be repeating the organization description column 1 million times, and each post's content about 100 times on average.
This can still be problematic without joins, as sending 1 million comment IDs in a query to the DB isn't going to work, so even the multiple queries version will need to do better than that
I wish postgres had FOR JSON (https://learn.microsoft.com/en-us/sql/relational-databases/j...)
Old man shouting at the literal IT clouds.
In 2021 it seemed the TypeORM project was badly maintained and quite buggy - https://news.ycombinator.com/item?id=26888369 . Is that still the case?
But the projects we did with it where not very high scale. Less than 5 requests per second. So a single server affair.
Seems all these shops starting out with MongoDB or the like always need to do a huge, complicated migration to a /proper/ optimized data store, when they could have started with a sane design in the first place...
Whereas MongoDB is "simple".
First thing to note, I couldn't find any reasoning for the change in the original post. This makes the speculating on it rather pointless, but clearly you are able to fill in the details quite easily somehow.
As a counter point. Last four businesses I've worked at all ran on MongoDB. More happenstance than something I've looked for.
They are all relatively small businesses and all ran fine, the biggest was ingesting 1-10GB/day of new data from their IOT fleet.
So this perspective that everything mongo is fundamentally broken and plain bad is pretty foreign to my real world experience.
These shops all liked what they were doing and there was zero talk of scaling issues or changing out the database.
Good lord
At least the underlying data is well modeled, just need to rip out the JS library and replace it… but with what?
[0] https://mikro-orm.io/
This is expected pricing. The pricing is based on the load on the db, not based on results.
Usually there are two ways working with this:
DB supports showing estimated query plan , which explains how query would be ran without running it. Allowing to capture any missing indexes or filtered indexes.
DB allows only querying against indexes (this is usually done by no-sql dbs which have flexibility to deny bad queries).
I picked Prisma because I'm not actually a backend developer and the tradeoffs were okay for my use case.
I wish performance was better, but a few indexes here and there went a long way.
The only surprising bit for me was peak memory usage - it comes in bursts and can get to hundreds of MB when doing even a simple query.
The queries themselves are also kind of out there in terms of complexity but so far I haven't found a reason to rewrite them in raw SQL, since that isn't where our performance bottlenecks are at the moment.
The lack of rollbacks is one legitimate pain point and I wish they addressed it.
Could you elaborate on this part?
https://github.com/prisma/prisma/discussions/4617#discussion...
Tl;dr it's a design decision, not everyone (including me) agrees.
I do get the feeling they went YOLO and built their solution on an over-engineered approach with a stack they were unfamiliar with, and when they hit problems, they YOLO'd into another solution without properly evaluating why, which feels like a team of junior developers run amok.
They also seemed to not really understand relational databases, and why foreign keys are kind of important when you are choosing a relation database implementation (yes you can do without them, but it's best to understand them first).
With that said I was planning to use prisma and went and looked at issues on GitHub. They’re egregious … and I will not be using prisma
The library also looks like it was written in a yolo way as well.
My guess is the graphql background has permanently stunted it, as the project manager keeps wanting examples for performance issues instead of realizing the entire fact joins are missing to begin with is idiotic and absurd.
So my guess is they’re piecemeal fixing one off issues instead of fixing the problem at its root, probably because prisma code is not designed with that in mind from the beginning
Since building a SQL generator (https://aihelperbot.com) as a side project, I have become much more proficient in SQL and even though I am also locked into Prisma, I use the `queryRaw` all the time to execute raw SQL queries. You can understand the code without knowing Prisma API. It is more performant. For more complex SQL queries, I use the SQL generator for initial suggestions and adapt if needed.
For the next projects I build I want to use the minimal Postgres client (https://github.com/brianc/node-postgres) combined with a lightweight migration library.
Also prisma raw query builder and migrations are really good.
I don't see all the hate, you can mix and match both approaches just fine.
EF converts your code into SQL with the help of the compiler, and is not purely a runtime affair. This is what makes it relatively more usable.
To illustrate, if JS/TS were to approach the problem in a similar way, you'd do this:
1. Allow queries to be written like this:
2. Use acorn/espree to parse this into an AST3. Convert the AST into SQL, hand over to runtime helpers.
Doing so will allow queries to be written in the most natural way possible.
I was saying that ORMs should make use of parsers and modern language features (such as first class functions) to offer a unified interface to collections in memory and collections in the db. Essentially treating the table as a list on which you can use operators you're already familiar with.
By doing so, your core API is timeless. Because you didn't invent an API.
It always strikes me as odd that teams can embrace GitHub, TypeScript, and VS Code and then react negatively to C# and .NET being Microsoft platforms.
EF Core is perhaps the most powerful, most mature ORM at this point. .NET Web APIs are incredibly productive and performant with really good DX. C# isn't that far off from TypeScript syntactically and structurally.
I feel like teams that are "graduating" from TypeScript on Node probably want C# on .NET but end up exploring Go or Rust instead.
Rebranding would have helped in sooo many ways, even for the devs that moved forward from .NET Framework.
For starters, it would have at least made search results less confusing and more contextual. There were years of confusion between .NET Framework, .NET Standard, .NET Core, and .NET 5. From a basic SEO perspective, rebranding would have been huge.
It was a huge mistake to not rebrand it entirely.
Contrasting this with Go where I landed a gig after two weeks of learning by myself and felt productive right away.
A small repo here: https://github.com/CharlieDigital/js-ts-csharp
And a practical example of a Playwright web scraper in C# and TypeScript: https://github.com/CharlieDigital/playwright-scrape-api
"Too many keywords" is the weirdest objection to a programming language versus actually using the language to build something practical.
Edit: I'm actually curious as to which keywords you personally found the most challenging to grok or what tipped the scales of "this is too complicated".
EF Core expression trees are nice, however, the change tracking behind the scenes are an absolute menace. I've gotten errors more times than I'd like, because EF Core had loaded a key in its store, and another query got the same key, resulting in an Exception because both touched the same record in the change tracker. Even if it was two separate requests.
I've become a purist sort of, using sqlc for golang, dapper for c# and sqlx for rust.
ORMs are 'fine' for simple crud apps, but once you go a bit deep in the sql, it becomes impossible to reason about. Telling an engineer that they can just run `explain analyze` on their query, and them having no idea how to actually reproduce their sql is not great.
This is also just my opinion, I've been in the same place, I used to love EF, but now I dread having to debug that stuff =D
Last time it was sending a non pointer to gorm in golang, panicing the application :,)
Then they are screwing around with AWS Lambdas and package size, but what is the advantage of the Lambdas if they don't make your deployment easier? Just have a ECS and run whatever you want for better price/performance ratio and much higher performance ceiling.
I don't get which of their tools are improving developer productivity or software quality. It looks like every tool stands in their way. Why won't they just learn SQL and use it? What's is so low productivity about it? It's the highest level data manipulation language there is. Now they are bringing Kysely for supposed type safety. Yeah, good luck debugging, optimizing or changing the queries in any way. You'll have to convert from SQL to TypeScript and back whenever you'd want to develop the query in your SQL console.
What I see is a bunch of poor choices. Shiny tools standing in the way of getting stuff done.
It’s hard to remember that, at this point, most tech influencers are shitposting or are full of shit.
Additionally, this article is really poorly written.
> Last week, we completed a migration that switched our underlying database from MongoDB to Postgres.
Okay cool, but why? MongoDB is a very capable and fast database.
> It was a shock finding out that Prisma needs almost a “db” engine layer of its own. Read more about it here: https://www.prisma.io/docs/concepts/components/prisma-engine...
If you did any research on Prisma rather than diving in head-first, you'd realize this is a core part of why Prisma exists.
> we discovered that at a low level, Prisma was fetching data from both tables and then combining the result in its “Rust” engine. This was a path for an absolute trash performance.
Can you confirm this is actually the case? Can you show some benchmarks re: this claim? Or are you just assuming this is the case? We're using Prisma in production and it is blazing fast.
Isn't that standard practice for inserts?
I started with Aurora and DataApi and switched to Prisma and PlanetScale. I have every intention of staying with PlanetScale but I’ve considered rewriting my codebase to not use Prisma. All the raw queries would be easy enough and I think I’d be happier with a type safe query _builder_ instead of a full ORM but I could be wrong. I’m almost glad Prisma’s aggregate helpers are such trash since it made me just write my own queries instead of standing on my head to make Prisma’s work.
One thing I won’t miss is scouring my codebase for .prisma folders that might contain an outdated schema file that the engine is somehow picking up. I even have an npm script that I run to find and delete all but the main one I know is supposed to be there.
https://rdbjs.org
It has a very small fooprint - well suited for Amazon lambda. We now support client over HTTP (in a safe manner), making it accessible in-browser.
Key Features:
No code generation required
Full intellisense, even when mapping tables and relations
Powerful filtering - with any, all, none at any level deep
Supports JavaScript and TypeScript
ESM and CommonJS compatible
Succint and concise syntax
Works over http