It's great that we're talking about tracing / OTEL (Open Telemetry) in the context of GraphQL. We've recently added OTEL support to WunderGraph, allowing our users to trace through the Gateway and their services. Some more info on our OSS implementation can be found here: https://docs.wundergraph.com/docs/guides/monitoring-and-obse...
In addition, we've deployed all the required infrastructure like an OTEL Collector and OTEL backend using Clickhouse to offer a managed end-to-end solution including dashboard for traces and a trace overview. A live demo can be found here (sign up for free into the demo org): https://cloud.wundergraph.com/wundergraph-demo/apollo-federa...
> one of the reasons I was told it was great was because it solves/avoids the n+1 problem
So, the thing about GraphQL is that it's a _specification_, in the same way that REST or gRPC are specifications.
There's nothing inherent in either GraphQL or REST that admits or prevents the N+1 problem. You have endpoints with business logic (or in GQL's case, "resolvers") and there's no telling what's inside of those methods code.
It's true that GraphQL was built to integrate with tools like a "Dataloader" to solve "N+1", but you could just as validly be using a Dataloader in your REST or gRPC API endpoints.
I have used Dataloaders in the past in both Go and Typescript and it just about solves the issue automatically. Dataloader is a specific library that is copied for different languages that allows you to write code that looks like N+1, but it will group the N part into batches, so it becomes just 1+1. It’s kind of weird to explain, but it should be easy to understand once you start working with them.
Dataloaders replace N+1 with E queries (where E is the number of separate entity types to fetch) and an in-memory merging of all datasets (huge if you're not careful).
They are a solution, but not necessarily a good one. And GraphQL doesn't lend itself to goos solutions.
N+1 only makes sense for E=2. For each parent in the list, fetch the child’s data. If there were multiple children, or if the children had their own children fetched 1 at a time, it would no longer be N+1.
So I think 1+1 (or 1+N/batch size) is still correct.
Alternatively you can use things like Postgraphile or Hasura and truly eliminate both N+1 and 1+1. CTEs and JOINs really are underrated. Both solutions allow easy review of the SQL they generate, they do all of the CRUD work for you, and new resolvers are just a function away.
MySQL isn't invited to the party however. Just Postgres.
It doesn’t solve it, but it lets you build a system that solves it for all future queries going forward. Think of it as a platform for taking endpoint specifications (ie GraphQL queries) and satisfying them efficiently. You have to build the part which makes it efficient, but once that’s done, it’s done.
GraphQL eliminates the n+1 problem for the client by pushing it to the server. Generally, the client<->backend connection is slow and spiky, and the backend<->db connection is fast and stable, so this trade-off made sense for Facebook as they had more and more mobile users connecting via 3G networks in developing countries.
N+1s are a fundamentally a data modeling and querying issue.
People who complain about GraphQL having a lot of N+1s are complaining about their lack of fundamental knowledge. You can build a GraphQL with minimal N+1s and you can build a “standard” API with loads of N+1s
For the first production-scale GraphQL servers I built, I just had it dump ascii art representations of the downstream request waterfalls during dev. This was enough to visualise and fix any n+1 issues.
13 comments
[ 3.3 ms ] story [ 38.3 ms ] threadIn addition, we've deployed all the required infrastructure like an OTEL Collector and OTEL backend using Clickhouse to offer a managed end-to-end solution including dashboard for traces and a trace overview. A live demo can be found here (sign up for free into the demo org): https://cloud.wundergraph.com/wundergraph-demo/apollo-federa...
But from this it sounds like that's something you have to detect and code for directly... which we'd have happily done in the existing code anyway?
https://news.ycombinator.com/item?id=36382212
There's nothing inherent in either GraphQL or REST that admits or prevents the N+1 problem. You have endpoints with business logic (or in GQL's case, "resolvers") and there's no telling what's inside of those methods code.
It's true that GraphQL was built to integrate with tools like a "Dataloader" to solve "N+1", but you could just as validly be using a Dataloader in your REST or gRPC API endpoints.
They are a solution, but not necessarily a good one. And GraphQL doesn't lend itself to goos solutions.
So I think 1+1 (or 1+N/batch size) is still correct.
MySQL isn't invited to the party however. Just Postgres.
People who complain about GraphQL having a lot of N+1s are complaining about their lack of fundamental knowledge. You can build a GraphQL with minimal N+1s and you can build a “standard” API with loads of N+1s