GraphQL is the peanut butter to Reacts chocolate at FB.
It works there because
1. Every user is logged in. Is there anything you can do at FB without giving up something to the Zuck?
2. Because it's all behind a login, you can front load the first login/request with a giant SPA and then run all the custom queries you want.
3. everything at FB is some sort of blended context (my user, someone else's user, a permissions interaction)... security is baked in to every field.
4. Because of permissions, and login requirement it's hard to be a bad actor (FB just locks you out, nothing is public).
If you have a SPA and logged in user requirement and that robust permissions model then GraphQL might make sense... Otherwise its a shining example of conways law and might not be fit for your org... The same can be said for react too.
presumably there is still some authorization requirements though? The logged in user is authorized to see details about Friend 123, but not about Non-Friend 456?
What if Facebook wanted to open up parts of the site for users without an account, would that require a major reengineering? I've wondered why Facebook (and Instagram) are so strict about not showing anything to logged-out users, could technical reasons be part of that decision?
I think this currently exists in some shape or other, I don’t have a facebook account but I can click on profiles of businesses and get their opening hours and pictures of their menus or whatever. In think there must be some stuff with a public context.
Candidly, I think a lot of the "security" is baked into the data model of what FB is... Are we linked on the graph, are we linked directly, what do your permissions allow me to do based on that relationship. It isn't a difficult query to wrap most requests in. I dont think FB needs that many people keeping an eye on this.
That having been said, outside that data model, your absolutely correct that its going to be costly to maintain those extra layers of relationships.
No, GraphQL makes sense at facebook because at their scale, dealing with the consequence of allowing all possible queries was less work that having to create dedicated enpoints for all the possible client queries.
People completely missed the point of GraphQL, which is you TRADE flexibility for the client for added cost on the server.
Which in a app and team as huge as facebooks' made sense, especially since they have the so-called facebook apps that could do... anything.
> People completely missed the point of GraphQL, which is you TRADE flexibility for the client for added cost on the server.
I mean, isn't this THE selling point of GraphQL? How was it missed so badly? I think every elementary resource about the technology covers this in the pros and cons section...
GraphQL was the shiny new thing that everyone was excited about and I was also trying to learn and use it too, but I've never found a compelling use-case to use it over REST API. Now that you shared your experiences with it, it give me more reason to stay away from it. Plus, I think it's more productive to stay with the stack I'm very fluent at.
I never got into GraphQL. It always felt like a lot of complexity for little gain (I'm full-stack). People always jump on tools created by tech giants but they solve different problems than the vast majority of companies.
Same with Redux. Solved problem in FB. Increases complexity tremendously for 90% of other websites. I remember seeing the tutorial being showed in Todo app and thinking "wtf is this garbage needed for here".
I'm glad Redux hype train is over and nobody is using it on new projects no more.
I think the hype is mostly over because of useReducer. It's the simplest thing that lets you structure your code in the way you want if you'd had chosen Redux (or similar).
If it takes more than a minute to understand how a single button example works, one should give up. Maybe some day I'll come crawling back to Redux, that'll be fine, but so far it hasn't happened. Same with Angular.
I think Angular fits very well for certain public, that public being Java and .NET developers. Extremely similar concepts. For rest, yes, it's a disaster.
GraphQL basically only really works with monoliths that share the same access-pattern (either everyone logged in, or everyone logged out), it's otherwise a pain to merge multiple different graphql-schemas into a single one (or at least I'm not aware of an elegant way to achieve that).. The worst of two worlds is a micro-services back-end with a graphql Api interface, it makes testing individual services and the Api as a whole quite annoying.
I do really like the Api definition part of it though - but I found something like typespec now to be the perfect middle-ground, it allows out you describe your apis similar to a graphql Api without enforcing the graphql engine on you though.
I agree with the article, and want to add on that the browser network debugging tools are a pain to use with GraphQL (every request to /api/graphql, can't even quickly filter by endpoint name). I landed instead on OpenAPI, which can feel like a bit of a hoop to jump through sometimes when writing endpoints (but then, so can GraphQL), but the result is equally nice. And it's much easier to get authorization right with REST APIs.
I wonder if GraphQL would have been as popular if it was a much smaller shop than facebook launching it. Feels like having a name like FB gets you a tech credibility pass.
For the browser network debugging tools: I'm using the Apollo GraphQL client and added a link that adds the operation name into the URL. So my URLs look something like "/graphql?op=getStuff" and my GraphQL queries: "query getStuff { ... }"
There are browser extensions which make it easier to debug grapgql. A new pane is added to the browser debug panel. Last I used them more than two years ago, it was still not as good as the built-in network tab for rest queries. Still better than the default for the graphql queries.
Haven't used GraphQL but the idea of exposing queries from the client directly to the DB is totally bananas, even behind a login with a separate auth. Even just explaining your DB structure is a thing you should not do.
Graph models need not be your tables . They should reflect your data models, the same kind of models that would have been exposed in a traditional RESTful API?
Similar concepts of allowing queries and response formats was there in other standards and not new ?
OData is fairly popular in REST world , even older , SOAP XMl had supported with custom query language one robust one I remember T-SQL used in Taleo the old Oracle hiring software
There are many examples of clients doing joins and queries in other standards , it is hardly unique to GraphQL
Not a fan of GraphQL (the problems mentioned in the post are very real), but GraphQL specification does not say anything about the DB design / queries. It is still an API layer and you are free to model it in any way you want.
Indeed, from one of the authors of graphQL on this very forum:
"That would be one way to implement the system in a DB-centric way. However we believe that intermediate application code is pretty critical to any GraphQL implementation." [1]
"GraphQL is a client-server dance that needs server-side capabilities, not just CRUD semantics." [2]
thats not a graphql thing. my startup uses graphql and it does not map anywhere near closer to 1:1 with our database. what you're talking about is the litany of RAD api backend apps that take your db and expose a graphql endpoint.
> the idea of exposing queries from the client directly to the DB is totally bananas
I don’t necessarily disagree but going to play devil’s advocate here on this common foundational viewpoint. Why is this bad? In fact, why not expose the DB directly to the client?
“Additional layers” and “don’t explain DB structure” is security through obscurity. If a DB manages access control directly (field-level or partition-level granularity), any permissible query should run. To prevent pathological queries, add role-based resource quotas and back-off policies.
What about zero day vulns? Well these are going to be a problem regardless of the presence of various infra layers on top of the DB; you’ll have to monitor for data exfiltration to detect these types of attacks in either case.
There’s not a database I’m aware of that handles all of the above considerations, but oftentimes security issues exist in the overlooked gaps that arise between the complex interplay of many dependent systems. If you move the responsibility for security to the DB itself (other than authentication), you’ve removed a lot of the “confusion” that can be exploited, and overall security then depends on just a few factors: 1) correct assignment of DB access permissions, 2) valid external authentication, 3) quality of data exfiltration monitoring.
You might say you’re one simple DB misconfiguration away from disaster, but again, this is already a problem. The benefit of this alternative approach is that you have now shifted the bulk of the security work to one place: very careful rollouts of changes to DB access permissions. You would conduct static analysis to determine the “diff” of how data access has changed and perform sandbox tests. In exchange for this additional work, you can quit worrying about the security of anything downstream of the DB, assuming authentication is set up correctly.
I'm not saying security by obscurity is true security. It's one thin layer, though.
>> If you move the responsibility for security to the DB itself (other than authentication)
And what about all the possible access levels within your auth? Should every DB view have to check a user table to authenticate that the viewer has the priority level to select on it? If it doesn't, all I need to know is the name of a view I'm not supposed to see. What happens if you add a new security level... you change every single view definition? What if your auth + user levels are in a completely different DB in another time zone? Why put that burden on the DB if you get flooded with bad queries from hundreds of clients? That's why I say it's crazy, instead of writing an API endpoint that runs auth first and knows which views or tables are accessible from there and to which users, all in one place. Yes you have slightly more work conforming the client data definitions to whatever is returned from the API, but that seems like a very small price to pay to put all your security and filtering/validation in an easy-to-read middle layer. Of course you should use views and DB perms, but those are not sufficient if someone steals a login or session unless there's a middle layer checking their IP and hash and potential double-taps and lots of other markers that it might be a rogue query before it goes to the database.
Fully agree with the article. I think the summary is that GraphQL was a great idea from the frontend's perspective, but it was never fully worked out from a backend's standpoint.
I've been using tRPC for 6+ months on a large web and mobile project with great results. The only downside is you're restricted to using TypeScript on both frontend and backend, but I was already going to do that anyway.
GraphQL always felt to me like it solved the easy part of the problem and offered nothing to solve the hard part. I have a similar response to things like Zanzibar. At the end of the day I want to go and get some stuff out of a database, and that complexity just ends up dominating everything.
That is for me gut feeling about AI .. but that is not discussion about that topic.
So in general had the same feeling about graphQL because I still had to write layer of code that gets stuff from database, get it limited or paginated. Maybe once you have enough endpoints then you can change your front end against already existing GraphQL - but for me that was never the case as 90% of work is adding new tables, new endpoints so graphQL feels like additional work with additional footguns for almost no benefits.
Worked on two GraphQL projects; I was quickly cured from the hype. I recognize a lot of points in this article.
In both these projects the GraphQL had started small. I came in during a more mature phase of these projects (2 and 4 years). That's where the requirements are harder, more specific, and overall complexity has grown. Adoption and demand on the API were growing quickly. Hence you logically spend more time debugging, this is true for any codebase.
But GraphQL has everything in it to make such problems even harder. And both these projects had clear signs of "learning-on-the-go" with loads of bad practices (especially for the N+1 problem). Issue descriptions were much vaguer, harder to find in logs and performance issues popped up in the most random places (code that had been running and untouched for ages).
Fun fact; in both these projects the original devs who set it up were no longer involved. Probably spreading their evangalism further elsewhere.
RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
GraphQL is very good for places where frontend and backend developers are isolated from each other (separate teams). Or rather places where you have data-producers and data-consumers as separate teams. If you have a big enough org eventually there will be many of such teams, interdisciplinary teams are not feasible at scale for everything.
It allows teams to work with less communication overhead. It solves more of a human problems than a technical problems, if someone think there is no value in that and bare metal performance is paramount that person never worked in a big org.
> RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
In some ways yes, in others no. For example it can be near impossible to see if a deprecated field in a REST API is still being used and by which clients it is being used. With GraphQL this is fairly simple.
Unfortunately GraphQL way of working is very different from normal REST APIs and often requires more complex server-side caching. The N+1 problem needs to be figured out upfront for every data-storage system used in the backend.
Couldn't disagree more. GraphQL encourages tight-coupling--the Frontend is allowed to send any possible query to the Backend, and the Backend needs to accommodate all possible permutations indefinitely and with good performance. This leads to far more fingerpointing/inefficiency in the log run, despite whatever illusion of short-term expediency it creates.
It is far better for the Backend to provide Frontend a contract--can do it with OpenAPI/Swagger--here are the endpoints, here are the allowed parameters, here is the response you will get--and we will make sure this narrowly defined scope works 100% of the time!
Not to mention the fact that GraphQL allows anyone messing with your API to also execute any query they want. Then you start getting into query whitelisting which adds a lot of complexity.
Can you explain what you mean by this? The GraphQL API you expose allows only a certain schema. Sure, callers can craft a request that is slow because it's asking for too much, but
- Each individual thing available in the request should be no less timely to handle than it would via any other api
- Combining too many things together in a single call isn't a failing of the GraphQL endpoint, it's a failing of the caller; the same way it would be if they made multiple REST calls
Do you have an example of a call to a GraphQL API that would be a problem, that wouldn't be using some other approach?
Then we just come back full round trip to REST where the backend clearly defines what is allowed and what is returned. So using GraphQL it is unnecessary complicated to safeguard against a caller querying for all of the data and then some. For example the caller queries nested structures ad infinitum possibly even triggering a recursive loop that wakes up somebody at 3am.
But GraphQL doesn't allow for infinitely nested queries; the query itself has to include as much depth as it wants in the response.
> Then we just come back full round trip to REST
Except that GraphQL allows the back end to define the full set of fields that are available, and the front end can ask for some subset of that. This allows for less load; both on the network and on what the back end needs to fetch data for.
From a technical perspective, GraphQL is (effectively) just a REST API that allows the front end to specify which data it wants back.
You are correct and I agree with you. GraphQL can be used effectively like this and I've seen one example where GraphQL is used like this. New endpoint can be defined very quickly and it is essentially like a REST API with the possibility of the client specifying what data it wants back (as you described).
The other extreme end example is to expose by default the entire data model (PostGraphile) and then getting lost in the customisation and authorisation.
The problem is that the client team can - without notice - change their query patterns in a way that creates excess load when deployed.
When you use the "REST" / JSON-over-HTTP pattern which was more common in 2010, changes in query patterns necessarily involve the backend team, which means they are aware of the change & have an opportunity to get ahead of any performance impact.
My blocker on ever using GraphQL is generally if you've got enough data to need GraphQL you're hitting a database of some kind... and I do not generally hand direct query access to any client, not even other projects within the same organization, because I've spent far too much time in my life debugging slow queries. If even the author of a system can be surprised by missing indices and other things that cause slow queries, both due to initial design and due to changes to how the database decides to do things as things scale up, how can I offload the responsibility of knowing what queries will and will not complete in a reasonable period of time to the client? They get the power to run anything they want and I get the responsibility of having to make sure it all performs and nobody on either side has a contract for what is what?
I've never gotten a good answer to that question, so I've never even considered GraphQL in such systems where it may have made sense.
I can see it in something big like Jira or GitHub to talk to itself, so the backend & frontend teams can use it to decouple a bit, and then if something goes wrong with the performance they can pick up the pieces together as still effectively one team. But if that crosses a team boundary the communication costs go much higher and I'd rather just go through the usual "let's add this to the API" discussions with a discrete ask rather than "the query we decided to run today is slow, but we may run anything else any time we feel like it and that has to be fast too".
It seems there is a recent trend of using adapters that expose data stores over graphql automatically, which is kind of scary.
The graphql usage I'm used to works more or less the same as REST. You control the schema and the implementation, you control exactly how much data store access is allowed, etc. It's just like REST except the schema syntax is different.
The main advantage of GraphQL IMO is the nice introspection tools that frontend devs can use, i.e. GraphiQL and run queries from that UI. It's like going shopping in a nice supermarket.
Not particularly scary. For something like Hasura, resolvers are opt-in, not opt-out. So that should alleviate some of your concerns off the bat.
For Postgraphile, it leans more heavily on the database, which I prefer. Set up some row-level access policies along with table-level grant/revoke, and security tends to bubble up. There's no getting past a UI or middleware bug to get the data when the database itself is denying access to records. Pretty simple to unit test, and much more resistant to data leakage when the one-off automation script doesn't know the rules.
I also love that the table and column comments bubble up automatically as GraphiQL resolver documentation.
Agreed about the introspection tools. I can send a GraphiQL URL to most junior devs with little to no SQL experience, and they'll get data to their UI with less drama than with Swagger interfaces IMO. (Though Swagger tends to be pretty easy too compared to the bad old days.)
But as people noted, it's not the "can this get the data" unit testing that's a problem here. It's the performance issues.
> I can send a GraphiQL URL to most junior devs with little to no SQL experience, and they'll get data to their UI with less drama
But that's like giving direct (read) database access to someone that was taught the syntax of SQL but not the performance implications of the different types of queries. Sure, they can get the data they want; but the production server may fall over when someone hits the front end in a new way. Which is, I think, what a lot of people are talking about when they talk about GraphQL having load issues based on the front end changing their call.
None of those are automatic. Any API you write still has to remember to put in LIMIT clauses, to avoid OFFSET. And then you have to actually write the API rather than have it generated for you.
There are no free lunches, especially with regard to security and sanity checks.
> It seems there is a recent trend of using adapters that expose data stores over graphql automatically, which is kind of scary.
I think that's the part where I have a disconnect. To me, both REST and GraphQL likely need to hit the database to get their data, and I would be writing the code that does that. Having the front end's call directly translated to database queries seems insane. The same would be true if you wrote a REST API that hit the database directly and took table/field names from query parameters; only... we don't do that because it would be stupid.
> changes in query patterns necessarily involve the backend team,
How does this follow? A client team can decide to e.g. put up a cross-sell shelf on a low-traffic page by calling a REST endpoint with tons of details and you have the same problem. I don't see the difference in any of these discussions, the only thing different is the schema syntax (graphql vs. openapi)
GQL is far more flexible wrt what kind of queries it can do (and yes, it can be constrained, but this flexibility is the whole point). Which means that turning a cheap query into an expensive one accidentally is very easy.
A hand-coded REST endpoint will give you a bunch of predefined options for filtering, sorting etc, and the dev who implements it will generally assume that all of those can be used, and write the backing query (and create indices) accordingly.
> and yes, it can be constrained, but this flexibility is the whole point
Flexibility within the constraints of what the back end can safely support.
To me, the "whole point" of GraphQL is to be able to have the client ask for only the data they need, rather than have a REST API that returns _everything_ and letting the front end throw away what they don't need (which incurs more load).
If you can't support returning certain configurations of data, then... don't.
And the client team using a REST API can do the exact same thing, by making more calls. There's no real difference between "more calls" and "same amount of calls, but requests more data".
That being said, it's a lot easier to setup caching for REST calls.
> It is far better for the Backend to provide Frontend a contract
It sure is better for the backend team, but the client teams will need to have countless meetings begging to establish/change a contract and always being told it will come in the next sprint (or the one after, or in Q3).
> This leads to far more fingerpointing/inefficiency in the log run, despite whatever illusion of short-term expediency it creates.
It is true it can cause these kind of problems, but they take far, far, far less time than mundane contract agreement conversations. Although having catastrophic failures is usually pretty dire when they do happen, but there are a lot of ways of mitigating as well like good monitoring and staggered deployments.
It is a tradeoff to be sure, there is no silver bullet.
This is what I wanted to say too. If your backend team is incapable of rapidly adding new endpoints for you, they probably are going to create a crappy graphql experience and not solve those problems either. So many frontend engineers on here saying that graphql solves the problem they had with backend engineers not being responsive or slow, but that is an org problem, not a technology problem.
At TableCheck, our frontend engineers started raising backend PRs for simple API stuff. If you use a framework like Rails, once you have the initial API structure sketched out, 80% of enhancements can be done by backend novices.
Yup. And the solution to that org problem is for the front engineers to slow down, and help out the "backend" engineers. The complexity issues faced by the back-end are only getting worse with time, the proper solution is not adding more complexity to the situation, but paying down the technical debt in your organization.
If your front-end engineers end up twiddling their thumbs (no bugs/hotfixes), perhaps there is time (and manpower) to try to design and build a "new" system that can cater to the new(er) needs.
GraphQL is the quintessential move fast and break things technology, I have worked in orgs and know other people who have done so in other orgs where getting time from other teams is really painful. It is usually caused by extreme pressure to deliver things.
What ends up happening is the clients doing work arounds to backend problems which creates even more technical debt
I never understood why this was such a big deal... "Hey, we need an endpoint to fetch a list of widgets so we can display them." "Okay." Is that so difficult? Maybe the real problem lies in poor planning and poor communication.
And the REST API can still get hammered by the client - they could do an N + 1 query on their side. With GraphQL at least you can optimize this without adding a new endpoint.
You can whitelist queries in most systems though. In development mode allow them to run whatever query, and then lock it in to the whitelist for production. If that type of control is necessary.
Most REST APIs I've seen in the wild just send the entire object back by default because they have no idea which fields the client needs. Most decent REST API implementations do support a field selection syntax in the query string, but it's rare that they'll generate properly typed clients for it. And of course OpenAPI has no concept of field selection, so it won't help you there either.
With my single WordPress project I found that WP GraphQL ran circles around the built-in WP REST API because it didn't try to pull in the dozens of extra custom fields that I didn't need. Not like it's hard to outdo anything built-in to WP tho...
> In some ways yes, in others no. For example it can be near impossible to see if a deprecated field in a REST API is still being used and by which clients it is being used. With GraphQL this is fairly simple.
You should log deprecation warnings.
But also if the client is composing urls/params manually then you are not doing REST, you are doing RPC.
Rest APIs should mainly use HATEOAS hyperlinks to obtain a resource. that is clients almost always call links you have provided in other reponses (starting from a main entrypoint)
The problem is you delegate a lot of the query building to the client, hoping that it will not suddenly change your performance profile by being creative and that you will have not missed an obviously expensive use case coming.
That's a huge bet, especially given that GraphQL is expensive in the first place, and given that the more you grow the API in size, the less you can actually map the cartesian product of all request params.
Ah but that's the beauty of GraphQL, a query can actually fetch data from several systems: the db, the cache, the search engine, etc. It's all abstracted away.
But let's say you have a timeout, and they have a retry, then suddenly, your server is now spammed by all the clients retry again and again a query that worked a week ago, but today is too heavy because of a tiny change nobody noticed.
And even if it's not the case, you can now break the client at any time because they decide to use a feature you gave them, but that you actually can't deal with right now.
Sure but queries are crafted in the client, that may know nothing about this, while a rest api, the requests are restricted and the queries are more likely under control of the server, which mean the backend will likely decide what fetches what and when.
It takes a lot of work to actually ensure all possible combinations of graphql params hit exactly what you want in the backend, and it's easy to mess with it in the frontend.
I'm not that much into GraphQL but I vaguely remember libraries that provide some kind of atteibutes you apply to entities/loaders and then pre-execution an estimated cost is calculated (and aborted if over a specified threshold).
API Platform for PHP is one of those graphql implementations that has a query cost limiter built in (it's binary, it just rejects queries that go over your configured complexity threshold). Shopify's graphql api is even fancier, where every query costs X amount of a limited number of "credits". The structure of gql itself makes costs easier to estimate (you have as many joins as you have bracket pairs, more or less), and some servers can recognize a directive in the schema to declare the "real" cost.
There’s a free query depth. There’s ways to do query cost but federating that then becomes really annoying. Coming from the security side, there’s always a long curve of explaining and an even longer curve of mitigating.
I always am happy when I get an elegant query working. Often however I just find I wasted time looking for a clean 1 query solution when iteration by caller was the only solution.
I'm not sure this is any more or less of a problem for REST APIs. What if your engineers change $client/$server and the new version makes really expensive queries? Well, ask them not to do that, then when some of them inevitably ignore you, start to review their code, terminate long-running queries, batch or pool fanouts so they don't take anything down, monitor new releases and roll back if anything breaks, etc.
If you're providing an external API like GitHub does, then that's a different story and I agree.
Obviously depends on the API but a REST API that maps relatively cleanly to database queries is going to make it very clear on both the client and the server when it’s not scaling well.
If, at page load, I’m making 100 HTTP requests to fetch 100 assets then as a client side developer I’m going to know that’s bad practise and that we really ought to have some kind of multi-get endpoint. With GraphQL that gets muddy, from the client side I’m not really sure if what I’m writing is going to be a massive performance drag or not.
There's a lot of relevant differences between REST & GraphQL. It is possible to construct a REST endpoint that simply can't do any of those things, and such construction is a mid-level developer task at best. For instance, pagination of "all posts ever" is not uncommon, and clients won't be shocked to deal with it. GraphQL is enough harder to characterize the performance of that it definitely qualifies as a change in quantity that is itself a change in quality. Hypothetically, both approaches are vulnerable to all the same issues, but GraphQL is far more vulnerable.
Sorry, what? The original suggestion was that a developer would change things and it would cause performance problems. That same developer can change either a REST system or a GraphQL system and introduce the same performance issues in the same way, probably by adding a horrible N+1 query, or unbounded parallelism, or unbounded anything else.
Yeah, the client can't change the query if you don't let it specify a query, this is trivially true, but the developer can go break an API endpoint with the exact same result while trying to achieve the exact same business outcome.
The much more constrained input of the REST query means that the effect of changes on the API are much more comprehensible. Performance testing a particular REST endpoint is generally practical, and if a dev doesn't do it, the responsibility is reasonably placed on them. GraphQL means that you may do something you think is completely innocent like changing some index but for some query you didn't anticipate it trashes the performance. The range of things the dev of a GraphQL endpoint must be keeping track of is much larger than a REST endpoint, arguably exponentially so (though generally with a low power factor in practice, the possible queries are still exponentially complicated), and taking on any form of exponential responsibility is generally something that you should do only as a last resort, even if you do think your powers will stay low.
GraphQL only exposes what you ask it to. There are plenty of pagination plugins for GraphQL frameworks just as there are plugins to REST framework.
GraphQL can be restrictive as REST if you want it to be.
The point is GraphQL can be "as restrictive" as REST, but if you want to enable more efficient queries by knowing all the data that the frontend is requesting, you can. But the opposite isn't true of REST. With REST if you want more advanced functionality like that you have to define your own specification.
> more complex, more expensive to maintain, consuming more resources,
Idk. Strawberry GQL and most GQL libraries are maybe equally as complex as the REST libraries for the same language. Strawberry and FastAPI I would say are equal in complexity and configuration.
It would be hard for me to say GQL is more expensive or consumes more resources. Opposite of the purpose and most uses of GQL.
If you have separation between front and back end, then the back end team can elect to serve REST APIs which only permit selecting, filtering, grouping and pagination that they know they can support within defined latency bounds for a given traffic level.
Thing get more problematic when there's vertical ownership for a feature, where the UI needs just a few extra things and you end up with a REST response which is fatter and fatter, in the interest of avoiding round trips and client-side joins.
The problem with killing correct queries that take too long is that it shows up as intermittent failure that's dependent on server load and data cardinality. You might not find it in testing and it ships a bad experience to the customer before bugs are found. Whereas APIs which can't be so easily misused make it much harder to ship bugs.
> the back end team can select to serve REST APIs which only permit selecting, filtering, grouping and pagination that they know they can support within defined latency bounds for a given traffic level.
Why do you think that they can't do that with GraphQL? GraphQL isn't open ended. Its a highly restricted syntax for calling nested resources. If a resource is expensive simply don't nest it and make it a top level field and it is the same as REST?
Lots of nested resources are by default efficiently served by GraphQL because they are mostly returning single object foreign keys. Something that would take extra calls with REST.
GraphQL can have the same restrictions and performance guarantees as REST but the later is not necessarily true because in REST there is no standard way to define nested resource access.
I think the point here is, if you have to involve a backend team to add restrictions to the graphql endpoints and try to make good educated guesses where those might be, then the idea of frontend not needing backend engineers to query whatever they need becomes less of an advantage. So is the complexity of setting up graphql and then having your backend team try and make sure no frontend engineers can do terrible queries better for the software than custom rest APIs where needed and standard resource APIs everywhere else. Obviously it depends on the project. But I find the complexity of setting up and managing graphql often isn’t worth the pain, especially with schema first resource api designs and tooling like Google’s AIP linter.
> if you have to involve a backend team to add restrictions to the graphql endpoints and try to make good educated guesses where those might be, then the idea of frontend not needing backend engineers
No because if you dont do that you have to involve more engineers anyways to build the REST endpoints and keep modifying the rest endpoints.
GraphQL is also default restrictive (i.e. exposes nothing). You don't need to add engineers to make it restrictive.
In Startups typically:
-> Frontend changes most frequently
-> Backend "utility functions " changes less
-> Data model changes the least
Without Graphql your "backend" ends up needing to have a lot of work and changes because it is constantly needing to be updated to the needs of the most frequent changes happening on the frontend.
With GraphQL the only time you need to update the backend is when those "utility" functions change (i.e. 3rd party api calls, etc) or the data model changes.
So you end up needing substantially less backend engineers.
> With GraphQL the only time you need to update the backend is when those "utility" functions change (i.e. 3rd party api calls, etc) or the data model changes.
This is akin to saying that "directly exposing the database is easier, you only have to change things if the data changes".
And yes this is true, but when the data changes, or the environment changes, the paradigm falls apart a bit, no? Which is what the backend code was for, insulation from that.
> In Startups typically:
Yes, so for a short lived, non-scaled application its far easier to do it one way, and maybe that's fine for most small apps (that will never scale far). I suspect a lot of the push back comes from larger, less nimble, more backwards-compat focused organizations/apps.
> This is akin to saying that "directly exposing the database is easier
Far from it actually. I am saying that in practice the data and queries that you perform on your Database actually tend to stabilize and you add less and less as time goes on.
By Allowing the frontend to select what combination of these pre-approved queries that you already approved it can use, you have to do less and less backend work when compared to REST where you have to do backend work for every query combination you want to serve.
> maybe that's fine for most small apps (that will never scale far).
I mean saying GQL doesn't scale for big apps is over looking one of the largest Corporate Software Orgs (FB) created and use it in production purposefully for managing large software APIs.
> By Allowing the frontend to select what combination of these pre-approved queries that you already approved it can use
Sure, so you are just filtering raw database access then. That doesn't make it any different - and, you still need to approve and filter these queries, so what exactly have you saved? I.e. either the front end engineers can change these filters, or not, so it amounts to the same thing in the case they can.
> I mean saying GQL doesn't scale for big apps is over looking one of the largest Corporate Software Orgs (FB) created and use it in production purposefully for managing large software APIs.
That's not a great argument, though, saying a large company with many resources is capable of supporting something does not make it a sustainable technical decision. They likely also have a very specific work structure they use to make it for them.
In fact thats a strong reason not to use it, if it requires enterprise level resources to use it effectively. There is a big difference between technologies that scale to enterprise and technologies that require enterprise...
It still comes down to, if you can achieve 99% of the same thing with autogenerated REST apis and a couple page specific apis, what, exactly, is worth the considerable increase in complexity for that remaining 1%? Making things regularly more complex is a hallmark of failed, bad technologies, and I suspect GraphQL will see the dustbin like SOAP did...
>You are bouncing back between it is ony for startups and it requires enterprise level maintenance. It can be used easily for both.
No, I never said that. You are the one that brought FB into the equation.
Just because it can be used for something does not mean that it should.
I said that that approach doesn't scale well, especially for frequent data/model changes. For small apps, where as you say, you have few data changes, by all means embed your database as closely as possible to you end user code.
Sqlite inside a c app or electron, e.g. No need for any API at all! Just raw query access.
Its nice GQL to generate stuff for small non-changing web apps, I'm sure. But once you get into more performance oriented, data-migration-style stuff, if there's not good support for changing the data and reacting to the environment, then adding complexity (GQL) to an already complex situation is a Bad Idea.
You never said what this 1% was, autogeneration is not a bonus when you already have to manually filter and route things. The simpler solution gets you there as well, with less fuss.
You think you don't have page specific apis, but if you are doing the manual filtering, then you still have them, you are just "hiding" them inside another language, that doesn't have a clear benefit? At least you can't say what it is, without going in circles, another sign GQL is probably ultimately a garbage technology...
But you actually don't need to keep modifying the REST endpoints for most projects, that's what everybody is saying.
The vast majority of projects don't gain anything from this flexibility, because you don't have suddenly a 1000 of farmvilles copy cat that need their own little queries. You just have bob that need an order by.
Even a SQL query can suffer the same fate. Ever tried writing a SQL query against a distributed database that isn’t optimized for that read path?
I think that’s what’s really pointing out the root cause issues here, it’s not purely GraphQL’s problem, it’s the problems inherent to distributed systems.
> What if your engineers change $client/$server and the new version makes really expensive queries?
Yes, so the cost benefit here is not in favor of GraphQL. If both technologies ultimately suffer from the same issues (what to do about unpredictable clients), but one is far more complex to implement and use (GraphQL), then there's a clear winner. Spoiler, its not GraphQL.
Page specific endpoints, I would argue, can do 99% of what GraphQL was trying to do. If you want to use it as some sort of template language for constructing page specific endpoints, that could be useful (the same way xml schema is useful for specifying complex xml documents).
But you can optimize a page specific endpoint, and do it with REST-style endpoint to boot.
Having a bunch of "simple" calls and optimizing for the "complex" ones that you need using metrics/analysis is what you should be doing, not creating a complex API that is far harder to break down into "simple" cases.
When you build a GraphQL server, you’re creating a system that outputs page-specific endpoints. They can be generated just-in-time (the default) or at build time (the general recommendation).
The engineering work involved shifts from building individual endpoints to building the endpoint factory. This shift may or may not be worth the trade off, but there are definite advantages, especially from the perspective of whomever is building the client. And once you factor in the ease at which you can introduce partial streaming with defer and streamable (granted they’re still WIP spec-wise), the experience can be pretty sublime.
This? Yeah, that seems neat, for command/batch queuing.
I'd be curious how it compares to e.g. rest apis returning refs to e.g. webrtc streams or tcp/udp ones for non-browser. I presume the main advantage would be client side.
When the client’s data requirements change, isn’t there always a risk that the data loading performance profile will change?
Surely that is always the case, if the client is composing multiple REST requests, or if there’s one RPC method per client page, or with any other conceivable data loading scheme.
We have similar issues in our codebases - but not just in GraphQL, but also in our PHP and Elixir code, and to some extent our Typescript stuff.
I think the "learning-on-the-go" symptom, where you can sometimes literally read down a file and watch some developer learn the language as they add more and more functions to the file with a gradual increase in skill (or, to put it less charitably, as they slowly get less bad at writing code) is probably a very common thing, and not just a GraphQL issue.
> RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
REST API's are a proven solution for the problem of other apps, including front-ends, needing data from a data store. Using JSON is much improved over the days of XML and SOAP. Beyond that there haven't been advancements in technology that cause fundamental shifts in that problem space. There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.
There are so many developers out there that can't stand plugging in proven solutions to problems and just dealing with the trade-offs or minor inconveniences. Nothing is going to be perfect and most likely a lot of the software we write will cease to be running in a decade.
REST APIS suck for nested resources. GraphQL is a huge breakthrough in managing them.
Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.
With internal REST for companies I have seen so many single page specific endpoints. Gross.
> There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.
You could argue almost any tech solution in a non-pure tech play is largely in consequentially as long as the end goal of the org is met, but managing REST APIS were a huge point of friction at past companies.
Either it goes through a long review process to make sure things are structured "right" (ie lots of opinions that nobody can sync on) or people just throw up rest endpoints willynilly until you have no idea what to use.
GraphQL is essentially the "Black" for Python Syntax but for Web APIs. Ever seen engineers fight over spaces vs tabs, 8 vs 4 spaces, whether a space before a colon? those fights happened a lot and then `black` came out and standardized it so there was nothing to fight over.
GraphqL makes things very clear and standard, but can't please everyone.
> With internal REST for companies I have seen so many single page specific endpoints. Gross.
Hardly gross. It is what it is and it’s universal across the domain. I bet Windows has internal APIs or even external ones that were created just for one page/widget/dialog of one app. It’s the nature of things at times.
An engineer had to spend time to make that specific API for that page instead of the frontend consumer using what was already defined and get all the resources with one call and 0 backend engineer needed for that new page.
Is the flexibility worth the tradeoffs? Maybe in a company where you are adding new pages all the time with deeply nested relational data needs. But I would argue this is more rare than not. And I often find that frontend engineers aren’t as familiar with database queries and the load they are putting on the system with some of the graphql queries they are making. Flexibility for frontend has its own tradeoffs and I totally understand why a frontend engineer doesn’t want to have to wait for an endpoint to be finished. But this article outlines some of the issues you encounter later as you scale your team and system.
We use a schema first design where I am at and if a frontend person needs a new endpoint because the resource-only endpoints aren’t enough then they submit a pull request to the schema repo with a design for their endpoint they need. It gets approved and boilerplate is auto generated. Yes you have to wait longer, but 90% of the time (for our software) the resource endpoints work great.
I'm not sure where this narrative comes from that GraphQL immediately means that the frontend time will have no idea what they are doing and will induce load on the system.
95% of my nested graphql fields are based on foreign key indexes so its almost no additional load to the system (only 1 query per "level" of GraphQL) to query the nested objects.
I restrict GraphQL to 5 levels and now I have 5 queries per API call instead of 5 Queries over 5 API calls.
The backend team exposes the fields that they know are efficient to query. They can restrict the depth of the GraphQL, number of fields, etc.
On the other hand, it may take that engineer less time to create a page-specific endpoint than it would be to create something more generic that might serve other purposes (that may never come to pass), which may also involve talking to other teams, checking what future plans are, etc.
And that's assuming it's a new endpoint; if there's an existing endpoint that does almost what's necessary, they may need to check in with that team about what modifications to the endpoint would be acceptable, etc.
Single-page endpoints aren't great, but often times they're acceptable because they end up being a half-day task instead of a week-long slog.
What? GraphQL is purpose built to solve that in 1 Query. Not doing it in 1 query is on you not the protocol.
In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.
> What? GraphQL is purpose built to solve that in 1 Query. Not doing it in 1 query is on you not the protocol.
1 graphql query maybe. But that translated to a dozen SQL queries.
> In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.
The point you're missing is that for 1 graphql query the API did N+1 SQL queries, and then also joined them in JavaScript.
In the REST case the front end can switch to the efficient custom endpoint when it is implemented. In the graphql case it will never get any faster because the API has to stay generic.
The entire point of graphql servers is that they're basically ORMs (or use an underlying ORM) that turn complex nesting into a single query's worth of joins. It won't beat hand-crafted sql from an expert, but if that's your preferred approach, debates on the relative merits of different query frameworks are all academic to you anyway.
I've never seen this kind of graphql server implementation that can automatically boil down a complex nested query to sensible SQL. It sounds like the best of both worlds. Do you have links?
Typically libraries use a Dataloader + ORM to implement this which gets you pretty far, outside of that some libs like Strawberry with automatically optimize your queries for you if you return Django ORM Querysets.
Any query you were going to build and serve with Rest can be made with these two methods or even a raw dataloader and manual SQL string.
The inventors of GraphQL did not intend it to be mapped directly to a database.
"That would be one way to implement the system in a DB-centric way. However we believe that intermediate application code is pretty critical to any GraphQL implementation." [1]
"GraphQL is a client-server dance that needs server-side capabilities, not just CRUD semantics." [2]
Specifically, GQL implementations always map to a `resolve(source, args, context, info)` function (the names can be anything, the types are what matter). In that sense, you also get a standard server interface similar to wsgi/psgi/rack, but much more fine-grained (there can be a resolver for every field in the query).
Symfony API Platform boils to ORM-generated SQL via Doctrine, which is verbose, but not overly clever. So the only link I could give you there would be to the Doctrine query builder docs (I won't subject you to API Platform's docs). I imagine a more sophisticated graphql dialect like what Prisma supports can generate some pretty gnarly sql, but that's ORMs for you. But exposing your data model directly through graphql is also not recommended, same with REST APIs (I can't claim the high ground, my biggest graphql project exposes models directly. The tradeoff was worth it in this case). So in the end you're writing custom controllers/resolvers anyway, and field resolvers are where graphql really starts to shine.
This is like, an hour per endpoint. For maybe 30 endpoints (high figure for many apps) throughout the lifespan of your application. But let's say 3 hours: you're talking about 90 hours in total, over a period of 2 or more years. It's really not that much.
And GraphQL isn't free either; you need to actually implement that. It pervades your entire stack, too – it's not something you can just "put on top of it".
I think that in many cases, perhaps even most, GraphQL is a "spend 8 hours to automate a half hour task" kind of affair. Are there cases where that's not the case? Probably. Maybe your specific use case is one of them. But for the general case? I'm entirely unconvinced.
And no one will be spending days working on it, they will work on other things while waiting for code reviews and such.
> GraphQL libs in Python are equally as complex as the FastAPI etc.
You need to account for the fact that anything can query anything, whereas a more traditional REST-type API will have much more limited codepaths.
This starts with just basic database maintenance and indexing, which in many case will be significantly more complex, to all sorts of other problems. This article already enumerates them so I'm not going to repeat them here.
You can't just handwave all of that away with "oh there's a library you can use".
If you think all of that is worth it then that's fine. As with many things it's a trade-off, and a bit of a judgement call too. But please, don't pretend this complexity doesn't exist, and that the trade-off doesn't exist.
Every page is special. Using generalized API is waste of resources. Generalized APIs are hard to use and develop. It takes a genius to create a good generalized API (say SQL) and even those generalized APIs are not usable in web context (because of authorization and DDoS issues).
It's better to think about optimizing of creation specialized APIs.
A REST endpoint can be analogous to a specific method in code. just as much as a GraphQL field.
What people are excited about is that the frontend can request all the data it needs at once and the backend can efficiently serve it. Something not possible with REST without reimplementing something similar to GraphQL.
It's definitely possible, just not without tight coordination between frontend and backend. When your development teams are broken into silos with strong separation, this isn't feasible and GraphQL starts to make some sense. Otherwise, you just create an ad hoc endpoint that serves the exact data you need and call it a day.
I suspect that people don't think about it too hard and presume that they should have only one model - that is that their REST model should map directly to their DB model which should map directly to their internal domain model (the idea that there might be two-or-more-domains and not-all-of-them-are-public hasn't even yet crossed their minds).
When you realize that sometimes your REST model may be a view-model that maps to a different storage model optimized for online-transaction-processing, neither of which map to your internal domain model directly (there are adapters for that) _then_ you get somewhere ... but of course you then have to fight off the `transformToRestModel(transformToDomainModel(retrieveDbModel(theId)))` when the three models happily coincide for a time.
Unpopular opinion: I'm actually a fan of singe page specific endpoints. You get much easier debugging, easier to audit security, easier performance optimization an the imho pretty small price to pay is that it's "not elegant" and a bit of backend code
I think it's elegant. I loved MVVM in the postback world, and with SPAs, I see view-specific endpoints as the view-model continuing to reside on the server while the view was split off to the client-side.
Agreed. Specific endpoints. I was on a project recently where a 40+ yr old dev was going crazy over "pure REST." It's embarrassing. Sure, have your pure REST but FFS create specific endpoints if you they make sense.
You are literally Transferring the REpresented State of a given page/view/screen.
Screen-specific REST endpoints will make their way as a default in to a JS-based framework in 2025 and people will pretend like this is some breakthrough advancement.
Yup. Especially when your api really only has 1 (possibly 2 in the case of a mobile app) real clients. People like to pretend their apis are generic but they aren’t. There’s a good argument to stop writing generic apis for your single application.
Typically the endpoints clients are multiple frontend developers. If the frontend is blocked for every page they need waiting on the backend to expose data that massively increases the costs of features and reduces the time for delivery.
My experience is that the problem is avoided in theory, but not in practice.
Making a good API on a large system with many clients is always difficult. GraphQL makes it easier in theory, but if you have average level devs working on it, they’ll make a bigger mess than if they use simple REST. The latter will still be complex, but at least it’s easier to have observability.
Yeah. And a gentle nudge that Conway's Law is about lines of communication in general, not specifically the org chart.
I used to be in charge of the stored procedures that served as the API to a shared database system used by many teams. (So, not exactly the same thing, but I'd argue that the sprocs vs ORM debate isn't entirely dissimilar from the REST/RPC vs GraphQL debate.) My turnaround time for getting database API changes into staging for application teams to play with was typically less than one work day. But that happened because I had a lot of latitude to just get things done, because the company valued rapid delivery, and therefore made sure we had what we needed to deliver things rapidly, both from a technical and a business process perspective. This was in a very highly regulated industry where every change required paperwork that would be audited by regulators, too.
I've also worked at places where this sort of thing took ages. Typically the worst places for work backing up were organizations that used Scrum, because the whole "sprints" thing meant that the _minimum_ time to get something from another team was about 1.5 times the sprint duration. And even then you could only manage that if you could deliver a request that already met all of that particular team's requirements for being able to point it, and could convince their Product Owner that it was a higher priority than whatever their pet project is this quarter, and the stars align so that you perfectly time the submission of your request with respect to that team's Scrum process's frame rule.
The thing I want to point out here is that absolutely none of that bullshit was ever the API technology's fault.
The modern web development practices are just insane.
The GP's idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.
On the other extreme, we have the HTTP 1.0 developers saying something like "networks are plenty of fast, we can waste a bit of it with legible protocols that are easier to make correct", while the HTTP 2.0 ones are all in "we must cram information into every single bit!"
Every place you look, things are completely bananas.
> idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.
For me, what's crazy is that there are "web" developers who can't just add the endpoint they need while working on a frontend feature, or "web" developers who can't just add an element or a page for testing the backend endpoint.
What ever happened to full-stack developers? The "frontend" and "backend" developer split is so incredibly inefficient that it's really jarring—you take something that should take 2 hours and magically, through tickets, delegation, and waiting for results (then repeat that for debugging, who knows how many times!), make it into a 2-3 day task.
I once reproduced (black-box style!) a two-week effort by a three-man team in six hours of work simply because I had PyCharm and IDEA opened side-by-side and could write code on both sides at the same time.
If someone has a good explanation for why the full-stacks that were once the norm went almost extinct, I'd be happy to give it a read!
Because people have a limited appetite for complexity.
I wrote some front-end stuff back in the days but I've lost track of whatever is happening these days. jQuery to append some stuff took five minutes to learn, but learning react hooks takes a determined effort.
Likewise, adding a field to a graphql type is simple, but doing it with authorization, controlling n+1s, adding tests etc.. requires front-end folks to actually invest time in learning whatever back-end they're dealing with this time.
Everything is just a lot more complicated these days, and if you've been around for a while you may not be excited anymore by the churn, but rather fed up.
This is why the Rails community should be applauded in my book, for their dogged determination that we should keep it a “one person” framework. Yes it may not be as performant, type safe or flashy on the front end but my god it’s productive.
At my startup there are 7 devs who can all do tickets across the stack and as we grow I think it would be good if we could resist the pressure to silo and specialize
It really is. Regrettably, I've drifted away from it in large part because of client requirements for more "modern" and "maintainable" solutions (e.g. Python or Node; I'll take Python every time, thanks). Django comes very close in terms of productivity (and is better in some ways: auth, admin, etc.) but the Rails CLI, generators and community (not sure if this is still relevant) give it the edge.
The "recent" (last 10+ years) movement towards "lightweight" libraries (instead of frameworks) that require you to either reinvent the wheel, copy-and-paste or use some random "getting-started" template every time you start a new project is disheartening. As others have said above, I think it's partially resume-driven-development and people wanting to tinker (both of which I do appreciate).
Something which continues to surprise me is that there hasn't really been a "modern" successor to Rails. Granted, I haven't kept pace with developments in the Node/TypeScript world but, last time I looked, Sails was on the right track but wasn't very actively developed and I was shot down (in favor of Express spaghetti) when I suggested it. There's also a smattering of Rust web frameworks but none that quite fit the bill and come with all of the batteries and opinions included. I keep saying I'm going to do a Summer of Code project which attempts this but ... life.
There are some Java frameworks that are kinda similar? Take a look at Micronaut for example. It has data access interfaces that use ActiveRecord's naming approach, controllers, view renderers, and a whole lot more on top.
Thanks. I'll take a look. Didn't mean to overlook Java, I'm just less familiar with that environment. I've dabbled with some JVM languages and could also see Kotlin being a very nice option for such a framework. Though, to be fair, I know Java has come a long way in terms of expressivity, syntactic sugar, etc. (better performance is a given).
There's no modern successor to Rails because Rails itself os still modern and very much up to date. The recent introduction of Stimulus allows to implement a SPA completely on rails (if they wanted) with the same simplicity present everywhere else in the framework.
If you absolutely need 2 people for that, they should be side by side.
But yes, that break-down of the problem is insane. People artificially split an atomic problem in two, and go create all of that extra craziness to try to solve the communication problem they made.
And then people go and push UX and UI tasks into the frontend developer, and ops tasks on the backend since them are there... What is insane again, because those are really incompatible tasks that can't be done with the same mindset.
And since it's standard, backend tools won't support the frontend and the other way around. So the insanity gets frozen on our tooling.
For context: I work on large-scale browser apps that are closer in complexity to something like Linear or Obsidian than to your standard WordPress blog with some forms. E.g I'm currently working on a browser-based GIS tool for the financial sector.
I started my career as a full-stack developer, but went all-in on frontend because I felt I was spreading myself too thin. At one point I found that I could choose to be almost good enough at doing two different things or extremely good at one thing. I chose the latter option.
Modern browser apps are complex beasts, at least if you want to do them right. You obviously have to worry about all the technical bits --- HTML, CSS, JavaScript, your view library of choice, platform APIs like <canvas> and WebAudio, cross browser testing, bundle sizes, performance optimizations, techniques like optimistic rendering, all that good stuff.
On top of that, you also need to work closely with designers to make sure they know the features and limitations of the platform(s) they're designing for. More often than not, you end up being a sort of bridge between the backend devs, designers, and product managers.
A lot of times you end up doing design too, whether you like it or not. I've learned a lot about UI/UX design just because I often have to fill in the gaps where a designer forgot to include a certain error state, or didn't test their design on tablet screens, or didn't account for cases where a certain API might not be available.
I tried for many years to learn as much as I could about Django as well as React and friends. But it eventually got too much. I found that I wasn't able to keep up with both ecosystems, and I was producing code that wasn't very good. I could certainly build things quickly because I was familiar with all parts of the stack, but it came at the cost of code quality, security, stability, and robustness. I eventually decided to hang up my backend developer hat and focus exclusively on what goes on inside the browser (which can be a lot by itself these days!)
It's probably possible for a single individual to build a high-quality server-rendered MPA with some forms without making a mess of it. But that says more about how good Rails/Django/Laravel are than about the capabilities of any single individual. I don't think a single person could build a product like Linear end-to-end without cutting corners.
> ... I don't think a single person could build a product like Linear end-to-end without cutting corners.
Cutting corners is a feature. I bet the Linear team is as pained as any, internally, at the tradeoffs they're making.
There is no way to know "what to get right" without going through it. So for 80% of the dev cycle the job is to cut corners to get to the 80/20, rinse and repeat forever.
This isn't against excellence and the dream of a beautiful product experience as your reply seems to convey.
IMO the fact that being a full-stack dev is so taxing is an indication that the stack as a whole is just way too complex and overengineered. Which is rather obvious from looking at the state of affairs on the frontend side of things. Desktop GUI devs don't usually have those problems.
Thing is, even when web apps are essentially used as desktop app replacement (Electron etc), all the complexity discussed here is still there.
As far as looking OS-native, this is (unfortunately) increasingly less relevant as OSes themselves drop desktop UX consistency even internally. But that aside, Qt is still around and is a very mature framework for "near native" UI, and then there's Avalonia and others. Even Gtk is surprisingly decent on Windows these days. Although I'm not sure what this all has to do with the original subject, since web apps most certainly don't look native anywhere.
I don't think the claim is that a single developer should be able to build an entire enterprise product, but rather that a single developer should be able to implement the software side of a task end-to-end. The latter is a reasonable expectation when your software is monolithic.
Many organizations would rather pay 3 people $120,000 each instead of paying 1 person $300,000 to do the same work, for a variety of reasons. Some good, some bad.
Microservice architectures and the associated explosion in complexity on both ends are to blame. When it takes twice the time to build something, it is natural to hire twice as many developers. Increased complexity drives specialization.
Sounds good in theory. In practice, what I've seen is people don't just do single page endpoints with shared services, they follow this all the way down into the services and so you end up with lots of duplicate looking code. This isn't a problem until the application gets big enough and now you have business logic that should be the same spread out across an app. This leads to weird bugs where something that should work the same way doesn't depending on which page you're accessing things from. Of course, I don't a solution, solving things like this is what makes software engineering hard.
> In practice, what I've seen is people don't just do single page endpoints with shared services, they follow this all the way down into the services and so you end up with lots of duplicate looking code
There is no easy or simple or final solution. The solution is to do the hard work of software engineering and catch this stuff at code review time or to plan and do refactoring later when you realize something needs normalization. I wish developers would just accept that software engineering is hard and do the hard work necessary.
There's that whole "the best software engineer is lazy because he will automate stuff", but that does not give anyone a license to automate stuff in an unmaintainable way. Automating stuff is hard and there is no easy way out.
Came here for that. People in this thread basically reinventing GraphQL lol.
"We need a http API endpoint that gives you all the data for one page. And also be able to reuse parts of it for other pages." Yeah bro, this is GraphQL.
The backend for frontend pattern. Something most apps where frontend and backend are in the same organization (fullstack or otherwise) should have. It does wonder to maintainability and performance.
Even though we use GQL here, we still have a B4F, so it's browser -> B4F -> GQL -> Database.
The tooling we use make it trivial so it doesn't add any development overhead (often reduces it. No CORS bullshit for one), and our app goes ZOOOOOOM.
The point is GQL is currently the standard for querying dynamic nested data shapes not that it is the first or was the first.
Look at OData download stats on Pypi it had 2 downlaods the last day. Graphql-core for python? 624,201. that is not even on the same planet.
If you don't use GQL and want a system of querying nested data you will be using a less used protocol that is analogous to GQL so you might as well use the standard.
OData is still currently a standard for querying (among other things) nested data shapes. The point was it is a matter of fact that as a standard it predates GraphQL and as a matter of ecosystem perspective whether or not you think it is a more or less used protocol.
To trade anecdotes for anecdotes: On Nuget, the Microsoft.Data.OData package marked deprecated and marked with CVEs for critical bugs still has an average of 36.3K per day downloads. (Its successors combine for about double that.) The top GraphQL library (of the same name, unadorned) still only has 9.3K per day downloads. In .NET if you want a system of querying nested data (that is also "REST compatible"), why would you use a less used protocol like GraphQL?
> Any more levels and you have now reinvented GraphQL
Sure, but reinventing the wheel can be good, particularly when the existing wheel technology is oblong, lumpy and constructed of stone and cheese.
I’m just bitter than GraphQL endpoints return 200 on errors. If you’re returning 200 on errors, then you’re not really doing HTTP; you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.
> you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.
Browser does not allow to access any ports with any protocols.
Often intermediate software gets in the way as well. For example with Kubernetes it's trivial to expose HTTP service using Ingress and it's not trivial to expose other protocols.
Another HTTP plus is that it's trivially secured by TLS and every developer knows API for that. Using TLS for your own protocol or QUIC is absolutely not trivial and probably more like arcane knowledge.
One place I was at we used REST with a hydration service that ran as a proxy in front of the requests. That gave us most of the benefits of GraphQL and we only implemented it for the main resources (users, photos, and maybe one other thing). To minimize latency/faults the hydration service ran locally on every webapp/API server. I wasn't around for far too long after though to see how it turned out as it grew (if it did at all, the company sort-of went defunct--development-wise anyway).
I also recall, we had similar N+1 query problems in the REST API endpoints irrespective of hydrating the returned resources.
The biggest benefit of GraphQL I can see from a user perspective is that it lowers total latency especially on mobile with fewer round trips.
I currently use GraphQL and have no problems with it specifically, I was merely sharing an experience using REST. Perhaps it adds a bit of latency/overhead due to the implementation/language that's used but with larger requests rounds downward.
It doesn't. That's literally what GraphQL was designed for: To provide a resolver that sits between high latency clients and all of the various REST/RPC services in order to roll up all of the requests at datacenter speeds into one in order to overcome the latency issues.
But you still need all of the various single page REST/RPC endpoints to make use of GraphQL as it is intended. Some developers out there skip the REST/RPC part, making GraphQL their entire service, violating its principles.
If it works it works, but it does not come tradeoff free.
? why do you need a REST endpoint with GraphQL? Nearly every language has a GraphQL engine that integrated directly with the database and exposed functions as GQL directly without REST at all.
You don't, obviously – we already talked about how many don't. But if you don't have a graph of "microservices" in which to query, what do you need GraphQL for? Especially if all you are doing is returning the results from a database. The database will already have its own API. Why not just use it? The native API is always going be better than your attempt to translate.
Because exposing your Database to the outside world is asinine. GQL sits between letting the frontend query semi-customizable queries and not having any customizability to select related resources.
Where is this API coming from? You have to build it. I'm saying you should make those APIs graphql apis though a language framework and not REST apis because GQL consuming GQL is much better than GQL consuming REST.
Someone has to build it, but as long as you are using a popular DMBS the work is no doubt already done. All you need to do is stick your application-y bits in the middle – same as you have to do with GraphQL. Computers aren't exactly magic. It's all just 1s and 0s at the end of the day. You can do just about anything and make it work. But there are tradeoffs to different approaches. If GraphQL is the best choice for working with databases, why don't DMBSes use it natively?
I'm curious as to why you believe that exposing direct SQL over the database is "asinine" but GQL is fine, given that either one is very generic, and e.g. the security issues are very similar (as the article points out).
That is how REST works but is the opposite of the way GQL works.
You don't pile into existing defintions but extend the current definitions out. A team needing new data doesn't affect the consumption of other teams. which is not the case of REST where if one team needs to change a REST endpoint to return different shape of data, they have to verify the new shape with other teams.
With REST, the endpoints are team based if you have even a semi-competent CTO so you never have this problem, you just check who owns the controller and that's it.
With GQL though, good luck retracing who owns what field and what does it affect if you change the definition of what it returns, especially that you you are not even using it on the same language.
Bonus points here if you are managing something outside of your control like a mobile app.
GQL you would only care about if someone removed a field not if someone added a field. How would adding a field change existing GQL calls return? Doesn't make sense.
Also, Its about 1 line to set up CI to extract all GQL queries from a typescript project and do static analysis to check against the schema.
But again you only care if someone deletes a field, and even if you have to delete it, at least the GQL schema has built in annotations for deprecation, something not in REST.
Actually deleting happens rather rarely compared to adding.
Its much easier to trace and debug what teams are using GQL fields than REST. What if one team is piggy backing on another teams exising rest endpoint and you dont know? same problem that would require some analysis of all code calling and endpoint to determine if a field is safe to delete. GQL makes this much simpler than REST.
If an app's API requires more than a couple of nested structures then, without knowing the specifics of the domain, I would venture a guess at a poorly defined data model is the issue vs. API technology.
For example, a lot of times people build out nice normalized table structures for online transactional apps. The UI/UX is pretty straight forward because end users typically only CRUD an object maybe a couple nested objects at a time. The API's are straight forward as well, likely following single responsibility principles, etc. Then comes along requirements to build UI's for analytics and/or reporting types of things where nearly the entire schema is needed depending on what the end user wants to do. Its the wrong data model for doing those types of things. What should be done is ETL the data from the OLTP schema into a data warehouse style schema where data is de-normalized so that you can build reporting, etc.
> Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.
> With internal REST for companies I have seen so many single page specific endpoints. Gross.
As someone pointed out in reply to another comment, GraphQL is "a technological solution to an organizational problem." If that problem manifests as abuse of REST endpoints, you can disguise it with GraphQL, until one day you find out your API calls are slow for more obscure, harder to debug reasons.
> Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.
With REST, though, that pain is visible to both sides. Frontend engineers generally don't want to make N+1 REST calls in a tight loop; it's a performance problem that they see and that is very visible in their Dev Tools. Backend engineers with good telemetry may not know why they get the bursts of N+1 calls that they see without asking the Frontend or digging into Frontend code, but they can still see the burstiness of the calls and have some idea that something could be optimized, that something is maybe too chatty.
There are multiple ways with REST to handle things: pagination, "transclusions", hyperlinks, and more. Certainly "single page endpoints" is a way as well, no matter how gross it is from REST theory, it's still a pragmatic solution for many in practice.
REST certainly can please everyone, given pragmatic compromises, even if it isn't very clear or standard.
> With internal REST for companies I have seen so many single page specific endpoints. Gross
That's an established pattern (backend for frontend). Like all patterns there are trade-offs and considerations to make, but it's certainly not a priori "gross".
Perhaps that's the case for server-side transactions and ORMs over REST and gRPC to coordinate rather than allowing the front-end to diddle with data however it likes.
And, resources with an index option obviously should have a db index or unique index.
The challenges with GraphQL are that it makes it too easy to DoS services, leak internal data, break referential integrity, and there were a great deal of tools, infrastructure, and monitoring systems already available for REST (and gRPC to a degree).
Company standards for REST and coding style can and should be set in the diff review pipeline. Another facet is setting standards to minimize duplication of effort or exposing a greater attack surface.
I think us engineers have an inherent desire to try to innovate, and I think that is a good thing. Some problems will require a lot of wrong turns before finding the right path, but that is simply the nature of innovation
Agreed it's a good thing. Writing software today is still a slog that turns into a spaghetti bowl without focused intent by highly skilled devs. I think most devs realize this and really want there to be something better, and the only way to find something good is to find a lot of the bad around it first.
I like the thought experiment of adding a new persisted/editable field to an entity in a web app, if you've done full-stack you know all the layers that need to be touched to accomplish this and how lame most of the work turns out to be. After doing that 20 times while iterating, any dev worth their salt starts to wonder why it sucks so bad still and how it could be made better, and some will actually try.
JSON winning over XML is like saying CSV won over MySQL. They aren't equivalent.
Much like CSV, JSON isn't particularly standardised and different parsers and writers will do different things in some situations. Usually it doesn't matter, but when it does you're probably in for a lot of pain.
If you handle structured data and the structures might change over time, JSON isn't a good fit. Maybe you'll opt for JSON Schema, maybe that'll work for your use case, but with XML you can be quite sure it'll be reliable and well understood by generations of developers.
The tooling is generally very good, commonly you can just point your programming language to the XSD and suddenly you have statically typed classes to program against. Perhaps you'd like to store the data in a RDBMS? You can probably generate the DB schema from the XSD. If you want you can just throw JSON into MongoDB instead, but there will be very important tradeoffs. Same goes for UI, you can write some XSLT based on the XML schema and suddenly you get web views directly from API responses. Or you could use those classes you generated and have your GUI code consume such objects.
None of this is as easy with JSON as it is with XML, similar to how many things aren't as easy with CSV as with a RDBMS.
What's missing in ECMA-404? Never had a problem with JSON parsers or writers, using it all day every day for decades. It's crappy in some ways, sure, like lack of full floating point support, but standardization is not an issue.
XML is mostly already lost on the current generation of developers though, much less future developers. Protobuf and cousins generally do typed interchange more efficiently with less complexity.
There are a lot of proponents that some or all of the "JSON5" [1] improvements should be standardized by ECMA as well. Especially because there is a mish-mash of support for such things in some but not all parsers. (Writers are a different matter.) Primarily comments and trailing commas, are huge wish list items and the biggest reasons for all of the other "many" variant parsers (JSONC, etc).
This is more of a concern for JSON configs and other such uses that are directly exposed to humans, but not really for machine-generated and machine-consumed data.
It's focused almost entirely on syntax and ignores semantics. For example, for numbers, all it says is that they are base-10 decimal floating point, but says nothing about permissible ranges or precision. It does not tell you that, for example, passing 64-bit numbers in that manner is generally a bad idea because most parsers will treat them as IEEE doubles, so large values will lose precision. Ditto for any situation where you need the decimal fraction part to be precise (e.g. money).
RFC 8259 is marginally better in that it at least acknowledges these problems:
This specification allows implementations to set limits on the range
and precision of numbers accepted. Since software that implements
IEEE 754 binary64 (double precision) numbers [IEEE754] is generally
available and widely used, good interoperability can be achieved by
implementations that expect no more precision or range than these
provide, in the sense that implementations will approximate JSON
numbers within the expected precision. A JSON number such as 1E400
or 3.141592653589793238462643383279 may indicate potential
interoperability problems, since it suggests that the software that
created it expects receiving software to have greater capabilities
for numeric magnitude and precision than is widely available.
Note that when such software is used, numbers that are integers and
are in the range [-(2**53)+1, (2**53)-1] are interoperable in the
sense that implementations will agree exactly on their numeric
values.
But note how this is still not actually guaranteeing anything. What it says is that implementations can set arbitrary limits on range and precision, and then points out that de facto this often means 64-bit floating point, so you should, at the very least, not assume anything better. But even if you only assume that, the spec doesn't promise interoperability.
In practice the only reliable way to handle any numbers in JSON is to use strings for them, because that way the parser will deliver them unchanged to the API client, which can then make informed (hopefully...) choices on how to parse them based on schema and other docs.
OTOH in XML without a schema everything is a string already, and in XML with a schema (which can be inline via xsi:type) you can describe valid numbers with considerable precision, e.g.: https://www.w3.org/TR/xmlschema-2/#decimal
GraphQL does define the size of its numeric types: Ints are 32 bits, Floats are 64, so if you have a bigint type in your db, you'd best be passing it around as a string. Any decent GQL implementation does at least check for 32-bit Int overflow. Several people have independently come up with Int53 types for GQL to use the full integer-safe range in JS, but the dance to make custom scalars usable on any given stack can be tricky.
Frankly, if a developer can't figure out XML then they aren't worth their salary. Age is no excuse here; as a developer your job involves figuring out how to work with technology you haven't used before.
It differs from the RFC, notably "text" is valid JSON according to ECMA but not the RFC. I've come across JSON parsers stumbling on the bottom part of ASCII, for example. JSON -> internal representation -> JSON commonly leads to loss of information.
Sure, protobuf is nice, but more limited in scope and closer to a JSON alternative than an XML alternative.
I use JSON every other day and have been for decades.
Those two are kinda orthogonal, and while there was some overlap for adoption, it was fairly common to serve XML over REST early on (because more languages and frameworks had proven-quality XML parsers out of the box, so it was easier for the clients to handle).
JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend. Then other languages caught up with their own implementations, although in some cases it took a while - e.g. for .NET you had to use third-party libraries until 2019.
To claim that pretty much every .NET project adding Newtonsoft.JSON as a first step was somehow a problem is just strange. No adequate team would claim this to be a problem.
It was made so that the ecosystem could continue to evolve, particularly in terms of performance and security hardening. But okay, what was the reason System.Text.Json introduced in your opinion? What were the egregious problems with Newtonsoft.Json?
Sure, you can disable it, but the fact that it is opt-out to begin with - i.e. that by default the parser will try to creatively interpret any string it sees in JSON input and convert in a locale-specific manner that also quietly loses data - is, frankly, insane through and through. I've personally run into this issue many times in existing code - it usually happens when people first start using the library and just never run into any inputs that would trigger this behavior while testing. Then once that code is shipped, someone somewhere just happens to have the data that triggers it.
And if you look at the comments to that issue, there are numerous mentions from other GitHub repos due to bugs it caused for them, including some Microsoft projects.
The cherry on that cake was author's response indicating that he doesn't even understand why this design is problematic in the first place: "I like what it does, I have no plans to change it, and I would do it again if given the chance." I wouldn't trust any parser written with this kind of attitude.
> JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend
Browsers had XML parsers before they could handle JSON directly, and at the beginning there were complaints that JSON was harder to use for that reason. The reason why JSON won rapidly even for backend apps which never loaded it in JSON was ergonomics: every part of the XML world from the parsers to XPath/XSLT/XQuery to the rat’s nest of standards was plagued by the hairy-shirt “this is hard and should feel hard” attitude that has thankfully become less common. I saw so many people just burn out in the entire ecosystem because they got tired of unhelpful errors, pointless usability bugs around namespaces, low-quality or missing examples, and especially how common tools just stopped getting improved.
I maintain that the format would have been far more popular if all of the effort spent on standards work after the turn of the century had been suspended and the time directed to fixing things like the usability of namespaces in almost every parser, and hiring at least one person to work on libxml2 so developers could actually use features which shipped after 1999. Unfortunately it seemed like there were a ton of architects who really wanted to spend time building castles in the air and they just seemed to assume that someone else would do the boring parts of implementing it, but those people all jumped on JSON pretty quickly. I worked with a bunch of people who weren’t developers and the cycle of initial enthusiasm fading into “doesn’t this kind of suck?” with XML was depressing to watch having seen so much initial promise.
I think IDLs and gRPC in particular are a meaningful advancement in the problem space. The ecosystem and productivity that they enable via programatic tooling were really eye-opening for me (The OpenAPI ecosystem is, imo, extremely poor). They also have better support for modern techniques like streaming and even load-balanced streaming.
>> Fun fact; in both these projects the original devs who set it up were no longer involved. Probably spreading their evangalism further elsewhere.
Ah this brings up bitter memories. Team I was managing was roped in to become the first graphql framework (nay platform) the ivory tower (central architecture run by the cto) team was building and trying to shove down the rest of the company. This was during the graphql craze around 5 years ago. The principal engineer leading it was even preemptively promoted to distinguished engineer for future contributions.
Fast-forward 5 years project was disbanded due to massive migration, cost and complexity problems. DE still lauded for experimentation heroism and evangelism. I think he is now pushing the latest flavors of the month to go for the next promo!
>And both these projects had clear signs of "learning-on-the-go" with loads of bad practices
I think two projects having loads of bad practices is too small a dataset to really assume anything, you sort of need to see widespread "bad practices" in the tech to be able to determine that the bad practice is actually the norm practice and there is perhaps a flaw in the tech that encourages that norm.
You're not wrong, but at the same time I think it's a decent data point that getting it right is not straight-forward and that, practical speaking, you at the very least need to think very carefully before actually using it.
"The idea of it" is sometimes fine, but then there's also "the practicality of it", and sometimes that's a very different thing.
Remember the old microkernel vs. monolithic debate; everyone more or less agrees that in principle, a microkernel is better. But the practicality of it is a lot more complex, so monolithic kernels and hybrid ones are much more common. Microservices vs monolithic is essentially the same debate, and I've seen a lot of Microservices with very poor implementations and a lot of problems. That doesn't mean the idea is bad in itself, but if it's hard to execute well, then you do need to be very careful.
There's tons more examples of this. You also see this sort of thing in e.g. politics, where what's "more fair" vs. "what's actually achievable without introducing heaps of bureaucracy and overhead" are sometimes very different things.
In the case of GraphQL, I think it's pretty obvious that the general idea, as described from a high level, is a good one. But the practicalities of it are a lot less straight-forward, as this article explains reasonably well IMHO.
I think one of the issues is actually that GraphQL has no specific implementation, it's not like you have an SQL database and that's it, so one thing is that when you are consuming a GraphQL api it is impractical for various implementation related reasons from the vendor.
With GraphQL you are doing a lot of extra work query wise than you do with REST, which is supposed to translate into various benefits. But each vendor has different implementation details that affects whether you actually receive those benefits.
This is different than REST, the benefits of REST are generally the same for every vendor because implementation details and API are extremely simple, what is different is the data structure you receive back.
In GraphQL both the data structure and the implementation details are different. GraphQL I think doesn't make sense if you have multiple data sources from different vendors, because of this increase in complexity and not being to trust how good their implementation actually is until you've gotten into it.
I knew we were in trouble when we started having to sort the query criteria in order to support caching of requests.
If I use graphQL again it’ll only be for admin features. Anything where very few users will use it and very infrequently. Preferably in spots where caching works against the workflow. OLAP vs OLTP.
GraphQL is really about reducing friction between teams. High functioning distributed systems all have two qualities in common: work stealing and back pressure. Without back pressure there is nothing to stop the project from running off a cliff.
It’s when you can reassign a task earmarked for one machine to another after the fact. There’s a latency and throughput problem to keeping all tasks on a central server until a worker finishes an existing task.
But if you locally queue, and one worker gets unlucky and gets the longest tasks, then the whole cluster sits idle while this worker starts and finishes multiple tasks. If you reallocate the ones that haven’t started or have timed out, then the overall time comes down quite a bit (and in the latter case also covers workers that crash).
For development teams it could mean moving the boundary on a service or API so your team does more of the feature work than originally planned, because the other team keeps getting jammed up on bugs or other operational issues.
That fun fact bothers me at a fundamental level. I've seen it happen so many times and is really painful when the people were promoted out of the place. Even worse when they are good developers, but often in a cowboy style where they confuse their energy for some fundamental efficiency of what they were doing.
Kudos to the author for reevaluating his opinion and changing heart on a technology he admits to have championed before.
IMO GraphQL is a technological dead end in much the same way as Mongo is.
They were both conceived to solve a perceived problem with tools widely adopted at the time, but ended up with something which is even worse, while the tools they were trying to replace rapidly matured and improved.
Today OpenAPI REST and Postgres are rightfully considered as the defaults, you even have PostgREST combining them, while most of those who adopted Mongo or GraphQL have either long migrated or are stuck discussing migrations.
That is pretty funny, But that video is 11 years old. It can't still be like that? can it? Seems like people are down on Mongo in the last year, and I'm trying to catch up.
WiredTiger was kinda Mongo's InnoDB and has made "your data will actually still be there later" rather more true than it used to be.
I think the key thing is that people using MySQL were having trouble with deep data and found MongoDB's document oriented approach much easier, but these days people are tending to start with PostgreSQL, which can handle that nicely.
(MySQL/MariaDB are far better than they used to be as well, though I find most stuff I read online doesn't take advantage of that as much as it might)
There's also probably a factor of Mongo solving pain points people had when they switched to it, and there being lots of excitement around that, where today the same people have run into the pain points of Mongo often enough that it's no longer nearly so exciting a prospect.
I wouldn't honestly be surprised if we're now at a point where people are -more- negative about Mongo than it really deserves, and I say that as somebody who viscerally hated it on sight and would still rather avoid dealing with it myself it if at all possible.
(oh and MongoDB the -company- has always done their best to be a good corporate community citizen, sponsoring all sorts of cool things as a result, and while I think the license change was a shame I -still- think they're doing their best, just in an environment where they wouldn't have a best to try to do in the first place if they didn't avoid being killed by AWS)
It's a shame that this out of date/meme stuff continues to give MongoDB a bad rap. It's a great DB if you need to be flexible/move fast and avoid migration headaches (speaking first hand, this has dragged dev cycles quite a bit). Most startups/saas/web apps would benefit greatly from using MongoDB purely from a reduction of complexity standpoint.
The current version of MongoDB, imo, makes you super productive and scales without a ton of thinking. If you're working in Node.js, it's even more useful as the query language works just like a JS/JSON object so writing queries is super fast (compared to SQL where you have to spend a lot of mental cycles figuring out how to map object/array data).
I've found that denormalizing data (not even necessarily copying/duping data, but trying to centralize storage of it) when using MongoDB is the way to get the most value out of it. If you try to treat it like an RDB (which does work but can cause issues with complex queries), you'll run into headaches. If you just design stuff to be nested, though (and use the built-in APIs to query that nested data), it works incredibly well.
GraphQL by itself has a lot of issues, but Hasura is IMO a power tool. It gives you CRUD with a security model and a lot of bells and whistles out of the box, and paired with Apollo client on the front end it's pretty quick to set up and use. I still use random REST endpoints, and I'm not interested in federation, but as a quick way to get an app going it's great.
Same, with a shout out to PostGraphile as well. As an aside, I'm sorry but I roll my eyes every time I encounter data loaders and the N+1 problem it's meant to address but which really is a consequence of insistence on following the resolver execution model. GraphQL is a query language. Just compile it to SQL (or Cypher, or SPARQL, or whatever)...when that's possible.
I wouldn’t have really called these issues “broken by design”…
Rough edges sure.
No reliable way to delete processed messages. Well, who’s to say they were processed? It’s a persistent queue, stuff sticks around by construction.
Besides, this can be managed with tombstones and turning on compaction for that topic.
How would you want to “handle” queue overflow? You’ve either got storage space, or you don’t, this feels a bit like asking “how do I make my bounded queue unbounded”. You don’t, that’s an indicator you’re trying to hold it wrong.
The configs could be a bit easier to drive, but confusing and massive configs is pretty par for the course for Java apps ime.
The queue, which should keep a reference count for messages.
> How would you want to “handle” queue overflow?
At the producer end, of course.
> You’ve either got storage space, or you don’t
Kafka assumes you have infinite storage space. That might be okay for toy projects or for the insane architectures you see in the enterprise space, but not for a serious project.
It may not be an exact analog, but "swagger UI" for openapi is the best I've seen like GraphiQL. Example https://petstore.swagger.io/ . Not sure of other alternatives.
No analog for "subgraph federation systems", unless a load balancer will suffice.
People need to stop judging the viability of something based on how satisfying it feels to use it in a toy project. When time is money, you'll see what really works.
At least GraphQL supposedly works for Facebook, and I tried it out before deciding it wasn't a default. I never even bothered with MongoDB. I've had to repeatedly veto using both in projects, cause someone thought it'd be cool and thought that was a good enough reason. Finally it's not cool anymore, but there will be another thing.
Maybe. Can that one army fix GraphQL for all the teams in the company? Cause I've seen things work that way, but I've also seen tools that are pitched as "maintained in one place for everyone" but are actually a complexity burden on every single team, especially if/when its usage changes.
Stack Overflow trends isn't really a good metric, but setting that aside, you need to sum the time series for Swagger and OpenAPI. There's still plenty of people who call OpenAPI 3.0 "Swagger". And strictly speaking, Swagger is "OpenAPI 2.0".
No idea why you bundle Mongo in there. I use Mongo in multiple production apps and I've never, ever looked back. Wouldn't even consider RDBMS's at all after my experience with Mongo unless I absolutely had to.
Seems like at that point exposing each query as an OpenAPI endpoint would achieve pretty much the same thing.
Then again having GraphQL as the definition for them is probably still not bad, I'll just have to write something that converts them to SQL::Abstract 2 trees once I get around to porting it to TS.
It would be the same thing except with Benje's approach, you're basically using GraphQL as a developer tool to create those end points instead of writing code to do it.
And you don't have to write something to convert them to SQL if you're using PostgreSQL, because Benje's already written it for you.
postgraphile does look like it'll handle basic cases pretty nicely but I've gone through the docs and didn't find anything like an explanation of what SQL queries it ends up mapping to - do you happen to know if there's one I missed, or a list of examples of GraphQL + corresponding SQL, or something?
PostGraphile compiles GraphQL directly to SQL. The SQL is "custom" in that it's specific to the GraphQL operation, though naturally it does follow rules. For example, Hasura does the same thing, and among the rules that it follows is that it uses `LEFT LATERAL JOIN` between tables (at least, on PostgreSQL). Full disclosure, I work for Hasura, so I'm not super familiar with the style of SQL PostGraphile generates but one thing you can do is just have PostGraphile report back the generated SQL for inspection:
> PostGraphile compiles GraphQL directly to SQL. The SQL is "custom" in that it's specific to the GraphQL operation
Yes, that's why I said GraphQL and -corresponding- SQL, I was hoping to find something that showed me the SQL for each of half a dozen or a dozen examples ... though the debug option there will let me point the out-of-the-box CLI at a pre-existing database and have a look at as many examples as I like, so that's pretty close to what I was looking for.
Would also be interested to see a bunch of examples of what Hasura generates if you have those to hand (I'm going to poke through the Hasura Community Edition docs but if you have the specific FM to R handy ... :)
I still love GraphQL, but much of the love comes from using tools like PostGraphile which generates the API for you based on your database schema. I then add my own Javascript plugins as necessary. Going back to REST and hand writing everything gives me the shivers, how much time am I spending just translating data A to data B?
Authorization: I do it in the database using roles, row level security and column level security. It works well and I defer everything to PostgreSQL's security controls, it feels like the right place to do it and I don't have to worry about it going out of fashion, PostgreSQL is here to stay. Anybody else who talks to the database directly is also subject to the same authorization rules, which is nice.
Introspection: this should really be disabled on production services. Only enable it for development.
N+1 problem: I don't really have a problem with N+1 because PostGraphile converts the request into an efficient query. In other cases this problem presents itself in REST too and the article proposes hoisting N+1 queries to the controller, but that's just really moving the problem around, and you can do this with GraphQL too.
The other problems, yeah sure they are present and a worry if you're running some highly visible/very loaded public API.
Agree with this 100%, Postgraphile is awesome. I started a new project recently and was writing “REST” APIs because I’d been reading that people were put off by GraphQL, but it was a complete pain - instead of exposing my data and querying it as needed, I had to try to guess up front what I should expose in what object. It was the bad old days all over again, writing adhoc code on the server to meet the needs of the client… switching back to GraphQL - and postgraphile - was a relief.
As someone who has used GeaphQL extensively, I really don’t understand most of the complaints, which seem like they’d be common to any complex API surface. Sure you can write a query that triggers a server bug, but that happens with REST too. Yes, your server needs to be hardened against these queries… so what?
And security is hard, granular security doubly so. If you need to do field level authorisation then the problem is that you need a policy engine, not a different query technology.
Can only agree with author on all points. GraphQL was nice for the types, and precious little else. But we have better solutions utilizing just the types now, so it was still valuable.
The reason it is like that is because GraphQL is intended for highly distributed backend where at any time one of those backend services might be unavailable and failing the whole request when only a single deeply nested field is missing would lead to bad user experience.
In short GraphQL is optimized for keeping as much functionality working as possible no matter how much instability there is in the backend.
This is a common problem in REST endpoints that consume data from multiple services as well. But the request usually fails completely. On the other hand REST request don't usually touch as many services as a GraphQL request.
Whenever you see REST in the article, think RPC. The author even says REST is simpler than GraphQL but I suspect they hadn’t implemented a REST API… ever.
Pure REST API is a huge pain in the ass to use and leads to bad performance because of waterfall requests. It is like the number one thing GraphQL solves for most people, decide which nested relationships to inline into a response.
The simplicity makes it bad to use, if you work around the simplicity (arbitrary inlining, extra fields to choose what to inline, etc) then you end up with non-standard complexity.
I’ve done it once in my two decades of being paid to code and only because it was small enough to not impact any timelines… nobody cared anyway. Otherwise REST now means ‘RPC over HTTP with JSON marshaling’ for better or worse.
Never bothered to learn GraphQL - anything being pushed by big tech companies usually in 80% of the cases works for them, but its an overkill for smaller scale projects.
Stick to tried and true - monolith, asp.net, angular or blazor front-end, relational database.
I have never had any major issues or problems with it.
Think of it this way: GraphQL, Docker, and Kubernetes solve problems that very, very, very few GIANT tech companies have, but not every app will scale to millions of users.
So you’re not familiar with AngularJS, the incompatible version 1? Angular 2 was effectively a complete rewrite - cleverly using the same name, thereby making it even more difficult for us - and the rewrite was done because Google Decided. Never again will I trust a Google front end library.
Because of my experience, I feel that your position is making a slim distinction based on your preferences. All the tech you use is built for big corporations, why pick on one set of tech just because you don’t have the specific problems it solves?
For me, for example, K8s properly solves a bunch of problems for my small SaaS business, not least of which is that I can upgrade my three piddly servers without taking my customers offline. My SSL certs get upgraded automatically without downtime. My CICD pipeline is simple. Logging is much easier. And so on.
I don’t understand the disdain for modern, managed K8s at all.
I would have thought all of those were fairly obvious. It is more complex and gives the client a lot more power and control, so the trade-off is inevitable.
For the service I primarily work on, we've stuck firmly with REST for our internal APIs despite some mild pressure from elsewhere in the company to move to GraphQL. I personally believe it's been the right choice. Not only would it be probably an order of magnitude more complicated to setup and maintain, our REST APIs already are very performant.
We've got little to gain, and the lack of flexibility from our POV is a relatively good thing. It gives us direct insight into desired use cases, and since the APIs are internal it lets us provide guidance on potentially better ways of implementing things as well as in some cases being able to provide direct optimizations for specific tasks.
So, TL;DR: if you want to exploit the flexibility offered by GraphQL, you need to pay for that with implantation complexity.
I’ve never used GraphQL in my life yet can foresee the issues described in this post. If you try to retrofit GraphQL feature X, Y, or Z into your REST API, you’ll run into these exact same problems.
If you stick with a classic REST API, you pay for that simplicity in other ways. Who here is on team “I have a plethora of use-specific endpoints”, and who here is on team “I built dynamic field selection and relationship traversal into API framework”?
And, yeah. Dealing with N+1 database queries is loads easier if you constrain the potential pathways to the point that you can just hard-code your database optimisations instead of having to build a dynamic query optimiser. So you end up with a bunch of basic resource-mirror endpoints and then the N+1 database queries become N+1 HTTP requests. But hey, at least the requests are roughly equally expensive so you can simplify your rate limiting!
As usual, it’s a question of trade offs, and it’s not the easy-path “you aren’t Facebook, so you don’t need it!” that people throw around here on the reg’. And, certainly, those that look down their nose and say “I know that I was right to not care for this, bloody magpie developers and their shiny toys” are certainly speaking with a misplaced sense of superiority.
* Not null fields in a distributed system are a lie. Clients write code assuming something is not null, so a query that fetches data from N sub systems breaks the entire page when one of them fails.
Performance:
* People say you only need to fetch just what the page wants but in practice clients create re-usable fragments for every object in the system and use it everywhere. Any time a new field is added, every api call fetches that new field adding latency everywhere
* clients are unaware of the underlying topology and add fields that are usually harmless but have bad tail latencies.
* The completely generic nature of the API means the backend can't optimize performance for a specific page. E.g. if one API has nasty tails, but it's not that important for this page, the backend could time it out at some reasonable value instead of holding up the entire request
* Not null fields in a distributed system are a lie
If something is null that's not supposed to be null, then the entire operation should be called into question. It's probably not safe to proceed so it's a good thing he entire page breaks, you don't want users to continue based on wrong information. If you define something in the schema that it's possible that it's null, but then the frontend dev ignores the fact that it can be null, why is it GraphQL's fault then that the page breaks?
* clients create re-usable fragments for every object
As a frontend developer I don't know why you would do that, but if your frontend devs are doing that then yes they are doing it wrong... However switching to REST with statically defined endpoints doesn't solve the over/underfetching problem, but as backend developer you do get to gatekeep that aspect. So yeah the devs should really be just doing it right.
About nulls: you're right, but because of authorization, everything can be null - if you don't have access to something, we need to remove it from the result (instead of just throwing an error, as you may still get enough information to do what you need to do) - and that will always be a fetcher error in GraphQL if the field was non-nullable. And because you shouldn't really know or care beforehand which fields may be "hidden" from the end user due to authorization, you need to make everything nullable or risk making a breaking change later.
The problem with not null is people classify them from a domain perspective and not from a distributed systems perspective where basically everything can be null but only some fields mean rendering the page is impossible. It also depends page by page what fields make it unrenderable
In my experience, nothing other than nullable DB fields should be nullable in the GQL schema. Everything else like inter-service problems, auth problems, etc, should be modeled as “result boxes” via union types (somewhat equivalent to Maybe/Optional types). This lets your schema model possible failure cases via strong types without ambiguity and results in resilient front-end code.
Note that the error types added to the union should only be as granular as relevant to the client. Most places will be just Foo | FooNotFound | FooError because your UI doesn’t care why there was an error and you don’t want to unnecessarily leak backend info when it’s not relevant.
I wish this was more strongly recommended in the GQL org docs, because so many people learn it the hard way and migrating is not easy.
> clients create re-usable fragments for every object in the system and use it everywhere
I quite like Relay's pattern where components define their data requirements, and this is passed up the component tree to some query at the root. Avoids a situation where the query defined at the root asks for unnecessary fields because it's so far away from where the data is actually required.
720 comments
[ 3.6 ms ] story [ 483 ms ] threadIt works there because
1. Every user is logged in. Is there anything you can do at FB without giving up something to the Zuck?
2. Because it's all behind a login, you can front load the first login/request with a giant SPA and then run all the custom queries you want.
3. everything at FB is some sort of blended context (my user, someone else's user, a permissions interaction)... security is baked in to every field.
4. Because of permissions, and login requirement it's hard to be a bad actor (FB just locks you out, nothing is public).
If you have a SPA and logged in user requirement and that robust permissions model then GraphQL might make sense... Otherwise its a shining example of conways law and might not be fit for your org... The same can be said for react too.
For most people "security is baked in to every field" is going to be very expensive.
That having been said, outside that data model, your absolutely correct that its going to be costly to maintain those extra layers of relationships.
People completely missed the point of GraphQL, which is you TRADE flexibility for the client for added cost on the server.
Which in a app and team as huge as facebooks' made sense, especially since they have the so-called facebook apps that could do... anything.
I mean, isn't this THE selling point of GraphQL? How was it missed so badly? I think every elementary resource about the technology covers this in the pros and cons section...
I'm glad Redux hype train is over and nobody is using it on new projects no more.
I do really like the Api definition part of it though - but I found something like typespec now to be the perfect middle-ground, it allows out you describe your apis similar to a graphql Api without enforcing the graphql engine on you though.
Apollo federation and implementation of supergraph and nested graphs is pretty robust
We use both in production at reasonable scale and complexity 10+ roles few hundred object models, 100s of request/sec
We were basically just enhancing an external graph with some of our own stuff added on though so fairly straightforward.
I wonder if GraphQL would have been as popular if it was a much smaller shop than facebook launching it. Feels like having a name like FB gets you a tech credibility pass.
Similar concepts of allowing queries and response formats was there in other standards and not new ?
OData is fairly popular in REST world , even older , SOAP XMl had supported with custom query language one robust one I remember T-SQL used in Taleo the old Oracle hiring software
There are many examples of clients doing joins and queries in other standards , it is hardly unique to GraphQL
"That would be one way to implement the system in a DB-centric way. However we believe that intermediate application code is pretty critical to any GraphQL implementation." [1]
"GraphQL is a client-server dance that needs server-side capabilities, not just CRUD semantics." [2]
[1] https://news.ycombinator.com/item?id=9879870
[2] https://news.ycombinator.com/item?id=14351800
I don’t necessarily disagree but going to play devil’s advocate here on this common foundational viewpoint. Why is this bad? In fact, why not expose the DB directly to the client?
“Additional layers” and “don’t explain DB structure” is security through obscurity. If a DB manages access control directly (field-level or partition-level granularity), any permissible query should run. To prevent pathological queries, add role-based resource quotas and back-off policies.
What about zero day vulns? Well these are going to be a problem regardless of the presence of various infra layers on top of the DB; you’ll have to monitor for data exfiltration to detect these types of attacks in either case.
There’s not a database I’m aware of that handles all of the above considerations, but oftentimes security issues exist in the overlooked gaps that arise between the complex interplay of many dependent systems. If you move the responsibility for security to the DB itself (other than authentication), you’ve removed a lot of the “confusion” that can be exploited, and overall security then depends on just a few factors: 1) correct assignment of DB access permissions, 2) valid external authentication, 3) quality of data exfiltration monitoring.
You might say you’re one simple DB misconfiguration away from disaster, but again, this is already a problem. The benefit of this alternative approach is that you have now shifted the bulk of the security work to one place: very careful rollouts of changes to DB access permissions. You would conduct static analysis to determine the “diff” of how data access has changed and perform sandbox tests. In exchange for this additional work, you can quit worrying about the security of anything downstream of the DB, assuming authentication is set up correctly.
>> If you move the responsibility for security to the DB itself (other than authentication)
And what about all the possible access levels within your auth? Should every DB view have to check a user table to authenticate that the viewer has the priority level to select on it? If it doesn't, all I need to know is the name of a view I'm not supposed to see. What happens if you add a new security level... you change every single view definition? What if your auth + user levels are in a completely different DB in another time zone? Why put that burden on the DB if you get flooded with bad queries from hundreds of clients? That's why I say it's crazy, instead of writing an API endpoint that runs auth first and knows which views or tables are accessible from there and to which users, all in one place. Yes you have slightly more work conforming the client data definitions to whatever is returned from the API, but that seems like a very small price to pay to put all your security and filtering/validation in an easy-to-read middle layer. Of course you should use views and DB perms, but those are not sufficient if someone steals a login or session unless there's a middle layer checking their IP and hash and potential double-taps and lots of other markers that it might be a rogue query before it goes to the database.
Another library to check out: Drizzle-ORM
So in general had the same feeling about graphQL because I still had to write layer of code that gets stuff from database, get it limited or paginated. Maybe once you have enough endpoints then you can change your front end against already existing GraphQL - but for me that was never the case as 90% of work is adding new tables, new endpoints so graphQL feels like additional work with additional footguns for almost no benefits.
In both these projects the GraphQL had started small. I came in during a more mature phase of these projects (2 and 4 years). That's where the requirements are harder, more specific, and overall complexity has grown. Adoption and demand on the API were growing quickly. Hence you logically spend more time debugging, this is true for any codebase.
But GraphQL has everything in it to make such problems even harder. And both these projects had clear signs of "learning-on-the-go" with loads of bad practices (especially for the N+1 problem). Issue descriptions were much vaguer, harder to find in logs and performance issues popped up in the most random places (code that had been running and untouched for ages).
Fun fact; in both these projects the original devs who set it up were no longer involved. Probably spreading their evangalism further elsewhere.
RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
It allows teams to work with less communication overhead. It solves more of a human problems than a technical problems, if someone think there is no value in that and bare metal performance is paramount that person never worked in a big org.
> RPC and REST are just more straightforward to monitor, log, cache, authorize and debug.
In some ways yes, in others no. For example it can be near impossible to see if a deprecated field in a REST API is still being used and by which clients it is being used. With GraphQL this is fairly simple.
Unfortunately GraphQL way of working is very different from normal REST APIs and often requires more complex server-side caching. The N+1 problem needs to be figured out upfront for every data-storage system used in the backend.
It is far better for the Backend to provide Frontend a contract--can do it with OpenAPI/Swagger--here are the endpoints, here are the allowed parameters, here is the response you will get--and we will make sure this narrowly defined scope works 100% of the time!
- Each individual thing available in the request should be no less timely to handle than it would via any other api
- Combining too many things together in a single call isn't a failing of the GraphQL endpoint, it's a failing of the caller; the same way it would be if they made multiple REST calls
Do you have an example of a call to a GraphQL API that would be a problem, that wouldn't be using some other approach?
> Then we just come back full round trip to REST
Except that GraphQL allows the back end to define the full set of fields that are available, and the front end can ask for some subset of that. This allows for less load; both on the network and on what the back end needs to fetch data for.
From a technical perspective, GraphQL is (effectively) just a REST API that allows the front end to specify which data it wants back.
The other extreme end example is to expose by default the entire data model (PostGraphile) and then getting lost in the customisation and authorisation.
When you use the "REST" / JSON-over-HTTP pattern which was more common in 2010, changes in query patterns necessarily involve the backend team, which means they are aware of the change & have an opportunity to get ahead of any performance impact.
I've never gotten a good answer to that question, so I've never even considered GraphQL in such systems where it may have made sense.
I can see it in something big like Jira or GitHub to talk to itself, so the backend & frontend teams can use it to decouple a bit, and then if something goes wrong with the performance they can pick up the pieces together as still effectively one team. But if that crosses a team boundary the communication costs go much higher and I'd rather just go through the usual "let's add this to the API" discussions with a discrete ask rather than "the query we decided to run today is slow, but we may run anything else any time we feel like it and that has to be fast too".
The graphql usage I'm used to works more or less the same as REST. You control the schema and the implementation, you control exactly how much data store access is allowed, etc. It's just like REST except the schema syntax is different.
The main advantage of GraphQL IMO is the nice introspection tools that frontend devs can use, i.e. GraphiQL and run queries from that UI. It's like going shopping in a nice supermarket.
For Postgraphile, it leans more heavily on the database, which I prefer. Set up some row-level access policies along with table-level grant/revoke, and security tends to bubble up. There's no getting past a UI or middleware bug to get the data when the database itself is denying access to records. Pretty simple to unit test, and much more resistant to data leakage when the one-off automation script doesn't know the rules.
I also love that the table and column comments bubble up automatically as GraphiQL resolver documentation.
Agreed about the introspection tools. I can send a GraphiQL URL to most junior devs with little to no SQL experience, and they'll get data to their UI with less drama than with Swagger interfaces IMO. (Though Swagger tends to be pretty easy too compared to the bad old days.)
> I can send a GraphiQL URL to most junior devs with little to no SQL experience, and they'll get data to their UI with less drama
But that's like giving direct (read) database access to someone that was taught the syntax of SQL but not the performance implications of the different types of queries. Sure, they can get the data they want; but the production server may fall over when someone hits the front end in a new way. Which is, I think, what a lot of people are talking about when they talk about GraphQL having load issues based on the front end changing their call.
And you can both put queries on an allow list, control max query depth, and/or throttle on query cost.
All of which are features which give you some way to respond to the performance issues you avoid by planning your API up-front.
There are no free lunches, especially with regard to security and sanity checks.
I think that's the part where I have a disconnect. To me, both REST and GraphQL likely need to hit the database to get their data, and I would be writing the code that does that. Having the front end's call directly translated to database queries seems insane. The same would be true if you wrote a REST API that hit the database directly and took table/field names from query parameters; only... we don't do that because it would be stupid.
How does this follow? A client team can decide to e.g. put up a cross-sell shelf on a low-traffic page by calling a REST endpoint with tons of details and you have the same problem. I don't see the difference in any of these discussions, the only thing different is the schema syntax (graphql vs. openapi)
A hand-coded REST endpoint will give you a bunch of predefined options for filtering, sorting etc, and the dev who implements it will generally assume that all of those can be used, and write the backing query (and create indices) accordingly.
Flexibility within the constraints of what the back end can safely support.
To me, the "whole point" of GraphQL is to be able to have the client ask for only the data they need, rather than have a REST API that returns _everything_ and letting the front end throw away what they don't need (which incurs more load).
If you can't support returning certain configurations of data, then... don't.
That being said, it's a lot easier to setup caching for REST calls.
It sure is better for the backend team, but the client teams will need to have countless meetings begging to establish/change a contract and always being told it will come in the next sprint (or the one after, or in Q3).
> This leads to far more fingerpointing/inefficiency in the log run, despite whatever illusion of short-term expediency it creates.
It is true it can cause these kind of problems, but they take far, far, far less time than mundane contract agreement conversations. Although having catastrophic failures is usually pretty dire when they do happen, but there are a lot of ways of mitigating as well like good monitoring and staggered deployments.
It is a tradeoff to be sure, there is no silver bullet.
If your front-end engineers end up twiddling their thumbs (no bugs/hotfixes), perhaps there is time (and manpower) to try to design and build a "new" system that can cater to the new(er) needs.
What ends up happening is the clients doing work arounds to backend problems which creates even more technical debt
It's really not, it's not exposing your whole DB or allowing random SQL queries.
> It is far better for the Backend to provide the Frontend a contract
GraphQL does this - it's just called the GraphQL "schema". It's not your entire database schema.
And the REST API can still get hammered by the client - they could do an N + 1 query on their side. With GraphQL at least you can optimize this without adding a new endpoint.
With my single WordPress project I found that WP GraphQL ran circles around the built-in WP REST API because it didn't try to pull in the dozens of extra custom fields that I didn't need. Not like it's hard to outdo anything built-in to WP tho...
You should log deprecation warnings.
But also if the client is composing urls/params manually then you are not doing REST, you are doing RPC.
Rest APIs should mainly use HATEOAS hyperlinks to obtain a resource. that is clients almost always call links you have provided in other reponses (starting from a main entrypoint)
That's a huge bet, especially given that GraphQL is expensive in the first place, and given that the more you grow the API in size, the less you can actually map the cartesian product of all request params.
But let's say you have a timeout, and they have a retry, then suddenly, your server is now spammed by all the clients retry again and again a query that worked a week ago, but today is too heavy because of a tiny change nobody noticed.
And even if it's not the case, you can now break the client at any time because they decide to use a feature you gave them, but that you actually can't deal with right now.
It takes a lot of work to actually ensure all possible combinations of graphql params hit exactly what you want in the backend, and it's easy to mess with it in the frontend.
I always am happy when I get an elegant query working. Often however I just find I wasted time looking for a clean 1 query solution when iteration by caller was the only solution.
If you're providing an external API like GitHub does, then that's a different story and I agree.
If, at page load, I’m making 100 HTTP requests to fetch 100 assets then as a client side developer I’m going to know that’s bad practise and that we really ought to have some kind of multi-get endpoint. With GraphQL that gets muddy, from the client side I’m not really sure if what I’m writing is going to be a massive performance drag or not.
Yeah, the client can't change the query if you don't let it specify a query, this is trivially true, but the developer can go break an API endpoint with the exact same result while trying to achieve the exact same business outcome.
GraphQL only exposes what you ask it to. There are plenty of pagination plugins for GraphQL frameworks just as there are plugins to REST framework.
GraphQL can be restrictive as REST if you want it to be.
The point is GraphQL can be "as restrictive" as REST, but if you want to enable more efficient queries by knowing all the data that the frontend is requesting, you can. But the opposite isn't true of REST. With REST if you want more advanced functionality like that you have to define your own specification.
You get something more complex, more expensive to maintain, consuming more resources, and configure it to basically be REST with extra steps.
Idk. Strawberry GQL and most GQL libraries are maybe equally as complex as the REST libraries for the same language. Strawberry and FastAPI I would say are equal in complexity and configuration.
It would be hard for me to say GQL is more expensive or consumes more resources. Opposite of the purpose and most uses of GQL.
Thing get more problematic when there's vertical ownership for a feature, where the UI needs just a few extra things and you end up with a REST response which is fatter and fatter, in the interest of avoiding round trips and client-side joins.
The problem with killing correct queries that take too long is that it shows up as intermittent failure that's dependent on server load and data cardinality. You might not find it in testing and it ships a bad experience to the customer before bugs are found. Whereas APIs which can't be so easily misused make it much harder to ship bugs.
Why do you think that they can't do that with GraphQL? GraphQL isn't open ended. Its a highly restricted syntax for calling nested resources. If a resource is expensive simply don't nest it and make it a top level field and it is the same as REST?
Lots of nested resources are by default efficiently served by GraphQL because they are mostly returning single object foreign keys. Something that would take extra calls with REST.
GraphQL can have the same restrictions and performance guarantees as REST but the later is not necessarily true because in REST there is no standard way to define nested resource access.
No because if you dont do that you have to involve more engineers anyways to build the REST endpoints and keep modifying the rest endpoints.
GraphQL is also default restrictive (i.e. exposes nothing). You don't need to add engineers to make it restrictive.
In Startups typically:
Without Graphql your "backend" ends up needing to have a lot of work and changes because it is constantly needing to be updated to the needs of the most frequent changes happening on the frontend.With GraphQL the only time you need to update the backend is when those "utility" functions change (i.e. 3rd party api calls, etc) or the data model changes.
So you end up needing substantially less backend engineers.
This is akin to saying that "directly exposing the database is easier, you only have to change things if the data changes".
And yes this is true, but when the data changes, or the environment changes, the paradigm falls apart a bit, no? Which is what the backend code was for, insulation from that.
> In Startups typically:
Yes, so for a short lived, non-scaled application its far easier to do it one way, and maybe that's fine for most small apps (that will never scale far). I suspect a lot of the push back comes from larger, less nimble, more backwards-compat focused organizations/apps.
Far from it actually. I am saying that in practice the data and queries that you perform on your Database actually tend to stabilize and you add less and less as time goes on.
By Allowing the frontend to select what combination of these pre-approved queries that you already approved it can use, you have to do less and less backend work when compared to REST where you have to do backend work for every query combination you want to serve.
> maybe that's fine for most small apps (that will never scale far).
I mean saying GQL doesn't scale for big apps is over looking one of the largest Corporate Software Orgs (FB) created and use it in production purposefully for managing large software APIs.
Sure, so you are just filtering raw database access then. That doesn't make it any different - and, you still need to approve and filter these queries, so what exactly have you saved? I.e. either the front end engineers can change these filters, or not, so it amounts to the same thing in the case they can.
> I mean saying GQL doesn't scale for big apps is over looking one of the largest Corporate Software Orgs (FB) created and use it in production purposefully for managing large software APIs.
That's not a great argument, though, saying a large company with many resources is capable of supporting something does not make it a sustainable technical decision. They likely also have a very specific work structure they use to make it for them.
In fact thats a strong reason not to use it, if it requires enterprise level resources to use it effectively. There is a big difference between technologies that scale to enterprise and technologies that require enterprise...
It still comes down to, if you can achieve 99% of the same thing with autogenerated REST apis and a couple page specific apis, what, exactly, is worth the considerable increase in complexity for that remaining 1%? Making things regularly more complex is a hallmark of failed, bad technologies, and I suspect GraphQL will see the dustbin like SOAP did...
> It still comes down to, if you can achieve 99% of the same thing with autogenerated REST apis and a couple page specific apis
Because you can get 100% by autogenerating GQL APIs and 0 page specific apis.
No, I never said that. You are the one that brought FB into the equation.
Just because it can be used for something does not mean that it should.
I said that that approach doesn't scale well, especially for frequent data/model changes. For small apps, where as you say, you have few data changes, by all means embed your database as closely as possible to you end user code.
Sqlite inside a c app or electron, e.g. No need for any API at all! Just raw query access.
Its nice GQL to generate stuff for small non-changing web apps, I'm sure. But once you get into more performance oriented, data-migration-style stuff, if there's not good support for changing the data and reacting to the environment, then adding complexity (GQL) to an already complex situation is a Bad Idea.
You never said what this 1% was, autogeneration is not a bonus when you already have to manually filter and route things. The simpler solution gets you there as well, with less fuss.
You think you don't have page specific apis, but if you are doing the manual filtering, then you still have them, you are just "hiding" them inside another language, that doesn't have a clear benefit? At least you can't say what it is, without going in circles, another sign GQL is probably ultimately a garbage technology...
The vast majority of projects don't gain anything from this flexibility, because you don't have suddenly a 1000 of farmvilles copy cat that need their own little queries. You just have bob that need an order by.
I think that’s what’s really pointing out the root cause issues here, it’s not purely GraphQL’s problem, it’s the problems inherent to distributed systems.
Yes, so the cost benefit here is not in favor of GraphQL. If both technologies ultimately suffer from the same issues (what to do about unpredictable clients), but one is far more complex to implement and use (GraphQL), then there's a clear winner. Spoiler, its not GraphQL.
Page specific endpoints, I would argue, can do 99% of what GraphQL was trying to do. If you want to use it as some sort of template language for constructing page specific endpoints, that could be useful (the same way xml schema is useful for specifying complex xml documents).
But you can optimize a page specific endpoint, and do it with REST-style endpoint to boot.
Having a bunch of "simple" calls and optimizing for the "complex" ones that you need using metrics/analysis is what you should be doing, not creating a complex API that is far harder to break down into "simple" cases.
The engineering work involved shifts from building individual endpoints to building the endpoint factory. This shift may or may not be worth the trade off, but there are definite advantages, especially from the perspective of whomever is building the client. And once you factor in the ease at which you can introduce partial streaming with defer and streamable (granted they’re still WIP spec-wise), the experience can be pretty sublime.
This? Yeah, that seems neat, for command/batch queuing.
I'd be curious how it compares to e.g. rest apis returning refs to e.g. webrtc streams or tcp/udp ones for non-browser. I presume the main advantage would be client side.
Surely that is always the case, if the client is composing multiple REST requests, or if there’s one RPC method per client page, or with any other conceivable data loading scheme.
What do you mean by "backend developer" ? The one who creates the GraphQL endpoints for UI to consume ?
I think the "learning-on-the-go" symptom, where you can sometimes literally read down a file and watch some developer learn the language as they add more and more functions to the file with a gradual increase in skill (or, to put it less charitably, as they slowly get less bad at writing code) is probably a very common thing, and not just a GraphQL issue.
REST API's are a proven solution for the problem of other apps, including front-ends, needing data from a data store. Using JSON is much improved over the days of XML and SOAP. Beyond that there haven't been advancements in technology that cause fundamental shifts in that problem space. There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.
There are so many developers out there that can't stand plugging in proven solutions to problems and just dealing with the trade-offs or minor inconveniences. Nothing is going to be perfect and most likely a lot of the software we write will cease to be running in a decade.
Ever seen an engineer do a loop and make n+1 REST calls for resources? It happens more often then you think because they don't want to have to create a backend ticket to add related resources to a call.
With internal REST for companies I have seen so many single page specific endpoints. Gross.
> There have been different opinions about structuring REST calls but those aren't going to cause any real forward progress for the industry and are inconsequential when it comes to business outcomes.
You could argue almost any tech solution in a non-pure tech play is largely in consequentially as long as the end goal of the org is met, but managing REST APIS were a huge point of friction at past companies.
Either it goes through a long review process to make sure things are structured "right" (ie lots of opinions that nobody can sync on) or people just throw up rest endpoints willynilly until you have no idea what to use.
GraphQL is essentially the "Black" for Python Syntax but for Web APIs. Ever seen engineers fight over spaces vs tabs, 8 vs 4 spaces, whether a space before a colon? those fights happened a lot and then `black` came out and standardized it so there was nothing to fight over.
GraphqL makes things very clear and standard, but can't please everyone.
Hardly gross. It is what it is and it’s universal across the domain. I bet Windows has internal APIs or even external ones that were created just for one page/widget/dialog of one app. It’s the nature of things at times.
An engineer had to spend time to make that specific API for that page instead of the frontend consumer using what was already defined and get all the resources with one call and 0 backend engineer needed for that new page.
We use a schema first design where I am at and if a frontend person needs a new endpoint because the resource-only endpoints aren’t enough then they submit a pull request to the schema repo with a design for their endpoint they need. It gets approved and boilerplate is auto generated. Yes you have to wait longer, but 90% of the time (for our software) the resource endpoints work great.
I'm not sure where this narrative comes from that GraphQL immediately means that the frontend time will have no idea what they are doing and will induce load on the system.
95% of my nested graphql fields are based on foreign key indexes so its almost no additional load to the system (only 1 query per "level" of GraphQL) to query the nested objects.
I restrict GraphQL to 5 levels and now I have 5 queries per API call instead of 5 Queries over 5 API calls.
The backend team exposes the fields that they know are efficient to query. They can restrict the depth of the GraphQL, number of fields, etc.
And that's assuming it's a new endpoint; if there's an existing endpoint that does almost what's necessary, they may need to check in with that team about what modifications to the endpoint would be acceptable, etc.
Single-page endpoints aren't great, but often times they're acceptable because they end up being a half-day task instead of a week-long slog.
In graphql land you'd be doing multiple SQL queries, "joining" in the API layer, and spending 50ms per API call.
In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.
1 graphql query maybe. But that translated to a dozen SQL queries.
> In practice with REST the frontend engineer didn't want to wait and tried to use the existing REST endpoints, did N+1 API HTTPS calls and then joined them client side in javascript.
The point you're missing is that for 1 graphql query the API did N+1 SQL queries, and then also joined them in JavaScript.
In the REST case the front end can switch to the efficient custom endpoint when it is implemented. In the graphql case it will never get any faster because the API has to stay generic.
Any query you were going to build and serve with Rest can be made with these two methods or even a raw dataloader and manual SQL string.
"That would be one way to implement the system in a DB-centric way. However we believe that intermediate application code is pretty critical to any GraphQL implementation." [1]
"GraphQL is a client-server dance that needs server-side capabilities, not just CRUD semantics." [2]
[1] https://news.ycombinator.com/item?id=9879870
[2] https://news.ycombinator.com/item?id=14351800
GQL implementations don't map to the database but to application code.
And GraphQL isn't free either; you need to actually implement that. It pervades your entire stack, too – it's not something you can just "put on top of it".
I think that in many cases, perhaps even most, GraphQL is a "spend 8 hours to automate a half hour task" kind of affair. Are there cases where that's not the case? Probably. Maybe your specific use case is one of them. But for the general case? I'm entirely unconvinced.
Why waste any time?
> And GraphQL isn't free either; you need to actually implement that
Rest isn't free. You have to actually implement also that and end up with a more limited API.
GraphQL libs in Python are equally as complex as the FastAPI etc..
> GraphQL libs in Python are equally as complex as the FastAPI etc.
You need to account for the fact that anything can query anything, whereas a more traditional REST-type API will have much more limited codepaths.
This starts with just basic database maintenance and indexing, which in many case will be significantly more complex, to all sorts of other problems. This article already enumerates them so I'm not going to repeat them here.
You can't just handwave all of that away with "oh there's a library you can use".
If you think all of that is worth it then that's fine. As with many things it's a trade-off, and a bit of a judgement call too. But please, don't pretend this complexity doesn't exist, and that the trade-off doesn't exist.
It's better to think about optimizing of creation specialized APIs.
What people are excited about is that the frontend can request all the data it needs at once and the backend can efficiently serve it. Something not possible with REST without reimplementing something similar to GraphQL.
I think this explains about three quarters of all engineering problems I've seen in corporate context, give or take.
When you realize that sometimes your REST model may be a view-model that maps to a different storage model optimized for online-transaction-processing, neither of which map to your internal domain model directly (there are adapters for that) _then_ you get somewhere ... but of course you then have to fight off the `transformToRestModel(transformToDomainModel(retrieveDbModel(theId)))` when the three models happily coincide for a time.
Screen-specific REST endpoints will make their way as a default in to a JS-based framework in 2025 and people will pretend like this is some breakthrough advancement.
Most of our frontend features now don't have backend changes and we were able to increase the ratio of frontend to backend devs.
Making a good API on a large system with many clients is always difficult. GraphQL makes it easier in theory, but if you have average level devs working on it, they’ll make a bigger mess than if they use simple REST. The latter will still be complex, but at least it’s easier to have observability.
I used to be in charge of the stored procedures that served as the API to a shared database system used by many teams. (So, not exactly the same thing, but I'd argue that the sprocs vs ORM debate isn't entirely dissimilar from the REST/RPC vs GraphQL debate.) My turnaround time for getting database API changes into staging for application teams to play with was typically less than one work day. But that happened because I had a lot of latitude to just get things done, because the company valued rapid delivery, and therefore made sure we had what we needed to deliver things rapidly, both from a technical and a business process perspective. This was in a very highly regulated industry where every change required paperwork that would be audited by regulators, too.
I've also worked at places where this sort of thing took ages. Typically the worst places for work backing up were organizations that used Scrum, because the whole "sprints" thing meant that the _minimum_ time to get something from another team was about 1.5 times the sprint duration. And even then you could only manage that if you could deliver a request that already met all of that particular team's requirements for being able to point it, and could convince their Product Owner that it was a higher priority than whatever their pet project is this quarter, and the stars align so that you perfectly time the submission of your request with respect to that team's Scrum process's frame rule.
The thing I want to point out here is that absolutely none of that bullshit was ever the API technology's fault.
The GP's idea that a frontend developer would send a ticket to somebody so they can get all the data they need... it's just crazy.
On the other extreme, we have the HTTP 1.0 developers saying something like "networks are plenty of fast, we can waste a bit of it with legible protocols that are easier to make correct", while the HTTP 2.0 ones are all in "we must cram information into every single bit!"
Every place you look, things are completely bananas.
For me, what's crazy is that there are "web" developers who can't just add the endpoint they need while working on a frontend feature, or "web" developers who can't just add an element or a page for testing the backend endpoint.
What ever happened to full-stack developers? The "frontend" and "backend" developer split is so incredibly inefficient that it's really jarring—you take something that should take 2 hours and magically, through tickets, delegation, and waiting for results (then repeat that for debugging, who knows how many times!), make it into a 2-3 day task.
I once reproduced (black-box style!) a two-week effort by a three-man team in six hours of work simply because I had PyCharm and IDEA opened side-by-side and could write code on both sides at the same time.
If someone has a good explanation for why the full-stacks that were once the norm went almost extinct, I'd be happy to give it a read!
I wrote some front-end stuff back in the days but I've lost track of whatever is happening these days. jQuery to append some stuff took five minutes to learn, but learning react hooks takes a determined effort.
Likewise, adding a field to a graphql type is simple, but doing it with authorization, controlling n+1s, adding tests etc.. requires front-end folks to actually invest time in learning whatever back-end they're dealing with this time.
Everything is just a lot more complicated these days, and if you've been around for a while you may not be excited anymore by the churn, but rather fed up.
At my startup there are 7 devs who can all do tickets across the stack and as we grow I think it would be good if we could resist the pressure to silo and specialize
It really is. Regrettably, I've drifted away from it in large part because of client requirements for more "modern" and "maintainable" solutions (e.g. Python or Node; I'll take Python every time, thanks). Django comes very close in terms of productivity (and is better in some ways: auth, admin, etc.) but the Rails CLI, generators and community (not sure if this is still relevant) give it the edge.
The "recent" (last 10+ years) movement towards "lightweight" libraries (instead of frameworks) that require you to either reinvent the wheel, copy-and-paste or use some random "getting-started" template every time you start a new project is disheartening. As others have said above, I think it's partially resume-driven-development and people wanting to tinker (both of which I do appreciate).
Something which continues to surprise me is that there hasn't really been a "modern" successor to Rails. Granted, I haven't kept pace with developments in the Node/TypeScript world but, last time I looked, Sails was on the right track but wasn't very actively developed and I was shot down (in favor of Express spaghetti) when I suggested it. There's also a smattering of Rust web frameworks but none that quite fit the bill and come with all of the batteries and opinions included. I keep saying I'm going to do a Summer of Code project which attempts this but ... life.
But yes, that break-down of the problem is insane. People artificially split an atomic problem in two, and go create all of that extra craziness to try to solve the communication problem they made.
And then people go and push UX and UI tasks into the frontend developer, and ops tasks on the backend since them are there... What is insane again, because those are really incompatible tasks that can't be done with the same mindset.
And since it's standard, backend tools won't support the frontend and the other way around. So the insanity gets frozen on our tooling.
I started my career as a full-stack developer, but went all-in on frontend because I felt I was spreading myself too thin. At one point I found that I could choose to be almost good enough at doing two different things or extremely good at one thing. I chose the latter option.
Modern browser apps are complex beasts, at least if you want to do them right. You obviously have to worry about all the technical bits --- HTML, CSS, JavaScript, your view library of choice, platform APIs like <canvas> and WebAudio, cross browser testing, bundle sizes, performance optimizations, techniques like optimistic rendering, all that good stuff.
On top of that, you also need to work closely with designers to make sure they know the features and limitations of the platform(s) they're designing for. More often than not, you end up being a sort of bridge between the backend devs, designers, and product managers.
A lot of times you end up doing design too, whether you like it or not. I've learned a lot about UI/UX design just because I often have to fill in the gaps where a designer forgot to include a certain error state, or didn't test their design on tablet screens, or didn't account for cases where a certain API might not be available.
I tried for many years to learn as much as I could about Django as well as React and friends. But it eventually got too much. I found that I wasn't able to keep up with both ecosystems, and I was producing code that wasn't very good. I could certainly build things quickly because I was familiar with all parts of the stack, but it came at the cost of code quality, security, stability, and robustness. I eventually decided to hang up my backend developer hat and focus exclusively on what goes on inside the browser (which can be a lot by itself these days!)
It's probably possible for a single individual to build a high-quality server-rendered MPA with some forms without making a mess of it. But that says more about how good Rails/Django/Laravel are than about the capabilities of any single individual. I don't think a single person could build a product like Linear end-to-end without cutting corners.
Cutting corners is a feature. I bet the Linear team is as pained as any, internally, at the tradeoffs they're making.
There is no way to know "what to get right" without going through it. So for 80% of the dev cycle the job is to cut corners to get to the 80/20, rinse and repeat forever.
This isn't against excellence and the dream of a beautiful product experience as your reply seems to convey.
For one, clients (mobile, desktop) are drastically different with all sorts of downstream implications:
- Differing screen size requiring responsive design
- Internet speeds magnitudes different
- Intermittent internet
- Uneven feature support due to different versions of browsers
- Everything needs to be JS at the end of the day
Desktop apps generally don’t have to worry about any of these issues.
Also, desktop GUI frameworks are typically either fragmented (one per OS) or don’t look OS-native.
As far as looking OS-native, this is (unfortunately) increasingly less relevant as OSes themselves drop desktop UX consistency even internally. But that aside, Qt is still around and is a very mature framework for "near native" UI, and then there's Avalonia and others. Even Gtk is surprisingly decent on Windows these days. Although I'm not sure what this all has to do with the original subject, since web apps most certainly don't look native anywhere.
The idea that resources and the underlying data needs to map 1-1 is wrong.
There is no easy or simple or final solution. The solution is to do the hard work of software engineering and catch this stuff at code review time or to plan and do refactoring later when you realize something needs normalization. I wish developers would just accept that software engineering is hard and do the hard work necessary.
There's that whole "the best software engineer is lazy because he will automate stuff", but that does not give anyone a license to automate stuff in an unmaintainable way. Automating stuff is hard and there is no easy way out.
"We need a http API endpoint that gives you all the data for one page. And also be able to reuse parts of it for other pages." Yeah bro, this is GraphQL.
Even though we use GQL here, we still have a B4F, so it's browser -> B4F -> GQL -> Database.
The tooling we use make it trivial so it doesn't add any development overhead (often reduces it. No CORS bullshit for one), and our app goes ZOOOOOOM.
GET /myresource?extra=foo,bar
sure you over fetch a bit if you have multiple accessors.
But agreed, if you have highly nested data especially when accessed with multiple different query purposes then REST might not be the best fit.
I think GraphQL has been positioned as a general purpose tool and for that I am with the author, REST is a better go-to for most usecases.
Any more levels and you have now reinvented GraphQL
https://www.odata.org/
Look at OData download stats on Pypi it had 2 downlaods the last day. Graphql-core for python? 624,201. that is not even on the same planet.
If you don't use GQL and want a system of querying nested data you will be using a less used protocol that is analogous to GQL so you might as well use the standard.
To trade anecdotes for anecdotes: On Nuget, the Microsoft.Data.OData package marked deprecated and marked with CVEs for critical bugs still has an average of 36.3K per day downloads. (Its successors combine for about double that.) The top GraphQL library (of the same name, unadorned) still only has 9.3K per day downloads. In .NET if you want a system of querying nested data (that is also "REST compatible"), why would you use a less used protocol like GraphQL?
Sure, but reinventing the wheel can be good, particularly when the existing wheel technology is oblong, lumpy and constructed of stone and cheese.
I’m just bitter than GraphQL endpoints return 200 on errors. If you’re returning 200 on errors, then you’re not really doing HTTP; you’re just using HTTP as a substrate for some other protocol, in which case you might as well just open a port and serve that protocol directly and not pretend to be HTTP.
Browser does not allow to access any ports with any protocols.
Often intermediate software gets in the way as well. For example with Kubernetes it's trivial to expose HTTP service using Ingress and it's not trivial to expose other protocols.
Another HTTP plus is that it's trivially secured by TLS and every developer knows API for that. Using TLS for your own protocol or QUIC is absolutely not trivial and probably more like arcane knowledge.
I also recall, we had similar N+1 query problems in the REST API endpoints irrespective of hydrating the returned resources.
The biggest benefit of GraphQL I can see from a user perspective is that it lowers total latency especially on mobile with fewer round trips.
there's lots of other benefits for GQL: multiple queries per request, mutation/query separation, typed errors, subscriptions support.
Single page endpoints is exactly what you want if you have more than 5 engineers in your company anyways.
It ensures that the endpoints are maintainable and future-proof when people are working on different features.
How does GQL prohibit this? It encourages it by focusing on 1 stable API for everyone instead of a custom API endpoint for each case.
But you still need all of the various single page REST/RPC endpoints to make use of GraphQL as it is intended. Some developers out there skip the REST/RPC part, making GraphQL their entire service, violating its principles.
If it works it works, but it does not come tradeoff free.
> The database will already have its own API
Where is this API coming from? You have to build it. I'm saying you should make those APIs graphql apis though a language framework and not REST apis because GQL consuming GQL is much better than GQL consuming REST.
That's why GraphQL examples usually focus on querying data from users and not how you are going to manage 10 different views of the same data.
That is how REST works but is the opposite of the way GQL works.
You don't pile into existing defintions but extend the current definitions out. A team needing new data doesn't affect the consumption of other teams. which is not the case of REST where if one team needs to change a REST endpoint to return different shape of data, they have to verify the new shape with other teams.
With GQL though, good luck retracing who owns what field and what does it affect if you change the definition of what it returns, especially that you you are not even using it on the same language.
Bonus points here if you are managing something outside of your control like a mobile app.
Also, Its about 1 line to set up CI to extract all GQL queries from a typescript project and do static analysis to check against the schema.
But again you only care if someone deletes a field, and even if you have to delete it, at least the GQL schema has built in annotations for deprecation, something not in REST.
Yeah sure you can make it work with anything if you spend the extra effort but the ownership really isn't as defined as in REST.
Is there code which have fuzzy or no ownership? Are there changes which affect other teams? Suddenly those became much harder questions to answer.
Its much easier to trace and debug what teams are using GQL fields than REST. What if one team is piggy backing on another teams exising rest endpoint and you dont know? same problem that would require some analysis of all code calling and endpoint to determine if a field is safe to delete. GQL makes this much simpler than REST.
I know there's some diverging opinions here but there's one which sounds definitely easier than the other.
As for deletes, I work in a company with a good hundred devs so that happens weekly at least.
For example, a lot of times people build out nice normalized table structures for online transactional apps. The UI/UX is pretty straight forward because end users typically only CRUD an object maybe a couple nested objects at a time. The API's are straight forward as well, likely following single responsibility principles, etc. Then comes along requirements to build UI's for analytics and/or reporting types of things where nearly the entire schema is needed depending on what the end user wants to do. Its the wrong data model for doing those types of things. What should be done is ETL the data from the OLTP schema into a data warehouse style schema where data is de-normalized so that you can build reporting, etc.
> With internal REST for companies I have seen so many single page specific endpoints. Gross.
As someone pointed out in reply to another comment, GraphQL is "a technological solution to an organizational problem." If that problem manifests as abuse of REST endpoints, you can disguise it with GraphQL, until one day you find out your API calls are slow for more obscure, harder to debug reasons.
With REST, though, that pain is visible to both sides. Frontend engineers generally don't want to make N+1 REST calls in a tight loop; it's a performance problem that they see and that is very visible in their Dev Tools. Backend engineers with good telemetry may not know why they get the bursts of N+1 calls that they see without asking the Frontend or digging into Frontend code, but they can still see the burstiness of the calls and have some idea that something could be optimized, that something is maybe too chatty.
There are multiple ways with REST to handle things: pagination, "transclusions", hyperlinks, and more. Certainly "single page endpoints" is a way as well, no matter how gross it is from REST theory, it's still a pragmatic solution for many in practice.
REST certainly can please everyone, given pragmatic compromises, even if it isn't very clear or standard.
That's an established pattern (backend for frontend). Like all patterns there are trade-offs and considerations to make, but it's certainly not a priori "gross".
https://learn.microsoft.com/en-us/azure/architecture/pattern...
And, resources with an index option obviously should have a db index or unique index.
The challenges with GraphQL are that it makes it too easy to DoS services, leak internal data, break referential integrity, and there were a great deal of tools, infrastructure, and monitoring systems already available for REST (and gRPC to a degree).
Company standards for REST and coding style can and should be set in the diff review pipeline. Another facet is setting standards to minimize duplication of effort or exposing a greater attack surface.
I like the thought experiment of adding a new persisted/editable field to an entity in a web app, if you've done full-stack you know all the layers that need to be touched to accomplish this and how lame most of the work turns out to be. After doing that 20 times while iterating, any dev worth their salt starts to wonder why it sucks so bad still and how it could be made better, and some will actually try.
Much like CSV, JSON isn't particularly standardised and different parsers and writers will do different things in some situations. Usually it doesn't matter, but when it does you're probably in for a lot of pain.
If you handle structured data and the structures might change over time, JSON isn't a good fit. Maybe you'll opt for JSON Schema, maybe that'll work for your use case, but with XML you can be quite sure it'll be reliable and well understood by generations of developers.
The tooling is generally very good, commonly you can just point your programming language to the XSD and suddenly you have statically typed classes to program against. Perhaps you'd like to store the data in a RDBMS? You can probably generate the DB schema from the XSD. If you want you can just throw JSON into MongoDB instead, but there will be very important tradeoffs. Same goes for UI, you can write some XSLT based on the XML schema and suddenly you get web views directly from API responses. Or you could use those classes you generated and have your GUI code consume such objects.
None of this is as easy with JSON as it is with XML, similar to how many things aren't as easy with CSV as with a RDBMS.
XML is mostly already lost on the current generation of developers though, much less future developers. Protobuf and cousins generally do typed interchange more efficiently with less complexity.
[1] https://json5.org/
RFC 8259 is marginally better in that it at least acknowledges these problems:
But note how this is still not actually guaranteeing anything. What it says is that implementations can set arbitrary limits on range and precision, and then points out that de facto this often means 64-bit floating point, so you should, at the very least, not assume anything better. But even if you only assume that, the spec doesn't promise interoperability.In practice the only reliable way to handle any numbers in JSON is to use strings for them, because that way the parser will deliver them unchanged to the API client, which can then make informed (hopefully...) choices on how to parse them based on schema and other docs.
OTOH in XML without a schema everything is a string already, and in XML with a schema (which can be inline via xsi:type) you can describe valid numbers with considerable precision, e.g.: https://www.w3.org/TR/xmlschema-2/#decimal
Sure, protobuf is nice, but more limited in scope and closer to a JSON alternative than an XML alternative.
I use JSON every other day and have been for decades.
JSON won in the end mostly because it was easier to handle in JS specifically, which is what mattered for the frontend. Then other languages caught up with their own implementations, although in some cases it took a while - e.g. for .NET you had to use third-party libraries until 2019.
Sure, you can disable it, but the fact that it is opt-out to begin with - i.e. that by default the parser will try to creatively interpret any string it sees in JSON input and convert in a locale-specific manner that also quietly loses data - is, frankly, insane through and through. I've personally run into this issue many times in existing code - it usually happens when people first start using the library and just never run into any inputs that would trigger this behavior while testing. Then once that code is shipped, someone somewhere just happens to have the data that triggers it.
And if you look at the comments to that issue, there are numerous mentions from other GitHub repos due to bugs it caused for them, including some Microsoft projects.
The cherry on that cake was author's response indicating that he doesn't even understand why this design is problematic in the first place: "I like what it does, I have no plans to change it, and I would do it again if given the chance." I wouldn't trust any parser written with this kind of attitude.
Browsers had XML parsers before they could handle JSON directly, and at the beginning there were complaints that JSON was harder to use for that reason. The reason why JSON won rapidly even for backend apps which never loaded it in JSON was ergonomics: every part of the XML world from the parsers to XPath/XSLT/XQuery to the rat’s nest of standards was plagued by the hairy-shirt “this is hard and should feel hard” attitude that has thankfully become less common. I saw so many people just burn out in the entire ecosystem because they got tired of unhelpful errors, pointless usability bugs around namespaces, low-quality or missing examples, and especially how common tools just stopped getting improved.
I maintain that the format would have been far more popular if all of the effort spent on standards work after the turn of the century had been suspended and the time directed to fixing things like the usability of namespaces in almost every parser, and hiring at least one person to work on libxml2 so developers could actually use features which shipped after 1999. Unfortunately it seemed like there were a ton of architects who really wanted to spend time building castles in the air and they just seemed to assume that someone else would do the boring parts of implementing it, but those people all jumped on JSON pretty quickly. I worked with a bunch of people who weren’t developers and the cycle of initial enthusiasm fading into “doesn’t this kind of suck?” with XML was depressing to watch having seen so much initial promise.
Go's XML parser straight-up emits broken XML when trying to output tags that have prefixed namespaces.
If only that were true in my experience.
Ah this brings up bitter memories. Team I was managing was roped in to become the first graphql framework (nay platform) the ivory tower (central architecture run by the cto) team was building and trying to shove down the rest of the company. This was during the graphql craze around 5 years ago. The principal engineer leading it was even preemptively promoted to distinguished engineer for future contributions.
Fast-forward 5 years project was disbanded due to massive migration, cost and complexity problems. DE still lauded for experimentation heroism and evangelism. I think he is now pushing the latest flavors of the month to go for the next promo!
I think two projects having loads of bad practices is too small a dataset to really assume anything, you sort of need to see widespread "bad practices" in the tech to be able to determine that the bad practice is actually the norm practice and there is perhaps a flaw in the tech that encourages that norm.
"The idea of it" is sometimes fine, but then there's also "the practicality of it", and sometimes that's a very different thing.
Remember the old microkernel vs. monolithic debate; everyone more or less agrees that in principle, a microkernel is better. But the practicality of it is a lot more complex, so monolithic kernels and hybrid ones are much more common. Microservices vs monolithic is essentially the same debate, and I've seen a lot of Microservices with very poor implementations and a lot of problems. That doesn't mean the idea is bad in itself, but if it's hard to execute well, then you do need to be very careful.
There's tons more examples of this. You also see this sort of thing in e.g. politics, where what's "more fair" vs. "what's actually achievable without introducing heaps of bureaucracy and overhead" are sometimes very different things.
In the case of GraphQL, I think it's pretty obvious that the general idea, as described from a high level, is a good one. But the practicalities of it are a lot less straight-forward, as this article explains reasonably well IMHO.
With GraphQL you are doing a lot of extra work query wise than you do with REST, which is supposed to translate into various benefits. But each vendor has different implementation details that affects whether you actually receive those benefits.
This is different than REST, the benefits of REST are generally the same for every vendor because implementation details and API are extremely simple, what is different is the data structure you receive back.
In GraphQL both the data structure and the implementation details are different. GraphQL I think doesn't make sense if you have multiple data sources from different vendors, because of this increase in complexity and not being to trust how good their implementation actually is until you've gotten into it.
If I use graphQL again it’ll only be for admin features. Anything where very few users will use it and very infrequently. Preferably in spots where caching works against the workflow. OLAP vs OLTP.
GraphQL is really about reducing friction between teams. High functioning distributed systems all have two qualities in common: work stealing and back pressure. Without back pressure there is nothing to stop the project from running off a cliff.
But if you locally queue, and one worker gets unlucky and gets the longest tasks, then the whole cluster sits idle while this worker starts and finishes multiple tasks. If you reallocate the ones that haven’t started or have timed out, then the overall time comes down quite a bit (and in the latter case also covers workers that crash).
For development teams it could mean moving the boundary on a service or API so your team does more of the feature work than originally planned, because the other team keeps getting jammed up on bugs or other operational issues.
IMO GraphQL is a technological dead end in much the same way as Mongo is.
They were both conceived to solve a perceived problem with tools widely adopted at the time, but ended up with something which is even worse, while the tools they were trying to replace rapidly matured and improved.
Today OpenAPI REST and Postgres are rightfully considered as the defaults, you even have PostgREST combining them, while most of those who adopted Mongo or GraphQL have either long migrated or are stuck discussing migrations.
I think the key thing is that people using MySQL were having trouble with deep data and found MongoDB's document oriented approach much easier, but these days people are tending to start with PostgreSQL, which can handle that nicely.
(MySQL/MariaDB are far better than they used to be as well, though I find most stuff I read online doesn't take advantage of that as much as it might)
There's also probably a factor of Mongo solving pain points people had when they switched to it, and there being lots of excitement around that, where today the same people have run into the pain points of Mongo often enough that it's no longer nearly so exciting a prospect.
I wouldn't honestly be surprised if we're now at a point where people are -more- negative about Mongo than it really deserves, and I say that as somebody who viscerally hated it on sight and would still rather avoid dealing with it myself it if at all possible.
(oh and MongoDB the -company- has always done their best to be a good corporate community citizen, sponsoring all sorts of cool things as a result, and while I think the license change was a shame I -still- think they're doing their best, just in an environment where they wouldn't have a best to try to do in the first place if they didn't avoid being killed by AWS)
That sounds similar to Elastic's story. Did MongoDB go through that as well?
The current version of MongoDB, imo, makes you super productive and scales without a ton of thinking. If you're working in Node.js, it's even more useful as the query language works just like a JS/JSON object so writing queries is super fast (compared to SQL where you have to spend a lot of mental cycles figuring out how to map object/array data).
I've found that denormalizing data (not even necessarily copying/duping data, but trying to centralize storage of it) when using MongoDB is the way to get the most value out of it. If you try to treat it like an RDB (which does work but can cause issues with complex queries), you'll run into headaches. If you just design stuff to be nested, though (and use the built-in APIs to query that nested data), it works incredibly well.
It is just my personal observation formed from working with Mongo and migrating systems away from it.
2. No reliable way to handle queue overflow.
Combine both and you are 100% guaranteed to have an incident. (I guess it keeps devops and sysadmins employed, though.)
Rough edges sure. No reliable way to delete processed messages. Well, who’s to say they were processed? It’s a persistent queue, stuff sticks around by construction. Besides, this can be managed with tombstones and turning on compaction for that topic.
How would you want to “handle” queue overflow? You’ve either got storage space, or you don’t, this feels a bit like asking “how do I make my bounded queue unbounded”. You don’t, that’s an indicator you’re trying to hold it wrong.
The configs could be a bit easier to drive, but confusing and massive configs is pretty par for the course for Java apps ime.
The queue, which should keep a reference count for messages.
> How would you want to “handle” queue overflow?
At the producer end, of course.
> You’ve either got storage space, or you don’t
Kafka assumes you have infinite storage space. That might be okay for toy projects or for the insane architectures you see in the enterprise space, but not for a serious project.
Can you suggest alternatives to graph introspection and related UI tools like GraphiQL, and the subgraph federation systems?
OpenAPI 3.0 has this concept of remote references, which can be URLs to other OpenAPI specs hosted anywhere. https://swagger.io/docs/specification/using-ref/
No analog for "subgraph federation systems", unless a load balancer will suffice.
At least GraphQL supposedly works for Facebook, and I tried it out before deciding it wasn't a default. I never even bothered with MongoDB. I've had to repeatedly veto using both in projects, cause someone thought it'd be cool and thought that was a good enough reason. Finally it's not cool anymore, but there will be another thing.
* openapi was basically nonexistent when GQL came out. It certainly wasn't "the tool they were trying to replace"
* Postgres and GQL are not in any way mutually exclusive
* Today, openapi is still tiny compared to GQL. At least as measured by StackOverflow question tags:
https://trends.stackoverflow.co/?tags=graphql,openapi,soap,m...
I swear, the older I am, the more convinced I am that people who don't use a RDBMS just don't work on complex systems. Period.
https://chillicream.com/docs/hotchocolate/v13/security
Honestly most of the "problems" the OP discusses has solutions in the documentation.
Then again having GraphQL as the definition for them is probably still not bad, I'll just have to write something that converts them to SQL::Abstract 2 trees once I get around to porting it to TS.
And you don't have to write something to convert them to SQL if you're using PostgreSQL, because Benje's already written it for you.
https://www.graphile.org/postgraphile/debugging/#via-postgra...
Yes, that's why I said GraphQL and -corresponding- SQL, I was hoping to find something that showed me the SQL for each of half a dozen or a dozen examples ... though the debug option there will let me point the out-of-the-box CLI at a pre-existing database and have a look at as many examples as I like, so that's pretty close to what I was looking for.
Would also be interested to see a bunch of examples of what Hasura generates if you have those to hand (I'm going to poke through the Hasura Community Edition docs but if you have the specific FM to R handy ... :)
Anyway, here's an example with Hasura:
https://gist.github.com/dventimihasura/b3f3c9e7ef1442d8e7f16...
Authorization: I do it in the database using roles, row level security and column level security. It works well and I defer everything to PostgreSQL's security controls, it feels like the right place to do it and I don't have to worry about it going out of fashion, PostgreSQL is here to stay. Anybody else who talks to the database directly is also subject to the same authorization rules, which is nice.
Introspection: this should really be disabled on production services. Only enable it for development.
N+1 problem: I don't really have a problem with N+1 because PostGraphile converts the request into an efficient query. In other cases this problem presents itself in REST too and the article proposes hoisting N+1 queries to the controller, but that's just really moving the problem around, and you can do this with GraphQL too.
The other problems, yeah sure they are present and a worry if you're running some highly visible/very loaded public API.
As someone who has used GeaphQL extensively, I really don’t understand most of the complaints, which seem like they’d be common to any complex API surface. Sure you can write a query that triggers a server bug, but that happens with REST too. Yes, your server needs to be hardened against these queries… so what?
And security is hard, granular security doubly so. If you need to do field level authorisation then the problem is that you need a policy engine, not a different query technology.
In short GraphQL is optimized for keeping as much functionality working as possible no matter how much instability there is in the backend.
This is a common problem in REST endpoints that consume data from multiple services as well. But the request usually fails completely. On the other hand REST request don't usually touch as many services as a GraphQL request.
The simplicity makes it bad to use, if you work around the simplicity (arbitrary inlining, extra fields to choose what to inline, etc) then you end up with non-standard complexity.
Stick to tried and true - monolith, asp.net, angular or blazor front-end, relational database.
> asp.net, angular or blazor front-end
Since when are these not technologies pushed by big tech companies?
A company I worked with got badly burned by AngularJS.
I have never had any major issues or problems with it.
Think of it this way: GraphQL, Docker, and Kubernetes solve problems that very, very, very few GIANT tech companies have, but not every app will scale to millions of users.
Because of my experience, I feel that your position is making a slim distinction based on your preferences. All the tech you use is built for big corporations, why pick on one set of tech just because you don’t have the specific problems it solves?
For me, for example, K8s properly solves a bunch of problems for my small SaaS business, not least of which is that I can upgrade my three piddly servers without taking my customers offline. My SSL certs get upgraded automatically without downtime. My CICD pipeline is simple. Logging is much easier. And so on.
I don’t understand the disdain for modern, managed K8s at all.
We've got little to gain, and the lack of flexibility from our POV is a relatively good thing. It gives us direct insight into desired use cases, and since the APIs are internal it lets us provide guidance on potentially better ways of implementing things as well as in some cases being able to provide direct optimizations for specific tasks.
Reliability:
* Not null fields in a distributed system are a lie. Clients write code assuming something is not null, so a query that fetches data from N sub systems breaks the entire page when one of them fails.
Performance:
* People say you only need to fetch just what the page wants but in practice clients create re-usable fragments for every object in the system and use it everywhere. Any time a new field is added, every api call fetches that new field adding latency everywhere
* clients are unaware of the underlying topology and add fields that are usually harmless but have bad tail latencies.
* The completely generic nature of the API means the backend can't optimize performance for a specific page. E.g. if one API has nasty tails, but it's not that important for this page, the backend could time it out at some reasonable value instead of holding up the entire request
If something is null that's not supposed to be null, then the entire operation should be called into question. It's probably not safe to proceed so it's a good thing he entire page breaks, you don't want users to continue based on wrong information. If you define something in the schema that it's possible that it's null, but then the frontend dev ignores the fact that it can be null, why is it GraphQL's fault then that the page breaks?
* clients create re-usable fragments for every object
As a frontend developer I don't know why you would do that, but if your frontend devs are doing that then yes they are doing it wrong... However switching to REST with statically defined endpoints doesn't solve the over/underfetching problem, but as backend developer you do get to gatekeep that aspect. So yeah the devs should really be just doing it right.
Note that the error types added to the union should only be as granular as relevant to the client. Most places will be just Foo | FooNotFound | FooError because your UI doesn’t care why there was an error and you don’t want to unnecessarily leak backend info when it’s not relevant.
I wish this was more strongly recommended in the GQL org docs, because so many people learn it the hard way and migrating is not easy.
I quite like Relay's pattern where components define their data requirements, and this is passed up the component tree to some query at the root. Avoids a situation where the query defined at the root asks for unnecessary fields because it's so far away from where the data is actually required.
https://relay.dev/docs/principles-and-architecture/thinking-...