The backend scaling/complexity problems originate in the database, its kind of cute the way they put it: "Making the graph-oriented way of getting data fit over an SQL database can be tricky." Tricky vastly understates this: we are talking about ORM, the "vietnam of computer science"
Clojure ecosystem has an immutable database, Datomic, which is designed for functional programming and fully solves the impedance mismatch. Datomic "Pull queries" is essentially GraphQL – and even predates GraphQL by a year!
Unlike GraphQL, Datomic is also a real database, competitive with SQL and can express relational filters and joins. Datomic is designed for the read-heavy data modeling loads that, today, SQL (and mongo, etc) is used for in anger. https://www.datomic.com/
That threw me away, worse is that the cloud version and on-premise are different products and not compatible. Is not something like RDS where you can just move your postgres setup to your own server or some other cloud service.
As it just happened to you, you can see they are putting some effort in hiding on-premise from their front page. All their effor seems to be going to the cloud version.
I love clojure and appreciate what Datomic brings to the table, but it seems like you need the whole package (clojure backend) in order for the benefits to pay off. GraphQL is a much less risky proposal for most companies, since it can be implemented more iteratively, and is storage-layer agnostic.
Just different risks. Adopting GraphQL without also adopting Facebook's exotic graph datastore is the unknown-unknown sort of risk that eventually leads to intractable problems as demonstrated by Hibernate and a long history of ORM related failures, and it isn't even on the radar of 90% of GraphQL adopters, if HN comment activity is any indicator.
could you elaborate on that? we aren't using an ORM and are able to successfully batch queries to improve performance. Couple that with an understanding that the developer should not use overly-nested queries and its been working out fairly well so far, although we're not dealing with a massive user load yet.
Finally a graph QL that can actually return graphs. Otoh, I don’t get why it is breaking news at all. Figuring out that in some cases you better interchange graphs and implementing a simple json-query-walker is a task for a day or two. You don’t need to go through The Era of REST or The Age of RPC to do that, neither is it a stellar innovation. Where are all the hackers gone?
We maintain an Ember app that uses JSON API, as well as a graphQL schema that both touch the same data. JSON API is well suited to Ember, since it manages data relationships client-side with its heavy data layer. GraphQL makes the backend a bit more complex (but not overly so IMO), but the tradeoff is that it abstracts the backend better than JSON API and allows new clients to be spun up more quickly since the data aspect is already taken care of, and the clients only need to be a view layer.
We are using it heavily, and with a few extensions (like a batch modification endpoint) it scales pretty well. I'm not sure what measure of size are you referring to.
I don't see why "rich UI/UX-based applications" is listed as "mobile-first". Sure, GraphQL is probably great for highly dynamic client applications, but why would that be limited to mobile?
As far as I can tell, GraphQL primarily allows you to trade client flexibility for back-end complexity. If you're willing to take the additional complexity on the back-end, GraphQL allows you API clients to iterate far quicker, and removes (or at least lessens) the dependency of the client on the team maintaining the back-end. If that's not worth it to you, don't use GraphQL.
I think the reason which the author glossed over is because it reduces the number of RTTs between the front and back end. On desktop, you typically have a much better internet connection where packets are less likely to get dropped and will generally arrive faster meaning that the performance impact of having a large number of RTTs is much less than on mobile.
One huge benefit is that the client controls the response shape and the same server will power multiple different response schemes without requiring any changes. Very nice for supporting N types of clients.
Correct me if I'm wrong, but having read around the topic of GraphQL with a slowly-changing, largely static catalogue type appication in mind (ideal fit for CDN in front of REST); I remember a few horror stories around GraphQL not being CDN-friendly.
Perhaps if you have a dynamic or real-time data-lake or messaging-style app with quickly-stale data that would need to be talking to regional server clusters anyway, then GraphQL has benefits on top for schema independence etc. But I wonder if its CDN caching story has improved (not just client-side as mentioned.)
Some GraphQL clients will create POST requests by default with the query as the data (Apollo does this), which should normally break caching, however there is no reason you couldn't append the query as a querystring param in a GET, which would help with that.
I'm helping to move a big 10B per year ecommerce site to graphql. This lack of tooling support around graphql (vs REST) is a big minus currently.
Some examples:
CDN is a big one that you mention. Our vendor (the biggest in the industry) has no support for GraphQL beyond what they do for REST or html pages. We've talked about moving the query into the path or as a parameter but then we have to figure out how to determine what can be cached. With REST it is easy: cache /sku, /product, etc but never /guest, /order, etc.
Tools that do performance monitoring (AppDyn, datadog, etc) don't grok GraphQL well. Quick example is response codes. What's the http status code of a graph response that has a partial failure? If you make it 200, then those tools won't see any errors. We put the error codes in the response body and those tools aren't parsing that for message failures.
>Tools that do performance monitoring (AppDyn, datadog, etc) don't grok GraphQL well.
If you haven't seen Apollo Engine[0], you might want to checkit out. Maybe its too expensive or problematic in some way for your scale, but I've used it quite successfully (on a much smaller scale). You implement their status-format for your responses, and place a proxy between your graphql server and client requests, which then siphons of the status data to them.
Regarding CDN, it would be interesting to see if someone tries to do some "computing on the edge" (i.e Cloudflare workers) to calculate/modify cache info from the graphql response.
EDIT: Apparently Apollo have a proposed standard for indicating cache status inside of responses[1], which then Engine can use to calculate correct cache headers for the CDN.
Our case is similar, albeit smaller, and we're only at the stage of considering the next step (from static, webpacked JSON.) Our application relies on several regionalised hefty product+metadata JSON files. Data changes, while infrequent (~2 per week), involve republishing. Especially for the (recently-added) mobile app, large downloads are a UX issue. Changes aren't easily isolated by file segregation, and can cross-cut, although of course we're looking at the right level of granularity (we do have a product hierarchy and price dimension.) The apps already rely on REST for a smaller part of their dynamic functionality, so providing the main bulk through CDN-cached REST is an option; GraphQL was superficially attractive but the CDN issues are a showstopper for adopting it, for now.
Does anyone know how GraphQL manages (or if it even can) to sort and filter across microservices? It seems impossible to me to do from any client. If you have a list view that is made up of data from microservice a, b, and c and you try to sort on a column from microservice b, how does microservice b know how to sort then data when there are additional constraints applied to data on other microservices such as a filter applied to microservice a. Does that make sense?
Basically I feel very frustrated when it comes to sorting and filtering across microservices, some people at work think that GraphQL will solve this and I have found absolutely nothing to support the claim that it would help us with this. Thoughts?
One thing I'd like to add, if your primary data store resides in a document-oriented DB like Mongo or Dynamo, GraphQL provides that missing "relational" piece, and allows you to create a strong schema without having to manually do "joins" in application code. I realize attempting to "relationalize" a document store isn't how you were supposed to use it, when I started at my current company thats what they had going on. GraphQL was essential for improving the sanity of how we grab data from dynamo, and gave us a good way to iteratively migrate over to Postgres where appropriate.
I'll start off by noting that I haven't been keeping up as much with the latest developments within the GraphQL ecosystem so some of my comments may be outdated.
They ignored one of the most thorough GraphQL implementations: Sangria [0], which provides almost everything you need out of the box to implement a fairly sophisticated application. I guess that Scala isn't very cool. I'd reluctantly agree that most of the community seems to be centered around the JS ecosystem, but I don't think it's fair to so easily dismiss the dozens of GraphQL implementations out there. Perhaps I'm wrong about this, but based on the author's bio it sounds like they're most experienced with JS and Ruby, which means that they're probably heavily biased towards those two languages. I don't think they've seriously evaluated each library extensively and they should be transparent about it if that's the case. I've used GraphQL with Elixir, Node, and Ruby, but took a lot of inspiration from Sangria's docs when working on my apps.
I don't know if it was meant as a criticism, but I actually think that the declarative syntax used with Elixir and Ruby is quite nice. The examples provided for the reduced "Developer Experience" are rather weak in my view. I think the author might just be most comfortable with JS.
> Secondly, as said in the second point, a GraphQL API server needs optimisations and maintenance.
> Making the graph-oriented way of getting data fit over an SQL database can be tricky.
IMO, this is severely understating the complexity. Depending on the type of application, you should probably be prepared to have at least one engineer dedicated full-time to maintaining your GraphQL server.
It's a lot easier to write a secure REST API with good performance than it is to achieve the same thing with GraphQL. The author doesn't even mention anything about the huge headache involved with protecting against malicious queries.
I immediately fell in love with the idea of GraphQL when I first heard about it, but as I started to use it I found it wasn't some magical solution. You'll probably end up having to figure out a lot of stuff on your own since the technology is so young, so be prepared to invest a lot of time in it. It depends heavily on your use-case and your team's collective experience, but I'd probably lean towards avoiding GraphQL within a small team.
If race conditions aren't a concern, you can wrap a REST API with a small GraphQL server. The inverse is also possible.
There's no mention of file uploads. Although many GraphQL libraries include some form of file uploading capability, I'd suggest against using that and just creating separate endpoints for each upload type. Having a separate endpoint makes it way easier to place reasonable limits on input size, as well as rate-limiting those actions adequately. You can also handle streaming uploads, which I don't think most GraphQL libraries can handle; if I recall correctly the Node GraphQL server parses the whole body before executing any resolver.
I used OData in a past life. From what I've seen GraphQL is more capable/tunable. OData feels a bit more constrained to relational models/particular ORMs, and GraphQL is more flexible to homogenize data from multiple sources.
I've seen some very bad OData performance issues, and generally wouldn't recommend its use outside of intranet/enterprise/non-public projects these days. GraphQL seems at first glance worse, but the ability to name queries alone provides a lot of tuning power (making profiling easier, and allowing for the potential to performance tune individual query patterns) and most of the good GraphQL engines provide at least some basic cost estimation systems to quota high cost queries, whereas the cost estimation of OData operators is trickier.
Though really a lot of it comes down to productivity needs. There's still cases where everything is on a single SQL server and Breeze against a simple OData endpoint on the SQL schema is, well, a breeze, for a simple client app. But I'm growing increasingly convinced that GraphQL is better in the long run, and I think a key need right now is much better server-side .NET GraphQL libraries that work well with EF, and have some of the profiling and cost estimation tools you see over in Apollo Engine and the like.
This is kind of touched on by other comments here, but after using GraphQL heavily on a bunch of different projects for both native apps and web UI front ends, one of the biggest benefits I've seen is how it enables the individual teams (front end and back end) to work faster, more efficiently, and more independently.
Our dev process for using GraphQL:
1. Product and design come up with screen designs and workflows.
2. Importantly, the front end teams then take a first stab at writing the GraphQL type contracts, fully doc'ed (we use a Node backend, so it's easy for the front end folks, even native app devs, to know enough JS to write these type strings). They then confer with backend folks on the contract to get agreement, and front end folks write mock endpoints for these contracts. This allows the front end teams to get off to the races early with a working (albeit mocked out) backend. Since the GraphQL type contracts are optimized to closely match the front end screens, it makes the front ends much simpler. With native apps it also means there are a lot more things we can fix with just a server change, not requiring an app store release.
3. Backend folks then start implementing the GraphQL resolvers, which in our architecture is mainly orchestrating microservices. The great thing about GraphQL here is that it essentially serves as a "decoupling layer", giving the backend folks a lot more freedom to change and switch up the microservices as long as the GraphQL contract is still respected.
4. Once the GraphQL resolvers are done, we switch out the mock resolvers. Usually there are a couple of edge cases or other bugs to fix, but for the most part we've seen very little disconnect.
It sounds like the advantage you're citing is a separation of code into client and server, with a well-specified API between the two. You can write a well-specified API with either REST or GraphQL. Is there something about GraphQL in particular that is helping here?
I guess it's makes the front-end a whole lot easier to implement.
To fetch all the data that the app needs to render on a single view, the code would only need to make a single API call to a graphql endpoint and you only have to deal with a single, neatly formatted response.
As opposed to the traditional way of making multiple calls to different REST endpoints, you had to write quite a bit of code to transform & merge the data from each response before using them to render your views.
> To fetch all the data that the app needs to render on a single view, the code would only need to make a single API call to a graphql endpoint and you only have to deal with a single, neatly formatted response.
If the frontend devs come up with the GraphQL schema definition, they could also come up with a single REST request definition with very specific DTOs needed for that particular view/workflow. You would have the same single, neatly formatted response (caching included). There is no difference what technology you use here, as long as your API is designed UI-first and your API design (or the REST facade for microservice orechstration) is developed according to the UI.
If you go API-first, you will end up doing lot of REST calls to get the data you need for your view, but you would encounter the same problems if you were to model your GraphQL queries API-first, before you design the UI workflow.
> If the frontend devs come up with the GraphQL schema definition, they could also come up with a single REST request definition with very specific DTOs needed for that particular view/workflow
The GraphQL schema is a unified, standardized language that describes these objects, types and relations. How many different flavours of REST exist out there? Every company interprets REST slightly differently, uses different conventions etc. Imagine every company in the world that uses REST in any capacity used the same documentation tools (eg. swagger), same conventions to describe relations, where all the tools could automatically generate code and types from the API definitions, with zero effort from the development team. What a wonderful world.
While I agree about people being very vocal about the "right" REST this will not be an issue in this scenario. If you design UI-first and want that little "single HTTP request" that contains all data, you will very likely just define a single DTO entity which will be JSON data - very widely used and understood. There is not much ambiguity in REST here to do it "wrong".
If we talk about write operations/commands, you have the same issues with GraphQL mutations as with REST, there is more than one way to do it, semantics might vary, people will create special fields on the DTO payloads to carry error codes, transactional ids etc. No standards here on both sides of the curtain. I've seen people doing multiple HTTP POSTs do something that should be transactional and a single HTTP POST with rollback on failure; and I am sure there are GraphQL APIs out there that do the same operation performing multiple GraphQL mutations that should be a single, safe transaction etc.
Plus, while REST semantics is debatable and many flavours exist, some common ground like error codes and defined cache semantics exist that integrate well with existing solutions (caching CDN, Loadbalancer etc.) while GraphQL is very lose on this end.
If you're using graphQL just for that...
You can expose a POST endpoint that can follow relation (GET) links recursively, starting from a base resource.
It's a bit of work but still far less than implementing a good schema, resolvers, etc. And much less technological coupling to a particular solution (apollo, etc).
> And much less technological coupling to a particular solution (apollo, etc).
You are confusing a specification (GraphQL) with an implementation (Apollo).
You don't need apollo to access a graphql server. A plain old HTTP POST is all that you need. From a web browser, use the builtin `fetch()` function and be done. No coupling whatsoever.
I'm not confusing anything, but regardless of which solution ones chooses, the implementation is nothing but trivial and people usually turn to fully ready implementations like apollo.
GraphQL allows the front end developers to effectively write their own ad-hoc queries (on top of whatever primitives the backend devs provide them), whereas in REST each query would have to be a custom endpoint on the backend. Or possibly many custom endpoints depending on how REST-y you want to be.
The big issue with GraphQL that I see is maintaining the API contracts over time. The GraphQL queries are defined and stored somewhere in your JS frontend code, iOS or Android App Code. If the backend team refactors the API because some requirements changed, there is no way the backend team can easily verify that the GraphQL queries still run successfully.
Whereas when using a REST API approach, all the Endpoints are within the backend domain. If some properties of your DAO change or get dropped, the DTO<->DAO mapping in the backend will likely break during compile time and the backend team would know that their change affects the contracts implemented by the clients. With GraphQL, such changes would go unnoticed until the frontend dev teams test against new API versions.
While you could use stored GraphQL queries to overcome this, I don't see the benefit here. You backend devs would need to be able to "speak" (learn) GraphQL (maintain the unit tests etc.), whereas in a REST world your frontend devs would need to learn REST.
First off, schemas are stored so they are accessible to the GQL server (not with FE code).
Maintaining API contracts is much easier with GraphQL than with REST (even if you layer on top a contract system like Swagger). The major advantage to GraphQL (which has a powerful schema system) is that you can explicitly know which fields are being accessed. The query itself represents what the client understands (capabilities). This makes it much easier to maintain backwards compatibility. You can even monitor traffic per field to deprecate features.
Re: BE code breaks, that’s what integration tests are for.
How is ANY of this different that REST? REST is very explicit about the available fields and data as it is represented by the API endpoints and their DTO formats. And with GraphQL you can tell (from the server side) which data a client requests based on the query, but there is no guarantee that the client uses all this data to display in the UI view, same as you can't know whether the client uses all the data with REST. If you start UI-first, your REST Endpoints also match exactly the UI requirements of your App, and your DTOs will contain exactly what is needed to render a specific UI. No advantage in GraphQL world here.
> Re: BE code breaks, that’s what integration tests are for.
Agreed, but for this to work you have to have the GraphQL queries that reside in your frontend code in the backend test suite. While it would be good practice to run your iOS/Android/Web App against builds of the backend, you hardly find this level of integration tests in the wild. With REST you test the queries by testing the API endpoints which can be simple HTTP operations with object/result assertions.
I think the difference is somewhat slight but actually really critical.
With REST, a common pattern is that you have our RESTful APIs return "business objects", but then down in the UI layer of the app you have a bunch of logic to turn these into "presentation objects". There are a couple problems with this:
1. If you have to support multiple front ends, like Android and iOS, you have to dupe all this logic.
2. If you have native apps, it makes it MUCH more difficult to change your backend APIs without breaking things.
3. Related to #2 above, GraphQL provides a much better paradigm for maintaining backwards compatibility than REST.
What other folks have said is somewhat true, with GraphQL a lot of what you do is "push up" code from the client up into the server. However, especially with native apps where you can't necessarily control when users upgrade, it is a huge benefit to be able to update that code serverside instead of clientside.
Except that the backwards compatible point is nonsense and actually a PRO REST argument.
If your database schema changes (or the orchestrated microservices), i.e. a field in a table gets dropped (lets say new security policies make this field not readable to everyone) you have NO way of making that change without breaking your GraphQL queries. Because you cannot update that queries that are stored on your users Apps, you will end up having more and more hacks in your GraphQL Server to account for all possible query combinations that are out in the wild. And because you have to account for those hacks, you will need to maintain a list of all old and deprecated queries that exist(ed) in your frontend code and make it available to the backend devs, that maintain the hacks in the GraphQL server adapter.
With REST, you would have to do the same hacks of course to maintain backwards compatibility - but as the REST Endpoint (and the queries they represent) reside in the backend and are not updated or removed, it is way easier to keep track of the hacks to maintain compatibility.
This is absolutely not correct, but it is a common misunderstanding I've seen with how people use GraphQL.
I've posted this before, but I despise how GraphQL has "QL" in the name, so folks think it is similar to SQL in that you should expose your database tables as-is with GraphQL queries. You should absolutely not do this.
You should design your GraphQL contracts to closely match the views displayed in your UIs - it even says this in the GraphQL spec: "Product‐centric: GraphQL is unapologetically driven by the requirements of views and the front‐end engineers that write them. GraphQL starts with their way of thinking and requirements and builds the language and runtime necessary to enable that."
If your GraphQL types closely hew to the design of your views, it shouldn't matter how you store them under the covers, and clients should have no need to know any details about the underlying storage schema.
1. (same)
2. Based on the screen designs and workflows, front end dev team writes the REST contract that they want. They mock the endpoints (i.e. using nock [0] or Angular's MockBackend) for the contracts and come up with the DTO definitions.
3. Backend folks start implementing the REST endpoints by orchestrating microservices (or by creating a REST service facade for an existing REST API).
4. Once the backend REST endpoints are done, switch out the mock endpoints for the real ones
There is no need or benefit for using GraphQL here, all this can be done with REST already. Tooling around REST is great, i.e. you could use TypeScript to generate .d.ts files for your REST Dtos based on the swagger API schema [1]. Or with plain JS, use a JS runtime library that validates the DTO schematics/REST contracts (both your mocked ones and the real ones returned by the backend) with i.e. joi [2].
OCFX, it appears you've been shadowbanned. Your question is very good so reposting it here:
> Does anyone know how GraphQL manages (or if it even can) to sort and filter across microservices? It seems impossible to me to do from any client. If you have a list view that is made up of data from microservice a, b, and c and you try to sort on a column from microservice b, how does microservice b know how to sort then data when there are additional constraints applied to data on other microservices such as a filter applied to microservice a. Does that make sense?
Basically I feel very frustrated when it comes to sorting and filtering across microservices, some people at work think that GraphQL will solve this and I have found absolutely nothing to support the claim that it would help us with this. Thoughts?
I "vouched" for that question (you also have enough karma to do that). Click on the timestamp of the question and you'll get an option to vouch for it as adding to the conversation.
As of now, GraphQL server implementations that I know of, does not manage sort and filters for you. It is up to your resolver to handle those. In case of stitching schema together(for example, your own schema plus the GitHub one), you would need to pass the sort/filter to their services.
Con: Unpredictable code paths (at least in complex use cases).
I've seen so many times GraphQL implementations or ad-hoc equivalents with user supplied property and filter requests, that regret giving the caller so much flexibility. IME, flexibility is a bad thing because in the API world, often once you cross a feature bridge, there's no going back. I say be very restrictive in your API for many reasons. And for that UI use case, I disagree with a complete client-side/server-side dev separation. Unlike data APIs, backend data requests for frontend use should be tightly tailored to what is used. GraphQL is just another way to forget what you've exposed and also allow clients to hit never before seen code paths to your backend. There are security, bloat, and maintenance concerns...keep the UI backend thin and light and inflexible and strict and limited yet agile and volatile and you'll be so happy you did a couple years down the road.
This. Nearly every GraphQL schema I've seen is a logical representation of the domain, just like how we typically approach schema design in OO apps. This is probably fine in many contexts where your GraphQL endpoint is consumed by other internal services.
However, if your GraphQL endpoint is used by consumer/public/external clients I feel like we need to give everyone permission to make a Schema matching the View Model, not the Model (proper). In practice GraphQL works quite nicely as a layer to translate the model proper into the view model. This makes MVVM much more attractive for SPA's than in object->relational worlds prior.
If you know your queries at compile time, then you can register each query with the server when you build as the hash of the query. Then clients only have to send up the hash and the server can look them up in its registry; plus you now have an easy way to cache (cache based on query hash plus user). If it's the first time you've seen a query, you can hash it and add it to the registry as well.
I know about apollo caching, but it's strictly inferior to HTTP caching which has tons of options (Vary, Etag, etc)
Caching per user is still going to mean tons of cache misses. Caching based on the query hash means you're going to cache subsets of the same data over and over as separate entries.
With resources over HTTP, you can separate the user specific part from the generic resource part and cache accordingly.
Plus, I like the option of caching somewhere else than the app server, like a distributed Varnish.
I'm not saying you can't cache with graphQL, it's just less natural and mature.
In practice, caching is non-trivial to implement for any sufficiently complex system even with REST. It depends on your use case of course, and how tolerant you are of stale data, but I have worked on very few frontends where HTTP caching is sufficient.
These seem like concerns that GraphQL backend implementations could address. With sufficient limitations on things like query size, nesting depth, and number of entities requested, it seems like you could have reasonable confidence that no pathological queries are valid.
Security doesn’t seem like a problem unique to GraphQL. Traditional REST APIs also need to be just as careful about authentication and access control.
> With sufficient limitations on things like query size, nesting depth, and number of entities requested, it seems like you could have reasonable confidence that no pathological queries are valid.
Rarely good enough and in general a dangerous mindset that just adding layers around is an acceptable solution to essentially giving the world a query builder (internal use only is different of course).
> Security doesn’t seem like a problem unique to GraphQL
It's about surface area of permutations as much as the data. Sure you can auth property paths, but many GraphQL setups can be gamed by malicious callers to strain systems in ways you never expected those client side devs to do when you gave it to them. If the only reason for this flexibility is because they can't be bothered to do a bit of server side tailoring, you have a dev fragmentation problem and a vertical integration problem that will cause security concerns one day anyways.
I never said they couldn't. What's baloney is pretending FB's approach towards GraphQL is replicated by all. Anyone that's seen the tech deployed in multiple places (and equivalent systems) has seen the cavalier attitude with which data can be accessed with seemingly infinite unique combinations. And it's especially true if you revisit them after some time to see they often grow, but rarely are paths or properties deleted. Sometimes it's due to API compat guarantees, but often it's due to the ridiculous divorce between actual client side usage and expected server side usage.
That you can do it right is not at issue, it's whether you do, and over time IME it's unaudited data proliferation and even when data is controlled, often unaudited permutations of all the possible queries that can be built.
I don't know, I've seen lots of REST APIs that have similar levels of query configurability (albeit without the predictable syntax of GraphQL). It's certainly not uncommon for REST APIs to support params for sorting, filtering, and including related entities (i.e. joining), at which point you could just as easily open yourself up to dangerous queries as you could with GraphQL.
Most definitely and I allude to them in my initial post because it's basically the same problem. Can't get rid of all parameters in all cases of course. Of the common configurability options I see in REST APIs, the ones that affect queries in the most complicated way is the customizable inclusion of extra relational properties.
None of these problems are GraphQL-specific though. Say someone wants to build a feature that requires some attribute or joined model we weren't exposing before. Maybe someone else tells them no for all the caching/performance/complexity reasons mentioned above. That's a slippery slope leading towards a culture where nothing ever gets done because all the senior engineers default to saying "no" to everything, which isn't a place people want to work and probably isn't great for the longterm outlook of the company.
So assuming the feature is going to get built, the implementation might involve sticking the extra attributes in an existing route, or might involve creating a new custom route just for this use. Both of these options have pro's and cons that need to be weighed in the given situation based on usage patterns. And both are possible with or without GraphQL. But they should require just a few lines of configuration in GraphQL vs more lines custom boilerplate code in your controller in a traditional MVC framework, which is a nice win for GraphQL.
The problems can arise whenever people start thinking some tool is basically magic where there's no wrong way to use it and it'll solve all your problems, instead of thinking criticially about how to use the tool and what tradeoffs involved. And this tends to happen much more easily with a hot, new tool like GraphQL than a more established one where people have more experience with the downsides.
> That's a slippery slope leading towards a culture where nothing ever gets done because all the senior engineers default to saying "no" to everything, which isn't a place people want to work and probably isn't great for the longterm outlook of the company.
No, and thinking in absolutes like this is what green lights extreme flexibility exposed to the public internet.
In general, all query possibilities from the public internet should be accountable and eagerly tested/auditable. And don't let complaints about stodgy seniors or flexibility make you do otherwise, development efficiency and tight public internet predictability are not mutually exclusive.
Again, to put it simply, it is very rare to need extreme flexibility in publicly accessible APIs. And articles like this one foolishly refuse to make that clear. In the meantime, super flexible data access schemes are lauded for their help to client side developers, scary. The org can be tightly integrated, don't let the ridiculous idea of having some devs not allowed to do any server side code force these loose-lipped systems on your users.
There are graph ql build systems where you can generate a list of queries used on the frontend, which should help with your concerns over total flexibility, while still allowing easier use of new queries.
If the frontend can use the API, can I, the guy with a curl command or headless browser, use it too? How are you generating that list of queries from my machine? Unless the surface is sufficiently small to get all permutations at which point no prob, my issue is with flexible GraphQL systems.
> If the frontend can use the API, can I, the guy with a curl command or headless browser, use it too?
No.
> How are you generating that list of queries?
I believe the queries in the device UI code are replaced with identifying hashes as part of the build and those prebaked queries are deployed to the GraphQL server.
> Unless the surface is sufficiently small at which point no prob, my issue is with flexible GraphQL systems.
This is still as flexible for UI developers since this is simply a build time optimization / limitation.
For publicly accessible GraphQL APIs I believe the common approach is to calculate a cost for the query and limit what users / queries can do that way.
In the case of turning flexibility at compile time to a fixed set at runtime and the server side only accepts that fixed set, I'd say that's definitely a lot less of a concern than the literal GraphQL queries I've seen in use at runtime by UIs. At that point, your only major concern is how many of those devs use those sans review by person familiar with the storage that could optimize it. After that, your fixed hash request hasn't improved things from a runtime standpoint over a fixed URL request. The disappointing internal team segregation that makes this a required path to go from actual query and programming languages to pseudo ones is concerning.
But for that common approach for APIs you mention, that cost calculator is a bit scary vs limiting the caller. Now if you're a big shop and you need to essentially expose a query language, ok. But for most, the flexibility is really not necessary and bites later with all the possible combinations of requests.
> But they should require just a few lines of configuration in GraphQL vs more lines custom boilerplate code in your controller in a traditional MVC framework, which is a nice win for GraphQL.
If it's a few lines of configuration in GraphQL, it's going to be a few lines of configuration or code in the "traditional MVC framework".
If it's a lot of code in the "traditional MVC framework", then it's going to be a lot of code in GraphQL as well.
The issue with using terms like "traditional MVC framework" is that you will have a hard time making statements that apply to all.
> That's a slippery slope leading towards a culture where nothing ever gets done because all the senior engineers default to saying "no" to everything
They can just say no to GraphQL then. You shouldn't be mixing culture/politics and technology arguments in one context.
”But they should require just a few lines of configuration in GraphQL”
…assuming the presence of the equivalent of a complete enough SQL engine in the server, and often not even then, because adding a new stored procedure that can be called may mean extensive rethinking of the indexes you maintain, the way tables are partitioned, etc.
Actually this is quite false. GraphQL queries are deterministic and Apollo offers great tools for query analysis.
For example the Github GraphQL API has a restriction on the amount of nodes an unauthenticated requests can query. Because of the deterministic nature you know the query complexity before execution time.
As a result you can come up with simple to implement mechanisms for restricting query complexity. This is far superior to any mechanism I have ever seen in REST APIs which are inherently far more restrictive because of the limitations of the design and much harder to predict. e.g. it's easy to exploit O(N) DOS type attacks in REST APIs that expose queries that are relational.
GraphQL supersedes REST in every way. For those who haven't realized that yet are simply missing context or experience.
> Actually this is quite false. GraphQL queries are deterministic and Apollo offers great tools for query analysis.
What is false? Who said anything about determinism?
Unless by deterministic you mean the ability to tell the future about what will be called. I'm talking about giving clients the flexibility instead of having predictability in the call. I'm saying that flexibility has costs especially as systems grow. It's not just GraphQL, lots of systems that are essentially query languages serialized to HTTP requests have this problem. Whether, once you know the call it's deterministic, or whether you can theoretically map all permutations doesn't absolve the practice of giving the clients so much free will.
BFF anti-pattern is a superlinear explosion of surface area to maintain. I agree with you that GraphQL does not actually fix it when backed by any mainstream database. If you solve this, you don't need to write backend boilerplate anymore, the whole thing can be done as an abstraction and you can fire the web developers. No clear answer has emerged yet. I think Datomic (c-f this thread) can be used as a basis to fix BFF anti pattern.
funny. BFF is not an anti pattern. number one it is a pattern.
it makes sure the frontend code is minimised.
if you want to maximised frontend sure go ahead don't use BFF.
but there are scenarios where you do want to minimise frontend code. for example when you have multiple frontends with different languages and code bases.
The holy grail (and goal of GraphQL) is one general purpose backend maintained by N engineers. Backend-for-frontend is M backends maintained by M*N engineers. This is why Google employs thousands of engineers.
yep. you can still have it with the BFF and GraphQL pattern. Since both would use the same general purpose services.
the key difference between GraphQL and BFF is, the GraphQL service itself is general purpose. hence the biz functionality resides in the frontend.
With the BFF pattern you try to move as much as possible of biz functionality into the backend.
- Graphql does allow you to specify data you need and make the user experience better but this is only true in theory. Since in graphql it is so easy to ask any data there are high chances of your queries not optimized and you end up doing a complex operation spanning multiple domains because you can do it. In REST since domains are tightly defined chances of non-optimized request are less.
- Another major con with GraphQL on mobile is there are no alternatives than Apollo client yet. This ties you to 1 solution for your entire application.
- There are companies that tried to enable Querying over rest. Though it is not as flexible as GraphQL.
I don’t see any difference between an app making making one non-optimised GraphQL request for the data it needs vs. making dozens of allegedly “optimised” REST requests for that same data. If the API designer didn’t anticipate that use case then it’s going to be slow either way. Probably slower with REST due to the extra HTTP traffic, loss of query batching, etc.
Re: specifying data - In my experience, you always want to overfetch anyway if you want your offline experience to be any level of reasonable.
It's pretty lame when you're offline, see an item in a list, click on the item, and then the new view has no additional information that wasn't already in the list.
That said, I've built a fairly sizable GraphQL API at my startup (https://banter.fm) and am pretty happy with it, though not 100% convinced I'd use it again.
Self Documenting - GraphQL documents the shape of responses properly. It's like Swagger if the spec always actually matched the implementation. This is a big win for everyone. Backend doesn't have to think about updating separate documentation, frontend always has an up to date and accurate spec.
I've still not seen tooling in the Python ecosystem to make a GraphQL API actually safe to expose to the internet -- without query complexity policing, a malicious client can construct queries that will take an effectively infinite amount of time to complete, and take out your DB connection.
Is this something that's generally solved in other languages? (I've seen a few plugins on NPM that tackle the problem). Or is this just not something that implementors are worrying about?
This is a great summary of the reasons why I initially dismissed GraphQL, and why I'm now a fan - having gone through a process of evaluating it against REST for a production API for a mobile app.
> A GraphQL API (server) implementation, out of the box, will have better performance than a standard REST API - for the clients. Since resolvers are called in parallel, data will load faster.
What does this mean? Are these theoretical REST endpoints fetching information serially, or something?
Almost all of our REST endpoints fetch all of their information in a single hand-optimized SQL query. How is GraphQL going to run faster than that?
I think anyone with experience will realise that you can get pretty much all the benefits of GraphQL with just a more sophisticated RESTish API.
There is however, one thing that I think GraphQL backends have that REST backends don't, and that's the idea of introspection. Every GraphQL backend must be able to serve it's API data models and types. And I think the community tooling that leverages this makes it an amazing experience for app developers. No SDKs, any GraphQL client works with any GraphQL backend. Codegen esp for Android type apps to skip boilerplate of mapping APIs to POJOs. IDE autocomplete. I really think that the killer feature of GraphQL is that.
We built a JSON DSL over a normal POST endpoint. Most GraphQL advantages sound "not really true" to me because you can always achieve the same thing (almost) yourself anyway.
Yup, agreed - the developer experience and the self-documenting nature of GraphQL is a big plus. I no longer use GraphQL, but this is something I miss about it.
As you mention though, people often don't realize that GraphQL is just a way to tell an endpoint that you want information in a generic way, and a pattern for having that endpoint resolve information in a generic way. It's an abstraction on top of a normal service call, and like most abstractions, it gives you consistency and ease-of-use but not any new capabilities you didn't have before.
Just wanted to chime in, OData had basically the same idea, however, from personal observation, somehow just didn't really become so popular, maybe because of its original association with Microsoft, etc. and also partially maybe its implementation. But it was pretty close, and today a lot of products support OData.
I did note recently while reviewing the Microsoft API standard guidelines they share on GitHub [1], that there is a non-OData standard, but OData like recommendation for working with queries across collections, which I find to be pretty good and pretty generic.
It seems like today you can do what you said - a sophisticated RESTish API without the model metadata piece pretty easily - but that metadata piece adds a lot of value.
This used to drive me crazy when GraphQL was first announced without a working download, compared to OData and several usable implementations.
At the time there was also a compelling js library supporting OData.. [1].
> Breeze.js: Rich Data Management for the JavaScript Client
> Client-side querying, caching, dynamic object graphs, change tracking and notification, model validation, batch save, offline support ...
Yeah, I have a project recently where GraphQL is actually a good fit conceptually, but I need a complete .NET client/server implementation and the open source ones are just about there, but I was still not confident enough to use it yet. Ugh.
That Breeze framework was actually always super impressive. I guess there's a company behind it too. I've implemented some stuff with Telerik Kendo UI and OData (data grids, pivot tables) and it also is still working really quite well.
I believe that article is referring when you need to fetch the data in differents places. Of course in your design, a single SQL query to you database would resolve it, probably you would use the same query for a Graphql resolve. But when a field need to be fetched from another database or document in a nosql, for example, graphql tends to be more organized, of course in a restful APi you would run some concurrents jobs and have the same result.
I believe that best benefits that I got from graphql is flexibility in the endpoint with some basic validation data type and a good structure in the backend. However, about performance is debatable.
> probably you would use the same query for a Graphql resolve
When you're hand-writing SQL for a specific client experience, you can make optimizations that GraphQL cannot make because you have knowledge about the constraints on the information in the given context that GraphQL doesn't have. This is a tradeoff that comes with querying information in a generic way vs. querying information in a way that is specifically tailored for the experience. As a result, you often end up with queries that are faster than can be assembled by GraphQL.
What I meant is that you probably would use the same SQL script to fetch from your database the data. GraphQL is not responsible for talk to your database. If you have a SQL database, you must specify the SQL script to fetch the data as well as RESTFUL. Pay attention that GraphQL is not a database technology, Graphql is a query language for APIS.
Yeah, understood. I’m saying that you often don’t use the same SQL script you would if you were using GraphQL, because when you’re writing a SQL script for a very specific user experience you can often make optimizations that only work given the constraints of that experience and the data it needs.
Oh gotcha. Yes, agree. In general a rest system will be faster than GraphQL. The GraphQL abstraction layer decrease the response time. Another point against GraphQL is if the object design has much resolver fetching database style, it will make a bunch of requests to the database that can overload the system or increase your bill.
What if I’m requesting a user’s profile information and a list of posts on their profile, and the list of posts takes 10x longer to fetch? Is it faster from a client perspective if I can’t render the user’s profile page until I wait until all of the posts are fetched, too?
In the general case, I’ve found that GraphQL is slower even in the case where you’re fetching all of a page’s info at once. Not only is it not necessarily always true that one client-server round trip is faster all-up than more granular requests even when you discount the user experience, this pattern often isn’t good general-case behavior for the reason above.
Seriously they are both bad ideas (graphql and rest). Just create a remote method doing exactly what you need with a well-defined metaprotocol. It really doesn’t have to be so hard.
If you go down the GraphQL path be aware you are trading the simplicity, flexibility and universality of HTTP, which you can debug via cURL, to a custom protocol and fat client owned by Facebook. It isn’t even an open standard. The last time Facebook tried to make HTML better we got FBML, remember that?
Be wary of tech that promises to solve all your problems that you yourself don’t understand and haven’t solved. You’re going to end up with at best another set of problems.
Most GraphQL presentations I have seen tend to construct the following strawman scenario:
- we have a REST API that forces the client to make too many requests
- it's impossible to fix the REST API for some reason
- the app is slow because of the multiple REST API calls
The followup:
- we bring in GraphQL and everything becomes super fast and efficient
- the people who couldn't provide an efficient REST API, can all the sudden provide a GraphQL interface
- the cost of implementing and maintaining GraphQL is usually glossed over
The reality is that GraphQL has a lot of drawbacks that are overlooked:
- it might be cheaper to fix the REST API than to implement GraphQL
- it's one extra service or module(if you are lucky) on your backend to look after
- protecting the database from bad queries becomes an order of magnitude harder
- you will have to maintain an additional GraphQL schema
- the client side tooling for GraphQL is far from simple
Depending on your needs, it might still be worth it, I would just wish that GraphQL presentations would be more honest about these.
115 comments
[ 3.4 ms ] story [ 180 ms ] threadClojure ecosystem has an immutable database, Datomic, which is designed for functional programming and fully solves the impedance mismatch. Datomic "Pull queries" is essentially GraphQL – and even predates GraphQL by a year!
Unlike GraphQL, Datomic is also a real database, competitive with SQL and can express relational filters and joins. Datomic is designed for the read-heavy data modeling loads that, today, SQL (and mongo, etc) is used for in anger. https://www.datomic.com/
https://www.datomic.com/on-prem.html
As it just happened to you, you can see they are putting some effort in hiding on-premise from their front page. All their effor seems to be going to the cloud version.
A quagmire of bad decisions made in haste, haunting us— years later.
As far as I can tell, GraphQL primarily allows you to trade client flexibility for back-end complexity. If you're willing to take the additional complexity on the back-end, GraphQL allows you API clients to iterate far quicker, and removes (or at least lessens) the dependency of the client on the team maintaining the back-end. If that's not worth it to you, don't use GraphQL.
If you have one server side code base but targets the web, the mobile, an customer api, a desktop client and embeded devices, it's a net win.
However, if you just have web + mobile, the complexity may not be worth it.
grpc is efficient, but it's the same rpc we've ever had with inflexible static calls and a bunch of roundtrips to do deep querying.
Kind of gives you the benefit of both.
Perhaps if you have a dynamic or real-time data-lake or messaging-style app with quickly-stale data that would need to be talking to regional server clusters anyway, then GraphQL has benefits on top for schema independence etc. But I wonder if its CDN caching story has improved (not just client-side as mentioned.)
Some examples: CDN is a big one that you mention. Our vendor (the biggest in the industry) has no support for GraphQL beyond what they do for REST or html pages. We've talked about moving the query into the path or as a parameter but then we have to figure out how to determine what can be cached. With REST it is easy: cache /sku, /product, etc but never /guest, /order, etc.
Tools that do performance monitoring (AppDyn, datadog, etc) don't grok GraphQL well. Quick example is response codes. What's the http status code of a graph response that has a partial failure? If you make it 200, then those tools won't see any errors. We put the error codes in the response body and those tools aren't parsing that for message failures.
If you haven't seen Apollo Engine[0], you might want to checkit out. Maybe its too expensive or problematic in some way for your scale, but I've used it quite successfully (on a much smaller scale). You implement their status-format for your responses, and place a proxy between your graphql server and client requests, which then siphons of the status data to them.
Regarding CDN, it would be interesting to see if someone tries to do some "computing on the edge" (i.e Cloudflare workers) to calculate/modify cache info from the graphql response.
EDIT: Apparently Apollo have a proposed standard for indicating cache status inside of responses[1], which then Engine can use to calculate correct cache headers for the CDN.
[0]: https://www.apollographql.com/engine [1]: https://github.com/apollographql/apollo-cache-control
(Disclosure: I'm the tech lead for Workers at Cloudflare.)
Basically I feel very frustrated when it comes to sorting and filtering across microservices, some people at work think that GraphQL will solve this and I have found absolutely nothing to support the claim that it would help us with this. Thoughts?
In cases like this you can always fall back to the naive solution: syncing the relevant data between microservices.
Unless there is something like PostgreSQL's foreign data wrappers available.
I'll start off by noting that I haven't been keeping up as much with the latest developments within the GraphQL ecosystem so some of my comments may be outdated.
They ignored one of the most thorough GraphQL implementations: Sangria [0], which provides almost everything you need out of the box to implement a fairly sophisticated application. I guess that Scala isn't very cool. I'd reluctantly agree that most of the community seems to be centered around the JS ecosystem, but I don't think it's fair to so easily dismiss the dozens of GraphQL implementations out there. Perhaps I'm wrong about this, but based on the author's bio it sounds like they're most experienced with JS and Ruby, which means that they're probably heavily biased towards those two languages. I don't think they've seriously evaluated each library extensively and they should be transparent about it if that's the case. I've used GraphQL with Elixir, Node, and Ruby, but took a lot of inspiration from Sangria's docs when working on my apps.
I don't know if it was meant as a criticism, but I actually think that the declarative syntax used with Elixir and Ruby is quite nice. The examples provided for the reduced "Developer Experience" are rather weak in my view. I think the author might just be most comfortable with JS.
> Secondly, as said in the second point, a GraphQL API server needs optimisations and maintenance.
> Making the graph-oriented way of getting data fit over an SQL database can be tricky.
IMO, this is severely understating the complexity. Depending on the type of application, you should probably be prepared to have at least one engineer dedicated full-time to maintaining your GraphQL server.
It's a lot easier to write a secure REST API with good performance than it is to achieve the same thing with GraphQL. The author doesn't even mention anything about the huge headache involved with protecting against malicious queries.
I immediately fell in love with the idea of GraphQL when I first heard about it, but as I started to use it I found it wasn't some magical solution. You'll probably end up having to figure out a lot of stuff on your own since the technology is so young, so be prepared to invest a lot of time in it. It depends heavily on your use-case and your team's collective experience, but I'd probably lean towards avoiding GraphQL within a small team.
If race conditions aren't a concern, you can wrap a REST API with a small GraphQL server. The inverse is also possible.
There's no mention of file uploads. Although many GraphQL libraries include some form of file uploading capability, I'd suggest against using that and just creating separate endpoints for each upload type. Having a separate endpoint makes it way easier to place reasonable limits on input size, as well as rate-limiting those actions adequately. You can also handle streaming uploads, which I don't think most GraphQL libraries can handle; if I recall correctly the Node GraphQL server parses the whole body before executing any resolver.
[0] https://sangria-graphql.org/
I've seen some very bad OData performance issues, and generally wouldn't recommend its use outside of intranet/enterprise/non-public projects these days. GraphQL seems at first glance worse, but the ability to name queries alone provides a lot of tuning power (making profiling easier, and allowing for the potential to performance tune individual query patterns) and most of the good GraphQL engines provide at least some basic cost estimation systems to quota high cost queries, whereas the cost estimation of OData operators is trickier.
Though really a lot of it comes down to productivity needs. There's still cases where everything is on a single SQL server and Breeze against a simple OData endpoint on the SQL schema is, well, a breeze, for a simple client app. But I'm growing increasingly convinced that GraphQL is better in the long run, and I think a key need right now is much better server-side .NET GraphQL libraries that work well with EF, and have some of the profiling and cost estimation tools you see over in Apollo Engine and the like.
Our dev process for using GraphQL:
1. Product and design come up with screen designs and workflows.
2. Importantly, the front end teams then take a first stab at writing the GraphQL type contracts, fully doc'ed (we use a Node backend, so it's easy for the front end folks, even native app devs, to know enough JS to write these type strings). They then confer with backend folks on the contract to get agreement, and front end folks write mock endpoints for these contracts. This allows the front end teams to get off to the races early with a working (albeit mocked out) backend. Since the GraphQL type contracts are optimized to closely match the front end screens, it makes the front ends much simpler. With native apps it also means there are a lot more things we can fix with just a server change, not requiring an app store release.
3. Backend folks then start implementing the GraphQL resolvers, which in our architecture is mainly orchestrating microservices. The great thing about GraphQL here is that it essentially serves as a "decoupling layer", giving the backend folks a lot more freedom to change and switch up the microservices as long as the GraphQL contract is still respected.
4. Once the GraphQL resolvers are done, we switch out the mock resolvers. Usually there are a couple of edge cases or other bugs to fix, but for the most part we've seen very little disconnect.
It sounds like the advantage you're citing is a separation of code into client and server, with a well-specified API between the two. You can write a well-specified API with either REST or GraphQL. Is there something about GraphQL in particular that is helping here?
To fetch all the data that the app needs to render on a single view, the code would only need to make a single API call to a graphql endpoint and you only have to deal with a single, neatly formatted response.
As opposed to the traditional way of making multiple calls to different REST endpoints, you had to write quite a bit of code to transform & merge the data from each response before using them to render your views.
If the frontend devs come up with the GraphQL schema definition, they could also come up with a single REST request definition with very specific DTOs needed for that particular view/workflow. You would have the same single, neatly formatted response (caching included). There is no difference what technology you use here, as long as your API is designed UI-first and your API design (or the REST facade for microservice orechstration) is developed according to the UI.
If you go API-first, you will end up doing lot of REST calls to get the data you need for your view, but you would encounter the same problems if you were to model your GraphQL queries API-first, before you design the UI workflow.
The GraphQL schema is a unified, standardized language that describes these objects, types and relations. How many different flavours of REST exist out there? Every company interprets REST slightly differently, uses different conventions etc. Imagine every company in the world that uses REST in any capacity used the same documentation tools (eg. swagger), same conventions to describe relations, where all the tools could automatically generate code and types from the API definitions, with zero effort from the development team. What a wonderful world.
If we talk about write operations/commands, you have the same issues with GraphQL mutations as with REST, there is more than one way to do it, semantics might vary, people will create special fields on the DTO payloads to carry error codes, transactional ids etc. No standards here on both sides of the curtain. I've seen people doing multiple HTTP POSTs do something that should be transactional and a single HTTP POST with rollback on failure; and I am sure there are GraphQL APIs out there that do the same operation performing multiple GraphQL mutations that should be a single, safe transaction etc.
Plus, while REST semantics is debatable and many flavours exist, some common ground like error codes and defined cache semantics exist that integrate well with existing solutions (caching CDN, Loadbalancer etc.) while GraphQL is very lose on this end.
You are confusing a specification (GraphQL) with an implementation (Apollo).
You don't need apollo to access a graphql server. A plain old HTTP POST is all that you need. From a web browser, use the builtin `fetch()` function and be done. No coupling whatsoever.
Whereas when using a REST API approach, all the Endpoints are within the backend domain. If some properties of your DAO change or get dropped, the DTO<->DAO mapping in the backend will likely break during compile time and the backend team would know that their change affects the contracts implemented by the clients. With GraphQL, such changes would go unnoticed until the frontend dev teams test against new API versions.
While you could use stored GraphQL queries to overcome this, I don't see the benefit here. You backend devs would need to be able to "speak" (learn) GraphQL (maintain the unit tests etc.), whereas in a REST world your frontend devs would need to learn REST.
Maintaining API contracts is much easier with GraphQL than with REST (even if you layer on top a contract system like Swagger). The major advantage to GraphQL (which has a powerful schema system) is that you can explicitly know which fields are being accessed. The query itself represents what the client understands (capabilities). This makes it much easier to maintain backwards compatibility. You can even monitor traffic per field to deprecate features.
Re: BE code breaks, that’s what integration tests are for.
> Re: BE code breaks, that’s what integration tests are for.
Agreed, but for this to work you have to have the GraphQL queries that reside in your frontend code in the backend test suite. While it would be good practice to run your iOS/Android/Web App against builds of the backend, you hardly find this level of integration tests in the wild. With REST you test the queries by testing the API endpoints which can be simple HTTP operations with object/result assertions.
With REST, a common pattern is that you have our RESTful APIs return "business objects", but then down in the UI layer of the app you have a bunch of logic to turn these into "presentation objects". There are a couple problems with this:
1. If you have to support multiple front ends, like Android and iOS, you have to dupe all this logic. 2. If you have native apps, it makes it MUCH more difficult to change your backend APIs without breaking things. 3. Related to #2 above, GraphQL provides a much better paradigm for maintaining backwards compatibility than REST.
What other folks have said is somewhat true, with GraphQL a lot of what you do is "push up" code from the client up into the server. However, especially with native apps where you can't necessarily control when users upgrade, it is a huge benefit to be able to update that code serverside instead of clientside.
If your database schema changes (or the orchestrated microservices), i.e. a field in a table gets dropped (lets say new security policies make this field not readable to everyone) you have NO way of making that change without breaking your GraphQL queries. Because you cannot update that queries that are stored on your users Apps, you will end up having more and more hacks in your GraphQL Server to account for all possible query combinations that are out in the wild. And because you have to account for those hacks, you will need to maintain a list of all old and deprecated queries that exist(ed) in your frontend code and make it available to the backend devs, that maintain the hacks in the GraphQL server adapter.
With REST, you would have to do the same hacks of course to maintain backwards compatibility - but as the REST Endpoint (and the queries they represent) reside in the backend and are not updated or removed, it is way easier to keep track of the hacks to maintain compatibility.
I've posted this before, but I despise how GraphQL has "QL" in the name, so folks think it is similar to SQL in that you should expose your database tables as-is with GraphQL queries. You should absolutely not do this.
You should design your GraphQL contracts to closely match the views displayed in your UIs - it even says this in the GraphQL spec: "Product‐centric: GraphQL is unapologetically driven by the requirements of views and the front‐end engineers that write them. GraphQL starts with their way of thinking and requirements and builds the language and runtime necessary to enable that."
If your GraphQL types closely hew to the design of your views, it shouldn't matter how you store them under the covers, and clients should have no need to know any details about the underlying storage schema.
1. (same) 2. Based on the screen designs and workflows, front end dev team writes the REST contract that they want. They mock the endpoints (i.e. using nock [0] or Angular's MockBackend) for the contracts and come up with the DTO definitions. 3. Backend folks start implementing the REST endpoints by orchestrating microservices (or by creating a REST service facade for an existing REST API). 4. Once the backend REST endpoints are done, switch out the mock endpoints for the real ones
There is no need or benefit for using GraphQL here, all this can be done with REST already. Tooling around REST is great, i.e. you could use TypeScript to generate .d.ts files for your REST Dtos based on the swagger API schema [1]. Or with plain JS, use a JS runtime library that validates the DTO schematics/REST contracts (both your mocked ones and the real ones returned by the backend) with i.e. joi [2].
[0]: https://github.com/nock/nock [1]: https://github.com/swagger-api/swagger-codegen [2]: https://github.com/hapijs/joi
> Does anyone know how GraphQL manages (or if it even can) to sort and filter across microservices? It seems impossible to me to do from any client. If you have a list view that is made up of data from microservice a, b, and c and you try to sort on a column from microservice b, how does microservice b know how to sort then data when there are additional constraints applied to data on other microservices such as a filter applied to microservice a. Does that make sense? Basically I feel very frustrated when it comes to sorting and filtering across microservices, some people at work think that GraphQL will solve this and I have found absolutely nothing to support the claim that it would help us with this. Thoughts?
I've seen so many times GraphQL implementations or ad-hoc equivalents with user supplied property and filter requests, that regret giving the caller so much flexibility. IME, flexibility is a bad thing because in the API world, often once you cross a feature bridge, there's no going back. I say be very restrictive in your API for many reasons. And for that UI use case, I disagree with a complete client-side/server-side dev separation. Unlike data APIs, backend data requests for frontend use should be tightly tailored to what is used. GraphQL is just another way to forget what you've exposed and also allow clients to hit never before seen code paths to your backend. There are security, bloat, and maintenance concerns...keep the UI backend thin and light and inflexible and strict and limited yet agile and volatile and you'll be so happy you did a couple years down the road.
However, if your GraphQL endpoint is used by consumer/public/external clients I feel like we need to give everyone permission to make a Schema matching the View Model, not the Model (proper). In practice GraphQL works quite nicely as a layer to translate the model proper into the view model. This makes MVVM much more attractive for SPA's than in object->relational worlds prior.
See: https://github.com/apollographql/apollo-link-persisted-queri... https://blog.apollographql.com/improve-graphql-performance-w...
Caching per user is still going to mean tons of cache misses. Caching based on the query hash means you're going to cache subsets of the same data over and over as separate entries.
With resources over HTTP, you can separate the user specific part from the generic resource part and cache accordingly.
Plus, I like the option of caching somewhere else than the app server, like a distributed Varnish.
I'm not saying you can't cache with graphQL, it's just less natural and mature.
Security doesn’t seem like a problem unique to GraphQL. Traditional REST APIs also need to be just as careful about authentication and access control.
Rarely good enough and in general a dangerous mindset that just adding layers around is an acceptable solution to essentially giving the world a query builder (internal use only is different of course).
> Security doesn’t seem like a problem unique to GraphQL
It's about surface area of permutations as much as the data. Sure you can auth property paths, but many GraphQL setups can be gamed by malicious callers to strain systems in ways you never expected those client side devs to do when you gave it to them. If the only reason for this flexibility is because they can't be bothered to do a bit of server side tailoring, you have a dev fragmentation problem and a vertical integration problem that will cause security concerns one day anyways.
That's baloney. GraphQL can have just as locked down code paths as any non-GraphQL API. Facebook does exactly this.
That you can do it right is not at issue, it's whether you do, and over time IME it's unaudited data proliferation and even when data is controlled, often unaudited permutations of all the possible queries that can be built.
So assuming the feature is going to get built, the implementation might involve sticking the extra attributes in an existing route, or might involve creating a new custom route just for this use. Both of these options have pro's and cons that need to be weighed in the given situation based on usage patterns. And both are possible with or without GraphQL. But they should require just a few lines of configuration in GraphQL vs more lines custom boilerplate code in your controller in a traditional MVC framework, which is a nice win for GraphQL.
The problems can arise whenever people start thinking some tool is basically magic where there's no wrong way to use it and it'll solve all your problems, instead of thinking criticially about how to use the tool and what tradeoffs involved. And this tends to happen much more easily with a hot, new tool like GraphQL than a more established one where people have more experience with the downsides.
No, and thinking in absolutes like this is what green lights extreme flexibility exposed to the public internet.
In general, all query possibilities from the public internet should be accountable and eagerly tested/auditable. And don't let complaints about stodgy seniors or flexibility make you do otherwise, development efficiency and tight public internet predictability are not mutually exclusive.
Again, to put it simply, it is very rare to need extreme flexibility in publicly accessible APIs. And articles like this one foolishly refuse to make that clear. In the meantime, super flexible data access schemes are lauded for their help to client side developers, scary. The org can be tightly integrated, don't let the ridiculous idea of having some devs not allowed to do any server side code force these loose-lipped systems on your users.
No.
> How are you generating that list of queries?
I believe the queries in the device UI code are replaced with identifying hashes as part of the build and those prebaked queries are deployed to the GraphQL server.
> Unless the surface is sufficiently small at which point no prob, my issue is with flexible GraphQL systems.
This is still as flexible for UI developers since this is simply a build time optimization / limitation.
For publicly accessible GraphQL APIs I believe the common approach is to calculate a cost for the query and limit what users / queries can do that way.
But for that common approach for APIs you mention, that cost calculator is a bit scary vs limiting the caller. Now if you're a big shop and you need to essentially expose a query language, ok. But for most, the flexibility is really not necessary and bites later with all the possible combinations of requests.
If it's a few lines of configuration in GraphQL, it's going to be a few lines of configuration or code in the "traditional MVC framework".
If it's a lot of code in the "traditional MVC framework", then it's going to be a lot of code in GraphQL as well.
The issue with using terms like "traditional MVC framework" is that you will have a hard time making statements that apply to all.
> That's a slippery slope leading towards a culture where nothing ever gets done because all the senior engineers default to saying "no" to everything
They can just say no to GraphQL then. You shouldn't be mixing culture/politics and technology arguments in one context.
…assuming the presence of the equivalent of a complete enough SQL engine in the server, and often not even then, because adding a new stored procedure that can be called may mean extensive rethinking of the indexes you maintain, the way tables are partitioned, etc.
For example the Github GraphQL API has a restriction on the amount of nodes an unauthenticated requests can query. Because of the deterministic nature you know the query complexity before execution time.
As a result you can come up with simple to implement mechanisms for restricting query complexity. This is far superior to any mechanism I have ever seen in REST APIs which are inherently far more restrictive because of the limitations of the design and much harder to predict. e.g. it's easy to exploit O(N) DOS type attacks in REST APIs that expose queries that are relational.
GraphQL supersedes REST in every way. For those who haven't realized that yet are simply missing context or experience.
What is false? Who said anything about determinism? Unless by deterministic you mean the ability to tell the future about what will be called. I'm talking about giving clients the flexibility instead of having predictability in the call. I'm saying that flexibility has costs especially as systems grow. It's not just GraphQL, lots of systems that are essentially query languages serialized to HTTP requests have this problem. Whether, once you know the call it's deterministic, or whether you can theoretically map all permutations doesn't absolve the practice of giving the clients so much free will.
- Graphql does allow you to specify data you need and make the user experience better but this is only true in theory. Since in graphql it is so easy to ask any data there are high chances of your queries not optimized and you end up doing a complex operation spanning multiple domains because you can do it. In REST since domains are tightly defined chances of non-optimized request are less.
- Another major con with GraphQL on mobile is there are no alternatives than Apollo client yet. This ties you to 1 solution for your entire application.
- There are companies that tried to enable Querying over rest. Though it is not as flexible as GraphQL.
It's pretty lame when you're offline, see an item in a list, click on the item, and then the new view has no additional information that wasn't already in the list.
That said, I've built a fairly sizable GraphQL API at my startup (https://banter.fm) and am pretty happy with it, though not 100% convinced I'd use it again.
Self Documenting - GraphQL documents the shape of responses properly. It's like Swagger if the spec always actually matched the implementation. This is a big win for everyone. Backend doesn't have to think about updating separate documentation, frontend always has an up to date and accurate spec.
Github handles this problem: https://developer.github.com/v4/guides/resource-limitations/
Is this something that's generally solved in other languages? (I've seen a few plugins on NPM that tackle the problem). Or is this just not something that implementors are worrying about?
Great post.
What does this mean? Are these theoretical REST endpoints fetching information serially, or something?
Almost all of our REST endpoints fetch all of their information in a single hand-optimized SQL query. How is GraphQL going to run faster than that?
I think anyone with experience will realise that you can get pretty much all the benefits of GraphQL with just a more sophisticated RESTish API.
There is however, one thing that I think GraphQL backends have that REST backends don't, and that's the idea of introspection. Every GraphQL backend must be able to serve it's API data models and types. And I think the community tooling that leverages this makes it an amazing experience for app developers. No SDKs, any GraphQL client works with any GraphQL backend. Codegen esp for Android type apps to skip boilerplate of mapping APIs to POJOs. IDE autocomplete. I really think that the killer feature of GraphQL is that.
We built a JSON DSL over a normal POST endpoint. Most GraphQL advantages sound "not really true" to me because you can always achieve the same thing (almost) yourself anyway.
As you mention though, people often don't realize that GraphQL is just a way to tell an endpoint that you want information in a generic way, and a pattern for having that endpoint resolve information in a generic way. It's an abstraction on top of a normal service call, and like most abstractions, it gives you consistency and ease-of-use but not any new capabilities you didn't have before.
I did note recently while reviewing the Microsoft API standard guidelines they share on GitHub [1], that there is a non-OData standard, but OData like recommendation for working with queries across collections, which I find to be pretty good and pretty generic.
It seems like today you can do what you said - a sophisticated RESTish API without the model metadata piece pretty easily - but that metadata piece adds a lot of value.
[1] https://github.com/Microsoft/api-guidelines/blob/vNext/Guide... - see sections on sorting, filtering etc.
At the time there was also a compelling js library supporting OData.. [1].
[1] https://github.com/breeze/breeze.jsUnfortunately I never had the opportunity to use it for a real project.
[Edit formatting]
That Breeze framework was actually always super impressive. I guess there's a company behind it too. I've implemented some stuff with Telerik Kendo UI and OData (data grids, pivot tables) and it also is still working really quite well.
When you're hand-writing SQL for a specific client experience, you can make optimizations that GraphQL cannot make because you have knowledge about the constraints on the information in the given context that GraphQL doesn't have. This is a tradeoff that comes with querying information in a generic way vs. querying information in a way that is specifically tailored for the experience. As a result, you often end up with queries that are faster than can be assembled by GraphQL.
In the general case, I’ve found that GraphQL is slower even in the case where you’re fetching all of a page’s info at once. Not only is it not necessarily always true that one client-server round trip is faster all-up than more granular requests even when you discount the user experience, this pattern often isn’t good general-case behavior for the reason above.
Be wary of tech that promises to solve all your problems that you yourself don’t understand and haven’t solved. You’re going to end up with at best another set of problems.
So it's not the request itself that is not debuggable, but maybe the query interpreter on your backend with the framework you're using.
- we have a REST API that forces the client to make too many requests - it's impossible to fix the REST API for some reason - the app is slow because of the multiple REST API calls
The followup:
- we bring in GraphQL and everything becomes super fast and efficient - the people who couldn't provide an efficient REST API, can all the sudden provide a GraphQL interface - the cost of implementing and maintaining GraphQL is usually glossed over
The reality is that GraphQL has a lot of drawbacks that are overlooked:
- it might be cheaper to fix the REST API than to implement GraphQL - it's one extra service or module(if you are lucky) on your backend to look after - protecting the database from bad queries becomes an order of magnitude harder - you will have to maintain an additional GraphQL schema - the client side tooling for GraphQL is far from simple
Depending on your needs, it might still be worth it, I would just wish that GraphQL presentations would be more honest about these.