No matter how careful you are with the initial architecture of your microservices system, it is inevitable to re-architect as you go. Specifically to properly distribute service responsibilities.
> No matter how careful you are with the initial architecture of your microservices system, it is inevitable to re-architect as you go.
Isn't this true for all software systems? If anything, microservice architectures just make it easier to re-architect parts of the system, thus lower the bar to actually do the work.
Microservices make it easier to change stuff inside each service, but only if the separation makes sense in the first place. If you have a distributed monolith, then it's actually harder. I personally think keeping the monolith around until it's very clear what the microservices are is the best way to go.
> Microservices make it easier to change stuff inside each service, but only if the separation makes sense in the first place.
With regards to stuff inside each service, microservices aren't really different from monoliths.
When service calls are involved, I'd argue that microservices make it even simpler to refactor/rewrite modules and entire services. Microservice architectures already impose boundaries between components that help limit the scope of any work, and teams are free to deploy and run in parallel, even in production, any service they wish to update/rewrite/launch. You can control which service is called with a load balancer, and don't bother much with the services you're developing.
What I meant is that you need a good separation of concerns for the microservices.
Each microservice is owned by a different team, so if you need to change something that touches 2 or more, you need to collaborate with other teams...this adds a lot of overhead. Not to mention that it becomes almost impossible to do real re-architecting: you can't really move data flows or code from one microservice to another. In my experience you're more likely to end up with duplicate flows, duplicate code, just because it's too damn hard to move "class X" from A to B. In a monolith you can and often do those kinds of changes.
It's also easier to have unit tests. It's easier to have integration tests that are still fast and self contained. Sure, you might say that unit tests are easy to write in any scenario but what I mean is maintaining them becomes easier, especially if you're doing Detroit style. It's easier to not break them when doing larger refactorings.
if you're not sure what the correct decomposition of responsibilities into services / components / libraries is, that suggests structuring the project in a way (at least initially, until you're feeling highly directional pain to suggest a refactoring direction) that the cost of rearranging responsibilities is low. as well as starting with a monolith, that also suggests starting with everything in the one source control repository to make it trivial to refactor and rearrange across swathes of code in lockstep.
in some organisations the amount of meetings and overhead scales proportionally to the number of network connections in the architecture diagram, so that also suggests replacing external dependencies with in-process function calls wherever possible.
But microservices make it a lot more complex. Start with a monolith in an easily refactorable language. Microservices are a solution to a problem you WISH you had.
> Accidental complexity is a bad thing and should be avoided if possible.
Yes. I absolutely agree.
Start with a monolith. Use server-side rendering if possible. Use SQLite instead of [wretched nosql abomination]. API development is a waste of time. The business doesn't give a shit about any of this. Just make it work, then make it "webscale".
Care to elaborate? In my mind, server-side rendering means: no more JavaScript framework to maintain (and that means: webpack + babel + a few hundred npm dependencies).
> In my mind, server-side rendering means: no more JavaScript framework to maintain
Not necessarily. All the major JavaScript frameworks also support server-side rendering, which is used to improve load times and perceived performance.
With server-side rendering you don't have to create yet another layer to actually transfer the data to the client just to render it. It usually also implies that you don't have build your data model in two different languages ($BE_LANG + JS/TS); of course there are things like JSON Schema, OpenAPI and whatnot to mitigate that, but they have their own share of problems.
Developing an API may also be a waste of time when your client-side application is the sole consumer of the server API. This is especially true when people try to build an API that does too much (often seen when building REST-style HTTP APIs) instead of a more usecase-based (or even view-centric) API.
> With server-side rendering you don't have to create yet another layer to actually transfer the data to the client just to render it.
That layer doesn't go away with server-side rendering. It is always there, even if you don't use a REST API. It just gets fused with views.
In fact, I'd argue that this perception that server-side rendering gets rid of that layer is a telltale sign that the project is already plagued with accidental complexity.
> It usually also implies that you don't have build your data model in two different languages ($BE_LANG + JS/TS);
Unless the front-end is written in pure HTML+CSS, this hypothetical simplification doesn't materialize. Also, you can't brush off or ignore the complexity added by templating engines.
> Developing an API may also be a waste of time when your client-side application is the sole consumer of the server API.
I don't get what point you're trying to make. Even if you are the only client for that API, don't you still need to put it together and maintain it?
> But that can be "good enough" for many applications
I disagree. If you expect any type of non-trivial user interaction from the front-end, you'll find most of your needs is not practically possible to implement without javascript running on the client-side.
> look at how the site we are discussing this on works as an example....
Please note that even HN loads javascript, which also makes AJAX calls to the backend.
> That layer doesn't go away with server-side rendering. It is always there, even if you don't use a REST API. It just gets fused with views.
When I wrote my comment I was actually thinking the same, so I agree with you here. However, I still think the (view) model is way closer to your business model without having to go through JSON (or similar) and without the hassle to ensure the client actually received what it expects. I'd argue it's way easier to change or debug in the general case.
> Unless the front-end is written in pure HTML+CSS, this hypothetical simplification doesn't materialize. Also, you can't brush off or ignore the complexity added by templating engines.
It depends on the application, but of course you will have a few highly interactive parts that needs JS/TS (although technologies like alpine.js/html can help here), but still not all models need to exists in the client.
> I don't get what point you're trying to make. Even if you are the only client for that API, don't you still need to put it together and maintain it?
Yes, I mean "elaborate" APIs, my point being that an API for a single consumer can be entirely focused on that consumer. I've seen a lot of people building very flexible APIs without the flexibility being needed at all. I'd argue that one is not tempted to do that as much when doing server-side rending.
Controller queries DB, builds a Model then passes that Model to a View.
View uses that Model to render HTML which returned to the client.
With dedicated frontend it turns into:
Controller queries DB, builds an API response.
Frontend receives API response and renders it into HTML.
So basically Model is API.
Issue with SSR is that it's not popular nowadays and most frameworks are stagnating. At this point it's better to stick to separate backend/frontend unless you don't want to touch it.
You don't have to build JSON Schema, OpenAPI and whatnot. Just spew some JSON and consume it with JavaScript.
> Issue with SSR is that it's not popular nowadays and most frameworks are stagnating. At this point it's better to stick to separate backend/frontend unless you don't want to touch it.
It certainly stagnated a bit in the past, but Rails is still alive and so are newcomers like Phoenix (especially LiveView) or even some new PHP stuff.
Also, a lot of talent is wasted when the old backend guys are barred from doing frontend work. They can be crazy efficient with classical server-side approaches.
> You don't have to build JSON Schema, OpenAPI and whatnot. Just spew some JSON and consume it with JavaScript.
This highly depends on the team structure. In projects where a team (or even single developer) works on both the frontend and backend at the same, it may be fine. However, when there is a dedicated frontend and backend team (in my current project we have a dedicated Android team), it makes sense to have some kind of schema (even when not using codegen). Otherwise you will have a lot of communication overhead.
Yes, true, for a lot of our enterprisey customers a simple HTML view (or PWA for that matter; as in visible a web page that appears as app) often suffices (although that's sometimes hard to sell, because there may be potential future requirements that need native capabilities). iOS users probably expect native applications though.
Good developers are often average designers. And the UI changes are often isolated independent from backend changes. To me, a clean API layer with a static SPA style frontend makes it much easier to iterate both UI and backend changes in parallel, without risk of one impacting the other. Development, testing and deployments can be separate and be independent of each other. That decoupling enables faster iterations. Sometimes, SSR seems to take us back to the days of JSP/ASP where view generation is tightly coupled with control/backend layer.
You're both right. The tradeoff you have to make: is the opportunity cost really worth it?
If you're a big company and are greenfielding a new design, you either need a huge amount of buyin (eg we're going to go heads down for 8 months and come up with a solution) OR you need to build something that provides incrimental improvements along the way. It's really hard for a new greenfield system to compete on features with one with 5 years of features on it.
If you don't have any users, it's far more valuable to demonstrate the business value as early as possible and then build a more scalable solution that's easier for a large team to manage once you've gotten a good chunk of funding.
Server side rendering also nicely avoids the multiple round trip problem the author mentioned given browsers are inherently request / response so forces one stream of html output.
While you can be careful about round trips with client side rendering, and graphql helps in that regard, it’s easy to create hidden bottlenecks.
> Doesn't this lead to an unnecessary increase in complexity without any immediate benefit?
Why do you think using server-side rendering leads to an unnecessary increase in complexity without any immediate benefit?
> Doesn't a well developed API mitigate, if not eliminate entirely, needless increases in complexity?
It does when multiple clients consume that API. The point of GP is, I think, that you shouldn't start with the assumption that you will have at some point multiple clients and prepare for it.
Gall’s law: A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.
It’s hard to define a specific brightline, but I’d say you tend towards “complexity from scratch” as you spend time doing things other than making features.
Some people try to write game engines from scratch before actually creating their game. Some people learn & adopt complex frontend frameworks before finishing their project’s landing page. Some people start new projects by immediately building all the abstraction layers they think they’ll need during the project before implementing a single useful feature.
I’ve fallen for each of those at least once, and I’ve seen lots of people doing the same. You’re not going to need it. Wait until it makes sense to justify the hidden time & reliability costs. Wait until you actually know what you need.
I don't have the perfect answer, but my criteria for services might be something like: Is it serving prod traffic? Did you deal with keeping it online, performant, and available in parallel with your development iterations? Did the the experience of keep it running provide critical feed back for further design choices?
Thanks for finally providing me with the source to this piece of wisdom that's been stuck in my head since I was a kid reading through some programming book I had. It's really stuck with me and I've been repeating it ever since, but I could never remember the attribution, just that I saw this quote in a book! I can't quite remember the book, but it was something like "Programming games with C++" and it came with a CD with a demo of Visual C++ 6.0 on it. Must have been around 2002-ish...
If you have never designed or developed software before, by all means try that.
Once you have reached at least junior level, start learning about how to split the system into subsystems and putting some additional thought into it.
The business doesn't give a shit about the implementation. Very true... that's why they hired you so that you fucking do. Just do your job.
If you want to work in finance, accounting, sales, investor relations, partnerships, business development or whatever, go do that. If you want to work creating software, start caring about how software is created, or at least don't ruin it for people that do.
If you decide to follow a business career path, you will find that they also have to care about their job, have attention to detail and care about good practices and compliance. In fact, that will also be the case for every occupation in every sector.
Systems divided up into subsystems include microservices architectures but that’s not the only way to get there.
Like so many fads in our industry microservices solve a problem that giant, famous companies have and are aped by thousands of companies that don’t, and will never, have those problems.
OK, then if not microservices, then the problem will be programming paradigms, programming languages, data structures and algorithms, interviews... everything that forces you to think will be a problem.
We have to stay mediocre and keep the bar low, to lower the entry barrier to software and depress wages.
Yes, we need to lower the barrier to entry. There are not enough programmers to go around. Mediocre programmers are much better than no programmers. Hiring managers at companies that can’t afford to get into bidding wars with FAAMG need to realize that.
You might prefer that they try anyway just to push up your salary a bit more, and I can’t blame you for wanting that, but all of us in this conversation aren’t going to strategically tailor our thoughts to help you achieve that goal.
> If you want to work creating software, start caring about how software is created, or at least don't ruin it for people that do.
How is explaining the way to build actual working shit "ruining it" for others? Do you want to live in some fantasy where Netflix's approach to engineering is applicable everywhere? I don't understand what some of you are trying to get out of this, but it sounds suspiciously like you want to have your cake while simultaneously having consumed it.
> Once you have reached at least junior level, start learning about how to split the system into subsystems and putting some additional thought into it.
Before learning about how to split something into multiple parts, you should start by learning about why you would want to do that.
> If you want to work in finance, accounting, sales, investor relations, partnerships, business development or whatever, go do that. If you want to work creating software, start caring about how software is created, or at least don't ruin it for people that do.
Again, before that you should learn about the why. You are writing software for someone. Why? For most people, especially when doing "web stuff", the why is going to be way more important than the how. Carefully crafted software is useless if it doesn't fit what the business wants. Better than that is bad software that still fits the use case. Even better is carefully crafted software that fits the use case.
Right, then let's create spaghetti code where everything is coupled together so that no other developer can understand each other's code, with comments in Klingon.
That is certainly a deliverable a company can use to grow a team of multiple developers and focus on making progress without being constantly interrupted by incidents, defects, downtime as customers roasting and demoralizing customer support representatives and leaving bad reviews online.
Those slow requests in your logs mean a human is looking at a spinning wheel asking why the fuck they bought your software. That is good for the business.
Your product being affected by a security breach is excellent for the business.
Your company infringing licenses because everyone is copying and pasting shit from the Internet is great for the business.
The mythical, romantic image about "business" is just the place where lazy developers run to when they want to stop caring about work.
Real business people want software that does not break everytime a sales person is giving a demo, want a good reputation and do not want to create opportunities for competitors to have a superior product.
Talking about business is not a valid replacement for doing the things that are actually favorable for the business.
All of your post is one long strawman. I could copy your structure by replacing "bad code" with "no business utility" but I don't see the point, as it would be a strawman too.
> Right, then let's create spaghetti code where everything is coupled together so that no other developer can understand each other's code, with comments in Klingon.
I reject the idea that you need to split your code into services as soon as multiple developers are working on it. I also reject the idea that everyone needs well-encapsulated piece of code to be able to work, with careful comments explaining everything and a great documentation. That's the ideal case, but life is not always ideal. Some code that you will have to work with is not perfect, and you have to understand it and modify it. Deal with it, that's your job. Sometimes the business needs one small feature delivered in a few days, not a month-long refactor to make "clean code".
I work on a codebase of 16 millions lines of code. In general it's a relatively good codebase, but it's far from perfect. Some things are easy to refactor, so we try to do it. Some other would require big architectural changes. Sure, I would love for the product to be perfect, but perfect evolves with time, and the codebase is 35 years old. You just have to live with imperfect code.
Large systems are like the living organisms that need to be cared and grown. We can't predict everything as our children grow, but we make sure they're moving on a right direction and help them with difficulties where possible.
Systems grow from toddlers to old men, so do the businesses. They just evolve and adapt, otherwise they die.
Performance isn't just latency - it's also about the requests and responses per second a service can handle.
GQL notoriously performs weaker to REST in this regard.
>When a user navigates to a page on a web app or deep links into a Single Page Application (SPA) or a particular view in a mobile app, the frontend application needs to call the backend service to fetch the data needed to render the view. With RESTful APIs, it is unlikely that a single call will be able to get all the data. Typically, one call is made, then the frontend code iterates through the results of that call and makes more API calls per result item to get all the data needed.
>For example, I don’t want to navigate through multiple screens to review my travel itinerary; I want to see the summary (including flights, car rental, and hotel reservation) all on one screen before I commit to making the purchase.
I disagree.
Sure, sometimes more than one restful call is made for just one page, but the example given, when architectured properly is just a user foreign key relationship to all its children (reservations flights, rentals, etc), which is just a single sql query.
I don't think the author really understand the flexibility of REST and how much it can emulate what GQL has to offer.
If you want specificity in your query fetching, just add query params or put them in the request body
And if it is the case that you really do need resources from different endpoints, what is preventing you to build that single endpoint in REST?
That is more related to the frontend, the frontend doesn't have to overfetch data.
But the graphql still will fetch that data and just filter what does out, it still has to get that data.
Example is a query like
```
{
currentUser {
id
name
todoLists {
title
items {
name
}
}
}
}
```
The resolver will likely get the whole user object from the database, then just send name and id. Then when it finished getting the user, it will then query for the todo lists, and then only send the title (even though it got the whole row for each todo list), then after it fetches those lists, it will query for the items. And retrieve the whole rows of each item from the database.
The data the server needed to fetch didn't change, just what the frontend receives. It is still loading and fetching all the data on that query and then graphql filters the results leaving the server.
Also in the above steps, you notice it queries AGAIN after a data set has been retrieve, this causes an N+1 problem.
It is not inherent in the specs or implementation that fixes these. If you want to avoid fetching the whole object you will need custom code, and to avoid N+1 problem, you need batching of data within requests that "caches" or consolidate nested requests like data-loader, and some form of response caching to help with these issues.
Not siding against the tech, just clarifying those cons.
Yes the client queries for only data it needs and server returns only data which client requested.
With this query,
{ currentUser { id name todoLists { title items { name } } } }
It is up to the server how it is implemented.
- The server can fetch all the data for the user, todolist and items from the database in one go and resolve the client query mentioned above. In this case there will be overfetching from the database if the client only requested user information.
The server can also fetch the data in 3 queries
1> First to fetch the user, lets say with id 1.
2> Then get all the todos for the for user id 1.
3> Then get all the items for all the todos in step 2. Batching/Dataloaders.
All these queries can be executed in parallel on the server side. Does this make the server complex? Yes but there is also benefit to this when the user only request currentUser it does not fetch any todolists or items from the database.
> Performance isn't just latency - it's also about the requests and responses per second a service can handle.
Performance is 99% about latency. And, when having this conversation, we are typically talking about I/O-bound latency.
Assume you have a trivial computer system that can only process 1 thing at a time. The number of things you can process per unit time is inversely proportional to the amount of time it takes to process each thing. Adding more threads/cores to the mix does not change this fundamental equation. Many practical business systems have a large component of processing that must occur in a serial fashion.
Think:
Multiple computers: Milliseconds, One computer: Nanoseconds
How much more work can you do if you can finish each unit in 100ns vs 5ms? How would this impact the user experience? Is the reason you cant finish in 100ns because you are doing global illumination on the CPU or because you are waiting for [hipster noSQL solution] to return some packets to your application server?
sure, I can accept that definition of latency.
However, the author, when describing latency, only specified the distance traveled to justify a graphql server without implicating anything else.
In this comment you've made a claim that GQL is not only weaker than REST here, but notoriously weaker. The only argument that you seem to have made here is that a REST call "can" do the same type of things that are built into GraphQL.
You haven't supported your position that GraphQL is weaker at all. Everything that I have read, seen and heard is exactly the opposite. GraphQL seems to be the "I will never go back to REST" experience for virtually everybody I know in the field who has used it.
The other issue is that you're fighting the nature of REST itself to make it do more. Sure, you can make any API call take in specific input and send out exactly the output that you need...but REST API guidelines tend to preach exactly the opposite approach. The call should return one thing and if you want more, you need to get it yourself.
I'm open to hearing your perspective on this, but if you're going to make a claim like that you really need to support it.
I may be wrong about this, but here's how I see things: I think the two main "variables that depend on" performance are latency and how much computer power you need. And the "variable that depend on" how much computer power you need are server costs, and the architecture of your application.
If we assume that REST serves more requests than GraphQL, but GraphQL is more flexible, that may be a trade off of dev time of the application vs a dev time of the infrastructure. Which would fit with more and more people using orchestration and things like that. That might be a very wrong assumption though.
personally I found gql to have a steeper learning curve in that it adds an abstract layer to the stack.
I call it an abstract layer, the author calls it a BFF (backend for the frontend).
As far as dev time excluding onboarding time, lets not forget why gql was created - facebook wanted to separate the data responses to their mobile and web platform
first of all, not all organizations are facebook
secondly, especially for smaller to medium sized startups, not all responses need to be separated, and "shaving" off data for an extra endpoint is not difficult in REST.
and I remember graphql's early website where they made the claim saying that you'll need to have some really huge number of endpoints in order to emulate what graphql has to offer, and since then they have taken that down because of how ridiculous that sounded.
In reality and practice, the number of endpoints that you need to "shave", again, especially for small to medium startups, is slim to none.
Isn't it more work to create REST services and GraphQl to implement BFF pattern. Also why is GraphQL bad for data services? The N+1 problem that author mentions is easily solvable by dataloaders. Why create two layers on backend?
If the underlying data store is SQL capable, then dataloaders are just a hack trying to do what the SQL engine already does in much more efficient way. Much better to map the GraphQL query to proper SQL query avoiding the round trips.
Yes, with SQL engine you can fetch a parent row with all its children in one go. However, sometimes the client does not need child rows at all and in this case there will overfetching from database and maybe it is fine when query is fast.
My main question was why implement GraphQL BFF layer on top of REST layer. How is this efficient, unless you have legacy REST service that you want expose as GraphQL service. If I am writing a service from scratch why would I create a REST service and then wrap it with GraphQL service?
I strongly believe in simplicity and maintainability of the applications, even at the cost of some flexibility. I like the simplicity of REST when compared to the GQL. Yeah, you have write some extra lines of code, but in return I get reliable tools, mature frameworks, simpler caching, easy to fine tune database queries, easy testability, etc.
63 comments
[ 0.16 ms ] story [ 165 ms ] threadIsn't this true for all software systems? If anything, microservice architectures just make it easier to re-architect parts of the system, thus lower the bar to actually do the work.
With regards to stuff inside each service, microservices aren't really different from monoliths.
When service calls are involved, I'd argue that microservices make it even simpler to refactor/rewrite modules and entire services. Microservice architectures already impose boundaries between components that help limit the scope of any work, and teams are free to deploy and run in parallel, even in production, any service they wish to update/rewrite/launch. You can control which service is called with a load balancer, and don't bother much with the services you're developing.
Each microservice is owned by a different team, so if you need to change something that touches 2 or more, you need to collaborate with other teams...this adds a lot of overhead. Not to mention that it becomes almost impossible to do real re-architecting: you can't really move data flows or code from one microservice to another. In my experience you're more likely to end up with duplicate flows, duplicate code, just because it's too damn hard to move "class X" from A to B. In a monolith you can and often do those kinds of changes.
It's also easier to have unit tests. It's easier to have integration tests that are still fast and self contained. Sure, you might say that unit tests are easy to write in any scenario but what I mean is maintaining them becomes easier, especially if you're doing Detroit style. It's easier to not break them when doing larger refactorings.
in some organisations the amount of meetings and overhead scales proportionally to the number of network connections in the architecture diagram, so that also suggests replacing external dependencies with in-process function calls wherever possible.
Yes. I absolutely agree.
Start with a monolith. Use server-side rendering if possible. Use SQLite instead of [wretched nosql abomination]. API development is a waste of time. The business doesn't give a shit about any of this. Just make it work, then make it "webscale".
Doesn't this lead to an unnecessary increase in complexity without any immediate benefit?
> API development is a waste of time.
Why do you think so? Doesn't a well developed API mitigate, if not eliminate entirely, needless increases in complexity?
Not necessarily. All the major JavaScript frameworks also support server-side rendering, which is used to improve load times and perceived performance.
Developing an API may also be a waste of time when your client-side application is the sole consumer of the server API. This is especially true when people try to build an API that does too much (often seen when building REST-style HTTP APIs) instead of a more usecase-based (or even view-centric) API.
That layer doesn't go away with server-side rendering. It is always there, even if you don't use a REST API. It just gets fused with views.
In fact, I'd argue that this perception that server-side rendering gets rid of that layer is a telltale sign that the project is already plagued with accidental complexity.
> It usually also implies that you don't have build your data model in two different languages ($BE_LANG + JS/TS);
Unless the front-end is written in pure HTML+CSS, this hypothetical simplification doesn't materialize. Also, you can't brush off or ignore the complexity added by templating engines.
> Developing an API may also be a waste of time when your client-side application is the sole consumer of the server API.
I don't get what point you're trying to make. Even if you are the only client for that API, don't you still need to put it together and maintain it?
But that can be "good enough" for many applications - look at how the site we are discussing this on works as an example....
I disagree. If you expect any type of non-trivial user interaction from the front-end, you'll find most of your needs is not practically possible to implement without javascript running on the client-side.
> look at how the site we are discussing this on works as an example....
Please note that even HN loads javascript, which also makes AJAX calls to the backend.
When I wrote my comment I was actually thinking the same, so I agree with you here. However, I still think the (view) model is way closer to your business model without having to go through JSON (or similar) and without the hassle to ensure the client actually received what it expects. I'd argue it's way easier to change or debug in the general case.
> Unless the front-end is written in pure HTML+CSS, this hypothetical simplification doesn't materialize. Also, you can't brush off or ignore the complexity added by templating engines.
It depends on the application, but of course you will have a few highly interactive parts that needs JS/TS (although technologies like alpine.js/html can help here), but still not all models need to exists in the client.
> I don't get what point you're trying to make. Even if you are the only client for that API, don't you still need to put it together and maintain it?
Yes, I mean "elaborate" APIs, my point being that an API for a single consumer can be entirely focused on that consumer. I've seen a lot of people building very flexible APIs without the flexibility being needed at all. I'd argue that one is not tempted to do that as much when doing server-side rending.
Controller queries DB, builds a Model then passes that Model to a View.
View uses that Model to render HTML which returned to the client.
With dedicated frontend it turns into:
Controller queries DB, builds an API response.
Frontend receives API response and renders it into HTML.
So basically Model is API.
Issue with SSR is that it's not popular nowadays and most frameworks are stagnating. At this point it's better to stick to separate backend/frontend unless you don't want to touch it.
You don't have to build JSON Schema, OpenAPI and whatnot. Just spew some JSON and consume it with JavaScript.
It certainly stagnated a bit in the past, but Rails is still alive and so are newcomers like Phoenix (especially LiveView) or even some new PHP stuff.
Also, a lot of talent is wasted when the old backend guys are barred from doing frontend work. They can be crazy efficient with classical server-side approaches.
> You don't have to build JSON Schema, OpenAPI and whatnot. Just spew some JSON and consume it with JavaScript.
This highly depends on the team structure. In projects where a team (or even single developer) works on both the frontend and backend at the same, it may be fine. However, when there is a dedicated frontend and backend team (in my current project we have a dedicated Android team), it makes sense to have some kind of schema (even when not using codegen). Otherwise you will have a lot of communication overhead.
If you're a big company and are greenfielding a new design, you either need a huge amount of buyin (eg we're going to go heads down for 8 months and come up with a solution) OR you need to build something that provides incrimental improvements along the way. It's really hard for a new greenfield system to compete on features with one with 5 years of features on it.
If you don't have any users, it's far more valuable to demonstrate the business value as early as possible and then build a more scalable solution that's easier for a large team to manage once you've gotten a good chunk of funding.
While you can be careful about round trips with client side rendering, and graphql helps in that regard, it’s easy to create hidden bottlenecks.
Why do you think using server-side rendering leads to an unnecessary increase in complexity without any immediate benefit?
> Doesn't a well developed API mitigate, if not eliminate entirely, needless increases in complexity?
It does when multiple clients consume that API. The point of GP is, I think, that you shouldn't start with the assumption that you will have at some point multiple clients and prepare for it.
https://en.m.wikipedia.org/wiki/John_Gall_(author)
where's the line between from scratch and evolved?
Some people try to write game engines from scratch before actually creating their game. Some people learn & adopt complex frontend frameworks before finishing their project’s landing page. Some people start new projects by immediately building all the abstraction layers they think they’ll need during the project before implementing a single useful feature.
I’ve fallen for each of those at least once, and I’ve seen lots of people doing the same. You’re not going to need it. Wait until it makes sense to justify the hidden time & reliability costs. Wait until you actually know what you need.
Once you have reached at least junior level, start learning about how to split the system into subsystems and putting some additional thought into it.
The business doesn't give a shit about the implementation. Very true... that's why they hired you so that you fucking do. Just do your job.
If you want to work in finance, accounting, sales, investor relations, partnerships, business development or whatever, go do that. If you want to work creating software, start caring about how software is created, or at least don't ruin it for people that do.
If you decide to follow a business career path, you will find that they also have to care about their job, have attention to detail and care about good practices and compliance. In fact, that will also be the case for every occupation in every sector.
Like so many fads in our industry microservices solve a problem that giant, famous companies have and are aped by thousands of companies that don’t, and will never, have those problems.
We have to stay mediocre and keep the bar low, to lower the entry barrier to software and depress wages.
You might prefer that they try anyway just to push up your salary a bit more, and I can’t blame you for wanting that, but all of us in this conversation aren’t going to strategically tailor our thoughts to help you achieve that goal.
How is explaining the way to build actual working shit "ruining it" for others? Do you want to live in some fantasy where Netflix's approach to engineering is applicable everywhere? I don't understand what some of you are trying to get out of this, but it sounds suspiciously like you want to have your cake while simultaneously having consumed it.
Before learning about how to split something into multiple parts, you should start by learning about why you would want to do that.
> If you want to work in finance, accounting, sales, investor relations, partnerships, business development or whatever, go do that. If you want to work creating software, start caring about how software is created, or at least don't ruin it for people that do.
Again, before that you should learn about the why. You are writing software for someone. Why? For most people, especially when doing "web stuff", the why is going to be way more important than the how. Carefully crafted software is useless if it doesn't fit what the business wants. Better than that is bad software that still fits the use case. Even better is carefully crafted software that fits the use case.
That is certainly a deliverable a company can use to grow a team of multiple developers and focus on making progress without being constantly interrupted by incidents, defects, downtime as customers roasting and demoralizing customer support representatives and leaving bad reviews online.
Those slow requests in your logs mean a human is looking at a spinning wheel asking why the fuck they bought your software. That is good for the business.
Your product being affected by a security breach is excellent for the business.
Your company infringing licenses because everyone is copying and pasting shit from the Internet is great for the business.
The mythical, romantic image about "business" is just the place where lazy developers run to when they want to stop caring about work.
Real business people want software that does not break everytime a sales person is giving a demo, want a good reputation and do not want to create opportunities for competitors to have a superior product.
Talking about business is not a valid replacement for doing the things that are actually favorable for the business.
> Right, then let's create spaghetti code where everything is coupled together so that no other developer can understand each other's code, with comments in Klingon.
I reject the idea that you need to split your code into services as soon as multiple developers are working on it. I also reject the idea that everyone needs well-encapsulated piece of code to be able to work, with careful comments explaining everything and a great documentation. That's the ideal case, but life is not always ideal. Some code that you will have to work with is not perfect, and you have to understand it and modify it. Deal with it, that's your job. Sometimes the business needs one small feature delivered in a few days, not a month-long refactor to make "clean code".
Systems grow from toddlers to old men, so do the businesses. They just evolve and adapt, otherwise they die.
Your body is constantly "refactoring" itself via catabolic processes, and it's also getting of different forms of waste.
GQL notoriously performs weaker to REST in this regard.
>When a user navigates to a page on a web app or deep links into a Single Page Application (SPA) or a particular view in a mobile app, the frontend application needs to call the backend service to fetch the data needed to render the view. With RESTful APIs, it is unlikely that a single call will be able to get all the data. Typically, one call is made, then the frontend code iterates through the results of that call and makes more API calls per result item to get all the data needed.
>For example, I don’t want to navigate through multiple screens to review my travel itinerary; I want to see the summary (including flights, car rental, and hotel reservation) all on one screen before I commit to making the purchase.
I disagree.
Sure, sometimes more than one restful call is made for just one page, but the example given, when architectured properly is just a user foreign key relationship to all its children (reservations flights, rentals, etc), which is just a single sql query.
I don't think the author really understand the flexibility of REST and how much it can emulate what GQL has to offer.
If you want specificity in your query fetching, just add query params or put them in the request body
And if it is the case that you really do need resources from different endpoints, what is preventing you to build that single endpoint in REST?
Also, which properties of GQL make it inherently slower than REST ... ? Both are abstract concepts. (Also I'd argue that GQL is REST.)
That feature is not exclusive to GQL.
But the graphql still will fetch that data and just filter what does out, it still has to get that data.
Example is a query like
``` { currentUser { id name todoLists { title items { name } } } } ```
The resolver will likely get the whole user object from the database, then just send name and id. Then when it finished getting the user, it will then query for the todo lists, and then only send the title (even though it got the whole row for each todo list), then after it fetches those lists, it will query for the items. And retrieve the whole rows of each item from the database.
The data the server needed to fetch didn't change, just what the frontend receives. It is still loading and fetching all the data on that query and then graphql filters the results leaving the server.
Also in the above steps, you notice it queries AGAIN after a data set has been retrieve, this causes an N+1 problem.
It is not inherent in the specs or implementation that fixes these. If you want to avoid fetching the whole object you will need custom code, and to avoid N+1 problem, you need batching of data within requests that "caches" or consolidate nested requests like data-loader, and some form of response caching to help with these issues.
Not siding against the tech, just clarifying those cons.
With this query, { currentUser { id name todoLists { title items { name } } } }
It is up to the server how it is implemented.
- The server can fetch all the data for the user, todolist and items from the database in one go and resolve the client query mentioned above. In this case there will be overfetching from the database if the client only requested user information.
The server can also fetch the data in 3 queries
1> First to fetch the user, lets say with id 1. 2> Then get all the todos for the for user id 1. 3> Then get all the items for all the todos in step 2. Batching/Dataloaders.
All these queries can be executed in parallel on the server side. Does this make the server complex? Yes but there is also benefit to this when the user only request currentUser it does not fetch any todolists or items from the database.
Performance is 99% about latency. And, when having this conversation, we are typically talking about I/O-bound latency.
Assume you have a trivial computer system that can only process 1 thing at a time. The number of things you can process per unit time is inversely proportional to the amount of time it takes to process each thing. Adding more threads/cores to the mix does not change this fundamental equation. Many practical business systems have a large component of processing that must occur in a serial fashion.
Think:
Multiple computers: Milliseconds, One computer: Nanoseconds
How much more work can you do if you can finish each unit in 100ns vs 5ms? How would this impact the user experience? Is the reason you cant finish in 100ns because you are doing global illumination on the CPU or because you are waiting for [hipster noSQL solution] to return some packets to your application server?
You haven't supported your position that GraphQL is weaker at all. Everything that I have read, seen and heard is exactly the opposite. GraphQL seems to be the "I will never go back to REST" experience for virtually everybody I know in the field who has used it.
The other issue is that you're fighting the nature of REST itself to make it do more. Sure, you can make any API call take in specific input and send out exactly the output that you need...but REST API guidelines tend to preach exactly the opposite approach. The call should return one thing and if you want more, you need to get it yourself.
I'm open to hearing your perspective on this, but if you're going to make a claim like that you really need to support it.
If we assume that REST serves more requests than GraphQL, but GraphQL is more flexible, that may be a trade off of dev time of the application vs a dev time of the infrastructure. Which would fit with more and more people using orchestration and things like that. That might be a very wrong assumption though.
As far as dev time excluding onboarding time, lets not forget why gql was created - facebook wanted to separate the data responses to their mobile and web platform
first of all, not all organizations are facebook
secondly, especially for smaller to medium sized startups, not all responses need to be separated, and "shaving" off data for an extra endpoint is not difficult in REST.
and I remember graphql's early website where they made the claim saying that you'll need to have some really huge number of endpoints in order to emulate what graphql has to offer, and since then they have taken that down because of how ridiculous that sounded.
In reality and practice, the number of endpoints that you need to "shave", again, especially for small to medium startups, is slim to none.
My main question was why implement GraphQL BFF layer on top of REST layer. How is this efficient, unless you have legacy REST service that you want expose as GraphQL service. If I am writing a service from scratch why would I create a REST service and then wrap it with GraphQL service?