I suspect that most people would be better off favoring inlined code over modules and microservices.
It's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
The overhead devs spend worrying about code organization may vastly exceed the amount of time floundering with messy programs.
Microservices aren't free, and neither are modules.
> With modules, I need to know where to search to begin with
You can just grep / search all the files.
> in which version.
grep / search doesn't search through time whether you're using one file or many modules. You probably want git bisect if you can't find where something is anymore (or 'git log' if you have good commit messages),
Of course it can't know about non-existent sources. But when the sources are there, it's light years ahead of a simple text search that is grep/ripgrep.
I'm unsure how to "sell" this idea. I don't want to force my view on others until I truly understand the problem that they're trying to solve with modules/microservices.
For searching multiple files, ripgrep works really well in neovim :)
Blow and Carmack are game programmers. They are brilliant, but their local programs and data are tiny compared to distributed systems over social graphs, where N^2 user-user edges interact.
You say that like real-time multiplayer gaming doesn't exist or something. Both of them have worked on those.... I think Carmack invented a lot of the techniques we use for those.
Yeah sure, the scale is smaller, but you can't get away with making the user wait 10 seconds to render text and images to a screen either. I think the software world might be a lot better place if more developers thought like game developers.
Let's not turn it into penis measuring contents, please. Code organization and requirements differ significantly between different programming niches, and the today's accepted practices are not some randomly invented caprices, but the result of the slow (and painful) evolution we've been fighting through past decades. Each niche has optimized over time for its own needs and requirements. My web apis have hundreds of controllers and keeping them in separate files makes it way easier to manage. I know that because we used to keep it all in a single file and it sucked, so over time I learned not to do it anymore. Does it mean that embedded systems devs should organize their code in the same way? I have no idea, that's up to them to decide, based on their specific environment, code and experience.
> Each niche has optimized over time for its own needs and requirements.
Sure, but those "needs and requirements" aren't necessarily aligned with things that produce good software, and I think a lot of software development these days is not aligned. Further, I think the evolutionary path of the web in particular has produced a monstrosity that we'd be better off scrapping and starting over with at this point, but that's a tangential discussion.
That you probably don't even need, but there's a paradigm of "every function should be it's own class" that some devs seem to follow that I will never understand.
I don't do one function one class mantra, but I absolutely need the separate controllers to group methods because each set of them does different things and returns different data. If in my 20+ years of web dev I learned one thing, it's that trying to be too smart with code optimizations and mixing different logic together is never a good idea - it will always backfire on you and everything you "saved" will be nullified by extra time and effort when you're forced to untangle it in future. The whole point of what I wrote was that there's no recipes that can be just uncritically applied anywhere, you need to adapt your style to your particular needs and experience. If you don't need many controllers, great for you... but don't presume you can just copy/paste your own experience on every other project out there, and we all are stupid for doing it differently...
Seems like you're talking about algorithms issues, not code complexity. If your code needs to scale (at all, never mind quadratically) with the size of the data, you're doing something very wrong.
I've worked extensively in both. Both are complex. Games typically have a complexity that requires more careful thinking at the microscopic level (we must do all this game state stuff within 16ms). Web service complexity requires careful thinking about overall systems architecture (we must be able to bring up 10x normal capacity during user surges while avoiding cost overruns). The solutions to these problems overlap in some ways, but are mostly rather different.
Their programs interact with different APIs and constraints on many different sets of hardware. 5 different major hardware targets, 8 operating systems, and hundreds of versions of driver software. It’s hard to do all that well and keep things validated to continue running with minimal ability to update and support it. Web programs barely work on two browsers. Server targets are generally reduced to a single target (Docker or a specific distribution of Linux).
Their tiny data is rendering millions of polygons with real-time lighting, asset loading in the background, low latency input handling, etc. at 60fps+. If my API responds to the user in 100ms, they’re ecstatic. If their game responds to the user in 100ms (5-10 dropped frames) even a few times, they’re likely to ask for a refund or write a bad review, hurting sales.
The constraints are different, but game programmers do the same kinds of optimization social networks do, just for different problems. They avoid doing the N^2 version of lighting, path finding, physics simulations, etc. Social networks avoid it when searching through their graphs.
I think the web and other big tech companies should try thinking about problems the way game programmers do.
Modules are lot cheaper if you have a solver for them.
Microservices are the same modules. Though they force-add distributiveness, even where it can be avoided, which is fundamentally worse. And they make integration and many other things lot harder.
Well, anything what allows you to express the wiring problem in formal terms and solve it. A dependency injection library. Implicit resolution mechanism in Scala. They may solve basic wiring problem.
distage can do more, like wire your tests, manage component lifecycles, consider configurations while solving dependency graph to alter it in a sound manner.
I do not agree. I believe that professionally "most people" don't work on the same code every day and don't work alone.
Modules are a mean of abstraction and are a classic application of "divide et impera" and you'll need them pretty soon to avoid keeping the whole thing in your head.
But different cultures of programming have a different meaning of what a module is, so, maybe, I'm misunderstanding your point
Ever since my time as a mathematician (I worked at a university) and using LaTeX extensively, I never understood the "divide your documents/code into many small files" mantra. With tools like grep (and its editor equivalents), jumping to definition, ripgrep et al., I have little problem working with files spanning thousands of lines. And yet I keep hearing that I should divide my big files into many smaller ones.
Mathematicians rarely collaborate by forking and improving other papers. They rewrite from scratch, because communicating ideas isn't considered the important part, getting credit for a new paper is.
But is it generally easier to read and understand, say 10 files of 1000 lines or 100 files of 100 lines or 1000 files of 10 lines, compared to one 10,000 lines file? (I don't know the answer, and don't have any strong opinion on this.)
But this is precisely my question: in what way does splitting the code into many small files help with that? Personally I find jumping between many files (especially when they are located in various directories on various levels in the fs) pretty annoying...
Of course, that probably depends on the nature of the project. Having all backend code of a web app in one file and all the frontend code in another would be very inconvenient. OTOH, I have no problems with having e.g. multiple React components in one file...
Obviously you can split your code into many files in a way that obfuscates the workings of the program pretty much comoletely. And I think you can write a single 10kloc file that is so well organized that it is easy to read. I just never have seen one...
I believe that files offer one useful level of compartmentalizing code that makes other people easier to understand what is going on by just looking at the file structure before opening a single file. The other guy can't grep anything before they have opened at least one file and found some interesting function/variable.
Actually, LaTeX has to recompile everything even after a tiny change (there are rare exceptions like TikZ, which can cache pictures, but even then it can put them into some file on its own AFAIR, so that I can still have one big file).
Now that I think about it, LaTeX is a very bad analogy anyway, since LaTeX documents usually have a very linear structure.
There are many other reasons why separation is better for humans (separation is organization) but these arguments about parallelism are valid for androids, humanoids and aliens.
I think what goes wrong with dividing is people truly just splitting the code into multiple files.
I think the code should be split conceptually thus not just copy/paste part of code from main file to submodules, but split the code around some functional bounderies or concepts so that each file is doing one thing and compose those concepts into more abstract concepts.
So that when I try to debug something I can decide where I want to zoom in and thus be able to quickly identify the needed files.
Because often you don't know what to grep for, or search is too general and returns lots of irrelevant results, or perhaps you're just in process of onboaring on the new project and you want just to browse the code and follow different logical paths back and forth...
and when dealing with the code that's well-organized and grouped into logically named files and dirs, you simply can navigate down the path and when you open a file all related code is there in one place without extra 10k lines of misc. code noise.
I think the big benefit is not the actual split into files, but the coincidental (driven both by features of some languages and also mere programmer convenience) separation of concerns, somewhat limiting the interaction between these different files.
If some grouping of functions or classes is split out in a separate file where the externally used 'interface' is just a fraction of these functions or classes, and the rest are used only internally within the file, then this segmentation makes the system easier to understand.
I often want to have multiple parts of the code open at once. Sometimes 5-10 different parts of the code (usually not so many as that, but it depends what I'm doing) so I can flip between them and compare code or copy and paste. Most editors I've used don't have good support for that within a single file (and having 5 tabs with the same name isn't going to make it very easy to tell which is which).
Very good point, though this is really a deficiency of "most editors". ATM I have 300+ files open in my Emacs (which is no wonder given that I work on several projects, and my current Emacs uptime is over 6 days, which is even fairly short). Using tabs is bad enough with 5+ files, tabs with 300+ would be a nightmare.
That said, Emacs can show different parts of the same file in several buffers, a feature which I use quite a bit (not every day, but still). And of course I can rename them to something mnemonic (buffer name ≠ file name). So I personally don't find this convincing at all.
One big reason is source control. Having many smaller files with well defined purpose reduces the number of edit collisions (merges) when working in teams.
Also, filenames and directories tree act as metadata to help create a mental map of the application. The filesystem is generally well represented in exploratory tools like file browser and IDE. While the same information can be encoded within the structure of a single file, one needs an editor that can parse and index the format, which may not be installed on every system.
Well, Git is pretty good at merging when the changes are in different places of the same file. Though your second point is a very good one.
FWIW, sometimes when I worked on a really large file, I put some ^L's ("new page" characters) between logical sections, so that I could utilize Emacs' page movement commands.
It's not just git, it's pull requests and code reviews that block on concurrent file updates. The workflow friction multiplies with the number of devs sharing the same code area. Small files minimize this by making more granular work units.
> Also, filenames and directories tree act as metadata to help create a mental map of the application
Correct. But this is an argument against splitting: once your folder structure reflects your mental model, you should no longer split, no matter how big individual files get. Splitting further will cause you to deviate from your mental model.
Also, it seems like we're arguing against a strawman: saying "big files are okay" is not the same as "you should only have big files". When people mean is that having a big file does not provide enough justification to split it. But it is still a signal.
It is a kind of cult really. Along with the rise of modern editors which somehow craps out at relatively large files. So small files are sold as well organized, modular, logically arranged codebase. You see these lot of adjectives to support short files. None of them seem to be obviously true to me.
Generally speaking those 10k lines files are shit code, regardless of context
Generally speaking it's only Enterprise(tm) code that has miniscule (not small, miniscule) files with shit code
As long as it's good code, no one cares. If you aren't working on Enterprise(tm) code odds are every bad code you'll look at has way too many lines in the abstraction unit that is being used
At the risk of getting lost in a swamp of not particularly good answers, it's most useful if you have scope control: If you have language keywords that allow you to designate a function as "The context/scope of this function never escapes this file," then multiple files suddenly become very useful, because as a reader you get strong guarantees enforced by a compiler, and
a much easier time understanding context. The same can be said of variables and even struct fields. In very large programs it can also be useful to say, "The scope of this function/variable/etc never escapes this directory".
If everything is effectively global to begin with, you're right, it might as well all be in one file. In very large programs the lack of scope control is going to be significant problem either way.
This is where object-oriented programming yields most of its actual value - scope control.
This is one of alarmingly few sane, articulate comments amid an extremely weird conversation about how maybe a giant mess of spaghetti is good, actually.
Yeah sure overengineering is a thing, but you're way off the path if you're brushing aside basic modularization.
I strongly disagree on this one. 10+K lines files are absolutely unreadable most of the time. Separating business logic than other parts of the application helps maintaining it and making everything evolve in parallel, without mixing things up. It also helps to clearly see where business logic happens.
The problem arises when you need to read the code of other modules or services. If you can rely on them working as they should, and interact with them using their well-defined and correctly-behaving interfaces, you won't need to read the code.
I'm a proponent of keeping things in a monolith as long as possible. Break code into files. Organize files into modules. A time may come when you need to separate out services. Often, people break things out too early, and don't spend enough effort on thinking how to break things up or on the interfaces.
> The problem arises when you need to read the code of other modules or services. If you can rely on them working as they should, and interact with them using their well-defined and correctly-behaving interfaces, you won't need to read the code.
You can say the exact same thing about C-headers though.
No, to me that's equally as bad. But 100k lines split across 500 well-named files is a lot easier to work with than 10+K line files or multi-repo code.
With an IDE you're can just look at the class hierarchy/data types rather than the files. As long as those are well organized, who cares hoe they span files?
For instance, in C# a class can span multiple files using "partial" or you can have multiple classes in a single file. It's generally not an issue as long as the code itself is organized. The only downside is the reliance on an IDE, which is pretty standard these days anyway.
I'm inbetween. 10K line files are usually extremely messy, but they can be written not to - a large number of well-organized, well-capsulated <100 LOC classes can be very readable if smashed together in one file. It just so happens that people who tend to write readable self-contained classes just don't put them in 10 KLOC files, but rather split them. And vice-versa, creating an association "10 KLOC files are unreadable", where it's not the length of the file, but rather the organization itself.
Same for business logic - very clear separation can be cumbersome sometimes, but otherwise it becomes messy if you're not careful. And careful people just tend to separate it.
Git diff absolutely does not understand function boundaries, it's diff algorithms routinely confuse things like adding a single new function, thinking that the diff should begin with a "}", instead of a function definition.
A line is a unit of change that git can report on.
If it's a separate file that is scoped to some specific concern, sure. But its tgat grouping by concern that is key. Not separation into another file. Extracting ra dom bits of code into separate files would be _worse_.
> A line is a unit of change that git can report on.
Yes and no.
Git doesn't store lines, it stores files. Git diff knows how to spit out line changes by comparing files.
So to run git blame on a 10k line file you're reading multiple versions of that 10k file and comparing. It's slow. Worse still is that trying to split said file up while trying to preserve history won't make the git blame any faster.
Yes and yes. While agree with the general points, note that they didn't say "unit that git stores", but "unit git can report on". Git can totally report on lines as a unit of change.
What muddies the waters here is languages like Java, where "10k lines" means "you've got a 10kLOC class there", and ecosystems like PHP's where while there's nothing in the language to require it, people and teams will insist on one class per file because hard rules are easier to understand and enforce than "let's let this area evolve as we increase our understanding".
As long as what's there is comprehensible, being able to evolve it over time is a very useful lever.
It varies a bit with language and tooling, but 10k lines is around the place where the size of your file by itself becomes a major impediment on finding anything and understanding what is important.
A 10k lines file is not something that will completely destroy your productivity, but it will have an impact and you'd better look out for it growing further, because completely destroying your productivity is not too far away. It is almost always good to organize your code when it reaches a size like this, and the exceptions are on contexts where you can't, never on contexts where it's worthless.
I generally agree. My argument is that 10K lines written one way can certainly be more readable than 10 files x 1K lines written in a different way, so the real differentiator is the encapsulation and code style, not KLOC/file per se.
Honest question: do you think the same exact 10+K lines of code are easier to read spread across 1,000 files? And why do you think the overhead of maintaining the extra code for module boundaries is worth it?
EDIT: And what editor do you use? I'm wondering if a lot of these differences come down to IDEs haha
> Honest question: do you think the same exact 10+K lines of code are easier to read spread across 1,000 files?
This is a fair point but assumes 1 particular use case. It is easier if you are just concerned with a bit of it. If you need to deal with all of it, yeah, good fucking luck. 10k LOC file or 1k 100 LOC files.
10+K lines spread across 1k file is equally as bad as 10+K line files IMO.
I tend to ensure each file serves exactly one purpose (e.g. in C# one file = one class, with a only few exceptions).
I use VS Code, but in every IDE with a file opening palette it's actually really fast: you want to look for the code to, let's say, generate an invoice, just search for "invoice" in the list of files and you'll find it immediatly.
(Also modules have their own problem, I was mainly talking in a general way since that's what the parent comment was talking about.)
The right answer is 20 files with 500 lines in each - i.e. few pages of clean/ readable/logical/well-factored code. Obviously it depends on the code itself - it's is fine to have longer files if highly correlated. Stateful classes should be kept short however as the cognitive load is very high.
I also find that updating code to take advantage of new/better language features / coding styles, etc. is impossible to do on a large code base at once. However, sprinkling these kind of things randomly leads to too much inconsistency. A reasonable sweet spot is to make each file self-consistent in this regard.
My experience stems from larger 500+ person-year projects with millions of lines of code.
The right answer is that there is no right answer. You shouldn't divide your code based on arbitrary metric like size, you should divide it based on concepts/domains.
If a particular domain gets big enough it probably means it contains sub-domains and can benefit from being divided too. But you cannot make that decision based on size alone.
Sure but no problem domain is not going to lead you to 10,000 single line files. Similarly it will likely lead to very few 10K line files. There will likely be a way to factor things into reasonable sized chunks. File sizes are not going to be that highly coupled to problem domain as there are multiple ways to solve the same problem.
Sure, but it can lead to 1000 lines which some people still think is too much.
The point is that a numeric upper bound on LoC is inherently subjective and pointless. Instead of measuring the right thing (concepts) you're measuring what's easy to measure (lines).
In fact, it usually makes things worse. I've seen it over and over: you have N tightly coupled classes in a single file which exceeds your LoC preference.
Instead of breaking the coupling you just move those classes into separate files. Boom, problem solved. Previously you had a mess, now you have a neat mess. Great success!
It's a valid hypothesis, but without empirical data the question is not easy to settle (and neither Carmack nor Blow are notable authorities on systems that have to be maintained by changing teams of hundreds of people that come and go, maintaining a large codebase over a couple of decades; if anything, most of their experience is on a very different kind of shorter-lived codebases, as game engines are often largely rewritten every few years).
Also, the question of inlined code mostly applies to programming in the small, while modules are about programming in the large, so I don't think there's much relationship between the two.
> neither Carmack nor Blow are notable authorities on systems that have to be maintained by changing teams of hundreds of people that come and go
Most systems don't have to be maintained by hundreds of people. And yet they are: maybe because people don't listen to folks like Carmack?
We like stories about huge teams managing huge codebases. But what we should really be interested in is practices that small teams employ to make big impact.
I didn't mean a team of hundreds maintaining the product concurrently, but over its long lifetime. A rather average codebase lifetime for server software is 15-20 years. The kind of codebase that Carmack has experience with has a lifetime of about five years, after which it is often abandoned or drastically overhauled, and it's not like games have an exceptional quality or that their developers report an exceptionally good experience that other domains would do well to replicate what games do. So if I were a game developer I would definitely be interested in Carmack's experience -- he's a leading expert on computer graphics (and some low-level optimisation) and has significant domain expertise in games, but he hasn't demonstrated some unique know-how in maintaining a very large and constantly evolving codebase over many years. Others have more experience than him in domains that are more relevant to the ones discussed here.
... then before you know it your code-base is 10 million lines long and you have no idea where anything is, what the side-effects of calling X are, what's been deprecated, onboarding is a nightmare, retention of staff is difficult, etc. You may be right for the smallest of applications, or whilst you're building an MVP, but if your application does anything substantial and has to live forever (a web app, for example), then you will have to get organised.
I think we are conflating (at least) two different issues here because we don't have a good way to deal with them separately. Closure on one hand and implicit logic on the other.
Most of the time when I split out a method what I want is a closure that is clearly separated from the rest of the code. If I can have such closures, where input and output is clearly scoped, without defining a new method, that might be preferable. Lambdas could be one way to do it but they still inherit the surrounding closure so it's typically not as isolated as a method.
It’s easy to miss the point of what the OP is saying here and get distracted by the fact this file is ridiculously huge. This file used to be a paltry 1k file, a 10k file, a 20k SLOC file… but it is where it is today because of the OP suggested approach.
Yeah, I think this a very much worst case scenario though.
<rant>
You will (for most statements) both be able to find a best and worst case. That's the catch with most generalized statements/ principles, e.g., DRY. The challenge is to find a good "enough" solution because perfection is usually either unfeasibly expensive or impossible (different viewpoints, ...)
</rant>
Though it's kinda hilarious that the source code of a MS project is not natively viewable on a MS platform.
> your task is to just make it a little bit faster. Where do you begin
With a trace from a profiler tool, which will tell you which line number is the hot spot. If run from any modern IDE, you can jump to the line with a mouse-click.
In essence, the file boundaries ought not to make any difference to this process.
>> In essence, the file boundaries ought not to make any difference to this process.
I find this the strongest counter argument. File boundaries shouldn’t matter - but in practice they do.
The ide will give up doing some of its analysis because this is not an optimised use case, the IDE vendors don’t optimise for it. Some IDEs will even give up just simple syntax highlighting when faced with a large file, never mind the fancy line by line annotations.
Do you have a feel for how large a file has to be for an IDE to choke on it? My vim setup can open a 13GB file in about ten seconds, with all syntax highlighting, line numbers, etc working. Once the file is open, editing is seamless.
Bitbucket is the biggest offender for me. It starts to disable syntax highlighting and context once the file hits about 500KB, which should be trivial to deal with.
Counter point: you have 2650 files, with a couple of 10 line functions in each. Your task is to just make it a little bit faster. Where do you start?
Answer: the same place - with a profiler. Logical code organization matters a lot more than “physical” break to specific files.
I have inherited a Java class spaghetti project in the past, with hunderds (perhaps thousands) of short classes, each which doesn’t do much but sits in its own file - and I would much prefer to work on an SQLite style codebase, even if I have to start with the amalgamation file.
Where do you begin if the code was in hundreds of separate modules? It's not clear if it's easier. It would take time to load the code into your brain regardless of the structure.
By the way, JavaScript parser in esbuild is a 16 thousand lines of code module too:
I can't say I agree with all your "okays", although If you prefix them with "In some cases it's okay", then I understand where you're coming from.
The problem is when it's OK and for how long. If you have a team of people working with a codebase with all those "okays", then they have to be really good developers and know the code inside out. They have to agree when to refactor a business login out instead of adding a hacky "if" condition nested in another hacky "if" condition that depends on another argument and/or state.
I guess what I'm trying to say that if those "okays" are in place, then there's a whole bunch of unwritten rules that come in place.
But I agree that microservices certainly aren't free (I'd say they are crazy expensive) and modules aren't free either. But all those "okays" can end up costing you your codebase also.
Respectfully, rants by niche celebrities are not something we should base our opinions on.
If you're a single dev making a game, by all means, do what you want.
If you work with me in a team, I expect a certain level of quality in the code you write that will get shipped as a part of the project I'm responsible for.
It should be structured, tested, malleable, navigable and understandable.
I feel like this is a knee jerk reaction to the hyperbole of the parent comment rather than the contents of the actual linked talks. I'm watching Jonathan Blow's talk linked above and your comment does not seem relevant to that. Jonathan's points so far seem very reasonable. Rather than arguing for 10000 lines of code it's arguing that there is such a thing as premature code split. Moving code into a separate method has potential drawbacks as well.
One suggested alternative is to split reusable code into a local lambda first and lift it into a separate code piece only once we need that code elsewhere. It seems to me that such approach would limit the complexity of the code graph that you need to keep in your head. (Then again, when I think about it maybe the idea isn't really that novel.)
>It should be structured, tested, malleable, navigable and understandable.
People have different thresholds for when their code reaches these states though, especially "understandable".
You can meaningfully define these (and test for them) on a small scale, in a team, but talking about all developers everywhere, these are all very loosely defined.
I’ve said this before about applying Carmack’s architectural input on this topic:
Games are highly stateful, with a game loop that iterates over the same global in-memory data structure as fast as it can. You have (especially in Carmack era games) a single thread performing all the game state updates in sequence. So shared global state makes a ton of sense and simplifies things.
Most web applications are highly stateless with request-oriented operations that access random pieces of permanently-stored data. You have multiple (usually distributed) threads updating data simultaneously, so shared global state complicates things.
That game devs gravitate towards different patterns for their code than web service devs should not be a surprise.
> I have historically used "style A" to allow for not prototyping in all cases, although some people prefer "style B". The difference between the two isn't of any consequence. Michael Abrash used to write code in "style C", and I remember actually taking his code and converting it to "style A" in the interest of perceived readability improvements.
> At this point, I think there are some definite advantages to "style C", but they are development process oriented, rather than discrete, quantifiable things, and they run counter to a fair amount of accepted conventional wisdom, so I am going to try and make a clear case for it. There isn't any dogma here, but considering exactly where it is and isn't appropriate is worthwhile.
> In no way, shape, or form am I making a case that avoiding function calls alone directly helps performance.
I love Carmack but I always thought his conclusion there was unsatisfying. I think the issue with really long functions is that they increase scope - in both the code and mental sense.
Now your MinorFunction3 code can access all the local variables used by MinorFunction1 and MinorFunction2, and there's no easy list of things that it might access which makes it harder to read.
Separate functions do have a nice list of things they might access - their arguments!
Of course this technically only applies to pure functions so maybe that's why it doesn't matter to Carmack - he's used to using global variables willy nilly.
Also sometimes the list of things a function might need to access gets unwieldy, which is when you can reach for classes. So no hard and fast rule but I think increased scope is the issue with it.
I used to be a fan of style C, but these days, I prefer either A or B, with the condition that no MinorFunction should be less than 5 lines of code. If a function is that small, and it's called from only one place, then it doesn't need to be a function.
Using A or B results in self-documenting code and I think DOES (or at least, CAN) improve readability. It also can help reduce excessive nesting of code.
> It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place.
Couldn't disagree more. As usual, it's a tradeoff. You could spend an infinite amount of time refactoring already fine programs. But complex code decreases developers productivity by orders of magnitude. Maybe it's not always worth refactoring legacy code, but you're always much better off if your code is modular with good separation of concerns.
> It's okay not to put "business logic" in a special place
It's not. This is the thing where you start thinking "YAGNI", yadayada, but you inevitably end up needing it. Layering with at least a service and a database/repositories is a no brainer for any non-toy app considering the benefits it brings.
> It's okay to have files with 10,000 lines
10.000 lines is a LOT. I consider files to become hard to understand at 1.000 lines. I just wc'd the code base I work on, we have like 5 files with more than 1.000 lines and I know all of them (I cringed reading the names), because they're the ones we have the most problems with.
As someone who has personally dealt with files as large as 60K lines, I disagree completely. I believe instead that structure and organization should be added as a business and codebase scales. The problem I think most orgs make is that, as they grow more successful, they don't take the time reorganize the system to support the growth, so, as the business scales 100x in employee account, employee efficiency is hampered by a code organization that was optimized for being small and nimble.
It gets worse when the people who made the mess quit or move on, leaving the new hires to deal with it. I've seen this pattern enough times to wonder if it gets repeated with most companies or projects.
I do agree that microservices and/or modules aren't magical solutions that should be universally applied. But they can be useful tools, depending on the situation, to organize or re-organize a system for particular purposes.
Anecdotally, I've noticed that smart people who aren't good programmers tend to be able to write code quickly that can scale to a particular point, like 10k-100k lines of code. Past that point, productivity falls rapidly. I do believe that part of being a skilled developer is being able to both design a system that scales to millions of lines of code across an organization, and to operate well on one designed by someone else.
Well said. You will see very fast if a dev is experienced or not by looking at code organization and naming. Although I deal with experienced ones that just like to live in clutter. You can be both smart and stupid at the same time.
Not sure I totally agree, but one strong point against factoring in-line code out into a function:
You have to understand everything that calls that function before you change it.
This is not always obvious. It takes time to figure out where it's called. IDEs make this easier but not bullet proof. Getting it wrong can cause major, unexpected problems.
Broadly my heuristic for this is, "Would it make sense to run these functions in the other other?".
If you split up MegaFunction(){} to Func1(){} Func2(){}, etc, but it never makes sense to call Func2 except after Func1, then you haven't actually created two functions, you've just created one function in two places.
Refactoring should be about logical separation not about just dicing a steak because it's prettier that way.
I think that's a reasonable heuristic, but I'd also say you have to take into account the human readability aspect of it. It sometimes does make sense IMO to split solely for that, if it allows you to "reduce" a complicated/confusing operation to a string name, leading to it being easier to understand at a glance.
This is how it's taught in school but the argument is that you shouldn't do that, instead just inline Func1 and Func2 and comment it better.
By chopping up MegaFunction like this, you've not actually separated Func1 and Func2 if they aren't really independent, they're just MegaFunction in disguise but now split across two places making it more difficult, not easier, to reason about.
If you need the state (implied or explicitly passed in and out) from having executed Func1 to run Func2 then you're just creating spaghetti code. You're taking a big ball of mud and smearing it around instead of actually tackling the abstraction.
This is how you end up with Func(a, b, c, d, &e, &f) which ends up changing your MegaFunction state.
Or just as bad, Func1,2,N are all private functions, never called anywhere else outside the class MegaFunction is in, so logically (and from the point of view of testability) it's no different to having them all inline.
If you're creating a function that's only ever called once, then instead of a function call you're probably better off with a comment to "name" that block and explain the process instead.
> you shouldn't do that, instead just inline Func1 and Func2 and comment it better.
Frankly It Depends(tm). Sometimes you can do this and not pass too much state, sometimes you can not. Sometimes your state is in a class, or global, and you just pass the class around.
I have somewhere a 3000 line state machine with the app state in a class - i just pulled out logically connected states in auxiliary files when it went over 1k lines. In my case it's easy to comprehend because groups of states are kinda separated logically.
I think you are missing one point: mental load. I doubt people keep all their files in one directory or all their emails in one folder or just have one large drawer with every possible kitchen utensil in a pile. The same is true for code. Organizing your code around some logical divisions allows for thunking. I will agree that some people take it too far and focus too much on "perfect". But even some rudimentary directories to separate "areas" of code can save a lot of unnecessary mental gymnastics.
I find there are practical negative consequences to having a 10,000 line file (well, okay; we don't have those at work, we have one 30k line file). It slows both the IDE and git blame/history when doing stuff in that file (w.r.t. I'll look at the history of the code when making some decisions). These might not be a factor depending on your circumstances (e.g. a young company where git blame is less likely to be used or something not IDE driven). But they can actually hurt apart from pure code concerns.
Organizing code and services so that a rotating crew of thousands of engineers can be productive is critical to companies like Amazon and Google and Netflix. Inlining code is a micro-optimization on top of a micro-optimization (that is, choosing to [re]write a service in C/C++), not an architectural decision.
As someone who works at a place that previously lived by:
>It's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
I absolutely disagree. It's "okay" if you're struggling to survive as a business and worrying about the future 5+ years out is pointless since you don't even know if you'll make it the next 6 months. This mentality of there being no need for discipline or craftsmanship leads to an unmanageable codebase that nobody knows how to maintain, everybody is afraid to touch, and which can never be upgraded.
You don't see the overhead of throwing discipline out the window because it's all being accrued as technical debt that you only encounter years down the road.
I think we're all confused over the definition. Also one might understand what all the proponents are talking about better if they think about this more as a process and not some technological solution:
All input I have is you want your code to run on many machines, in fact you want it to run the same on all machines you need to deliver and preferably more. Vertically and horizontally at the same time, so your services only call localhost but in many separate places.
This in turn mandates a distributed database. And later you discover it has to be capable of async-to-async = no blocking ever anywhere in the whole solution.
The way I do this is I hot-deploy my applications async. to all servers in the cluster, this is what a cluster node looks like in practice (the name next to Host: is the node): http://host.rupy.se if you click "api & metrics" you'll see the services.
With this not only do you get scalability, but also redundancy and development is maintained at live coding levels.
This is the async. JSON over HTTP distributed database: http://root.rupy.se (2000 lines hot-deployable and I can replace all databases I needed up until now)
We delivered many talks on that subject and implemented an ultimate tool for that: https://github.com/7mind/izumi (the links to the talks are in the readme).
The library is for Scala, though all the principles may be reused in virtually any environment.
One of the notable mainstream (but dated) approaches is OSGi.
This is the usual: "don't use a one size fits all solution, sometimes microservices are good, sometimes they're bad. Just be smart and think about why you're doing things before doing them".
How do I horizontally scale my modules across multiple machines? How do I release a new version of my module without waiting for hundreds of other teams to all fix their bugs? .99^100^365 is a very small number.
you scale the component containing your module to multiple nodes, same with microservices, same with monoliths. the only reason it might be hard is if some other module in the component is aggressively preallocating resources even when lightly used, and that is a problem to be solved by itself, orthogonal to the deployment strategy
Not every company can afford to run crappy code in production. PayPal and banks, for example. Being able to quickly roll back changes while still keeping developer velocity and moving forward is important, and it is very difficult when thousands of changes are going out every day in the same monolith.
But not all forward fixes are quick to write or even locate. Also, monoliths take much longer to go through CI (build + all the tests), and deployment tends to be much slower.
In cases when a rollback is necessay, you'll often find yourself in a cycle where a breakage goes out, the deployment is rolled back, a patch is applied, a new deployment goes out and a new breakage happens, the deployment is rolled back, and so on. I am exaggerating how often this happens, but it does happen when you have thousands of engineers, and it happened often enough at my workplace that developers were unhappy and management pushed for breaking up the monolith.
I’m surprised none of the big 3 cloud providers have come out with a cloud native language where modules or even classes gets deployed as micro services that scales horizontally.
Then you can have a mono repo that deploys to multiple micro services / cloud functions / lambdas as needed depending on code changes and programmers don’t have to worry about RPC or json when communicating between modules and can just call the damn function normally.
If you want to scale horizontally then why just run multiple copies of your monolith? That will work just fine unless you're really big, in which case you'll probably be wanted to write your own solution rather than depending on a cloud provider anyway.
The 2 generals problem illustrates the problems with this. You can’t just call a function over an unreliable communication channel and expect it to work the same way.
Solving this in the general case for every function call is difficult. Pure functions are idempotent, so you can retry everything until nothing fails.
But once you add side effects and distributed state, we don’t know how to solve this in a completely generalized and performant way.
Which is the actor model with supervision hierarchy (for clarity). I happen to agree, the actor model is the best approach to writing micro-services in my humble opinion. I would still call them 'micro services' though. Has the term 'micro-services' been overly constrained to RESTful only APIs? If so that would be a shame.
You don’t have to necessarily decide one way or the other. I have systems where the decision to put the module in the same process or to call it via RPC is done at runtime. Java’s dynamic proxies help with this, but it can be done in any language. The only downside is that one has to think about the size of messages crossing the API and that they need to be immutable values, not references to something in the process.
There is another downside, much more problematic in my experience - the failure modes of a distributed system (even over multiple processes in the same machine) are very different. The other process might be killed by OOM manager, user initiated kill signal, or a bug. All of a sudden, asking an object for its string identifier xyz.Name() becomes a possibly failing operation even though it succeeded a microsecond ago.
Yes, there are downsides to distributed systems. In some cases, they are still necessary. This approach allows us to make the decision to be distributed at runtime instead of at design time.
For our services, there is never a single point of failure. We use Resilience4j for retries, caching inside the dynamic proxy handler (using annotations to describe what is cacheable), and annotations to describe what can be done asynchronously.
In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
I was referring to your "only downside" statement (size and mutability of messages). No, that's far from the only downside, and in my experience, not even the most important one.
> In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
This is at odds with
> I have systems where the decision to put the module in the same process or to call it via RPC is done at runtime.
Either it's in the caller's choice (based on the documentation), or it's at runtime, but it can't be both and be equally reliable.
I meant "only downside" in terms of how to design the API parameters and return values.
I did not say it is the caller's choice nor that it is equally reliable. It is a system composition decision. It is a system that is sometimes distributed and treated as such from a reliability point of view. Sometimes the actual implementation is more reliable and faster.
If the same thing is said about A and B, it is not guaranteed that A=B, unless that thing specifies equality. Anyway I agree with the article that microservices are basically modules. They extend the domain of modules. They're usually overhyped.
Yes, generally that's what we want -- modularity. I think the article touches upon a key truth, that There is Nothing New Under the Sun -- we always deal with complexity and we always find the same ways of structuring our responses to it.
One thing I've observed in a microservice-heavy shop before was that there was the Preferred Language and the Preferred Best Practices and they were the same or very similar across the multiple teams responsible for different things. It lead to a curious phenomenon, where despite the architectural modularity, the overall SAAS solution built upon these services felt very monolithic. It seemed counter-productive, because it weakened the motivation to keep separation across boundaries.
You want to get away from sweeping generalisations because once the build/test/package/deployment tax of a modules approach bites, you do want to go monorepo microservices - it’s free clawback of time wasted building and deploying all the parts of a huge system that you didn’t change in your PR.
The article above, and most if not all the comments I read right before posting this, seem to be very quiet about what I thought was one of the main "distinctive elements" of Microservices.
I.e. the idea that each microservice has direct access to its own, dedicated, maybe duplicated storage schema/instance (or if it needs to know, for example, the country name for ISO code "UK" it is supposed to ... invoke another microservice that will provide the answer for this).
I always worked in pretty boring stuff like managing reservations for cruise ships, or planning production for the next six weeks in an automotive plant.
The idea of having a federation of services/modules constantly cross-calling each other in order to just write "Your cruise departs from Genova (Italy) at 12:15 on May 24th, 2023" is not really a good fit for this kind of problems.
Maybe it is time to accept that not everyone has to work on the next version of Instagram and that Microservices are probably a good strategy... for a not really big subset of the problems we use computers for?
The distinct elements are they compile separately and have versioning on their API calls.
No, you don't use separate microservices for writing out that text message.
The idea is pretty simple instead of writing one big program, you write many smaller programs.
It's useful around the 3 to 4 separate developers mark.
It avoids you having to recompile for any minor change, allows you run the tests for just the part you changed and allows you to test the microservices in isolation.
If you a production issue, the exception will be in a log file that corresponds to a single microservice or part of your codebase.
Microservices are a hard form of encapsulation and gives a lot of benefits when the underlying language lacks that encapsulation. e.g. Python.
No, you don't use separate microservices for writing out that text message.
But in order to find out that Genua is the name of the port from where the cruise is departing, the appropriate time (converted to the timezone of the port, or the timezone of the client who will see this message, depending on what business rule you want to apply) and that Genua is in IT=Italy... how many microservices do I have to query, considering that port data, timezones, ISO country codes and dep.date/time of the cruise are presumably managed on at least four different "data stores"?
ive never seen that in practice. you dont have a database for each individual lambda. thats insanity. you can have multiple microservices point to a shared datasource, its not illegal.
"2.1. Fundamentals
By definition, microservices should be loosely coupled, scalable, and independent in terms of development and deployment. Therefore, the database per service is a preferred approach as it perfectly meets those requirements. Let’s see how it looks:"
Please understand that I have worked only on monoliths and will probably retire while still working on monoliths. This kind of absurd positions only come up when someone comes to my office with some grand plan to convert the application I work on to something "more microservice oriented".
You are absolutely right: you do not know much about cruises.
The things you listed are ... 4-5 different applications, mostly running directly on the ship(s) and what is conspicuously missing are the parts that are managed shoreside, like:
Itinerary planning (your product is one or more cruises: therefore you need to preplan the itineraries, which ships to use, when you will enter each port and when you will leave it, and so on... try to imagine this like a mix between hotel management and flight company management)
Inventory management (i.e. Reservation).
You mention "bookings" like a microservice. This could work for ferry line, where you are basically selling a ticket for a single trip, most of the time with no personal accommodation (except maybe for the car).
A Cruise booking usually comes with a ton of ancillary services (e.g. I live in Berlin and I want to take a cruise in the Caribbeans... therefore I need a flight to get there and back. This could be provided by a charter flight or a normal airline ... in either cases the ticket will be part of the booking itself) - take in account that cruise customers are mostly middle-aged or older and relatively affluent, therefore the last thing they would like to do is to create their own itinerary by booking services on 4-5 different websites.
(But I suppose it is better to stop there, we are really OT now)
When you book a cruise, your expectation is to get everything (including hotel stays before or after the cruise, along with flights to and from it) as a single package provided by the cruise vendor (or a travel agency).
So the "reservation system" must take care of all of that.
When I said "4-5 different websites" I was trying to explain the point that a 60 yo high-income guy is usually not interested in getting the cruise itself on Carnival.com, then go look for excursions on CarribeansExcursion.net and to book flights on Lufthansa.de or AirFrance.fr.
It was a remark on the way the Cruise business works, and why it is so, not about architecture.
(But, once again, I really believe that we are way off topic... personally I do not really feel like creating a "Ask HN: are microservices a good choice for the cruise industry?" but if someone feels like submitting one I will try to contribute).
They solve specific problems. If they don't solve a problem you have, then using them is probably a mistake.
The thing is that the framing of "the problems we use computers for" misses the entire domain of problems that microservices solve. They solve organisational problems, not computational ones.
I have exactly the opposite problem though: the kind of problems I have worked on so far would not be "solved" by leveraging "large number of developers that can independently work on an equally large number of small, modular programs with a very well defined, concise interface".
And this is not because "my stuff is complicated and your stuff is a toy", either. It's more like "ERP or Banking Systems" were deployed decades ago, started as monoliths and nobody can really afford to rewrite these from scratch to leverage Microservices (or whatever the next fad will be).
(I am also not sure it is a good idea in general for transactions that have to handle/persist lots of state, but this could be debated).
The problem, in fact, is that "new guys" think that Microservices will magically solve that problem, too, want to use these because they are cool and popular (this year), and waste (and make me waste) lots of time before admitting something that was clear from day 1: Microservices are not a good fit for these scenarios.
(I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this).
Microservices as a named architectural pattern are over a decade old at this point. Anyone jumping on them because they're the new hotness is more than a little behind the times.
> I have exactly the opposite problem though: the kind of problems I have worked on so far would not be "solved" by leveraging "large number of developers that can independently work on an equally large number of small, modular programs with a very well defined, concise interface".
If you don't have multiple teams working on it, and you don't have specific chunks of functionality that you need to scale orthogonally, then you don't have the problems microservices solve. So don't use them. That seems uncontroversial to me.
> I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this
This is survivorship bias, in a sense. Microservices only make sense where access to CPU and network is cheap and on-demand (largely). That's only started to be an assumption you could really make since Amazon's EC2 convinced the world's CIOs that fungible tin was better than their own racks.
That means you don't see microservice architectures in older companies, and you don't see problems being solved with them even where it might have made sense retrospectively because IT Ops would never have been set up to enable it.
Today you don't need a big rewrite to justify introducing microservices where they're needed. That's a straw man. All you need is a team that will be able to move faster if their deployment is decoupled from everyone else's.
But fundamentally if your problem area smells like "bunch of business rules and a database" then, again: if you don't have the problem that the architecture solves, don't use it.
A micro service implementation of that would be CQRS, where the services to write updates, backend process (eg, notifying the crew to buy appropriate food), and query existing records are separated.
You might even have it further divided, eg the “prepare cruise” backend calls out to several APIs, eg one related to filing the sailing plans, one related to ensuring maintenance signs off, and one related to logistics.
Country codes, and other lookup tables, could easily be handled by a repository of config files. Or, if your company uses a single programming language, a library.
One strategy I've used is to designate one system as a source of truth (usually an ERP system) and periodically query its database directly to reload a cache. Every system works off their own periodically refreshed cache. Ideally, having all the apps query a read replica would prevent mistakes from taking down the source of truth.
I haven't done this, but I think using Postgres with a combination of read-only foreign tables and materialized views could neatly solve this problem without writing any extra code.
I don't know how far this would scale. I do know that coordination and procedures for using/changing the source of truth will fall apart long before technical limitations.
Instagram, Dropbox, many of the major tech companies still use a monolith.
Or you can think of it as trunk/branch architecture. One main "trunk" service and other branch services augmenting it, which is a simpler thing to reason about.
Now imagine a small shop of 20 devs deciding to build something more complicated.
I want modules that are highly decoupled. But it's often more painful than extracting some API from a monolith and defining a message passing strategy, interfaces, etc.
Another way to put it is that teams that share parts of the same codebase introduce low level bugs that affect each other, and most organizations are clueless about preventing it and in some cases do not even detect it.
This article seems to have missed 16 years of event-based architectures, domain driven design, bound contexts, CQRS, and pretty much every reason we use microservices.
Ideally, What I want is the ability to decide later whether a particular call should be local, or RPC, and I should not have to care about any of it until I need to start scaling. I should also not need to specify locally what is going on - my reference infrastructure should handle turning the call into a jump-call or a net-call. Ideally, the fiber system that simulates multithreading for me should have an extensibility point that allows my RPC infrastructure to hook into it and "do the right thing" by treating RPC calls as the equivalent of cross-proc co-routines, especially if my OS lets me transparently share memory between two processes: In a way that would let me share pointers and modules across them.
Someday I will find a way to untangle this wishlist enough to turn into a design.
CORBA made rpc calls opaque, which was nice in theory, but there are faults that need to be handled only for remote calls. It’s a lot of extra code to handle those faults that you don’t need for local functions.
Ideally yes. But there are two major differences between a local and a remote call:
1) A remote call – inherently – fail. However, some local calls never fail. Either because they are designed to never fail or because you have done the required checks before executing the call. A remote call can fail because the network is unreliable and there is no way around that (?).
2) A remote call – inherently – can be very slow. Again because of the unpredictable network. A local call may be slow as well but usually either because everything is slow or because it just takes a while.
So if you have a call that may or may not be local you still have to treat it like a remote call. Right?
I think having a certain set of calls that may or may not be executed locally is not that bad. Usually it will just be a handful of methods/functions that should get this "function x(executeLocally = false|true)" treatment - which is prob. an acceptable tradeoff.
The cool part about "some local call never fails" is that because of scale out, out-of-order multi-threaded execution, is that even "local calls" (for some definitions of local) can fail as your preconditions get invalidated in other threads, unless you're employing mutexes/barriers.
Those modern computers are a network mesh of small D/G/CPUs, and it's coming back to bite us sometimes.
IMO the real problem with microservices comes from dealing with distributed state - going from one data store with usually strong consistency guarantees - to many distributed stores that have no way of ensuring they are in sync is the hard part.
RPC vs local calls is trivial in comparison and you can get that level of transparency out of the box with functional programming - it's just data in data out.
> What I want is the ability to decide later whether a particular call should be local, or RPC
Using hindsight, most of the systems that pretend that the network part of invoking RPCs is "easy" and "simple" and "local" end up being very complex, slow and error prone themselves.
See DCOM, DCE, CORBA and EJBs RMI.
You need instead a efficient RPC protocol that doesn't hide the fact it is a RPC protocol - like Cap'n Proto (it's "time-traveling RPC" feature is very interesting).
Part of the reason is that anything that leaves the application package increases the error potential exponentially.
Also, modules "bottleneck" functionality, and allow me to concentrate work into one portion of the codebase.
I'm in the middle of "modularizing" the app I've been developing for some time.
I found that a great deal of functionality was spread throughout the app, as it had been added "incrementally," as we encountered issues and limitations.
The new module refines all that functionality into one discrete codebase. This allows us to be super-flexible with the actual UI (the module is basically the app "engine").
We have a designer, proposing a UX, and I found myself saying "no" too often. These "nos" came from the limitations of the app structure.
I don't like saying "no," but I won't make promises that I can't keep.
BTW: The module encompasses interactions with two different servers. It's just that I wrote those servers.
Microservices, while often sold as solving a technical problem, usually actually solve for a human problem in scaling up an organization.
There's two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).
The first problem, modules, can be solved at the language level. Modules can do that job, and that's the point of this blog post.
The second problem, scalability, is harder to solve at the language level in most languages outside those designed to be run in a distributed environment. But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them; the database can eventually be a bottleneck, but you can scale up databases a lot.
The real reason that microservices may make sense is because they keep people honest around module boundaries. They make it much harder to retain access to persistent in-memory state, harder to navigate object graphs to take dependencies on things they shouldn't, harder to create PRs with complex changes on either side of a module boundary without a conversation about designing for change and future proofing. Code ownership by teams is something you need as an organization scales, if only to reduce the amount of context switching that developers need to do if treated as fully fungible; owning a service is more defensible than owning a module, since the team will own release schedules and quality gating.
I'm not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables. Problems with state synchronization are one of the most common failure modes in distributed applications, where queues get backed up, retries of "bad" events cause blockages and so on.
> Microservices [..] actually solve for a human problem in scaling up an organization.
So does modularity.
"The benefits expected of modular programming are: (1) managerial_development time should be shortened because separate groups would work on each module with little need for communication..."
On the Criteria To Be Used in Decomposing Systems into Modules, D.L. Parnas 1972.
Parnas was writing under an assumption of BDUF, big-bang releases, and expensive hardware instances. As soon as you want to decouple module deployments and accommodate changing requirements over time you need something else. That "something else" might be "making sure your modular monolith has a good enough test suite that any team can deploy it with only their changes" or it might be "microservices for all", but Parnas' assumption that independently verified modules will need to be built once by independent teams then function correctly when assembled has been comprehensively squashed in the last 50 years.
You can deploy modules independently, but the technical and organisational measures you need in place to do it safely are an extra step you need to take whereas with microservices they're built in. Modules live in the same execution context, so you need a good shared ownership model for that context itself.
The point is that Parnas never conceived of doing it because he was writing about a world where the interfaces between modules were known ahead of time and were static.
With microservices, as long as you maintain the contract with caller services, you can deploy whenever you want. If you have some kind of issue, your team is solely responsible. If the caller services do weird things, they are responsible for fixing them.
If you are pushing changes to a module as part of a more monolithic, or wrapper service - if you do a deploy and break the whole big service, you are now responsible for the entirety of any issue, which is now more likely due to integration risk and unrelated changes from others, especially because now there need to be coordination across many teams integrating - hopefully via tests and observability. But this requires a high-degree of maturity for automated quality assurance and site reliability. If you don't have that, the operational risks are very high. So that alternative is having some ops-like function, or other integration team responsible. Or doing more top-down waterfall coordination.
Given the service maturity needed is rare, microservices distributes that operational ownership in a way where there is more accountability.
Every organisation I've worked at that had microservices has had all of them released every other Thursday at the same time by the same pipeline because they are tied to the organisation scrum schedule. Also, most changes are part of epics that span multiple microservices. If you are gonna argue that properly done microservices don't have that problem you are free to practice your perfect microservices alongside perfect scrum, oop, and communism.
I was arguing the opposite, coordinated deployments already presumes more maturity than I was considering. I've only been at one company with same-day deployments. All the other places have been free-for-alls where each group determines their own release schedule, with their own pipelines. Managing deployments between dependent services thus requires significant coordination overhead (or none, because you are stuck with what you get until the other teams deliver your needs on their own timeline).
In chaotic environments like that, microservices help with defining clear ownership of operational responsibility, though at the cost of clear responsibility for the overall distributed system. This comes at the cost of making it hard to develop, let alone ship, changes that impact multiple microservices (outside the scope of a single team, or closely related sibling teams).
The main point I was making was that, with a more monolithic system, managing that kind to organization disfunction, necessitates top-down, waterfall like control. So moving to microservices enables non-coordinated agility removing one layer of coordination problem (a the cost of other problems).
And also with modules you need the business processes to coordinate deployments across teams, because they all live in the same wrapper application. That's what stops them being independent.
You don't restart the network every time you deploy a microservice.
>And also with modules you need the business processes to coordinate deployments across teams, because they all live in the same wrapper application. That's what stops them being independent.
If we're being technical some languages support hot swapping modules so no restart would be needed. Setting that aside, restarting an application isn't anything that needs coordination today. You wouldn't even restart the application. You'd deploy a new version, swap over, and shut down the old version.
>You don't restart the network every time you deploy a microservice.
No, but something changes in the network configuration so that the other microservices are aware of the deployment.
> the technical and organisational measures you need in place to do it safely are an extra step you need to take whereas with microservices they're built in
They aren't built in, it's just that the need for them is impossible to ignore. Developers (and management) can't help but recognize and respect modularity in microservices because the separation of services and the APIs between them make the modularity obvious. When the separation between modules only exists in code, and even then only when seen through the eyes of someone who understands the modular architecture of the codebase, it is easily ignored, inevitably forgotten, and might as well not exist at all.
Module is not just code in a separate file. Litmus test: if you cannot have two modules be written in separate languages - you don't have modules. M
If all modules have to be deployed using the same build even though different build of the modules would have been API compatible - you don't have modules.
That sounds way too strict to me. A C++ app might use the C++ ABI to communicate with its modules, which would preclude writing them in any other language without jumping through too many hoops. But that's an ABI constraint and says nothing about the actual modularity of the application.
IMO a module is defined as something that has a self-contained API, and versioning rules for that API if the module is evolving.
For code modularity to serve the same purpose, I think there needs to be explicit language-level support, because on a medium-sized or larger project, when the modularity exists only in the minds of developers, it might as well not exist at all. You need virtually all of your developers to understand the design and respect the seriousness of violating it, and beyond a certain project size, that isn't possible.
Developers do a much better job with microservices. I think it's easy for them to respect the seriousness of designing and changing the API of a microservice. In contrast, they often don't respect or even realize the seriousness of making a change that affects the modular design of code.
Language-level support for defining and enforcing module interfaces might help developers invest the same level of care for module boundaries in a modular monolith as they do in a microservices architecture, but I've yet to work in a language that achieves this.
Totally agree.
Also microservices shines IF you need different release schedule for two services. If they are managed in the same project/team, the effort you pay is high, the advantage could not pay your bill, so be careful in such scenario.
yeah i've always thought that microservices shine when the org chart defines them. I've been on teams where we were responsible for multiple microservices and it was so annoying to have to bounce back and forth between code bases and deployments and then it was super easy to intertwine different microservices when you were in the them so often. I feel like if you don't have a team dedicated full time per microservice then you probably shouldn't be using that architecture.
This has been my major criticism of them; you cement the design of the app by creating an organizational structure around your software components.
What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail.
You also cannot produce any meaningful end-user value without involving several teams.
Microservices is just "backend, frontend, and database team" reincarnated.
My take: do Microservices all you want, but don't organize teams around the services!
Conway's law is an adage that states organizations design systems that mirror their own communication structure. It is named after the computer programmer Melvin Conway, who introduced the idea in 1967. His original wording was:
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
— Melvin E. Conway
Amazon is a good example of a huge organization absolutely crippled by micro-service fiefdoms driven by sr.mgr whims and not strong technical guidance from such PEs.
Depends entirely on the org. My org actually broke SOA (thousands of services and some intense technical debt) and is now in a years long process of moving toward a centralized service orchestration model. This couldn’t have happened without PE, Sr PE and DE involvement.
Now that sounds interesting. I assume you're not counting each API endpoint as a service, so can you shed any more light on this? The scale sounds mind-boggling. Thousands of separate services being pulled (more) together. Can you give an idea of the size or scope of these services?
Amazon's warehouses have incredible domain complexity which drives the complexity of the software architecture. It's also one of the oldest parts of the company with a direct lineage going back 20+ years. (for example, our inventory databases still have the same schema and are still using a RDBMS, albeit RDS Postgres instead of Oracle).
About five years ago we started rearchitecting to a centralized orchestration service which is being used to abstract this complexity from process teams. This is to avoid projects where 20+ service teams must be engaged just to launch a new solution.
How do you organize teams if not around services? As the GP points out, the whole point of SOA is to scale people. Yes, this makes rearchitecture hard, but that is always the case at scale. The problem of territoriality and resistance to change needs other solutions (tldr; clueful management and mature leadership level ICs who cut across many teams to align architecture vision and resolve disputes between local staff engineers)
Exactly my question. The teams are going to be organized around something. Even if not intentionally, it'll be true in practice, as people are going to end up knowing things. And if you fight that, it only gets worse.
Long ago I consulted for one company that reconstituted teams for every new project, so people got swapped around every few months, usually ending up in a new chunk of the code. Whatever the theoretical benefits, it meant that nobody felt a sense of ownership for any of the code. Combine that with aggressively short project deadlines and an absolute refusal to ever adjust them and the code rapidly dropped in quality. After all, you'd be on something else soon, so any mess wasn't your long-term problem. And why should you bother given that the code you started with wasn't pristine?
As far as I can tell, the best it gets is organizing around the most durable features of the business environment. E.g., audiences and their real-world needs. Next, long-lived solutions to those needs. And then you find the common and/or large needs of those teams and spin up service teams around them. And in the background, some strong culture around what the priorities really are, so that service teams don't get so self-important that they become the tail wagging the dog.
But I love all the war stories here and would love to hear more, as I think there are no perfect global solutions to organizational problems. I think it's mostly in the details of circumstance, history, and the people involved.
Organize around delivering value to the customer. This is the obvious split for different products, and it is natural for something like AWS where each service is a relatively small team. Sometimes the customer is internal and they need an API, and internal customer decides when they are satisfied, not the API passing tests. SOA will arise, but services need to be big rather than micro, have defined internal customers and their own team. Better to have teams with frontend, backend, designer, etc. who collectively complete features, rather than hoping a bunch of services integrate into a solution for the customer at the end. It is much easier to get the architecture right when people talk first and often about what the customer needs more than implementation details. I think once you have decided how data is going to be persisted and what the customer needs to accomplish, the architecture falls into place. The application code should be stateless so it is never a scaling issue. When the code is isolated by customer feature, the blast radius for future changes is much smaller.
Microsoft, Apple, Amazon I think do a good job at preaching the customer first model and organizing around that.
Agreed. I've worked at a scarce few companies who organized this way, but it seemed to work better than "frontend team, backend team, devops team." When each piece of the stack is a different team, it creates unnecessary hostility and miscommunication. Aligning engineers around products, such that your iOS dev and your backend dev are on the same team, seems to run more smoothly. YMMV.
Implicit to this discussion is the belief that growth inevitably means more people, and that the collateral damage of more people is worth the cost of spending so much time to corral said people. So industry clings to frameworks with a single blessed path, languages that provide only one way to do things, caps expressiveness (just mention macros and people start frothing from the mouth).
This isn't easily fixable, but I'd like technologists to at least be able to perceive the extent to which the surrounding business culture has permeated into technical culture. I'm probably considered old AF by a lot of you (41) but I'm becoming increasingly interested in tools/methodologies that enable fewer devs to do a lot more, even if it means that there are sharp edges.
It seems natural and correct to me that the business concerns and technical concerns would be mixed together. Engineers should understand the business needs of the organization and account for them, just as leadership should understand technical needs.
As projects get larger, the problems become more organizational in nature.
Fewer devs doing a lot more has always been my goal, and in hardware we would call this a vertical scaling versus horizontal scaling problem.
Horizontal scaling always incurs communication overheads faster, and as we know, developers have high communication overhead to begin with.
Human scaling is its own problem set and companies like to ramp projects up and then down again. Most of those people only think about how to get more people on board, not how to run the ship once attrition starts to condense responsibilities and reduce redundancy, especially for unpleasant tasks.
I’ve been trying to beat the idea of ergonomics into a smart but mule-headed developer (too old not to know better) for years now and it wasn’t until a combination of team shrink and having to work with code written by a couple of people who emulate his development style that all of a sudden he’s repeating things I said four years ago (of course with no self awareness of where those ideas came from). It remains to be seen if it affects how he writes code, or whether he starts practicing Do As I Say, Not As I Do, but I’m a short-timer now so it’s mostly an anthropological study at this point.
Organizing teams around microservices makes ... a lot of sense?
Also - talking of redesign should be a major lift for any product/service that has real paying clients. The risk of breaking is huge, and the fact that during that time you won't be delivering incremental value is also going to look bad.
Conways law:
Tech follows communication patterns (.i.e org setup)
hence, if you want a certain architecture, you likely need to execute a "reverse conway law" org first. To get the org into the target config, the software will follow.
It’s okay for one team to have several logical units of code. In fact from a Conway First model, it’s helpful because it allows horse trading when it turns out the team competencies don’t match responsibilities very well. I’m not dismantling my team, I’m just transferring module ownership to another team.
What does not work is splitting ownership of many tings for a long time. It’s too many plates to spin. But some things can be shared by a few people without tipping over into anarchy.
"This has been my major criticism of them; you cement the design of the app by creating an organizational structure around your software components."
Conway's Law is basically "You don't have a choice." You will cement the design of your app around your organizational design. (At least, beyond a certain org size. If you have only 4 engineers you don't really have the sort of structure in question here at all.) je42 narrowly beats me to the point that if this is a problem you can try to match your organization to your problem, but that takes a fairly agile organization.
"What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail."
Unfortunately, in the real world, this is not so much "a disadvantage to the microservice approach" as simply "an engineering constraint you will have to work around and take account in your design".
Despite what you may think after I've said that, I'm not a microservice maximalist. Microservices are a valuable tool in such a world but far from the only tool. As with everything, the benefits and the costs must be accounted for. While I've not successfully rearchitected an entire organization from my position as engineer, I have had some modest, but very real success in moving little bits and pieces around to the correct team so that we don't have to build stupid microservices just to deal with internal organizational issues. I don't mean this to be interpreted as a defeatist "you're doomed, get ready for microservices everywhere and just deal with it"; there are other options. Or at least, there are other options in relatively healthy organizations.
But you will have code that matches the structure of your organization. You might as well harness that as much as you can for the benefits it can provide because you're stuck with it whether you like it or not. By that I mean, as long as you are going to have teams structured around your services whether you like it or not, go in with eyes open about this fact, and make plans around it to minimize the inevitable costs while maximizing the benefits. Belief that there is another option will inevitably lead to suboptimal outcomes.
You can't engineer at an organizational level while thinking that you have an option of breaking the responsibility and authority over a codebase apart and somehow handing them out to separate teams. That never works long term. A lot of institutional malfunctioning that people correctly complain about on HN is at its core people who made this very mistake and the responsibility & authority for something are mismatched. Far from all of it, there are other major pathologies in organizations, of course. But making sure responsibility & authority are more-or-less in sync is one of the major checklist items in doing organization-level engineering.
Part of the issue is that not everyone understands the trade-offs or perhaps believes the trade-offs so you get teams/leaders haphazardly ignoring expert guidance and crafting careless micro-services around team boundaries that do not line up with value.
I find the responsibility & authority angle is very helpful in these conversations. It doesn't take a lot of convincing (in a reasonably healthy organization) to say that these should be broadly aligned, and that if you can show people a place where they are either currently grossly out of sync, or are going to be out of sync if we move forward with some particular iteration of the plan, they start thinking more about how to bring them in sync.
The good news is, pretty much everyone has been on at least one side of that balance being broken (that is, having responsibility without authority, or being subject to someone else's authority without responsibility), and a lot of people have been on both, and once you bring it to their attention that they're creating another one there is a certain amount of motivation to avoid it, even if it is a bit more work for a them. You can actually use people's experiences with bad organization to convince them to not create another one, if you are careful and thoughtful.
You never want code with shared ownership to tolerate feature creep. That’s impossible to keep on the rails. If you’re going to use SOLID anywhere, it’s in shared code.
If your org doesn’t suffer feature creep willingly, I believe that means you can have a lot more flexibility with respect to Conway’s Law. A low-churn project can maintain a common understanding. Not all of them will of course, but it’s at least possible. Feature factories absolutely cannot, and you shouldn’t even try.
What really kills a lot of projects though is an inverted dependency tree. And by inverted I mean that the most volatile code is at the bottom of the call graph, not the top. In this case every build, every deploy, or in the microservices scenario every request, can have different behavior from the previous one because some tertiary dependency changed under you. Now you need all sorts of triggers in your CI/CD pipeline to guarantee that integration tests are rerun constantly to figure out where the regressions are being introduced. Your build pipeline starts to look like one of those server rooms with loose wires everywhere and a bucket under the AC unit. Nothing is stable and everything is on the verge of being on fire at a moment’s notice.
I mentioned this elsewhere, but I think it's a good idea to read the actual text of conways law:
> Any organization that designs a system (defined broadly) will produce _a design whose structure is a copy of the organization's communication structure_.
Organization is the primary method by which we organize communication (indeed, that's its only point). That's why architecture follows organization. It is possible to "break" that boundary by organizing other forms of communication, for example the feature teams of SAFE (not a proponent of safe, just an example).
This is the "reverse Conway's law" that some others are referring to, and I alluded to. I've had success doing it in small places here and there. I'm yet to convince a management organization to do a really large-scale reorg because of some major pathology, though. Though I haven't tried often, either. Really just once. The costs are automatically so high for such a proposal that it is difficult for mere engineering needs to get to net-positive because it's hard to prove the benefits.
The kind of reorg we're talking about here is not the typical musical chairs, but a full realignment. This happens a lot less frequently in my experience.
> My take: do Microservices all you want, but don't organize teams around the services!
Exactly. In order to reap the benefits of modular/Microservices architecture, you need teams to be organized around product/feature verticals.
IMO it’s not about the internals of a product/service architecture, but about encapsulating, isolating, and formalizing inter-organizational dependencies.
Or hardware. I worked on an HPC product which used "microservices" of a kind but many of them were tied to specific hardware nodes. So much of what we think of as "microservices" relies on a fairly narrow set of assumptions, and mostly makes sense in specific contexts, i.e. cloud, APIs, etc.
FWIW, most of Google's Ads services share a database - the catch is it's F1 on top of Spanner. Latency is highish, but throughput is good.
In the outside world, for an application that may truly need to scale, I'd go MySQL -> Vitess before I'd choose separate data stores for each service with state. But I'd also question if the need to scale that much really exists; you can go a long way even with data heavy applications with a combination of sharding and multitenancy.
>> because they keep people honest around module boundaries
Imagine a world where every pip/nuget/cargo package was a k8s service called out-of-process and needed to be independently maintained. We would have potentially hundreds of these things running, independently secured via mTLS, observability and metrics for each, and all calls run out of process. This is the abysmally slow hellscape that some are naively advocating for without realizing it.
It is relatively obvious what should be a library (json, numpy, NLog, etc), and what should be a service (Postgres, Kafka, Redis, NATS, etc.) when dealing with 3rd party components. It is also obvious that team scaling is not determined by whether code exists in a library or service from this example since all 3rd party code is maintained by others.
However, once we are all in the same organization, we lose the ability to correctly make this determination. We create a service when a library would do in order to more strictly enforce ownership. I think an alternate solution to this problem is needed.
But of course team scaling is determined this way. Merely releasing a library version has zero business impact. To have impact, library teams must go around to every consumer team and beg them to accept a version bump. This is a) shitty work for which it is impossible to retain people, and b) intractable without an enormous labor force once the number of consumers is big enough.
Services can be continuously deployed by small owning teams.
I'm not sure at what point the monolith codebase became absolutely enormous, but I would bet that GitHub grew to 1M active users with a monolith just fine.
Micro-services might be necessary for the companies like Github, Twitter, Google that grow to be huge, but monoliths seem to work just fine for the vast majority of other companies.
> most companies will never experience the level of scale of 2013 Twitter
I fully agree with your argument. Then again, as mentioned elsewhere in this discussion, microservices are often not introduced to solve a scalability problem but an organizational one and there are many organizations that have more engineers than Twitter (had).
Personally, I still don't buy that argument because by solving one organizational problem one risks creating a different one, as this blog post[0] illustrates:
> […] Uber has grown to around 2,200 critical microservices
Unsurprisingly, that same post notes that
> […] in recent years people have begun to decry microservices for their tendency to greatly increase complexity, sometimes making even trivial features difficult to build. […] we experienced these tradeoffs first hand
I'm getting very strong Wingman/Galactus[1] vibes here.
This happens a lot. Organisational problems conflated for technical ones. Technical solutions to organisational politics. Etc. It's often easier to admit technical challenges than organisational ones. Also, different people get to declare technical and organisational challenges, and different people that get to design the solutions.
There's also a dynamic where concepts are created by the technical 1% doing vanguard or challenging tasks. The guys responsible for scaling youtube or whatnot. Their methods and ideas become famous and are then applied to less technically demanding tasks, then in less technical organisations entirely.
I think if we can be honest at the actual problem at hand, 80/20 fixes will emerge. IE, the "real" value is not the architecture per se, but the way it lets you divide the responsibilities in the organisation.
Services generally have a stateless request/response architecture, inherited from the success of HTTP, and contra the experiences of CORBA and COM, the latter being much better suited to Office Object Linking and Embedding and VB visual controls - local, assumed reliable, not across a network boundary.
Creating an object-oriented API for a shared library which encapsulates a module to the degree a service boundary would is not trivial and it's very rarely done, never mind done well. Most OO libraries expose an entire universe of objects and methods to support deep integration scenarios. Maintaining that richness of API over time in the face of many consumers is not easy, and it (versioning) is a dying art in the eternal present of online services. The 90s aren't coming back any time soon.
If you own a library which is used internally, and you add a feature which needs a lot more memory or compute, how do you communicate the need to increase resource allocation to the teams who use the library? How do you even ensure that they upgrade? How do you gather metrics on how your library is used in practice? How do you discover and collect metrics around failure modes at the module boundary level? How do you gather logs emitted by your library, in all the places it's used? What if you add dependencies on other services, which need configuring (network addresses, credentials, whatever) - do you offload the configuration effort on to each of the library users, who need to do it separately, and end up with configuration drift over time?
I don't think binary drops work well unless the logic they encapsulate is architecturally self-contained and predictable; no network access, no database access, no unexpected changes in CPU or memory requirements from version to version.
There's plenty of code like this, but it's not usually the level of module that we consider putting inside a service.
For example, an Excel spreadsheet parser might be a library. But the module which takes Excel files uploaded by the user and streams a subset of the contents into the database is probably better off as a service than a library, so that it can be isolated (security risks), can crash safely without taking everything down, can retry, can create nice logs about hard to parse files, can have resource metrics measured and growth estimated over time, and so on.
People keep regurgitating this “you aint going to be google” mantra but I worked there and in reality generic microservice stack is in a totally different league of complexity and sophistication of what google and co have. This argument is basically reductio ad absurdum
The argument is simply the empirical observation that the vast majority of microservices deployments don't operate anywhere near the scale that actually requires microservices and is not going to operate at that scale in the foreseeable future. When scalability is the primary argument in favor of microservices, how is that absurd?
You'll need to support loading two versions of each DLL into memory and invoking both on some percentage of servers to be able to replicate a microservice though.
The important part of microservices isn't just API boundaries, it's lifecycle management. This CAN be done with a DLL or JAR, but it's MUCH harder today.
I don't know about you, but I could deploy multiple versions of my Java webapp to Apache Tomcat over 15 years ago... and gracefully sunset the old version.
Working at a huge company I feel like every service owning its own data store (or at least only accessing/manipulating another’s through its API, not directly in the underlying store) is the only sensible way to do it, because you can never really envision what clients you will have ahead of time or what they will want to do, and that’s an easy recipe for headaches. But the smaller the company gets the further I’d go along the spectrum from full isolation to isolated services with shared stores to monolith.
> A better rule is for one service to own writes for a table
This breaks down when the database is essentially a generic graph. The worst solution I've seen to this is to have another service responsible for generic write operations and any service that wants to write data goes through that service -- you're essentially re-introducing the problem you're purporting to solve at a new layer with an added hop and most likely high network usage. The best solution I've seen is to obviously have the monolith. The enterprise solution I've seen, while not good by any means but nearly essential for promoting team breakdown and ownership, is to just let disparate services write as they see fit, supported by abstraction libraries and SOPs to help reinforce care from engineers.
> I'm not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables.
I’ll go one step further and say that you should treat your data stores as services in and of themselves, with their own well-considered interfaces, and perhaps something like PostgREST as an adapter to the frontend if you don’t really need sophisticated service layers. The read/write pattern you recommend is a great fit for this and can be scaled horizontally with read replicas.
Modules aren't an alternative to microservices in a reasonable way though. And for all modules solve the modularization problem at the code level, they don't really solve modularization at the service level. The main alternative to microservices is monoliths and for many applications I far prefer microservices to monoliths. I want modularization in how I scale my app, I don't want to spin up a whole bunch of servers that can serve almost any possible function in production. I'd prefer more delineated responsibilities. I don't think modularization solves this if after the modules you just throw all your modules in one big bucket and serve that up.
I think what is missing here is that it doesn't have to be an either-or at the organizational level.
If you have a particular piece of the system that needs to be scaled, you can take that module out when it becomes necessary. You can alter your system such that you deploy the entire codebase but certain APIs are routed to certain boxes, and you can use batch processing patterns with a set of code outside the codebase.
You can have an admin application and a user application, and both of them are monoliths. They may or may not communication using the database or events or APIs.
However, you don't make this on the single bounded context guideline.
Just because the monolith image contains all code doesnt mean each deployment needs to run all that code. Or even same version of the code. A deployment can run only a small portion of the code determined by deployment args. It can even run an older version than other deployments
> But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them
At my last job, there were quite a few times where being able to scale some small "microservice instance" up from 2 -> 4 instances or 4 -> 8 or 8 -> 12 was a lot easier/quicker than investigating the actual issue. It'd stop production outages/hiccups. It was basically encouraged.
Not sure how that can be done with a "it's all modules in a giant monolith".
A few ways, the easiest being to scale up the whole monolith with more instance. Another way is run multiple "services" using the same codebase, so you have workload segmentation, either via synchronous network calls, or async worker systems.
> A few ways, the easiest being to scale up the whole monolith with more instance
as far as I know, there's no way to granularly scale up a monolith. if the monolith has 20 or 50 or 100 modules and you need 1 or 2 of them scaled, you have to scale up the entire thing which is huge, expensive, etc.
> Another way is run multiple "services" using the same codebase, so you have workload segmentation, either via synchronous network calls, or async worker systems.
this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
> you have to scale up the entire thing which is huge, expensive, etc.
Yes, yet people still do it that way. This is tradeoff against the costs of microservices. I'm not saying it is worth it, but yes, sometimes you can just inefficiently throw hardware resources at scaling problems.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
Because you are incrementally changing an existing monolith, but still getting some of the benefits of scaling distinct workloads independently. If you do this right, it does lend it self to reworking the new "service" to be become its own microservice. Or you can just keep going with the shared codebase.
yes, it's very common design. sometimes called a "distirbuted monolith." it's not microservices because the number of additional services is usually small, and the codebase is still tightly coupled to itself (even if well factored in terms of modules). i.e., everything is still tested together, and there's still a single monolithic build and deployment process, and no team can go off and decide to write a service in a different language.
so for example in the context of Java, you have one main App/main() entry, probably bounding to a port (let's say 80 serving HTTP requests) acting as a router
it brokerages them to subclasses/handlers for CRUD/RPC/plumbing to database/other services
then in a separate process App/main() you have a cron job or a queue of some sort
you bundle these together as a monolith into one main()?
In the case of Java, imagine we have a big Maven project that has a bunch of different well factored modules that form the "libraries" of our application. At first, in our Maven project we also have a single module which is the deployable for "the monolith" -- it consumes all the other "lib" modules and gets packaged into a fat jar.
We deploy a few instances of the monolith fronted by a load balancer. There's not much scaling logic here -- when we see p75 request latency reach a certain threshold, we add another monolith instance.
A while later, we realize that the /foo route is getting a ton of spikey traffic, which makes it hard to scale. So, in the load balancer, we set up a new pool that just targets /foo, and in our Maven project we create a new module that will be the deployable for this pool. Now, /foo still requires most of our libs, so the jar we're bundling is still "the monolith" in essence, but maybe it doesn't require a few dependencies, and so is a little slimmer. We deploy that and everything is scaling great!
Then, we discover that in our main monolith, the background threads we were using to send emails are getting really clogged up, which is adversely effecting performance. We decide we want to move this work to an external persistent queue, like Kafka. Now, for our consumer, we create one more Maven module for a new jar. This time, we're lucky. Emails only need 2 of our libraries and so this is actually a pretty small deployable, but it's still being built from the same core set of libraries as the monolith.
so you end up with a monolith that might have a config flag or something to determine its purpose when it comes online, and then it can go down a few different code paths. pretty cool. is this something that is done on the fly and then later you don't let that code pattern stay alive (aka, is it not a smell/bandaid)?
my concern would be if you accidentally create bugs by basically creating new mini-monolith flavors as the modules were never expected (in the beginning of their creation) to run in a "partial" context
I was part of a project that did exactly this - same monolithic code base with config flags that transformed it into web server, worker or scheduler. It allows scaling all parts separately, but I can confirm it's easy to introduce bugs if you're not careful. Since you're sharing the same DB, migrations also need to be backwards-compatible.
A cool side-effect is that you can usually run the whole thing in one app for development by just enabling all profiles - as opposed to some microservice architectures where you need dozens of containers and DBs for replicating inter-service bugs.
The way it usually works, if 1 of your 100 modules needs to be scaled, it probably means the overall monolith only needs a small amount of extra resources, since that module is only using 1% of the total. So it's not like you need to double the instances of the whole thing.
The benefit though is you get a bit of 'free' scaling. Most of the modules don't even have to consider scaling at all, as long as they are growing ~average or slower. So this saves a lot of operational burden for many teams. Conversely with microservices every team has to consider scaling, no matter how simple their service is.
(If you do have 1 module out of 100 that takes up 40% of the resources, then it may be appropriate to split it up. Monolith doesn't literally mean "exactly one service", after all, just a small number of big ones.)
> The way it usually works, if 1 of your 100 modules needs to be scaled, it probably means the overall monolith only needs a small amount of extra resources, since that module is only using 1% of the total. So it's not like you need to double the instances of the whole thing.
I could be wrong but doesn't monolith usually refer to one really heavy app? As soon as you go from needing 1 instance to 2 (because 1 of the 100 inner "modules" needs to be scaled), I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith? Unless you can scale threads or something instead of the entire process... Or if the entire process doesn't initiate 100 modules as soon as it is started (although I imagine it would in most cases)
You do lose some memory to holding unnecessary code and whatever data caching that a module does. But it's not generally very much and large data caching can be pushed to Redis. If you're not buying your servers by the rack or data center it's ultimately negligible compared to development cost.
>doesn't monolith usually refer to one really heavy app?
No. Backend Server clusters are around almost as long as the internet exists.
>I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith?
That's right, but there's a gotcha. Everytime you split your monolith into parts, that doesn't mean that each part will consume 1/n of original monolith startup resource. There will be a floor of memory usage.
Consider 1 monolith that eats up 1GB to startup vs 4 microservices that uses 512MB each. Right from start you doubled the wasted memory from 1GB to 2GB. That only gets worse the more services are created.
Another problem is that microservice/cloud folks loves to create anemic machines. Usually setting up 512MB RAM and half a core cpu to a service that needs 380mb minimum :) Thats 75% overhead and 25% of working memory. It's bonkers.
> as far as I know, there's no way to granularly scale up a monolith. if the monolith has 20 or 50 or 100 modules and you need 1 or 2 of them scaled, you have to scale up the entire thing which is huge, expensive, etc.
not necessarily. yes, you do pay for executable bloat and some memory overhead from loading code into memory, but the argument here is that you can still deploy a "Service B" from the same deployable that only services certain kinds of requests in order to scale parts of the application horizontally.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
no, because again, you're still deploying the same single codebase, it's just the instance on the other end of the queue is configured just to consume and process messages.
I think it runs a bit deeper than that. Microservices are how the business views its self.
Look, once upon a time managers designed the system that workers implemented. The factory assembly line, or the policy manual.
But now the workers are CPUs and servers. The managers designing the system the workers follow are coders. I say coders are the new managers.
Now this leaves two problems. The first is that there are a lot of "managers" who are now doing the wrong job. (and explains perfectly why Steve Jobs gets involved in emails about hiring coders.) But that's a different post.
For me the second problem is microservices represent the atomic parts of a business. They should not be seen as a technical solution to anything, and because the first problem (managers arent managing workers anymore) there is no need to have the microservices built along Conways Law.
And so if you want to build a microservice it is not enough to consider the technical needs, it is not enough to automate what was there, you need to design an atomic building block of any / your company. They become the way you talk about the company, the way you think about the processes in a company.
An mostly the company becomes programmable. It is also highly likely the company becomes built mostly of interchangable parts.
> there is no need to have the microservices built along Conways Law
This is a misunderstanding of Conway's Law. Your code _will_ reflect the structure of your organization. If you use microservices so will their architecture. If you use modules, so will the modules.
If you want a specific architecture, have your organization reflect the modules/microservices defined in that architecture.
Quite. And many people still seem to confuse this. The way to exploit Conway's law is not to do anything _technical_ but rather to do something _organisational_. The "Reverse Conway Manoeuvre" is the term, IIRC, for this. Rather than trying to "build something along the lines of Conway's law" (which doesn't really make sense) -- one should determine a desired architecture and then set up the _organisation_ and teams to mirror that.
But, I am trying to feel my way towards the idea that that was true when managers arranged systems for workers to follow, and then we came along to automate the current process. But if we have a system where the workers are the CPUs, then the people doing the managing are the coders.
The point ebing is that if workers are CPUs and coders are managers, then why worry about how the managers of the coders are arranged. Get rid of that management layer. Conway is not implying the financiers of the organisation affect the architecture.
This basically means that the microservices a company is built out of should more readily align to the realities of the business model. And instead of shuffling organisations around it would behove executives to look at the microservices.
One can more easily measure orders transferred, etc, if the boundaries are clearer.
Plus conway is just a ... neat idea, not a bound fate.
There is a caveat with the architecture reflecting the organisation of the software teams, but that usually follows the other to-be-automated structure as well.
Managers arranging work is _not_ why Conway's law is true though. I think it behooves one to look at the actual text of the "law":
> Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
It doesn't matter WHO defines or does the work (the coders themselves in a team or the manager). It matters how communication is arranged. And communication naturally aligns itself along organizational and team boundaries.
When a group/person/whatever writes an API they are MOSTLY writing a document for other human beings. That is an act of communication. If you meet with someone everyday in a stand-up you're likely to put a less structured API between yourselves than if you meet with someone once a month, or once a year. You're also going to create a finer grained subdivision of the API for them as you're likely working on closely related posts of the problem.
Organization creates communication, and communication creates architecture. Therefore organization creates architecture.
> Plus conway is just a ... neat idea, not a bound fate
It is in fact bound fate. Code is written to communicate with others.
People come together, they decide to solve a problem. They divide up that problem and create functions (an API), objects (an API), modules (an API), and services/microservices (an API). At each later they've written something that communicates with others. The structure of that code reflects the necessary communication to the other parties for their part of the problem. This the code is structured following the lines of communication in the organization.
Organization is how we structure communication (that's it's primary point) and thus organization = architecture + some wiggle room for communication paths defined outside that (say a SAFE feature team).
Just to add it here, how evaluations and goals are set matter too. Those tend to be even more constrained by the formal organization than communication.
Very good point. Tying that back into the Conway's law angle on all of this...
I've been a couple levels up the chain at times in my career. When I've seen technical decisions at this level done well is when those people realize that their organizational decisions are inevitably technical decisions because of Conway's law. When the impact of Conway's law is not explicitly considered, things will often look random at the IC level. Too often, organizational leaders don't realize they are dictating technical outcomes with decisions that appear to just be about reporting structure.
An example is when a major new product feature comes along that is big enough to need multiple teams to build/own it for the long term. Where are those teams placed in the organization? Conway's law helps us see that a huge part of the future architecture is being determined by answering that question. Placing those new teams in an existing org mainly responsible for UX will result in a very different architecture than putting the new teams in an org mainly responsible for monetization. Can the teams be split across those orgs? Well, can you envision a reasonable architecture with an enforced divide along those lines? Without recognition of Conway's law, that last question might not be considered and the decision to split or not will be based on other factors.
Unfortunately that requires multidisciplinary thinking (technical architecture, resource allocation, communication structure, people development, ...). Such broad thinking is not always easy. When leadership starts neglecting one or more of these disciplines, more and more of the people doing the work on affected teams start perceiving the decisions as arbitrary and counter productive. Organizational churn (e.g., teams repeatedly changing location within the org over the course of a year or two) is often leadership looking at a different subset of these dimensions over time, yielding different "correct" answers.
I think microservices turn this around - the idea is that if we ignore them as technical units and treat them as organisational units we can invert conway's law - it's much easier to change how microservices talk to each other than it is to chnage how people do.
But if you chnage the dataflow in a few microservices so that the accounts team no longer deals / works directly with the sales team you have chnaged the organisation.
I just want to point out that for the second problem (scalability of CPU/memory/io), microservices almost always make things worse. Making an RPC necessarily implies serialization and deserialization of data, and nearly always also means sending data over a socket. Plus the fact that most services have some constant overhead of the footprint to run the RPC code and other things (healthchecking, stats collection, etc.) that is typically bundled into each service. And even if the extra CPU/memory isn't a big deal for you, doing RPCs is going to add latency, and if you get too many microservices the latency numbers can start really adding up and be very difficult to fix later.
Running code in a single process is MUCH lower overhead because you don't need to transit a network layer and you're generally just passing pointers to data around rather than serializing/deserializing it.
There are definitely some cases where using microservices does make things more CPU/memory efficient, but it's much rarer than people think. An example where you'd actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat.
I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.
Microservices are less efficient, but are still more scalable.
Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.
But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.
It's also way easier to design for. The web is an embarrassingly parallel problem once you remove the state. That's a big reason why you offload state to databases - they've done the hardest bits for you.
Running a single server often can’t meet availability expectations that users have. This is orthogonal to scalability. You almost always need multiple copies.
Yes but it’s literally a single point of failure. You probably want at least two servers in separate physical locations. Also how do you do deployments without interruption of service on a single server?
> That's a budget for a local crafts store website hosting, not "high availability" system
I'm not sure about that: you could still put something together within that budget, say, a few different VPSes across Hetzner and Contabo (or different regions within the same provider's offerings), with some circuit breaking and load balancing between those. Probably either a managed database offering, or a cluster of DBs running on similar VPSes.
Of course, this might mean that you have 1-3 instances of a service instead of 10-30, but as long as availability is the goal and not necessarily throughput, that can go pretty far.
> If you're a amateur something, that just does it for fun - sure
sed "s/amateur/comparatively poor, from a third world country, without VC money, or have cheap labor/g"
Not everyone can afford advanced tools or platforms, or even using something like AWS/Azure/GCP. Some of those can indeed be amateur use cases (e.g. side project or bootstrapped SaaS), others simply stretching your money for any number of considerations (e.g. non profit, limited budget etc.), but it's definitely possible. In some countries it probably makes more sense to just build your own solution, as long as you're not doing anything too advanced.
500 USD a month would get you approximately the following resources (taxes vary) on the aforementioned platforms:
Contabo
Nodes: 15 to 83 (depending on configuration)
CPU: 150 to 332 cores
RAM: 664 to 900 GB
SSD: 4150 to 6000 GB
Hetzner
Nodes: 7 to 110 (depending on configuration)
CPU: 86 to 192 cores
RAM: 192 to 384 GB
SSD: 2200 to 4800 GB
(this includes regular VPS packages, not storage optimized ones, or dedicated hosting etc.)
I'm not sure about you, but in my experience that could be enough for some pretty decent systems, albeit some storage heavy workloads would need the storage packages instead of the regular VPS ones. It's mostly a matter of picking a suitable topology and working towards your goals.
> Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
This (depending on the circumstances) does sound like a good point! Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
>> Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
> Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
This is the point - for-profit corporations, by definition, don't want to waste money, and if "high reliability" isn't required (or they don't know about it), they don't waste money. Most of the time, they don't waste the money.
However, if "being cheap" would means billions in lost income (and they know about it), they really want to have reliable, redundant infrastructure and systems around.
I could probably design and deploy an HA system for way less. Maybe less than $200/month. It wouldn't be the most performant, but would be HA in three regions.
But it leads me back to my original statement - extreme requirements for uptime don't come out of nothing.
If you're in a location where IT related labor is extremely cheap - you're just going to have people keep one server up.
I know I used to do exactly that, because the server was more than my annual income. But that didn't last long. After the first 20 minute downtime, the budget for HA solution was allocated. But before a certain point downtime wasn't expensive.
Non-profits would probably be the only reasonable exception, where HA and low budgets could coincide. Otherwise - nah...
Those are all fair points, perhaps even more so given the trend of compute and other resources generally becoming more cheap with time (things like Wirth's law and limited IPv6 support aside), thanks for expanding on your arguments!
well, to be fair, you could host a highly available local craft store website for under 500 dollars on $cloudprovider easily. You could also trivially get regional redundancy.
Go to Digitalocean or a similar provider. Launch a managed HA db. (starts at ~$120/month). Launch an autoscaling HA K8 cluster (starts from ~$80) , deploy 1-2 stateless pods. There you go.
You know, I don't really think microservices are fundamentally more scalable. Rather, they expose scaling issues more readily.
When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems."
When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?"
That allows you to better prioritize fixing those scaling problems.
That’s a really interesting point - something that could probably be addressed by module-level logging and metrics. That said, even as a pro-monolith advocate, I can see why it’s preferable to not allow any one module/service to consume all the resources for your service in the first place. The service boundaries in microservice architectures can help enforce resource limits that otherwise go unchecked in a larger application.
It's one I've ran into a few times in my company (which has a large number of these types of endpoints).
The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.
Sure depends on the architecture.
When the auth service is down, everything else should be down.
But when the "optional feature" service is down, a core component should be unaffected by that.
Split up services where it makes sense and don't over do it.
That's how I design my projects.
When the auth service is down, only new authentications should fail. Existing auth sessions should continue to function just fine. This exact failure mode occured at Google in early 2021 which caused a fairly big outage but not as big as it could have been because of this design choice.
How so? If functionality A is critical to functionality B, how will wrapping it in an HTTP call (microservices) reduce the damage from breaking functionality A?
I can see an advantage regarding resource hogging, but the flip side is the extra point of failure of network calls in microservices.
Not saying which is better, but deployment is orthogonal to logical dependence and correctness.
Most features in a modern app are not critical functionality, though.
For instance, in a shopping site, why should a crash in the recommendations engine result in a non-functional webpage (rather than a working purchase page with no recommendations)?
Personally I think microservices start to make sense when you have several hundred developers (an environment I'm currently keen to never enter again - $work has 5 devs and might one day have 6).
That makes sense. I am still trying to understand why does it differ between a monolith and microservices. The app in the monolith can make the call to non-critical functionalities time-limited and fault-tolerant, just like a network call has a time-out and can return nothing (in a simplified manner, it can wrap that call with a timer and an exception handler).
I agree that microservices are suitable for large organization, where the organization practically has multiple products (which could be purchased from a vendor or sold to another company).
If your monolithic service OOMs, hits a large GC pause causing dependent requests to time out, locks a shared file descriptor, or a bunch of other things then the monolithic service as a whole can hit a fault or stall even if other threads/tasks are still executing. While classes of errors like OOMs go away when multiple processes are executing.
I don't think it was well expressed, but to reuse my last example: OOM-killer ending the recommendations process mid-request is less of a big deal if the main store server can keep running and serving traffic.
If the recommendations team write code that causes the OOM-killer to end their process, making them run it on separate infrastructure insulates your "main store team" from the bugs they write.
It was about the OOM killer as the sibling comment says, yeah. I'm surprised you're so incredulous. OOM Killer and GC stalls are some things I've run up against in my career frequently. I'm sorry my comment didn't live up to your expectations, it was hastily typed on mobile.
His point was that the comment was unclear if you'd also read it hastily :-)
I imagine his logic was something like: "How can OOMs happen less often if you run more processes (possibly on the same machine)?", while your comment actually wants to say: "if a specific service is affected by an OOM, with microservices only that specific microservice goes down, since it's probably running on its own hardware".
A monolith can also scale vertically with mechanisms to redeploy on fatal errors. If all starts failing, you may have a problem. But you can get the same problems with a microservices that is in the critical path
Networks could have unexpected delays, routing errors and other glitches. At least with a monolith you can often find a stacktrace for debugging. I have seen startups that have limited traceability and logging when using micro services.
When a small startup has to manage "scalable" K8s infrastructure in the cloud, distributed tracing and monitoring is often not prioritized when you are a team of 5 developers trying to find a product market fit.
I am not against microservices (I work with them daily) but you just trade one type of stability problem with another
Right I'm not advocating for one over the other, I was just explaining issues solved by microservices. Now instead of the OOM Killer taking your service down, you have a flaky NIC on another microservice box and now you need to figure out how to gracefully degrade.
I love working with microservices at the scale of $WORK, but we're Big Tech. I can't imagine why a 5 person startup would want k8s and microservices. You don't need that scale until you have more than 2 teams, and you're pushing at the very least 15 engineers at that point and usually the sales and marketing staff to make that investment worth it.
Resource hogging is a huge class of errors, though. Everything from a bad client update DDoSing a feature, file handles, memory leaks, log storage (a little outdated now perhaps), and so many more...
It sounds like creating problem, then spending time=money on fixing it and calling it a win?
There is a point when it all starts to make sense. But that point is when you go into billions worth business, hundreds of devs etc. And going there has large cost, especially for small/medium systems. And that cost is not one off - it's a day-to-day cost of introducing changes. It's orders of magnitude chaper and faster (head cound wise) to do changes in ie. single versioned monorepo where everything is deployed at once, as single working, tested, migrated version than doing progressive releases for each piece keeping it all backward compatible at micro level. Again - it does make sense at scale (hundreds of devs kind of scale), but saying your 5 devs team moves faster because they can work on 120 micoservices independently is complete nonsense.
In other words micoservices make sense when you don't really have other options, you have to do it, it's not good start-with default at all; and frankly Sam Newman says it in "Building Microservices" and so do people who know what they're talking about. For some reason juniors want to start there and look at anything non-microservice as legacy.
> It sounds like creating problem, then spending time=money on fixing it and calling it a win?
It sort of is.
It's not a perfect world. One issue with monoliths is that, organizations like to take a "if it ain't broke, don't fix it" attitude towards things. Unfortunately, that leads to spotty service and sometimes expensive deployments. Those aren't always seen as being "broken" but just temporary problems that if you pull enough all nighters you can get through regularly.
It takes a skilled dev to be able to really sell a business on improving monoliths with rework/rewrites of old apis. Even if it saves money, time, all nighters. It's simply hard for a manager to see those improvements as being worth it over the status quo. Especially if the fact of running the monolith on big iron masks the resource usage/outages caused by those APIs.
> I don't really think microservices are fundamentally more scalable
It depends on what you are scaling. I think microservices are fundamentally more scalable for deployment, since changes can be rolled out only to the services that changed, rather than everywhere. Unless your language and runtime support hot-loading individual modules at runtime.
I'm talking about the scalability of actually delivering new code to servers (or "serverless" runtimes). Feature flags don't help with that.
Admittedly, it isn't a scalability problem you will run into right away.
But when you need to roll out an emergency fix, there is a big difference between deploying to thousands of servers that all have everything, and ten servers running a single service.
I disagree, in my opinion micro-services hinder scalability of deployment, and development - at least the way I see most businesses use them. Typically they break out their code into disparate repositories, so now instead of one deployment you have to run 70 different ci/cds pipelines to get 70 microservices deployed, and repo A has no idea that repo B made breaking changes to their API. Or lib B pulled in lib D that now pollutes the class-path of lib A, who has a dependency on lib B. Often you need to mass deploy all of your microservices to resolve a critical vulnerability (think log4shell)
The solution to this is to use the right tool, a build system that supports monorepos like Bazel. Bazel solves this problem wonderfully. It only builds / tests / containerizes / deploys (rules_k8s, rules_docker) what needs to be rebuilt, retested, recontainerized, and redeployed. Builds are much faster, developers have God like visibility to all of an organizations' code, and can easily grep the entire code base and be assured their changes do not break other modules if Bazel test //... passes. It is language agnostic so you can implement your services in whatever language best suits it. It allows you to more easily manage transitive dependencies, manage versions globally across your org's codebase.
Of course Bazel has a steep learning curve so it will be years before it is adopted as widely as Maven, Gradle etc. But in the banks I've worked at it would've saved them tens of millions of dollars.
Also git would need to catch up to large codebases. I think Meta released a new source control tool recently that is similar to git but could handle large monorepos.
Breaking APIs do not happen often at least in my projects.
We mostly add new Endpoints for new features (non breaking change).
We keep object definitions in a separate repo to not duplicate them and generate a maven dependence from that.
Pretty simple actually.
We mainly run on Java thought.
Would be more complicated when there are multiple languages.
Protobuf or Json schema could come to the rescue if needed.
Man, I wish my colleagues would read your comment (and at least question their believes for one brief moment)…
> I disagree, in my opinion micro-services hinder scalability of deployment
…and of anything related to testing:
- Want to fire up the application in your pipeline to run E2E tests? Congratulations, you must now spin up the entire landscape of microservices in k8s. First, however, you need to figure out which versions of all those microservices you want to test against in the first place, since every service is living in a separate repository and thus getting versioned separately.
- Want to provide test data to your application before running your tests? Well, you're looking at 100 stateful services – good luck with getting the state right everywhere.
If you have many app servers and they all run copies of the same app you can roll out new versions to a few servers at a time. You just have to handle the db update first but you need to do that with microservices anyway (they might use smaller databases and therefore making it somewhat easier).
> Microservices are less efficient, but are still more scalable.
Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.
> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices
200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :)
I believe that this level of service can no longer accurately be described as "micro".
Helps to have a language that natively uses more CPU cores and/or training for the devs.
Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.
This is exactly what I do. When it comes to your regular backend business server I write stateful multithreaded monolith in C++ sitting on the same computer as the database hosted on some multicore server with gobbles of RAM (those are cheap now). Performance is insane and is enough to serve any normal business for years or decades to come.
So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
>So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.
the middle management especially half cooked engineers who drank the cool aid and became managers are hard to reason with.
They want to be both the architect and the manager and anything you say would be over ruled and since they are the boss its hard to ignore them.
This service is a monolith because it has 10K code and it needs to be broken up.The product is at MVP and its rock solid on Java Spring and it hardly crashes.
We are never going to lose data based on the design choices we made.
None of that matters.
We need zero down time upgrade, when we had zero customers.
Even for non-FAANG, less-than-a-million-user business applications, there are two problems:
1. Your feature/function scope is not all fully defined at the start and is not static till the end of life. Software has to evolve with business. In this case, it is easier to build a loosely coupled shared nothing architecture that can scale easily than to build a shared-everything-all-in-one-binary monolith architecture.
2. Your customer base isn't one size fits all. You usually different growing businesses that need solution at different scale points but still with very high unit economics. This means you need a incremental scaling solution – this is where old-school big-chassis systems build blade scalable server architectures. But because of custom/proprietary backplane design they become unmanageably complex and buggy.
Instead, if you build an architecture that can scale the number of corporate users by adding cheap $2k pizza box 1u servers as the company grows, that's much more attractive. Also, you can keep your systems design flexible enough to recompile and take advantage of advancements in hardware tech every 18 months – this gives you better operating margins as your own business starts to grow.
This cannot be emphasized enough. The top of the line configuration you can get today is a dual EPYC 9654. That nets you 384 logical cores, up to 12 TB of RAM, and enough PCIe lanes to install ~40 NVMe drives, which yields 320 TB of storage if we assume 8 TB per drive. All for a price that is comparable to a FAANG engineer's salary.
And you're missing one little thing - if you get all of your processing on two of those, you save way more than 4 engineers salaries in development/maintenance costs.
Let alone - AWS Lets you get that machine for less than a junior engineer salary ($7.5 per hour, roughly equivalent to $32 of hourly wage = $67k)
How about 2 erlang OTP nodes (homogenous)? I don't have a real world use case of that load but I often imagine I would have 1:2 RAM ratio to be able to vertical scale each time. For example, start with 1TB(A):2TB(B), if that's not enough, scale A to 4TB. When load climbs start to exceeds, scale B to 8TB .. so on alternately.
Yeah, it's bin packing, not straight efficiency. Also, people seem to exaggerate latency for RPC calls. I often get the feeling that people who make these latency criticisms have been burned by some nominal "microservices" architecture in which an API call is made in a hot loop or something.
Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.
Couple of calls is normal. But if you make everything a microservice. And there are dependencies between them, then by design its destined that some hotter loop eventually will contain rpc.
Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone.
Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.
I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith.
Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.
>A load balancer only works together with servers running the same code.
Uh - what?
>A load balancer balances traffic between servers
Correct.
> of the same service or mononlith
Incorrect.
Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base.
But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts.
The load balancer internally makes these decisions.
Each set of upstream hosts in nginx is a single instance of load balancing. You aren't load balancing across services, you're splitting traffic by service and then load balancing across instances of that service.
The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.
I don't really think of route based "load balancing" as load balancing. That's routing, or a reverse proxy. Not load balancing. Load balancing is a very specific type of reverse proxy.
The point is, if a client makes a request to a server, the response should always be the same, no matter where the load balancer sends the request to. Which means it should run the same code.
Nginx doesn't even mention route based or endpoint based load balancing in their docs. Maybe they don't consider it load balancing either.
Friend, you don’t know what you’re talking about and if linking NGINX documentation literally describing load balancing algorithms applied across Kubernetes pods hosting endpoints doesn’t clear things up for you, I don’t think anything will.
> Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.
I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself).
Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity.
Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices.
Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong.
However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere):
SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them
MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging
MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing
Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to...
I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.
I was working at Amazon when they started transitioning from monolith to microservices, and the big win there was locality of data and caching.
All of the catalog data was moved to a service which only served catalog data so its cache was optimized for catalog data and the load balancers in front of it could optimize across that cache with consistent hashing.
This was different from the front end web tier which used consistent hashing to pin customers to individual webservers.
For stuff like order history or customer data, those services sat in front of their respective databases and provided consistent hashing along with availability (providing a consistent write-through cache in front of the SQL databases used at the time).
I wouldn't call those areas where it makes things more efficient rare, but actually common, and it comes from letting your data dictate your microservices rather than letting your organiation dictate your microservices (although if teams own data, then they should line up).
Thank you for pointing out caching. I was going to reply, at some point, that architecture in a well-designed distributed system considers both locality of data and the ability to cache data as a means of improving both latency and throughput and often does appreciably.
Most web sites I've seen in industry have just two servers, and only for active-active availability. Because of this, cache locality makes very little difference, because both nodes need to cache common data anyway.
Cache locality starts making a difference when you have over ten servers, and a significant fraction of the data is cacheable.
For "transactional" Enterprise CRUD apps, often there is very little that can be cached due to consistency concerns.
I don't see the affinity between microservices and locality. Any kind of distributed storage, including cache, is going to include a choice about how data are partitioned, ordered, and indexed. If you want certain records to be in memory on the same nodes, you should not need a separate cache cluster to accomplish that, let alone a separate cache service implementation.
I think Amazon (and Google and other FAANG entities) work at another scale than 99.9% of the rest of the world. The problems they face are different, including the impact of data locality.
I've seen many systems where there are a few hundred to a few thousand users, not hundred of millions or billions of users. There can also be teams of 5-10 developers who manages +20 microservices. I still don't think those projects have the same needs and could have done something else.
I started at Amazon in 2001 when we had around 400 servers running the website, it was large, but it was around the scale that most Fortune 500 companies operate at.
I'll agree that a team of developers with 20 microservices sounds obviously wrong (which is really my point that the site should be divided around data and the orgs should probably reflect the needs of the data--not the other way around). And Amazon back then didn't have that problem.
So tell me, who is making microservices for calls that could be done wholly in memory? How are you handling rollbacks and such for this in memory logic?
Aren't you still writing out to external data stores? If that's the case then its really not a comparison of in process to a RPC, it just two RPC hops now, no?
When you move to microservices -- or rather, when you split your data across several DBs -- you sometimes end up basically reimplementing DB synchronization in business logic layers that could (sometimes, and depending on scalability constraints) be solved by using DB transactions.
So instead of a single DB transaction, often with a single SQL roundtrip, microservices can sometimes force through coordination of separate transactions to several DBs.
I think all your points are correct but none of them address scalability. What do you do if your one efficient machine is too small for your problem? To be fair, not a lot of problems reach that point in practice, definitely not every problem a microservice is written for.
It might still be completely viable to rewrite something so that it's only 10% as efficient but you can scale it horizontally easily.
I do think using something like Flatbuffers or CapnProto where the SerDe is removed (aside from the travel over the network) could be a huge win. Another thing I have always wanted to try is to develop a system using microservices with a RPC layer that uses immutable objects so you can later combine the services and convert RPCs to function calls.
I really miss the old days where when an app was down it was obvious... Now apps just go unresponsive because of connectivity issues between services... We used to have frameworks that could communicate errors in one log, it was a beautiful thing. As cloud infrastructure became more powerful to better host so called "monolithic" systems, it was all scrapped to move to microservices, which were made for less powerful infrastructure. Everything was refactored to the point where it's expensive to go back... Pretty crazy how much money the industry wastes on refactoring...
On top of that, the tendency to get complacent with unstructured data in a lot of systems is really creating a very complicated lock-in when systems are developed for unique services on each cloud provider... Bowls of spaghetti.
Hire Solutions Architects for dev projects, make your apps future proof... I warn you. Too much microservice customization leads to vendor lock in and expensive operational costs... This is why a lot of apps get sunsetted early.
I very much agree with you on points 1 & 2. I'll add a 3rd - drying up repeated functionality such as repeated auth logic in completely separate applications.
I don't understand the argument around microservices purporting to fix scalability. Say your system has modules A, B and C in one app, and that A requires more resources while B and C are relatively unused. Won't B and C just run less and thereby use appropriately fewer resources from within the same system as A? Are microservices just an aesthetic separation (it feels nice to know you're "only" scaling up A)?
> A better rule is for one service to own writes for a table, and other services can only read that table,
Been there: how do you handle schema changes?
One of the advantages that having a separate schema per service provides is that services can communicate only via APIs, which decouples them allowing you to deploy them independently, which is at the heart of microservices (and continuous delivery).
The way I see it today: everyone complains about microservices, 12 factor apps, kubernetes, docker, etc., and I agree they are overengineering for small tools, services, etc., but if done right, they offer an agility that monoliths simply can't provide. And today it's really all about moving fast(er) than yesterday.
For us, we started off with a world where each service communicates to each other only via RabbitMQ, so all fully async. So theoretically each service should be able to be down for however it likes with no impact to anyone else, then it comes back up and starts processing messages off its queue and no one is the wiser.
Our data is mostly append-only, or if it's being changed, there is a theoretical final "correct" version of it that we should converge to. So to "get" data, you subscribe to messages about some state of things, and then each service is responsible for managing its own copy in its own db. This worked well enough until it didn't, and we had to start doing true-ups from time to time to keep things in sync, which was annoying, but not particularly problematic, as we design to assume everything is async and convergent.
The optimization (or compromise) we decided on, was that all of our services use the same db cluster, and that if the db cluster goes down, it means everything is down. Therefore, if we can assume the db is always up, even if a service is down, we consider it an acceptable constraint to provide a readonly view into other services db. Any writes are still sent async via MQ. This eliminates our syncing drifting problem, while still allowing for performant joins, which http apis are bad at and our system uses a lot of.
So then back to your original question, the way that this contract can break is via schema changes. So for us, since we use postgres, we created database views that we expose for reading. And postgres view updates are constrained that they must always be backwards compatible from a schema perspective. So then now our migration path is:
- service A has some table of data that you like to share
- write a migration to expose a view for service A
- write an update for service B to depend upon that view
- service B now needs some more data in that view
- write a db migration for service A that adds that missing data, but keeping the view fully backwards compatible
> So then back to your original question, the way that this contract can break is via schema changes. So for us, since we use postgres, we created database views that we expose for reading. And postgres view updates are constrained that they must always be backwards compatible from a schema perspective. So then now our migration path is:
> - service A has some table of data that you like to share
>
> - write a migration to expose a view for service A
>
> - write an update for service B to depend upon that view
>
> - service B now needs some more data in that view
>
> - write a db migration for service A that adds that missing data, but keeping the view fully backwards compatible
I don't think I understand. You need to update (and deploy) service B every time you perform a view update (from service A), although it's backward compatible?
if service B needs some new data from the view that isn't being provided, then you first run the migration on service A to update that view and add a column. Then you are able to update service B to utilize that column.
If you don't need the new column, then you don't need to do anything on service B, because you know that existing columns on the view won't get removed and their type won't change. You only need to make changes on service B when you want to take advantage of those additions to the view.
This only works if you apply backward compatible changes all the time. Sometimes you do want to make incompatible changes in your implementation. Database tables are an implementation detail, not an API which you're trying to expose as a view, etc.
But hey, every team and company has to find their strategy to do things. If this works for you, that's great!
I would never claim that our setup uses microservices. Probably just more plainly named "services".
And yes, that is correct, we agree that once we expose a view, we won't remove columns or change types of columns. Theoretically we could effectively deprecate a column by having it just return an empty value. Our use cases are such that changes to such views happen at an acceptable rate, and backwards incompatible changes also happen at an acceptable rate.
Our views are also often joins across multiple tables, or computed values, so even if it's often quite close to the underlying tables, they are intentionally to be used as an abstraction on top of them. The views are designed first from the perspective of, what form of data do others services need?
No, because that's not the service API. That's just a view over a table - an internal data structure used to represent some business domain model which should be properly exposed through some implementation-agnostic API. Service B should not care how service A implements it.
And the fact you must keep backward compatibility ( see the OP answer above) at the implementation level shows how fragile this approach is - you will never be able to change a database schema as you wish just because you have consumers that rely on the internal details of your implementation - a table. If you want to change a field from char to int, you can't. How is it important for service B to know that level of detail? An API could still expose a domain model as char, if you want to, and maybe introducing new fields, new methods, whatever way. Or maybe nothing at all, maybe it's not necessary because the database field is never exposed but only used internally (!!).
On the other hand, if you expose a database agnostic API (e.g., http, rpc, ... whatever) you can even swap the underlying database and nobody will notice.
A good rule of thumb is: if I change the implementation, do I need to ask/tell another team? If the answer is yes, that is not a microservice.
> a view would allow you to change a field's type.
Which proves my point: if I change a field type ( in the table ), I will have to change the view type. I need to change 2 services because one of them changed an implementation detail.
Yeah... The issue lies in cases where the decomposition is so extreme, that you end up not able to deploy independently.
And any benefit of a microservice owning it's own rDB is still, that schema changes aren't easily reversible. Specially when new, non predefined, data has been flowing in.
Stateless microservices are great, in the sense that you don't have to build multiple versions of APIs... but stateful microservices are just a PITA.
> There's two technical problems that microservices purport to solve
There's a third technical problem that microservices solve, and it's my favorite: isolation. With monoliths, you provide all of your secrets to the whole monolith, and a vulnerability in one module can access any secret available to any other module. Similarly, a bug that takes down one module takes down the whole process (and probably the whole app when you consider cascading failures). In most mainstream languages, every module (even the most unimportant, leaf node on the dependency tree) needs to be scoured for potential security or reliability issues because any module can bring the whole thing down.
This isn't solved at the language level in most mainstream languages. The Erlang family of languages generally address the reliability issue, but most languages punt on it altogether.
> The real reason that microservices may make sense is because they keep people honest around module boundaries.
Agreed. Microservices, like static type systems, are "rails". Most organizations have people who will take shortcuts in the name of expedience, and systems with rails disincentivize these shortcuts (importantly, they don't preclude them).
Another benefit of microservices is it allows you to have a permission boundary, which can restrict the scope of damage if a single server/service is compromised.
And for that matter it can (but not necessarily) limit how much damage a bad code release can do.
Of course you don't get those benefits for free just by using microservices, but impleminting those kind if boundaries in a monolith is a lot harder.
Really? How do you make it so that different modules in the same process have different IAM credentials and can't get the credentials for a different module? How do you make sure a buffer overflow in you analytics module doesn't allow an attacker to read memory from you login module? How do you make sure an RCE in your image upload code doesn't give an attacker access to credentials for your payment processor?
Maybe it is possible with some low-level system calls, but at the very least it is a lot more difficult than using separate VMs or containers.
Giving me an example that is created to work explicitly at system/container level isn't the "gotcha" you think it is.(IAM profiles have their own limitations)
Separate process vaults, HSMs and other techniques of offloading security credentials - are the same for microservice architecture, as "monolith".
The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
> Separate process vaults, HSMs and other techniques of offloading security credentials
How do give one module access to the vault/HSM without also giving any other code in the same process access? Even in the event of a security compromise. And that still doesn't address the problem of a vulnerability anywhere in the monolith potentially exposing any sensitive data in processes memory (such as user data including password).
> The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
Ok, replace "microservices" with "service oriented architecture". I'm comparing specifically against a monolith where you have an "uber executable doing literally everything".
> I'm comparing specifically against a monolith where you have an "uber executable doing literally everything".
I doubt that anyone here is advocating for a system that it literally everything in one OS process. I bet even you would say that a process with a RDMBS connection is a monolith, even though it doesn't fit your definition.
As for modules - they can run as a separate process on the same machine, to isolate security critical elements.
Conversely, I doubt that any reasonable microservices architecture has every instance of a microservice in its own subnet with a firewall and strict network access permissions... including one off generated API keys.
Most, at best, use a static API key per microsoervice inside one large "secure" network... which leads me back to my point - they're not exactly easier to secure. Methods may differ a little bit, but the techniques are the same.
Then your example of a buffer overflow is going to be as bad for microservices, as for monolith.
Unless you're going to invest in a variety of systems, programming languages and OSes in your stack - your one buffer overflow, turns into buffer overflow on every other microservice... making your claim of "easier to isolate" a little bit delusional. (Classic example of Log4J bug, where monolith or microservices - once you're breached "game over")
What it is is using the wrong solution for a problem. Vice-versa the number of devs who justify monolithic apps is absurd. The justification of microservices is just as problematic.
It is about striking a balance. No reason should something be overly compounded or overly broken up.
> scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).
I gave a talk [1] about scalability of Java systems on Kubernetes, and one of my recommendations is that Java-based systems - or any system on a runtime similar to the JVM, like CLR or even Go - should be scaled diagonally. Efficiency is the word.
While horizontal scaling can easily address most performance issues and load demand, in most cases it is the least efficient way for Java (and again, I risk saying .NET and Go), as these systems struggle with CPU throttling and garbage collectors.
In short, and exemplifying: one container with 2 CPUs and 2GB of RAM will allow the JVM to perform better, in general, than 2 containers with 1 CPU/1GB RAM each. That said, customers shouldn't be scaling horizontally to any amount more than what is adequately reasonable for resiliency, or unless the bottleneck is somewhere like disk access. For performance on the other hand, customers should be scaling vertically.
And Kubernetes VPA is already available. People just need to use it properly and smartly. If a Kubernetes admin believes a particular pod should double in number of replicas, the admin should consider: "would this microservice benefit even more from 1.5x more CPU/Memory than 2x more replicas?" and I bet to say that, in general, yes.
Basically any software, even something in C or C++ with manual memory management will scale more efficiently vertically than horizontally. Basically it would be better to run one twice as big as it would be to run two instances on the same machine. There is almost always some overhead or cache that will make the performance of fewer instances better. The only exception is resource contention but it is generally not too hard to fix the important cases.
My default recommendation has always been to make instances "as big as possible, but no bigger". You may need 3 instances for redundancy, fault tolerance and graceful updates but past that you should probably scale up to near the size of your machine. There are obviously lots of complications and exceptions (for example maybe using most 3 instances uses 90% of your machines so you can't bin pack other processes there so it is better to use 4 instances at 70% as the machines will be used more efficiently) but bigger by default is generally a good option.
One other thing they give you at a large organization is flexibility in your stack
If a team wants to try out a different language, or hosting model, or even just framework/tooling, those things can be really hard to do within the same codebase; much easier when your only contract is handling JSON requests. And if your whole organization is totally locked into a single stack, it's hard to keep evolving on some axes
(I'm generally against microservices, but this is one of the more compelling arguments I've heard for them, though it still wouldn't mean you need to eagerly break things up without a specific reason)
Tech zoo sometimes considered to be an anti-pattern in microservices. By introducing a different language into your organization, you decrease the mobility of developers between code bases and dilute technical knowledge.
Everybody shouldn't pull in their favorite stack just for fun, but it seems valuable to have the option of trying out new things that could turn out to be a better way forward for the org
Rewriting a module from one tech stack to another is not much harder when that module was a part of a monolith and not a separate service, except that you haven't paid the upfront cost of bootstrapping a new service, putting in RPC calls, etc. And in any case, starting a project as microservices is already a bad practice due to a number of reasons, the most important for me is that it's hard to change module boundaries, which you will most likely get wrong in a new project.
Forcing consistent language, runtime, tooling etc is generally a good thing as it reduces ongoing operational burden as well as some retention & hiring dilemmas.
Those human scaling problems are also security scaling problems; the Unix process model (and, in fact, the von Neumann architecture) doesn't separate modules in security capabilities, nor is it practical to do so.
Microservices allow your permissions to be clear and precise. Your database passwords are only loaded into the process that uses them. You can reason about "If there's an RCE in this service, here's what it can do". Trying to tie that to monoliths is hard and ugly.
>scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).
I would like someone to spell this out. It seems to me people are claiming that if a single binary serves some CPU-bound requests and some memory-bound requests, and you give it more memory, then the memory gets "wasted" on the CPU-bound part. Or if you give it more CPU, the CPU gets wasted on the memory-bound part. But this kind of assignment of resources to code paths seems to be a consequence of microservices. In a single computer, single binary situation resources should not get used up unless the workload actually wants to use them. A compute-heavy thread doesn't cost heap. A big heap doesn't slow down a compute-heavy thread. What am I missing?
I’d love for a language / framework that allows for an application to be composed of “modules” that can either be run in a single process, or deployed as multiple independently scalable processes, with a mostly transparent RPC system requiring minimal boilerplate.
My IDE should be able to easily traverse the call graph. My development environment should be simple to setup.
I’ve worked on microservices that required an insane amount of boilerplate to do simple things. Like 7 layers of controllers, clients, services, data services, etc, just to fetch a simple piece of data. And the developer experience of running dozens of services in a Kubernetes cluster running on my dev machine was awful.
Does anything like this exist?
I only dabbled many years ago but Erlang/OTP comes to mind.
And tRPC for TypeScript calls in browser and server.
Hey, we have a system like that at my company. Microservices are self contained and responsible for exposing their interface via SDKs created with a tRPC like library. Jump to definition works across the entire monorepo like a charm. Could you take a look and give me any feedback? https://github.com/rectech-holdings/umbrella-corp-boilerplat...
The industry has been trying to do "transparent remoting" for decades now. DCOM and CORBA were both basically that.
It turns out that "transparent RPC" is basically a contradiction in terms. As soon as you start doing things across process boundaries, and even more so across network boundaries, it requires a very different approach for API design - something that's very cheap locally, like passing objects by reference, becomes expensive and full of footguns.
If you reduce the feature set to the point where it can be transparently mapped to either local or RPC - which is, basically, function calls processing and returning data organized into arrays & trees (but not graphs) - there's still the issue that RPC has so many more failure points that you have to handle that would never light up in local.
This is all still doable; I have my doubts about practical the end result would be, though.
By “mostly transparent” I was mostly thinking type safety, ergonomics, no hand written boilerplate, etc.
Totally fine to support a limited set of primitives and collections passed by value, and require developers consider the failure modes.
In my experience HTTP+JSON or gRPC are usually lacking in at least one of those respects. If you constrain all the services to be the same language it becomes easier.
I made a framework along those lines for work which we used for many years to mediate ~1000 namespaced commands across dozens of repos. Whether it ran distributed or bundled (I called it "mega-service" mode) was completely controlled by config. We mostly used ZMQ for inter-service messaging, but I had implemented http, redis pub/sub, grpc, etc at various times to prove the concept.
Interestingly, we're pursuing a monorepo & multi-monolith setup for the next version of our platform. So lib-courier is no longer necessary to stitch it all together. It was fun while it lasted though. Once you understood the routing algo & code patterns, lots of stuff "just worked".
I am working on a project that uses a microservice architecture to make the individual components scalable and separate the concerns. However one of the unexpected consequences is that we are now doing a lot of network calls between these microservices, and this has actually become the main speed bottleneck for our program, especially since some of these services are not even in the same data center. We are now attempting to solve this with caches and doing batch requests, but all of this created additional overhead that could have all been avoided by not using microservices.
This experience has strongly impacted my view of microservices and for all personal projects I will develop in the future I will stick with a monolith until much later instead of starting with microservices.
How the hell... like... who decided to do Microservices in the first place if they didn't know this? This is such a rookie mistake. It's like somebody right out of high school just read on a blog the new way is "microservices" and then went ahead with it.
well, this is a surprisingly common case in my experience, except that the people were not "right out of high school" but "right out of a company funded seminar on microservices".
There is a high amount of correlation between people that start with microservices (instead of using them to solve specific problems after they have the problem) and people that lack any awareness about the resources they need.
>How the hell... like... who decided to do Microservices in the first place if they didn't know this?
Microservices can have both design utility and simultaneously been a major fad.
Lurking reddit and HN you can watch development fashions come and go. It's really really profoundly hard to hold a sober conversation on splitting merits from limitations in the middle of the boom.
I don’t mean to be snarky but how is that an “unexpected consequence”? Were the pros and cons just never considered when deciding to use micro services? Additional networks calls are one of the most obvious things to go on the cons list!
This is one of the reasons I dislike working in teams that lack 'old people'. Young devs still have a lot of mistakes to make that old devs have already made or seen. The young ones seem to see them as slow and difficult, but they also create stability and control.
In a start-up, having a team of young people will allow you to move fast, pivot and deliver. What you usually end up with though, is a tower built from spaghetti that's serving actual users. I see this again and again and then people get surprised that the software doesn't scale.
I'm so conflicted over becoming an older engineer (having recently moved back from a leadership role). On one hand, I have a wealth of knowledge and can see around corners that the youngsters have no chance of doing. OTOH, it is absolutely exhausting explaining in excruciating detail why I am right time-after-time as new mids join who just want to experiment. That and I am absolutely petrified of my alacrity fading with time.
I hear ya. If you feel you've lost your perspicacity, fear not; it's always in the last place you look.
But... yeah. A colleague (25+ years) has to deal with this more than I do, but have hit it some over the years, and it happens more the older I get.
Often he or I may be 'right' but only because of existing constraints in a system. Someone with a different view may even be 'correct' in a nuts-and-bolts technical sense. Yes, A is 8% faster than B. But... we have an existing system in B that meets our current needs, and team who know B, have written it, and maintained it for a couple years. Switching tech stacks to something no one knows for an 8% network speed boost is not remotely helpful.
My experience of being the old guy in the team (older also than the manager) was that the young whippersnappers[0] drive new tech adoption regardless of the views of the old fart; the manager waves it through to keep the young-un's happy, and the old fart doesn't find out until it's a fait accompli.
My experience was also that the young manager was very suspicious of the old fart. I've never had any problem working for managers younger than me; but some young managers don't know what to make of a team member that knows more than they do.
[0] No insult intended; almost everything I learned after the first 10 years, I learned from people younger than me.
You are right, looking back I also ask why it was not considered more indepth before. I was not involved in the decision making process as I recently joined, and to be honest I would have maybe not thought about it either (but will certainly in the future). I think the main reason this became such a big problem is because we underestimated the number of calls that would be made between the services. Service A was initally supposed to call Service B only as a last resort, so the 10 ms or so were fine, but in the end, the answers that Service B gave turned out to be very useful in our process so more and more calls were made and now they are strongly connected, but still separated services, which is ... not ideal.
Could this be solved by consolidating service A and service B into one unit while maintaining the overall architecture? I know it's just an example but the whole point of microservices is to have the flexibility to rearchitect pieces without having to rewrite the entire project, so potentially poor initial choices can be reworked over time.
Just wait until you decide you need some kind of distributed transaction support across multiple microservices… You should aim to spend a considerable amount of time in the design and discovery phases.
Some of the services not being in the same datacentre seems orthoganal. If that solves a problem you have, wouldn't it still be an issue in a non-microservices design?
> one of the unexpected consequences is that we are now doing a lot of network calls between these microservices
Not trying to be harsh here, but not expecting an increase of network call in a system where each component is tied together with... network calls sounds a bit naive.
> We are now attempting to solve this with caches and doing batch requests
So you have built a complex and bottle-necked application for the sake of scalability, then having to add caching and batching just to make it perform? That sounds like working backwards.
Obviously, I have no clue on the scale of the project you are working on, but it sure sounds like you could have built it as a monolith in half of the time with orders of magnitude more performance.
Scalability is a feature, you can always add it in the future.
Scalability is not a feature. If you "need" to scale but don't then you're not delivering to your target market, not delivering is not a lack of a feature it's a net loss to the organization. If you're Twitter/Instagram/Uber/whatever you cannot tell your users to not post or like or request a ride because "right now we don't have the scalability feature"
Yes, if you can't scale fast enough as you need to, it can hurt your business. Not being able to keep up with demand is a (luxury) problem that every business faces, not just in tech. They would often be called 'growing pains' in a business, and though they are bad, they rarely contribute to the failure of a company.
Starting a startup/service/platform with microservices before you even understand the bottlenecks/market fit/customers is usually not a good idea. You can come a very, very long way with a monolith before you hit performance and scalability limits. And once you do, you can always start breaking things up into smaller services for scalabity. Obviously you need to make sure you are scaling on time to keep up with demand.
'Nail it, then scale it', and 'premature optimization is the mother of all f-ups' are popular sayings for a reason.
I'm not making a case in favor of starting with microservices for your startup, that'd be insane, to say the least. I'm just disagreeing with "scalability is a feature". A feature is every addition on top of your minimum viable product. If at some point it becomes apparent that the business needs scalability, then scalability becomes your minimum viable product.
Musk may just be wrong but isn’t it also possible that the RPCs need to communicate with an edge node in India. Perhaps government regulation requires them to store/serve some user data from India if serving Indian customers? Or something like that?
In which case, the next step for your org is a mandate that all microservices be available in all datacenters.
Let each microservice owner figure out how to achieve their latency+reliability SLA's in every location - whether replicating datastores, caching, being stateless, or proxying requests to a master location.
Would be interesting to learn how your (or any other) team defines borders of a microservice. Iow, how “micro” they are and in which aspect. I guess without these details it will be hard to reason about it.
At my last job we created a whole fleet of microservices instead of a single modular project/repo. Some of them required non-trivial dependencies. Some executed pretty long-running tasks or jobs for which network latency is insignificant and will remain so by design. Some were few pages long, some consisted of similar-purpose modules with shared parts factored out. But there was no or little processes like “ah, I’ll just ask M8 and it will ask M11 and it will check auth and refer to a database. I.e. no calls as trivial as foo(bar(baz())) but done all over the infra.
Did you have a common, centralized data store or did each and every microservice manage their own instance?
(Because this is at the same time one of the defining elements of this architecture... and the first one to be opted out when you actually start using it "for real").
Yes and no. The "central" part of data flowed naturally through services (i.e. passed in requests and webhooks, not in responses). Microservices maintained local state as well, though it was mostly small and disposable due to whole-intra crashonly design. For example, we didn't hesitate to shut something down or hot-fix it, except for bus-factor periods when there's only few of them. They could also go down by themselves or by upstream, and routers avoided them automatically.
I agree that monoliths are the way to start many times, especially if you're not actually sure you'll ever need to scale. One reason we do sometimes plan on microservices from the start with projects is separation of security concerns. Easier to lock a public-facing microservice down tight and have what's ostensibly a non public-facing monolith call out to it.
Yeah, I think this is one of the very few places where splitting out something as a microservice makes sense. For example, you (mostly) never want to open/process/examine/glance at user-provided PDFs on a box with any sort of unfiltered network access. Ideally you do what you need to do within a sandbox that has _no_ network access, but that's really hard to do performantly.
The primary reason for this is that PDFs can contain executable code and the common tools used to process them are full of unpatched CVEs.
Here is an analogy that can inform the implications of this (so-called, imo) architecture:
Imagine if you are responsible, at runtime, for linking object files (.o) where each object is a just the compilation of a function.
Now why would anyone think this is a good idea (as a general solution)? Because in software organizations, the “linker’s” job is supposed to be done by the (“unnecessary weight”) software architect.
Microservices primarily serve as a patch for teams incapable of designing modular schemas. Because designing schemas is not entry level work and as we “know” s/e are “not as effective” after they pass 30 years of age. :)
> monolith
Unless monolith now means not-microservice, then be aware that there are a range of possible architectures between a monolith and microservices.
Yes. If you design a distributed system you need to consider the network traffic very carefully, and choose your segmentation in such a way that you minimize traffic and still achieve good scalability.
For this reason, I've been trying to push for building a monolithic app first, then splitting into components, and introducing libs for common functionality. Only when this is all done, you think about the communication patterns and discuss how to scale the app.
Most microservice shops I've been in have instead done the naïve thing; just come up with random functionally separate things and put them in different micro services; "voting service", "login service", "user service" etc. This can come with a very very high price. Not only in terms of network traffic, but also in debuggability, having a high amount of code duplication, and getting locked into the existing architecture, cementing the design and functionality.
Everything you mention (network traffic cost, code duplication (we instead resorted to auto-generate some shared code between services based on a open-api spec), locked into the architecture) applies to our use case...
And it is reassuring to hear that you seem to have success in avoiding these issues with a monolithic architecture, as I thought I was oldschool for starting to prefer monoliths again.
The execution model of local calls are quite different from remote calls. Because of these differences, many efforts have been made, but "transparent remoting" is still not achieved [1].
I agree with sentiment of the paper in that it can never be fully transparent. When I say "it should just be a process of identifying hot function calls..." this is admittedly a little contrived - there would need to be a more thorough analysis.
The main point seems to be that partial failure is too difficult to handle without explicitly declaring some objects as "remote".
> Merging the models by making local computing follow the model of distributed computing would require major changes in implementation languages (or in how those languages are used) and make local computing far more complex than is otherwise necessary
Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
What comes to mind is the event loop. The fact that in Node.js, almost every function call is now an asynchronous promise due to the non-blocking paradigm coupled with the convenience of async/await syntax. These event loop runtimes provide a universal way to hook into asynchronous calls.
I would imagine that in 1994, envisioning the code added to the languages of the day needed to handle timeout and asynchronous invocations would have felt overwhelming.
Also thinking about ORMs and databases. It seems TOPLink for SmallTalk was the first, released in 1995 after this paper was. ORMs seem kind of like "transparent remoting", as they usually can run against an in-memory database or db on a local system, or over an TCP connection, which then needs to deal with latency and failure. And for any app talking to a DB with an ORM, means that the failure cases of the ORM must prevade throughout the stack.
> Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
As the paper argues, it is not possible to make remote calls look like local ones. But it is trivially possible to make local calls look like remote ones! Use call-by-value parameter passing, and make all call asynchronous. The most successful solution I have heard of is Erlang's immutable-message-passing model.
It has been many years since I used it, but I think TOPLink Session and UnitOfWork interfaces threw exceptions signaling remote execution. This is like Erlang - you work with remote APIs; local execution is a special case.
No personal project has any business using microservices, unless it's specifically as a learning toy. Use a monolith. Monoliths can scale more than you can (provided you manage not to limit yourself to the single-thread single-process version of a monolith). Microservices are an organisational tool for when you need to release chunks independently.
I first wrote a program which ran on a web server about 25 years ago. In that time, computers have experienced about ten doublings of Moore's law, i.e. are now over a thousand times faster. Computers are very fast if you let them be.
Either way, one of my biggest pet peeves is the near-ubiquitous use of HTTP & JSON in microservice architectures. There's always going to be overhead in networked servies, but this is a place where binary protocols (especially ones like Cap'n Proto) really shine.
>We are now attempting to solve this with caches and doing batch requests, but all of this created additional overhead that could have all been avoided by not using microservices.
> especially since some of these services are not even in the same data center.
I think you need to answer why? If you can't put all of the services in one data center, then by definition you can't write a monolith either. If the monolith would happily run in one datacenter, then you should have all instances of your microservices in that one datacenter.
It surprised me that you would conclude that this is a problem with microservices. It's like if a particularly architect always punches you in the groin in every meeting, and you've concluded that architects are bad people, rather than this one architect is a bad person.
I think a lot of microservices are defined as separate apps and code repositories. This sounds good for initial build and deployment but the long term maintenance is the issue. Developers like the idea of independence of other teams writing parts of the overall solution but that trade-off can mean a lot of overhead in maintaining different code repositories of the same stack.
When a critical vulnerability comes out for whatever language you are using, you now have to patch, test and deploy X apps/repos vs much fewer if they are consolidated repositories written modularly. Same can be said for library/framework upgrades, breaking changes in versions, deprecated features, taking advantage of new features, etc.
Keeping the definition of runtimes as modular as the code can be instrumental in keeping a bunch of related modules/features in one application/repository. One way is with k8s deployments and init params where the app starts specific modules which then lends itself to be scaled differently. I'm sure there are home-grown ways to do this too without k8s.
This article is all over the place. The author acknowledges that microservices are about organizational clarity, then writes "In theory, anyway" without elaboration, and then goes on to talk about the latency impact of network-level IPC.
Why do we care about network latency, when we JUST established that microservices are about scaling large development teams? I have no problem with hackers ranting about slow, bloated and messy software architecture...but this is not the focus of discussion as presented in the article.
And then this conclusion:
> The key is to establish that common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be.
...so, like gRPC over HTTP? Last time I checked, gRPC is pretty well understood from an integration perspective. Much better than Enterprise Java Beans from the last century. Isn't this ironic? And where are the performance considerations for this backplane? Didn't we criticize microservices before because they have substandard performance?
Just saying "gRPC over HTTP" doesn't solve any of the sentence you quoted.
> common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be
Regardless of the tech used to implement, this paradigm needs to be solved to have a good system. That backplane is not an implementation, but a set of understood guiderails for inter-module communication.
Even with gRPC, both sides need to know what to call and what to provide and expect in response. That's the "conventions" part. Having consistency is more important than the underlying tech. Just simple ReST over HTTP works just as well as gRPC.
This article nails it, but I still like microservices because I’ve yet to see a team doing a modular architecture in memory keep from creating a spaghetti mess of interdependence.
Yes, the same often happens in microservices, but the extra complexity of a distributed system provides a slightly stronger nudge to decouple that means some teams at the margin do achieve something modular.
I’m something of a skeptic on modular monoliths until we as an industry adopt practices that encourage decoupling more than we currently do.
Yes, in theory they’re the same as microservices, without the distributed complexity, but in practice, microservices provide slightly better friction/incentives to decouple.
Software architecture can be tailored to a specific use case to best fit an application. Rather than strictly align to some thereoreical design principle, one can consider the use case and end goals and make the architecture match it.
But in general,just write some damn code. Presumably you have to write code, because building a software engineering department is one of the most difficult things you can do in order to solve a business problem. Even with the smartest engineers in the world (which you don't have), whatever you ship is inevitably going to end up an overly complex, expensive, bug-riddled maintenance nightmare, no matter what you do. Once you've made the decision to write code, just write the damn code, and plan to replace it every 3-5 years, because it probably will be anyway.
The way I usually describe my preferred heuristic to decide between modules and microservices is:
If you need to deploy the different parts of your software individually, and there’s a cost of opportunity in simply adopting a release train approach, go for microservices.
Otherwise, isolated modules are enough in the vast majority of cases.
670 comments
[ 4.6 ms ] story [ 330 ms ] threadIt's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
The overhead devs spend worrying about code organization may vastly exceed the amount of time floundering with messy programs.
Microservices aren't free, and neither are modules.
[1] Jonathan Blow rant: https://www.youtube.com/watch?v=5Nc68IdNKdg&t=364s
[2] Jon Carmack rant: http://number-none.com/blow/john_carmack_on_inlined_code.htm...
I easily find my way in messy codes with grep. With modules, I need to know where to search to begin with, and in which version.
Fortunately, I have never had the occasion to deal with microservices.
You can just grep / search all the files.
> in which version.
grep / search doesn't search through time whether you're using one file or many modules. You probably want git bisect if you can't find where something is anymore (or 'git log' if you have good commit messages),
But modules can be in different repositories I haven't cloned yet.
(And other tools like symbol search https://www.jetbrains.com/help/idea/searching-everywhere.htm...)
For searching multiple files, ripgrep works really well in neovim :)
[1] https://github.com/BurntSushi/ripgrep
Yeah sure, the scale is smaller, but you can't get away with making the user wait 10 seconds to render text and images to a screen either. I think the software world might be a lot better place if more developers thought like game developers.
Sure, but those "needs and requirements" aren't necessarily aligned with things that produce good software, and I think a lot of software development these days is not aligned. Further, I think the evolutionary path of the web in particular has produced a monstrosity that we'd be better off scrapping and starting over with at this point, but that's a tangential discussion.
That you probably don't even need, but there's a paradigm of "every function should be it's own class" that some devs seem to follow that I will never understand.
Their tiny data is rendering millions of polygons with real-time lighting, asset loading in the background, low latency input handling, etc. at 60fps+. If my API responds to the user in 100ms, they’re ecstatic. If their game responds to the user in 100ms (5-10 dropped frames) even a few times, they’re likely to ask for a refund or write a bad review, hurting sales.
The constraints are different, but game programmers do the same kinds of optimization social networks do, just for different problems. They avoid doing the N^2 version of lighting, path finding, physics simulations, etc. Social networks avoid it when searching through their graphs.
I think the web and other big tech companies should try thinking about problems the way game programmers do.
Microservices are the same modules. Though they force-add distributiveness, even where it can be avoided, which is fundamentally worse. And they make integration and many other things lot harder.
distage can do more, like wire your tests, manage component lifecycles, consider configurations while solving dependency graph to alter it in a sound manner.
Ever since my time as a mathematician (I worked at a university) and using LaTeX extensively, I never understood the "divide your documents/code into many small files" mantra. With tools like grep (and its editor equivalents), jumping to definition, ripgrep et al., I have little problem working with files spanning thousands of lines. And yet I keep hearing that I should divide my big files into many smaller ones.
Why, really?
Of course, that probably depends on the nature of the project. Having all backend code of a web app in one file and all the frontend code in another would be very inconvenient. OTOH, I have no problems with having e.g. multiple React components in one file...
I believe that files offer one useful level of compartmentalizing code that makes other people easier to understand what is going on by just looking at the file structure before opening a single file. The other guy can't grep anything before they have opened at least one file and found some interesting function/variable.
LaTeX is an obvious case: why pay to recompile the whole huge project, for each tiny change?
Now that I think about it, LaTeX is a very bad analogy anyway, since LaTeX documents usually have a very linear structure.
There are many other reasons why separation is better for humans (separation is organization) but these arguments about parallelism are valid for androids, humanoids and aliens.
I think the code should be split conceptually thus not just copy/paste part of code from main file to submodules, but split the code around some functional bounderies or concepts so that each file is doing one thing and compose those concepts into more abstract concepts.
So that when I try to debug something I can decide where I want to zoom in and thus be able to quickly identify the needed files.
and when dealing with the code that's well-organized and grouped into logically named files and dirs, you simply can navigate down the path and when you open a file all related code is there in one place without extra 10k lines of misc. code noise.
If some grouping of functions or classes is split out in a separate file where the externally used 'interface' is just a fraction of these functions or classes, and the rest are used only internally within the file, then this segmentation makes the system easier to understand.
That said, Emacs can show different parts of the same file in several buffers, a feature which I use quite a bit (not every day, but still). And of course I can rename them to something mnemonic (buffer name ≠ file name). So I personally don't find this convincing at all.
Also, filenames and directories tree act as metadata to help create a mental map of the application. The filesystem is generally well represented in exploratory tools like file browser and IDE. While the same information can be encoded within the structure of a single file, one needs an editor that can parse and index the format, which may not be installed on every system.
FWIW, sometimes when I worked on a really large file, I put some ^L's ("new page" characters) between logical sections, so that I could utilize Emacs' page movement commands.
Correct. But this is an argument against splitting: once your folder structure reflects your mental model, you should no longer split, no matter how big individual files get. Splitting further will cause you to deviate from your mental model.
Also, it seems like we're arguing against a strawman: saying "big files are okay" is not the same as "you should only have big files". When people mean is that having a big file does not provide enough justification to split it. But it is still a signal.
Generally speaking it's only Enterprise(tm) code that has miniscule (not small, miniscule) files with shit code
As long as it's good code, no one cares. If you aren't working on Enterprise(tm) code odds are every bad code you'll look at has way too many lines in the abstraction unit that is being used
If everything is effectively global to begin with, you're right, it might as well all be in one file. In very large programs the lack of scope control is going to be significant problem either way.
This is where object-oriented programming yields most of its actual value - scope control.
Yeah sure overengineering is a thing, but you're way off the path if you're brushing aside basic modularization.
If you have everything into one .c file, you need to recompile the whole thing every change.
I'm a proponent of keeping things in a monolith as long as possible. Break code into files. Organize files into modules. A time may come when you need to separate out services. Often, people break things out too early, and don't spend enough effort on thinking how to break things up or on the interfaces.
Don't you want determinism and deterministic simultations? If you do, you'll also need stub implementations (mocks, dummies) for your interfaces.
Some notes on that: https://blog.7mind.io/constructive-test-taxonomy.html
> A time may come when you need to separate out services.
Most likely it won't if you organise properly. For example, if each your component is an OSGi module.
You can say the exact same thing about C-headers though.
For instance, in C# a class can span multiple files using "partial" or you can have multiple classes in a single file. It's generally not an issue as long as the code itself is organized. The only downside is the reliance on an IDE, which is pretty standard these days anyway.
Same for business logic - very clear separation can be cumbersome sometimes, but otherwise it becomes messy if you're not careful. And careful people just tend to separate it.
It’s a good idea to break things down to files along logical boundaries. But got reporting isn’t a reason.
edit: "got diff" -> "git diff". DYAC and responding from mobile!
If it's a separate file that is scoped to some specific concern, sure. But its tgat grouping by concern that is key. Not separation into another file. Extracting ra dom bits of code into separate files would be _worse_.
Yes and no.
Git doesn't store lines, it stores files. Git diff knows how to spit out line changes by comparing files.
So to run git blame on a 10k line file you're reading multiple versions of that 10k file and comparing. It's slow. Worse still is that trying to split said file up while trying to preserve history won't make the git blame any faster.
As long as what's there is comprehensible, being able to evolve it over time is a very useful lever.
A 10k lines file is not something that will completely destroy your productivity, but it will have an impact and you'd better look out for it growing further, because completely destroying your productivity is not too far away. It is almost always good to organize your code when it reaches a size like this, and the exceptions are on contexts where you can't, never on contexts where it's worthless.
EDIT: And what editor do you use? I'm wondering if a lot of these differences come down to IDEs haha
This is a fair point but assumes 1 particular use case. It is easier if you are just concerned with a bit of it. If you need to deal with all of it, yeah, good fucking luck. 10k LOC file or 1k 100 LOC files.
I tend to ensure each file serves exactly one purpose (e.g. in C# one file = one class, with a only few exceptions).
I use VS Code, but in every IDE with a file opening palette it's actually really fast: you want to look for the code to, let's say, generate an invoice, just search for "invoice" in the list of files and you'll find it immediatly.
(Also modules have their own problem, I was mainly talking in a general way since that's what the parent comment was talking about.)
I also find that updating code to take advantage of new/better language features / coding styles, etc. is impossible to do on a large code base at once. However, sprinkling these kind of things randomly leads to too much inconsistency. A reasonable sweet spot is to make each file self-consistent in this regard.
My experience stems from larger 500+ person-year projects with millions of lines of code.
If a particular domain gets big enough it probably means it contains sub-domains and can benefit from being divided too. But you cannot make that decision based on size alone.
The point is that a numeric upper bound on LoC is inherently subjective and pointless. Instead of measuring the right thing (concepts) you're measuring what's easy to measure (lines).
In fact, it usually makes things worse. I've seen it over and over: you have N tightly coupled classes in a single file which exceeds your LoC preference.
Instead of breaking the coupling you just move those classes into separate files. Boom, problem solved. Previously you had a mess, now you have a neat mess. Great success!
Yes java developers can't do anything without their IDE. It helps them mask the 30000 nested directories they've created to "organize" the code.
Also, the question of inlined code mostly applies to programming in the small, while modules are about programming in the large, so I don't think there's much relationship between the two.
Most systems don't have to be maintained by hundreds of people. And yet they are: maybe because people don't listen to folks like Carmack?
We like stories about huge teams managing huge codebases. But what we should really be interested in is practices that small teams employ to make big impact.
Most of the time when I split out a method what I want is a closure that is clearly separated from the rest of the code. If I can have such closures, where input and output is clearly scoped, without defining a new method, that might be preferable. Lambdas could be one way to do it but they still inherit the surrounding closure so it's typically not as isolated as a method.
For sure, I have absolutely no idea how your comment relates to what I wrote.
It’s easy to miss the point of what the OP is saying here and get distracted by the fact this file is ridiculously huge. This file used to be a paltry 1k file, a 10k file, a 20k SLOC file… but it is where it is today because of the OP suggested approach.
<rant> You will (for most statements) both be able to find a best and worst case. That's the catch with most generalized statements/ principles, e.g., DRY. The challenge is to find a good "enough" solution because perfection is usually either unfeasibly expensive or impossible (different viewpoints, ...) </rant>
Though it's kinda hilarious that the source code of a MS project is not natively viewable on a MS platform.
With a trace from a profiler tool, which will tell you which line number is the hot spot. If run from any modern IDE, you can jump to the line with a mouse-click.
In essence, the file boundaries ought not to make any difference to this process.
I find this the strongest counter argument. File boundaries shouldn’t matter - but in practice they do.
The ide will give up doing some of its analysis because this is not an optimised use case, the IDE vendors don’t optimise for it. Some IDEs will even give up just simple syntax highlighting when faced with a large file, never mind the fancy line by line annotations.
Bitbucket is the biggest offender for me. It starts to disable syntax highlighting and context once the file hits about 500KB, which should be trivial to deal with.
Answer: the same place - with a profiler. Logical code organization matters a lot more than “physical” break to specific files.
I have inherited a Java class spaghetti project in the past, with hunderds (perhaps thousands) of short classes, each which doesn’t do much but sits in its own file - and I would much prefer to work on an SQLite style codebase, even if I have to start with the amalgamation file.
By the way, JavaScript parser in esbuild is a 16 thousand lines of code module too:
https://github.com/evanw/esbuild/blob/0db0b46399de81fb29f6fc...
The problem is when it's OK and for how long. If you have a team of people working with a codebase with all those "okays", then they have to be really good developers and know the code inside out. They have to agree when to refactor a business login out instead of adding a hacky "if" condition nested in another hacky "if" condition that depends on another argument and/or state.
I guess what I'm trying to say that if those "okays" are in place, then there's a whole bunch of unwritten rules that come in place.
But I agree that microservices certainly aren't free (I'd say they are crazy expensive) and modules aren't free either. But all those "okays" can end up costing you your codebase also.
If you're a single dev making a game, by all means, do what you want.
If you work with me in a team, I expect a certain level of quality in the code you write that will get shipped as a part of the project I'm responsible for.
It should be structured, tested, malleable, navigable and understandable.
One suggested alternative is to split reusable code into a local lambda first and lift it into a separate code piece only once we need that code elsewhere. It seems to me that such approach would limit the complexity of the code graph that you need to keep in your head. (Then again, when I think about it maybe the idea isn't really that novel.)
Great comment!
I personally find that most codebases are overstructured and undertested :)
In my experience, module boundaries tend to make code less malleable, less navigable, and less understandable.
People have different thresholds for when their code reaches these states though, especially "understandable".
You can meaningfully define these (and test for them) on a small scale, in a team, but talking about all developers everywhere, these are all very loosely defined.
Games are highly stateful, with a game loop that iterates over the same global in-memory data structure as fast as it can. You have (especially in Carmack era games) a single thread performing all the game state updates in sequence. So shared global state makes a ton of sense and simplifies things.
Most web applications are highly stateless with request-oriented operations that access random pieces of permanently-stored data. You have multiple (usually distributed) threads updating data simultaneously, so shared global state complicates things.
That game devs gravitate towards different patterns for their code than web service devs should not be a surprise.
> At this point, I think there are some definite advantages to "style C", but they are development process oriented, rather than discrete, quantifiable things, and they run counter to a fair amount of accepted conventional wisdom, so I am going to try and make a clear case for it. There isn't any dogma here, but considering exactly where it is and isn't appropriate is worthwhile.
> In no way, shape, or form am I making a case that avoiding function calls alone directly helps performance.
Now your MinorFunction3 code can access all the local variables used by MinorFunction1 and MinorFunction2, and there's no easy list of things that it might access which makes it harder to read.
Separate functions do have a nice list of things they might access - their arguments!
Of course this technically only applies to pure functions so maybe that's why it doesn't matter to Carmack - he's used to using global variables willy nilly.
Also sometimes the list of things a function might need to access gets unwieldy, which is when you can reach for classes. So no hard and fast rule but I think increased scope is the issue with it.
Using A or B results in self-documenting code and I think DOES (or at least, CAN) improve readability. It also can help reduce excessive nesting of code.
Couldn't disagree more. As usual, it's a tradeoff. You could spend an infinite amount of time refactoring already fine programs. But complex code decreases developers productivity by orders of magnitude. Maybe it's not always worth refactoring legacy code, but you're always much better off if your code is modular with good separation of concerns.
It's not. This is the thing where you start thinking "YAGNI", yadayada, but you inevitably end up needing it. Layering with at least a service and a database/repositories is a no brainer for any non-toy app considering the benefits it brings.
> It's okay to have files with 10,000 lines
10.000 lines is a LOT. I consider files to become hard to understand at 1.000 lines. I just wc'd the code base I work on, we have like 5 files with more than 1.000 lines and I know all of them (I cringed reading the names), because they're the ones we have the most problems with.
It gets worse when the people who made the mess quit or move on, leaving the new hires to deal with it. I've seen this pattern enough times to wonder if it gets repeated with most companies or projects.
I do agree that microservices and/or modules aren't magical solutions that should be universally applied. But they can be useful tools, depending on the situation, to organize or re-organize a system for particular purposes.
Anecdotally, I've noticed that smart people who aren't good programmers tend to be able to write code quickly that can scale to a particular point, like 10k-100k lines of code. Past that point, productivity falls rapidly. I do believe that part of being a skilled developer is being able to both design a system that scales to millions of lines of code across an organization, and to operate well on one designed by someone else.
You have to understand everything that calls that function before you change it.
This is not always obvious. It takes time to figure out where it's called. IDEs make this easier but not bullet proof. Getting it wrong can cause major, unexpected problems.
If you split up MegaFunction(){} to Func1(){} Func2(){}, etc, but it never makes sense to call Func2 except after Func1, then you haven't actually created two functions, you've just created one function in two places.
Refactoring should be about logical separation not about just dicing a steak because it's prettier that way.
By chopping up MegaFunction like this, you've not actually separated Func1 and Func2 if they aren't really independent, they're just MegaFunction in disguise but now split across two places making it more difficult, not easier, to reason about.
If you need the state (implied or explicitly passed in and out) from having executed Func1 to run Func2 then you're just creating spaghetti code. You're taking a big ball of mud and smearing it around instead of actually tackling the abstraction.
This is how you end up with Func(a, b, c, d, &e, &f) which ends up changing your MegaFunction state.
Or just as bad, Func1,2,N are all private functions, never called anywhere else outside the class MegaFunction is in, so logically (and from the point of view of testability) it's no different to having them all inline.
If you're creating a function that's only ever called once, then instead of a function call you're probably better off with a comment to "name" that block and explain the process instead.
Frankly It Depends(tm). Sometimes you can do this and not pass too much state, sometimes you can not. Sometimes your state is in a class, or global, and you just pass the class around.
I have somewhere a 3000 line state machine with the app state in a class - i just pulled out logically connected states in auxiliary files when it went over 1k lines. In my case it's easy to comprehend because groups of states are kinda separated logically.
Unfortunately most engineers don't like nuance, they want one-size-fits-all solutions.
I find there are practical negative consequences to having a 10,000 line file (well, okay; we don't have those at work, we have one 30k line file). It slows both the IDE and git blame/history when doing stuff in that file (w.r.t. I'll look at the history of the code when making some decisions). These might not be a factor depending on your circumstances (e.g. a young company where git blame is less likely to be used or something not IDE driven). But they can actually hurt apart from pure code concerns.
>It's okay to not organize your code. It's okay to have files with 10,000 lines. It's okay not to put "business logic" in a special place. It's okay to make merge conflicts.
I absolutely disagree. It's "okay" if you're struggling to survive as a business and worrying about the future 5+ years out is pointless since you don't even know if you'll make it the next 6 months. This mentality of there being no need for discipline or craftsmanship leads to an unmanageable codebase that nobody knows how to maintain, everybody is afraid to touch, and which can never be upgraded.
You don't see the overhead of throwing discipline out the window because it's all being accrued as technical debt that you only encounter years down the road.
https://github.com/tinspin/rupy/wiki/Process
All input I have is you want your code to run on many machines, in fact you want it to run the same on all machines you need to deliver and preferably more. Vertically and horizontally at the same time, so your services only call localhost but in many separate places.
This in turn mandates a distributed database. And later you discover it has to be capable of async-to-async = no blocking ever anywhere in the whole solution.
The way I do this is I hot-deploy my applications async. to all servers in the cluster, this is what a cluster node looks like in practice (the name next to Host: is the node): http://host.rupy.se if you click "api & metrics" you'll see the services.
With this not only do you get scalability, but also redundancy and development is maintained at live coding levels.
This is the async. JSON over HTTP distributed database: http://root.rupy.se (2000 lines hot-deployable and I can replace all databases I needed up until now)
We delivered many talks on that subject and implemented an ultimate tool for that: https://github.com/7mind/izumi (the links to the talks are in the readme).
The library is for Scala, though all the principles may be reused in virtually any environment.
One of the notable mainstream (but dated) approaches is OSGi.
with a good branching strategy
In cases when a rollback is necessay, you'll often find yourself in a cycle where a breakage goes out, the deployment is rolled back, a patch is applied, a new deployment goes out and a new breakage happens, the deployment is rolled back, and so on. I am exaggerating how often this happens, but it does happen when you have thousands of engineers, and it happened often enough at my workplace that developers were unhappy and management pushed for breaking up the monolith.
Then you can have a mono repo that deploys to multiple micro services / cloud functions / lambdas as needed depending on code changes and programmers don’t have to worry about RPC or json when communicating between modules and can just call the damn function normally.
Though there were many attempts to do it, I would just mention Erlang and Akka.
The answer to your question is close to the answer for "what's wrong with Akka"
Solving this in the general case for every function call is difficult. Pure functions are idempotent, so you can retry everything until nothing fails.
But once you add side effects and distributed state, we don’t know how to solve this in a completely generalized and performant way.
For our services, there is never a single point of failure. We use Resilience4j for retries, caching inside the dynamic proxy handler (using annotations to describe what is cacheable), and annotations to describe what can be done asynchronously.
In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
> In the cases where we use these, the caller is aware (via the interface documentation) that this is a distributed system where sometimes invocations happen really fast and reliably.
This is at odds with
> I have systems where the decision to put the module in the same process or to call it via RPC is done at runtime.
Either it's in the caller's choice (based on the documentation), or it's at runtime, but it can't be both and be equally reliable.
I did not say it is the caller's choice nor that it is equally reliable. It is a system composition decision. It is a system that is sometimes distributed and treated as such from a reliability point of view. Sometimes the actual implementation is more reliable and faster.
One thing I've observed in a microservice-heavy shop before was that there was the Preferred Language and the Preferred Best Practices and they were the same or very similar across the multiple teams responsible for different things. It lead to a curious phenomenon, where despite the architectural modularity, the overall SAAS solution built upon these services felt very monolithic. It seemed counter-productive, because it weakened the motivation to keep separation across boundaries.
I.e. the idea that each microservice has direct access to its own, dedicated, maybe duplicated storage schema/instance (or if it needs to know, for example, the country name for ISO code "UK" it is supposed to ... invoke another microservice that will provide the answer for this).
I always worked in pretty boring stuff like managing reservations for cruise ships, or planning production for the next six weeks in an automotive plant.
The idea of having a federation of services/modules constantly cross-calling each other in order to just write "Your cruise departs from Genova (Italy) at 12:15 on May 24th, 2023" is not really a good fit for this kind of problems.
Maybe it is time to accept that not everyone has to work on the next version of Instagram and that Microservices are probably a good strategy... for a not really big subset of the problems we use computers for?
No, you don't use separate microservices for writing out that text message.
The idea is pretty simple instead of writing one big program, you write many smaller programs.
It's useful around the 3 to 4 separate developers mark.
It avoids you having to recompile for any minor change, allows you run the tests for just the part you changed and allows you to test the microservices in isolation.
If you a production issue, the exception will be in a log file that corresponds to a single microservice or part of your codebase.
Microservices are a hard form of encapsulation and gives a lot of benefits when the underlying language lacks that encapsulation. e.g. Python.
But in order to find out that Genua is the name of the port from where the cruise is departing, the appropriate time (converted to the timezone of the port, or the timezone of the client who will see this message, depending on what business rule you want to apply) and that Genua is in IT=Italy... how many microservices do I have to query, considering that port data, timezones, ISO country codes and dep.date/time of the cruise are presumably managed on at least four different "data stores"?
e.g.: https://www.baeldung.com/cs/microservices-db-design
"2.1. Fundamentals By definition, microservices should be loosely coupled, scalable, and independent in terms of development and deployment. Therefore, the database per service is a preferred approach as it perfectly meets those requirements. Let’s see how it looks:"
Please understand that I have worked only on monoliths and will probably retire while still working on monoliths. This kind of absurd positions only come up when someone comes to my office with some grand plan to convert the application I work on to something "more microservice oriented".
Let's suppose we are Acme Cruise Lines running a Cruiseliner:
Microservice - National Coastguard Ship Arrival System Feed Handler
Database - Logs incoming messages on the feed
Microservice - Asian and Australian Joint Ship Monitoring System
Database - Logs incoming messages on the feed
Microservice - Cruiser Arrival and Departure Times
Database - Cruiser Arrival and Departures Times in a Standard Format
Microservice - Customer Bookings and Payments
Database - Customer Bookings and Payments
Microservice - Fuel Management System
Database - Ship Fuel Levels & Costs of fuel at Various Ports.
It's that high level of split up.
(AWS Lambdas aren't quite the same thing as microservices.)
The things you listed are ... 4-5 different applications, mostly running directly on the ship(s) and what is conspicuously missing are the parts that are managed shoreside, like:
Itinerary planning (your product is one or more cruises: therefore you need to preplan the itineraries, which ships to use, when you will enter each port and when you will leave it, and so on... try to imagine this like a mix between hotel management and flight company management) Inventory management (i.e. Reservation).
You mention "bookings" like a microservice. This could work for ferry line, where you are basically selling a ticket for a single trip, most of the time with no personal accommodation (except maybe for the car).
A Cruise booking usually comes with a ton of ancillary services (e.g. I live in Berlin and I want to take a cruise in the Caribbeans... therefore I need a flight to get there and back. This could be provided by a charter flight or a normal airline ... in either cases the ticket will be part of the booking itself) - take in account that cruise customers are mostly middle-aged or older and relatively affluent, therefore the last thing they would like to do is to create their own itinerary by booking services on 4-5 different websites.
(But I suppose it is better to stop there, we are really OT now)
When you book a cruise, your expectation is to get everything (including hotel stays before or after the cruise, along with flights to and from it) as a single package provided by the cruise vendor (or a travel agency). So the "reservation system" must take care of all of that.
When I said "4-5 different websites" I was trying to explain the point that a 60 yo high-income guy is usually not interested in getting the cruise itself on Carnival.com, then go look for excursions on CarribeansExcursion.net and to book flights on Lufthansa.de or AirFrance.fr.
It was a remark on the way the Cruise business works, and why it is so, not about architecture.
(But, once again, I really believe that we are way off topic... personally I do not really feel like creating a "Ask HN: are microservices a good choice for the cruise industry?" but if someone feels like submitting one I will try to contribute).
The thing is that the framing of "the problems we use computers for" misses the entire domain of problems that microservices solve. They solve organisational problems, not computational ones.
And this is not because "my stuff is complicated and your stuff is a toy", either. It's more like "ERP or Banking Systems" were deployed decades ago, started as monoliths and nobody can really afford to rewrite these from scratch to leverage Microservices (or whatever the next fad will be). (I am also not sure it is a good idea in general for transactions that have to handle/persist lots of state, but this could be debated).
The problem, in fact, is that "new guys" think that Microservices will magically solve that problem, too, want to use these because they are cool and popular (this year), and waste (and make me waste) lots of time before admitting something that was clear from day 1: Microservices are not a good fit for these scenarios.
(I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this).
> I have exactly the opposite problem though: the kind of problems I have worked on so far would not be "solved" by leveraging "large number of developers that can independently work on an equally large number of small, modular programs with a very well defined, concise interface".
If you don't have multiple teams working on it, and you don't have specific chunks of functionality that you need to scale orthogonally, then you don't have the problems microservices solve. So don't use them. That seems uncontroversial to me.
> I still think that "these scenarios" are prevalent in any company which existed before the 90s, but I might be wrong or biased on this
This is survivorship bias, in a sense. Microservices only make sense where access to CPU and network is cheap and on-demand (largely). That's only started to be an assumption you could really make since Amazon's EC2 convinced the world's CIOs that fungible tin was better than their own racks.
That means you don't see microservice architectures in older companies, and you don't see problems being solved with them even where it might have made sense retrospectively because IT Ops would never have been set up to enable it.
Today you don't need a big rewrite to justify introducing microservices where they're needed. That's a straw man. All you need is a team that will be able to move faster if their deployment is decoupled from everyone else's.
But fundamentally if your problem area smells like "bunch of business rules and a database" then, again: if you don't have the problem that the architecture solves, don't use it.
cough Aging directors in Health and Finance cough
A micro service implementation of that would be CQRS, where the services to write updates, backend process (eg, notifying the crew to buy appropriate food), and query existing records are separated.
You might even have it further divided, eg the “prepare cruise” backend calls out to several APIs, eg one related to filing the sailing plans, one related to ensuring maintenance signs off, and one related to logistics.
One strategy I've used is to designate one system as a source of truth (usually an ERP system) and periodically query its database directly to reload a cache. Every system works off their own periodically refreshed cache. Ideally, having all the apps query a read replica would prevent mistakes from taking down the source of truth.
I haven't done this, but I think using Postgres with a combination of read-only foreign tables and materialized views could neatly solve this problem without writing any extra code.
I don't know how far this would scale. I do know that coordination and procedures for using/changing the source of truth will fall apart long before technical limitations.
Does anyone have a microservice "map" of Instagram? I feel that would be helpful here.
Or you can think of it as trunk/branch architecture. One main "trunk" service and other branch services augmenting it, which is a simpler thing to reason about.
Now imagine a small shop of 20 devs deciding to build something more complicated.
Binaries are deployed and scaled independently as thrift services.
Tons of rpc.
Another way to put it is that teams that share parts of the same codebase introduce low level bugs that affect each other, and most organizations are clueless about preventing it and in some cases do not even detect it.
Someday I will find a way to untangle this wishlist enough to turn into a design.
1) A remote call – inherently – fail. However, some local calls never fail. Either because they are designed to never fail or because you have done the required checks before executing the call. A remote call can fail because the network is unreliable and there is no way around that (?).
2) A remote call – inherently – can be very slow. Again because of the unpredictable network. A local call may be slow as well but usually either because everything is slow or because it just takes a while.
So if you have a call that may or may not be local you still have to treat it like a remote call. Right?
I think having a certain set of calls that may or may not be executed locally is not that bad. Usually it will just be a handful of methods/functions that should get this "function x(executeLocally = false|true)" treatment - which is prob. an acceptable tradeoff.
Those modern computers are a network mesh of small D/G/CPUs, and it's coming back to bite us sometimes.
RPC vs local calls is trivial in comparison and you can get that level of transparency out of the box with functional programming - it's just data in data out.
Using hindsight, most of the systems that pretend that the network part of invoking RPCs is "easy" and "simple" and "local" end up being very complex, slow and error prone themselves.
See DCOM, DCE, CORBA and EJBs RMI.
You need instead a efficient RPC protocol that doesn't hide the fact it is a RPC protocol - like Cap'n Proto (it's "time-traveling RPC" feature is very interesting).
Part of the reason is that anything that leaves the application package increases the error potential exponentially.
Also, modules "bottleneck" functionality, and allow me to concentrate work into one portion of the codebase.
I'm in the middle of "modularizing" the app I've been developing for some time.
I found that a great deal of functionality was spread throughout the app, as it had been added "incrementally," as we encountered issues and limitations.
The new module refines all that functionality into one discrete codebase. This allows us to be super-flexible with the actual UI (the module is basically the app "engine").
We have a designer, proposing a UX, and I found myself saying "no" too often. These "nos" came from the limitations of the app structure.
I don't like saying "no," but I won't make promises that I can't keep.
BTW: The module encompasses interactions with two different servers. It's just that I wrote those servers.
There's two technical problems that microservices purport to solve: modularization (separation of concerns, hiding implementation, document interface and all that good stuff) and scalability (being able to increase the amount of compute, memory and IO to the specific modules that need it).
The first problem, modules, can be solved at the language level. Modules can do that job, and that's the point of this blog post.
The second problem, scalability, is harder to solve at the language level in most languages outside those designed to be run in a distributed environment. But most people need it a lot less than they think. Normally the database is your bottleneck and if you keep your application server stateless, you can just run lots of them; the database can eventually be a bottleneck, but you can scale up databases a lot.
The real reason that microservices may make sense is because they keep people honest around module boundaries. They make it much harder to retain access to persistent in-memory state, harder to navigate object graphs to take dependencies on things they shouldn't, harder to create PRs with complex changes on either side of a module boundary without a conversation about designing for change and future proofing. Code ownership by teams is something you need as an organization scales, if only to reduce the amount of context switching that developers need to do if treated as fully fungible; owning a service is more defensible than owning a module, since the team will own release schedules and quality gating.
I'm not so positive on every microservice maintaining its own copy of state, potentially with its own separate data store. I think that usually adds more ongoing complexity in synchronization than it saves by isolating schemas. A better rule is for one service to own writes for a table, and other services can only read that table, and maybe even then not all columns or all non-owned tables. Problems with state synchronization are one of the most common failure modes in distributed applications, where queues get backed up, retries of "bad" events cause blockages and so on.
So does modularity.
"The benefits expected of modular programming are: (1) managerial_development time should be shortened because separate groups would work on each module with little need for communication..."
On the Criteria To Be Used in Decomposing Systems into Modules, D.L. Parnas 1972.
http://sunnyday.mit.edu/16.355/parnas-criteria.html
He's right as far as Conway's Law goes, though.
Edit: failed to spell "I" correctly. Ouch.
The point is that Parnas never conceived of doing it because he was writing about a world where the interfaces between modules were known ahead of time and were static.
With microservices, as long as you maintain the contract with caller services, you can deploy whenever you want. If you have some kind of issue, your team is solely responsible. If the caller services do weird things, they are responsible for fixing them.
If you are pushing changes to a module as part of a more monolithic, or wrapper service - if you do a deploy and break the whole big service, you are now responsible for the entirety of any issue, which is now more likely due to integration risk and unrelated changes from others, especially because now there need to be coordination across many teams integrating - hopefully via tests and observability. But this requires a high-degree of maturity for automated quality assurance and site reliability. If you don't have that, the operational risks are very high. So that alternative is having some ops-like function, or other integration team responsible. Or doing more top-down waterfall coordination.
Given the service maturity needed is rare, microservices distributes that operational ownership in a way where there is more accountability.
Infrastructure that someone has to take care of.
In chaotic environments like that, microservices help with defining clear ownership of operational responsibility, though at the cost of clear responsibility for the overall distributed system. This comes at the cost of making it hard to develop, let alone ship, changes that impact multiple microservices (outside the scope of a single team, or closely related sibling teams).
The main point I was making was that, with a more monolithic system, managing that kind to organization disfunction, necessitates top-down, waterfall like control. So moving to microservices enables non-coordinated agility removing one layer of coordination problem (a the cost of other problems).
With modules you need some sort of wrapper application to bring it all together.
With microservices you need some sort of network layer so that the microservices can talk to each other.
You don't restart the network every time you deploy a microservice.
If we're being technical some languages support hot swapping modules so no restart would be needed. Setting that aside, restarting an application isn't anything that needs coordination today. You wouldn't even restart the application. You'd deploy a new version, swap over, and shut down the old version.
>You don't restart the network every time you deploy a microservice.
No, but something changes in the network configuration so that the other microservices are aware of the deployment.
They aren't built in, it's just that the need for them is impossible to ignore. Developers (and management) can't help but recognize and respect modularity in microservices because the separation of services and the APIs between them make the modularity obvious. When the separation between modules only exists in code, and even then only when seen through the eyes of someone who understands the modular architecture of the codebase, it is easily ignored, inevitably forgotten, and might as well not exist at all.
If all modules have to be deployed using the same build even though different build of the modules would have been API compatible - you don't have modules.
IMO a module is defined as something that has a self-contained API, and versioning rules for that API if the module is evolving.
Developers do a much better job with microservices. I think it's easy for them to respect the seriousness of designing and changing the API of a microservice. In contrast, they often don't respect or even realize the seriousness of making a change that affects the modular design of code.
Language-level support for defining and enforcing module interfaces might help developers invest the same level of care for module boundaries in a modular monolith as they do in a microservices architecture, but I've yet to work in a language that achieves this.
Often people are too lazy to issue API version change on a "bugfix", and take down multiple applications.
Funny thing, a modularized monolith may have identified some of those issues.
Modularization allows development to scale.
Microservices allow operations to scale.
What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail.
You also cannot produce any meaningful end-user value without involving several teams.
Microservices is just "backend, frontend, and database team" reincarnated.
My take: do Microservices all you want, but don't organize teams around the services!
Amazon's warehouses have incredible domain complexity which drives the complexity of the software architecture. It's also one of the oldest parts of the company with a direct lineage going back 20+ years. (for example, our inventory databases still have the same schema and are still using a RDBMS, albeit RDS Postgres instead of Oracle).
About five years ago we started rearchitecting to a centralized orchestration service which is being used to abstract this complexity from process teams. This is to avoid projects where 20+ service teams must be engaged just to launch a new solution.
Long ago I consulted for one company that reconstituted teams for every new project, so people got swapped around every few months, usually ending up in a new chunk of the code. Whatever the theoretical benefits, it meant that nobody felt a sense of ownership for any of the code. Combine that with aggressively short project deadlines and an absolute refusal to ever adjust them and the code rapidly dropped in quality. After all, you'd be on something else soon, so any mess wasn't your long-term problem. And why should you bother given that the code you started with wasn't pristine?
As far as I can tell, the best it gets is organizing around the most durable features of the business environment. E.g., audiences and their real-world needs. Next, long-lived solutions to those needs. And then you find the common and/or large needs of those teams and spin up service teams around them. And in the background, some strong culture around what the priorities really are, so that service teams don't get so self-important that they become the tail wagging the dog.
But I love all the war stories here and would love to hear more, as I think there are no perfect global solutions to organizational problems. I think it's mostly in the details of circumstance, history, and the people involved.
Microsoft, Apple, Amazon I think do a good job at preaching the customer first model and organizing around that.
This is actually the whole point (see: Reverse Conway).
This isn't easily fixable, but I'd like technologists to at least be able to perceive the extent to which the surrounding business culture has permeated into technical culture. I'm probably considered old AF by a lot of you (41) but I'm becoming increasingly interested in tools/methodologies that enable fewer devs to do a lot more, even if it means that there are sharp edges.
As projects get larger, the problems become more organizational in nature.
Horizontal scaling always incurs communication overheads faster, and as we know, developers have high communication overhead to begin with.
Human scaling is its own problem set and companies like to ramp projects up and then down again. Most of those people only think about how to get more people on board, not how to run the ship once attrition starts to condense responsibilities and reduce redundancy, especially for unpleasant tasks.
I’ve been trying to beat the idea of ergonomics into a smart but mule-headed developer (too old not to know better) for years now and it wasn’t until a combination of team shrink and having to work with code written by a couple of people who emulate his development style that all of a sudden he’s repeating things I said four years ago (of course with no self awareness of where those ideas came from). It remains to be seen if it affects how he writes code, or whether he starts practicing Do As I Say, Not As I Do, but I’m a short-timer now so it’s mostly an anthropological study at this point.
Also - talking of redesign should be a major lift for any product/service that has real paying clients. The risk of breaking is huge, and the fact that during that time you won't be delivering incremental value is also going to look bad.
hence, if you want a certain architecture, you likely need to execute a "reverse conway law" org first. To get the org into the target config, the software will follow.
What does not work is splitting ownership of many tings for a long time. It’s too many plates to spin. But some things can be shared by a few people without tipping over into anarchy.
Conway's Law is basically "You don't have a choice." You will cement the design of your app around your organizational design. (At least, beyond a certain org size. If you have only 4 engineers you don't really have the sort of structure in question here at all.) je42 narrowly beats me to the point that if this is a problem you can try to match your organization to your problem, but that takes a fairly agile organization.
"What happens if you need to redesign the architecture to meet new needs? That's right; it's not going to happen, because people will fight it tooth and nail."
Unfortunately, in the real world, this is not so much "a disadvantage to the microservice approach" as simply "an engineering constraint you will have to work around and take account in your design".
Despite what you may think after I've said that, I'm not a microservice maximalist. Microservices are a valuable tool in such a world but far from the only tool. As with everything, the benefits and the costs must be accounted for. While I've not successfully rearchitected an entire organization from my position as engineer, I have had some modest, but very real success in moving little bits and pieces around to the correct team so that we don't have to build stupid microservices just to deal with internal organizational issues. I don't mean this to be interpreted as a defeatist "you're doomed, get ready for microservices everywhere and just deal with it"; there are other options. Or at least, there are other options in relatively healthy organizations.
But you will have code that matches the structure of your organization. You might as well harness that as much as you can for the benefits it can provide because you're stuck with it whether you like it or not. By that I mean, as long as you are going to have teams structured around your services whether you like it or not, go in with eyes open about this fact, and make plans around it to minimize the inevitable costs while maximizing the benefits. Belief that there is another option will inevitably lead to suboptimal outcomes.
You can't engineer at an organizational level while thinking that you have an option of breaking the responsibility and authority over a codebase apart and somehow handing them out to separate teams. That never works long term. A lot of institutional malfunctioning that people correctly complain about on HN is at its core people who made this very mistake and the responsibility & authority for something are mismatched. Far from all of it, there are other major pathologies in organizations, of course. But making sure responsibility & authority are more-or-less in sync is one of the major checklist items in doing organization-level engineering.
The good news is, pretty much everyone has been on at least one side of that balance being broken (that is, having responsibility without authority, or being subject to someone else's authority without responsibility), and a lot of people have been on both, and once you bring it to their attention that they're creating another one there is a certain amount of motivation to avoid it, even if it is a bit more work for a them. You can actually use people's experiences with bad organization to convince them to not create another one, if you are careful and thoughtful.
If your org doesn’t suffer feature creep willingly, I believe that means you can have a lot more flexibility with respect to Conway’s Law. A low-churn project can maintain a common understanding. Not all of them will of course, but it’s at least possible. Feature factories absolutely cannot, and you shouldn’t even try.
What really kills a lot of projects though is an inverted dependency tree. And by inverted I mean that the most volatile code is at the bottom of the call graph, not the top. In this case every build, every deploy, or in the microservices scenario every request, can have different behavior from the previous one because some tertiary dependency changed under you. Now you need all sorts of triggers in your CI/CD pipeline to guarantee that integration tests are rerun constantly to figure out where the regressions are being introduced. Your build pipeline starts to look like one of those server rooms with loose wires everywhere and a bucket under the AC unit. Nothing is stable and everything is on the verge of being on fire at a moment’s notice.
> Any organization that designs a system (defined broadly) will produce _a design whose structure is a copy of the organization's communication structure_.
Organization is the primary method by which we organize communication (indeed, that's its only point). That's why architecture follows organization. It is possible to "break" that boundary by organizing other forms of communication, for example the feature teams of SAFE (not a proponent of safe, just an example).
How often does major reorganization happen? For most companies the answer is never.
The "what if" argument is what leads to all sorts of premature optimizations.
For most companies – regularly.
But the need for them does :)
Yeah, didn't get a lot done...
Exactly. In order to reap the benefits of modular/Microservices architecture, you need teams to be organized around product/feature verticals.
IMO it’s not about the internals of a product/service architecture, but about encapsulating, isolating, and formalizing inter-organizational dependencies.
That will always happen, and I've seen it happen in a monolith too (expressed as module ownership/ownership over part of the codebase).
It's inevitable in almost all organizations.
In the outside world, for an application that may truly need to scale, I'd go MySQL -> Vitess before I'd choose separate data stores for each service with state. But I'd also question if the need to scale that much really exists; you can go a long way even with data heavy applications with a combination of sharding and multitenancy.
Imagine a world where every pip/nuget/cargo package was a k8s service called out-of-process and needed to be independently maintained. We would have potentially hundreds of these things running, independently secured via mTLS, observability and metrics for each, and all calls run out of process. This is the abysmally slow hellscape that some are naively advocating for without realizing it.
It is relatively obvious what should be a library (json, numpy, NLog, etc), and what should be a service (Postgres, Kafka, Redis, NATS, etc.) when dealing with 3rd party components. It is also obvious that team scaling is not determined by whether code exists in a library or service from this example since all 3rd party code is maintained by others.
However, once we are all in the same organization, we lose the ability to correctly make this determination. We create a service when a library would do in order to more strictly enforce ownership. I think an alternate solution to this problem is needed.
Services can be continuously deployed by small owning teams.
Is this correct or are there other aspects that you are considering?
Even if Microservices are better for scale, most companies will never experience the level of scale of 2013 Twitter.
Are Microservices beneficial at much smaller levels of scale? Ex: 1M MAU
I'm not sure at what point the monolith codebase became absolutely enormous, but I would bet that GitHub grew to 1M active users with a monolith just fine.
Micro-services might be necessary for the companies like Github, Twitter, Google that grow to be huge, but monoliths seem to work just fine for the vast majority of other companies.
I fully agree with your argument. Then again, as mentioned elsewhere in this discussion, microservices are often not introduced to solve a scalability problem but an organizational one and there are many organizations that have more engineers than Twitter (had).
Personally, I still don't buy that argument because by solving one organizational problem one risks creating a different one, as this blog post[0] illustrates:
> […] Uber has grown to around 2,200 critical microservices
Unsurprisingly, that same post notes that
> […] in recent years people have begun to decry microservices for their tendency to greatly increase complexity, sometimes making even trivial features difficult to build. […] we experienced these tradeoffs first hand
I'm getting very strong Wingman/Galactus[1] vibes here.
[0]: https://web.archive.org/web/20221105153616/https://www.uber....
[1]: https://www.youtube.com/watch?v=y8OnoxKotPQ
This happens a lot. Organisational problems conflated for technical ones. Technical solutions to organisational politics. Etc. It's often easier to admit technical challenges than organisational ones. Also, different people get to declare technical and organisational challenges, and different people that get to design the solutions.
There's also a dynamic where concepts are created by the technical 1% doing vanguard or challenging tasks. The guys responsible for scaling youtube or whatnot. Their methods and ideas become famous and are then applied to less technically demanding tasks, then in less technical organisations entirely.
I think if we can be honest at the actual problem at hand, 80/20 fixes will emerge. IE, the "real" value is not the architecture per se, but the way it lets you divide the responsibilities in the organisation.
Each team gets to distribute libraries over repos (COM, JAR, DLL, whatever), no way around that unless they feel like hacking binaries.
Creating an object-oriented API for a shared library which encapsulates a module to the degree a service boundary would is not trivial and it's very rarely done, never mind done well. Most OO libraries expose an entire universe of objects and methods to support deep integration scenarios. Maintaining that richness of API over time in the face of many consumers is not easy, and it (versioning) is a dying art in the eternal present of online services. The 90s aren't coming back any time soon.
If you own a library which is used internally, and you add a feature which needs a lot more memory or compute, how do you communicate the need to increase resource allocation to the teams who use the library? How do you even ensure that they upgrade? How do you gather metrics on how your library is used in practice? How do you discover and collect metrics around failure modes at the module boundary level? How do you gather logs emitted by your library, in all the places it's used? What if you add dependencies on other services, which need configuring (network addresses, credentials, whatever) - do you offload the configuration effort on to each of the library users, who need to do it separately, and end up with configuration drift over time?
I don't think binary drops work well unless the logic they encapsulate is architecturally self-contained and predictable; no network access, no database access, no unexpected changes in CPU or memory requirements from version to version.
There's plenty of code like this, but it's not usually the level of module that we consider putting inside a service.
For example, an Excel spreadsheet parser might be a library. But the module which takes Excel files uploaded by the user and streams a subset of the contents into the database is probably better off as a service than a library, so that it can be isolated (security risks), can crash safely without taking everything down, can retry, can create nice logs about hard to parse files, can have resource metrics measured and growth estimated over time, and so on.
As for scalability, most orgs aren't Facebook nor Google scale, regardless of how much they want it to be true.
All of those issues can be tackled, when proper architecture design is done, instead of coding on the go and then will see attitude.
The important part of microservices isn't just API boundaries, it's lifecycle management. This CAN be done with a DLL or JAR, but it's MUCH harder today.
Sure if one likes to make things harder than they are supposed to be, there are lots of paths down that road.
It's invented by a software outsourcing firm to milk billable hours from contracts.
This breaks down when the database is essentially a generic graph. The worst solution I've seen to this is to have another service responsible for generic write operations and any service that wants to write data goes through that service -- you're essentially re-introducing the problem you're purporting to solve at a new layer with an added hop and most likely high network usage. The best solution I've seen is to obviously have the monolith. The enterprise solution I've seen, while not good by any means but nearly essential for promoting team breakdown and ownership, is to just let disparate services write as they see fit, supported by abstraction libraries and SOPs to help reinforce care from engineers.
I’ll go one step further and say that you should treat your data stores as services in and of themselves, with their own well-considered interfaces, and perhaps something like PostgREST as an adapter to the frontend if you don’t really need sophisticated service layers. The read/write pattern you recommend is a great fit for this and can be scaled horizontally with read replicas.
If you have a particular piece of the system that needs to be scaled, you can take that module out when it becomes necessary. You can alter your system such that you deploy the entire codebase but certain APIs are routed to certain boxes, and you can use batch processing patterns with a set of code outside the codebase.
You can have an admin application and a user application, and both of them are monoliths. They may or may not communication using the database or events or APIs.
However, you don't make this on the single bounded context guideline.
And I do agree that most people need less of those than they think.
At my last job, there were quite a few times where being able to scale some small "microservice instance" up from 2 -> 4 instances or 4 -> 8 or 8 -> 12 was a lot easier/quicker than investigating the actual issue. It'd stop production outages/hiccups. It was basically encouraged.
Not sure how that can be done with a "it's all modules in a giant monolith".
> A few ways, the easiest being to scale up the whole monolith with more instance
as far as I know, there's no way to granularly scale up a monolith. if the monolith has 20 or 50 or 100 modules and you need 1 or 2 of them scaled, you have to scale up the entire thing which is huge, expensive, etc.
> Another way is run multiple "services" using the same codebase, so you have workload segmentation, either via synchronous network calls, or async worker systems.
this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
Yes, yet people still do it that way. This is tradeoff against the costs of microservices. I'm not saying it is worth it, but yes, sometimes you can just inefficiently throw hardware resources at scaling problems.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
Because you are incrementally changing an existing monolith, but still getting some of the benefits of scaling distinct workloads independently. If you do this right, it does lend it self to reworking the new "service" to be become its own microservice. Or you can just keep going with the shared codebase.
it brokerages them to subclasses/handlers for CRUD/RPC/plumbing to database/other services
then in a separate process App/main() you have a cron job or a queue of some sort
you bundle these together as a monolith into one main()?
We deploy a few instances of the monolith fronted by a load balancer. There's not much scaling logic here -- when we see p75 request latency reach a certain threshold, we add another monolith instance.
A while later, we realize that the /foo route is getting a ton of spikey traffic, which makes it hard to scale. So, in the load balancer, we set up a new pool that just targets /foo, and in our Maven project we create a new module that will be the deployable for this pool. Now, /foo still requires most of our libs, so the jar we're bundling is still "the monolith" in essence, but maybe it doesn't require a few dependencies, and so is a little slimmer. We deploy that and everything is scaling great!
Then, we discover that in our main monolith, the background threads we were using to send emails are getting really clogged up, which is adversely effecting performance. We decide we want to move this work to an external persistent queue, like Kafka. Now, for our consumer, we create one more Maven module for a new jar. This time, we're lucky. Emails only need 2 of our libraries and so this is actually a pretty small deployable, but it's still being built from the same core set of libraries as the monolith.
my concern would be if you accidentally create bugs by basically creating new mini-monolith flavors as the modules were never expected (in the beginning of their creation) to run in a "partial" context
A cool side-effect is that you can usually run the whole thing in one app for development by just enabling all profiles - as opposed to some microservice architectures where you need dozens of containers and DBs for replicating inter-service bugs.
Microservice scaling is also an inefficient hardware scaling option.
The benefit though is you get a bit of 'free' scaling. Most of the modules don't even have to consider scaling at all, as long as they are growing ~average or slower. So this saves a lot of operational burden for many teams. Conversely with microservices every team has to consider scaling, no matter how simple their service is.
(If you do have 1 module out of 100 that takes up 40% of the resources, then it may be appropriate to split it up. Monolith doesn't literally mean "exactly one service", after all, just a small number of big ones.)
I could be wrong but doesn't monolith usually refer to one really heavy app? As soon as you go from needing 1 instance to 2 (because 1 of the 100 inner "modules" needs to be scaled), I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith? Unless you can scale threads or something instead of the entire process... Or if the entire process doesn't initiate 100 modules as soon as it is started (although I imagine it would in most cases)
No. Backend Server clusters are around almost as long as the internet exists.
>I would guess most of the time there's a high expense (lots of memory) in duplicating the entire monolith?
That's right, but there's a gotcha. Everytime you split your monolith into parts, that doesn't mean that each part will consume 1/n of original monolith startup resource. There will be a floor of memory usage. Consider 1 monolith that eats up 1GB to startup vs 4 microservices that uses 512MB each. Right from start you doubled the wasted memory from 1GB to 2GB. That only gets worse the more services are created. Another problem is that microservice/cloud folks loves to create anemic machines. Usually setting up 512MB RAM and half a core cpu to a service that needs 380mb minimum :) Thats 75% overhead and 25% of working memory. It's bonkers.
not necessarily. yes, you do pay for executable bloat and some memory overhead from loading code into memory, but the argument here is that you can still deploy a "Service B" from the same deployable that only services certain kinds of requests in order to scale parts of the application horizontally.
> this is interesting. a monolith with some form of IPC? why not do microservices at that point? that sounds like microservices-ish?
no, because again, you're still deploying the same single codebase, it's just the instance on the other end of the queue is configured just to consume and process messages.
Look, once upon a time managers designed the system that workers implemented. The factory assembly line, or the policy manual.
But now the workers are CPUs and servers. The managers designing the system the workers follow are coders. I say coders are the new managers.
Now this leaves two problems. The first is that there are a lot of "managers" who are now doing the wrong job. (and explains perfectly why Steve Jobs gets involved in emails about hiring coders.) But that's a different post.
For me the second problem is microservices represent the atomic parts of a business. They should not be seen as a technical solution to anything, and because the first problem (managers arent managing workers anymore) there is no need to have the microservices built along Conways Law.
And so if you want to build a microservice it is not enough to consider the technical needs, it is not enough to automate what was there, you need to design an atomic building block of any / your company. They become the way you talk about the company, the way you think about the processes in a company.
An mostly the company becomes programmable. It is also highly likely the company becomes built mostly of interchangable parts.
This is a misunderstanding of Conway's Law. Your code _will_ reflect the structure of your organization. If you use microservices so will their architecture. If you use modules, so will the modules.
If you want a specific architecture, have your organization reflect the modules/microservices defined in that architecture.
The point ebing is that if workers are CPUs and coders are managers, then why worry about how the managers of the coders are arranged. Get rid of that management layer. Conway is not implying the financiers of the organisation affect the architecture.
This basically means that the microservices a company is built out of should more readily align to the realities of the business model. And instead of shuffling organisations around it would behove executives to look at the microservices.
One can more easily measure orders transferred, etc, if the boundaries are clearer.
Plus conway is just a ... neat idea, not a bound fate.
There is a caveat with the architecture reflecting the organisation of the software teams, but that usually follows the other to-be-automated structure as well.
> Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.
It doesn't matter WHO defines or does the work (the coders themselves in a team or the manager). It matters how communication is arranged. And communication naturally aligns itself along organizational and team boundaries.
When a group/person/whatever writes an API they are MOSTLY writing a document for other human beings. That is an act of communication. If you meet with someone everyday in a stand-up you're likely to put a less structured API between yourselves than if you meet with someone once a month, or once a year. You're also going to create a finer grained subdivision of the API for them as you're likely working on closely related posts of the problem.
Organization creates communication, and communication creates architecture. Therefore organization creates architecture.
> Plus conway is just a ... neat idea, not a bound fate
It is in fact bound fate. Code is written to communicate with others.
People come together, they decide to solve a problem. They divide up that problem and create functions (an API), objects (an API), modules (an API), and services/microservices (an API). At each later they've written something that communicates with others. The structure of that code reflects the necessary communication to the other parties for their part of the problem. This the code is structured following the lines of communication in the organization.
Organization is how we structure communication (that's it's primary point) and thus organization = architecture + some wiggle room for communication paths defined outside that (say a SAFE feature team).
Just to add it here, how evaluations and goals are set matter too. Those tend to be even more constrained by the formal organization than communication.
I've been a couple levels up the chain at times in my career. When I've seen technical decisions at this level done well is when those people realize that their organizational decisions are inevitably technical decisions because of Conway's law. When the impact of Conway's law is not explicitly considered, things will often look random at the IC level. Too often, organizational leaders don't realize they are dictating technical outcomes with decisions that appear to just be about reporting structure.
An example is when a major new product feature comes along that is big enough to need multiple teams to build/own it for the long term. Where are those teams placed in the organization? Conway's law helps us see that a huge part of the future architecture is being determined by answering that question. Placing those new teams in an existing org mainly responsible for UX will result in a very different architecture than putting the new teams in an org mainly responsible for monetization. Can the teams be split across those orgs? Well, can you envision a reasonable architecture with an enforced divide along those lines? Without recognition of Conway's law, that last question might not be considered and the decision to split or not will be based on other factors.
Unfortunately that requires multidisciplinary thinking (technical architecture, resource allocation, communication structure, people development, ...). Such broad thinking is not always easy. When leadership starts neglecting one or more of these disciplines, more and more of the people doing the work on affected teams start perceiving the decisions as arbitrary and counter productive. Organizational churn (e.g., teams repeatedly changing location within the org over the course of a year or two) is often leadership looking at a different subset of these dimensions over time, yielding different "correct" answers.
But if you chnage the dataflow in a few microservices so that the accounts team no longer deals / works directly with the sales team you have chnaged the organisation.
plus it's way easier to monitor activity etc
https://ardalis.com/conways-law-ddd-and-microservices/#:~:te....
Running code in a single process is MUCH lower overhead because you don't need to transit a network layer and you're generally just passing pointers to data around rather than serializing/deserializing it.
There are definitely some cases where using microservices does make things more CPU/memory efficient, but it's much rarer than people think. An example where you'd actually get efficiency would be something like a geofence service (imagine Uber, Doordash, etc.) where the geofence definitions are probably large and have to be stored in memory. Depending on how often geofence queries happen, it might be more efficient to have a small number of geofence service instances with the geofence definitions loaded in memory rather than having this logic as a module that many workers need to have loaded. But again, cases like this are much less common than the cases where services just lead to massive bloat.
I was working at Uber when they started transitioning from monolith to microservices, and pretty much universally splitting logic into microservices required provisioning many more servers and were disastrous for end-to-end latency times.
Servers can only get so big. If your monolith needs more resources than a single server can provide, then you can chop it up into microservices and each microservice can get its own beefy server. Then you can put a load balancer in front of a microservice and run it on N beefy servers.
But this only matters at Facebook scale. I think most devs would be shocked at how much a single beefy server running efficient code can do.
Those machines are made very redundant.
All components in a Parallel Sysplex configuration can operate in redundant mode.
Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters.
That's why corporations that require high availability to not get sued for millions/billions use them.
Geographically Dispersed Parallel Sysplex.
I'm not sure about that: you could still put something together within that budget, say, a few different VPSes across Hetzner and Contabo (or different regions within the same provider's offerings), with some circuit breaking and load balancing between those. Probably either a managed database offering, or a cluster of DBs running on similar VPSes.
Of course, this might mean that you have 1-3 instances of a service instead of 10-30, but as long as availability is the goal and not necessarily throughput, that can go pretty far.
Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
500 USD a month would get you approximately the following resources (taxes vary) on the aforementioned platforms:
I'm not sure about you, but in my experience that could be enough for some pretty decent systems, albeit some storage heavy workloads would need the storage packages instead of the regular VPS ones. It's mostly a matter of picking a suitable topology and working towards your goals.> Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.
This (depending on the circumstances) does sound like a good point! Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
> Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.
This is the point - for-profit corporations, by definition, don't want to waste money, and if "high reliability" isn't required (or they don't know about it), they don't waste money. Most of the time, they don't waste the money.
However, if "being cheap" would means billions in lost income (and they know about it), they really want to have reliable, redundant infrastructure and systems around.
But it leads me back to my original statement - extreme requirements for uptime don't come out of nothing.
If you're in a location where IT related labor is extremely cheap - you're just going to have people keep one server up.
I know I used to do exactly that, because the server was more than my annual income. But that didn't last long. After the first 20 minute downtime, the budget for HA solution was allocated. But before a certain point downtime wasn't expensive.
Non-profits would probably be the only reasonable exception, where HA and low budgets could coincide. Otherwise - nah...
When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems."
When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?"
That allows you to better prioritize fixing those scaling problems.
The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.
Split up services where it makes sense and don't over do it. That's how I design my projects.
So if a database service is not available you simply return stale, cached data until the service is back up.
I can see an advantage regarding resource hogging, but the flip side is the extra point of failure of network calls in microservices.
Not saying which is better, but deployment is orthogonal to logical dependence and correctness.
For instance, in a shopping site, why should a crash in the recommendations engine result in a non-functional webpage (rather than a working purchase page with no recommendations)?
Personally I think microservices start to make sense when you have several hundred developers (an environment I'm currently keen to never enter again - $work has 5 devs and might one day have 6).
I agree that microservices are suitable for large organization, where the organization practically has multiple products (which could be purchased from a vendor or sold to another company).
You're going to have to explain what you mean by that a bit more... You surely cant mean it as it is written.
If the recommendations team write code that causes the OOM-killer to end their process, making them run it on separate infrastructure insulates your "main store team" from the bugs they write.
I imagine his logic was something like: "How can OOMs happen less often if you run more processes (possibly on the same machine)?", while your comment actually wants to say: "if a specific service is affected by an OOM, with microservices only that specific microservice goes down, since it's probably running on its own hardware".
Networks could have unexpected delays, routing errors and other glitches. At least with a monolith you can often find a stacktrace for debugging. I have seen startups that have limited traceability and logging when using micro services.
When a small startup has to manage "scalable" K8s infrastructure in the cloud, distributed tracing and monitoring is often not prioritized when you are a team of 5 developers trying to find a product market fit.
I am not against microservices (I work with them daily) but you just trade one type of stability problem with another
I love working with microservices at the scale of $WORK, but we're Big Tech. I can't imagine why a 5 person startup would want k8s and microservices. You don't need that scale until you have more than 2 teams, and you're pushing at the very least 15 engineers at that point and usually the sales and marketing staff to make that investment worth it.
There is a point when it all starts to make sense. But that point is when you go into billions worth business, hundreds of devs etc. And going there has large cost, especially for small/medium systems. And that cost is not one off - it's a day-to-day cost of introducing changes. It's orders of magnitude chaper and faster (head cound wise) to do changes in ie. single versioned monorepo where everything is deployed at once, as single working, tested, migrated version than doing progressive releases for each piece keeping it all backward compatible at micro level. Again - it does make sense at scale (hundreds of devs kind of scale), but saying your 5 devs team moves faster because they can work on 120 micoservices independently is complete nonsense.
In other words micoservices make sense when you don't really have other options, you have to do it, it's not good start-with default at all; and frankly Sam Newman says it in "Building Microservices" and so do people who know what they're talking about. For some reason juniors want to start there and look at anything non-microservice as legacy.
It sort of is.
It's not a perfect world. One issue with monoliths is that, organizations like to take a "if it ain't broke, don't fix it" attitude towards things. Unfortunately, that leads to spotty service and sometimes expensive deployments. Those aren't always seen as being "broken" but just temporary problems that if you pull enough all nighters you can get through regularly.
It takes a skilled dev to be able to really sell a business on improving monoliths with rework/rewrites of old apis. Even if it saves money, time, all nighters. It's simply hard for a manager to see those improvements as being worth it over the status quo. Especially if the fact of running the monolith on big iron masks the resource usage/outages caused by those APIs.
That often is, because new solitions tend to come with their own problems.
It depends on what you are scaling. I think microservices are fundamentally more scalable for deployment, since changes can be rolled out only to the services that changed, rather than everywhere. Unless your language and runtime support hot-loading individual modules at runtime.
Admittedly, it isn't a scalability problem you will run into right away.
But when you need to roll out an emergency fix, there is a big difference between deploying to thousands of servers that all have everything, and ten servers running a single service.
The solution to this is to use the right tool, a build system that supports monorepos like Bazel. Bazel solves this problem wonderfully. It only builds / tests / containerizes / deploys (rules_k8s, rules_docker) what needs to be rebuilt, retested, recontainerized, and redeployed. Builds are much faster, developers have God like visibility to all of an organizations' code, and can easily grep the entire code base and be assured their changes do not break other modules if Bazel test //... passes. It is language agnostic so you can implement your services in whatever language best suits it. It allows you to more easily manage transitive dependencies, manage versions globally across your org's codebase.
Of course Bazel has a steep learning curve so it will be years before it is adopted as widely as Maven, Gradle etc. But in the banks I've worked at it would've saved them tens of millions of dollars.
Also git would need to catch up to large codebases. I think Meta released a new source control tool recently that is similar to git but could handle large monorepos.
We keep object definitions in a separate repo to not duplicate them and generate a maven dependence from that. Pretty simple actually. We mainly run on Java thought. Would be more complicated when there are multiple languages.
Protobuf or Json schema could come to the rescue if needed.
> I disagree, in my opinion micro-services hinder scalability of deployment
…and of anything related to testing:
- Want to fire up the application in your pipeline to run E2E tests? Congratulations, you must now spin up the entire landscape of microservices in k8s. First, however, you need to figure out which versions of all those microservices you want to test against in the first place, since every service is living in a separate repository and thus getting versioned separately.
- Want to provide test data to your application before running your tests? Well, you're looking at 100 stateful services – good luck with getting the state right everywhere.
Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.
200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :)
I believe that this level of service can no longer accurately be described as "micro".
Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.
So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)
As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.
They want to be both the architect and the manager and anything you say would be over ruled and since they are the boss its hard to ignore them.
This service is a monolith because it has 10K code and it needs to be broken up.The product is at MVP and its rock solid on Java Spring and it hardly crashes. We are never going to lose data based on the design choices we made. None of that matters.
We need zero down time upgrade, when we had zero customers.
Instead, if you build an architecture that can scale the number of corporate users by adding cheap $2k pizza box 1u servers as the company grows, that's much more attractive. Also, you can keep your systems design flexible enough to recompile and take advantage of advancements in hardware tech every 18 months – this gives you better operating margins as your own business starts to grow.
Let alone - AWS Lets you get that machine for less than a junior engineer salary ($7.5 per hour, roughly equivalent to $32 of hourly wage = $67k)
Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.
Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone.
Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.
I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith.
Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.
Uh - what?
>A load balancer balances traffic between servers
Correct.
> of the same service or mononlith
Incorrect.
Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base.
But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts.
The load balancer internally makes these decisions.
I hope this clears up your misunderstanding.
The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.
Are you replying to the right comment? I made no such claim.
The point is, if a client makes a request to a server, the response should always be the same, no matter where the load balancer sends the request to. Which means it should run the same code.
Nginx doesn't even mention route based or endpoint based load balancing in their docs. Maybe they don't consider it load balancing either.
https://www.nginx.com/resources/glossary/load-balancing/
Describes exactly what I’m talking about.
Just because Nginx is doing it, doesn't mean it's load balancing. It's an extra function tacked onto a load balancer.
I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself).
Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity.
Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices.
Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong.
However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere):
Of course, there can be more nuance to it, like modular monoliths, that still have one codebase, but can have certain groups of functionality enabled or disabled. I actually wrote about that approach a while back, calling them "moduliths": https://blog.kronis.dev/articles/modulith-because-we-need-to...I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.
All of the catalog data was moved to a service which only served catalog data so its cache was optimized for catalog data and the load balancers in front of it could optimize across that cache with consistent hashing.
This was different from the front end web tier which used consistent hashing to pin customers to individual webservers.
For stuff like order history or customer data, those services sat in front of their respective databases and provided consistent hashing along with availability (providing a consistent write-through cache in front of the SQL databases used at the time).
I wouldn't call those areas where it makes things more efficient rare, but actually common, and it comes from letting your data dictate your microservices rather than letting your organiation dictate your microservices (although if teams own data, then they should line up).
But I don't have to now.
Cache locality starts making a difference when you have over ten servers, and a significant fraction of the data is cacheable.
For "transactional" Enterprise CRUD apps, often there is very little that can be cached due to consistency concerns.
I've seen many systems where there are a few hundred to a few thousand users, not hundred of millions or billions of users. There can also be teams of 5-10 developers who manages +20 microservices. I still don't think those projects have the same needs and could have done something else.
I'll agree that a team of developers with 20 microservices sounds obviously wrong (which is really my point that the site should be divided around data and the orgs should probably reflect the needs of the data--not the other way around). And Amazon back then didn't have that problem.
Aren't you still writing out to external data stores? If that's the case then its really not a comparison of in process to a RPC, it just two RPC hops now, no?
So instead of a single DB transaction, often with a single SQL roundtrip, microservices can sometimes force through coordination of separate transactions to several DBs.
It might still be completely viable to rewrite something so that it's only 10% as efficient but you can scale it horizontally easily.
On top of that, the tendency to get complacent with unstructured data in a lot of systems is really creating a very complicated lock-in when systems are developed for unique services on each cloud provider... Bowls of spaghetti.
Hire Solutions Architects for dev projects, make your apps future proof... I warn you. Too much microservice customization leads to vendor lock in and expensive operational costs... This is why a lot of apps get sunsetted early.
Been there: how do you handle schema changes?
One of the advantages that having a separate schema per service provides is that services can communicate only via APIs, which decouples them allowing you to deploy them independently, which is at the heart of microservices (and continuous delivery).
The way I see it today: everyone complains about microservices, 12 factor apps, kubernetes, docker, etc., and I agree they are overengineering for small tools, services, etc., but if done right, they offer an agility that monoliths simply can't provide. And today it's really all about moving fast(er) than yesterday.
Our data is mostly append-only, or if it's being changed, there is a theoretical final "correct" version of it that we should converge to. So to "get" data, you subscribe to messages about some state of things, and then each service is responsible for managing its own copy in its own db. This worked well enough until it didn't, and we had to start doing true-ups from time to time to keep things in sync, which was annoying, but not particularly problematic, as we design to assume everything is async and convergent.
The optimization (or compromise) we decided on, was that all of our services use the same db cluster, and that if the db cluster goes down, it means everything is down. Therefore, if we can assume the db is always up, even if a service is down, we consider it an acceptable constraint to provide a readonly view into other services db. Any writes are still sent async via MQ. This eliminates our syncing drifting problem, while still allowing for performant joins, which http apis are bad at and our system uses a lot of.
So then back to your original question, the way that this contract can break is via schema changes. So for us, since we use postgres, we created database views that we expose for reading. And postgres view updates are constrained that they must always be backwards compatible from a schema perspective. So then now our migration path is:
- service A has some table of data that you like to share
- write a migration to expose a view for service A
- write an update for service B to depend upon that view
- service B now needs some more data in that view
- write a db migration for service A that adds that missing data, but keeping the view fully backwards compatible
I don't think I understand. You need to update (and deploy) service B every time you perform a view update (from service A), although it's backward compatible?
If you don't need the new column, then you don't need to do anything on service B, because you know that existing columns on the view won't get removed and their type won't change. You only need to make changes on service B when you want to take advantage of those additions to the view.
But hey, every team and company has to find their strategy to do things. If this works for you, that's great!
It's just not a microservice by definition.
And yes, that is correct, we agree that once we expose a view, we won't remove columns or change types of columns. Theoretically we could effectively deprecate a column by having it just return an empty value. Our use cases are such that changes to such views happen at an acceptable rate, and backwards incompatible changes also happen at an acceptable rate.
Our views are also often joins across multiple tables, or computed values, so even if it's often quite close to the underlying tables, they are intentionally to be used as an abstraction on top of them. The views are designed first from the perspective of, what form of data do others services need?
And the fact you must keep backward compatibility ( see the OP answer above) at the implementation level shows how fragile this approach is - you will never be able to change a database schema as you wish just because you have consumers that rely on the internal details of your implementation - a table. If you want to change a field from char to int, you can't. How is it important for service B to know that level of detail? An API could still expose a domain model as char, if you want to, and maybe introducing new fields, new methods, whatever way. Or maybe nothing at all, maybe it's not necessary because the database field is never exposed but only used internally (!!).
On the other hand, if you expose a database agnostic API (e.g., http, rpc, ... whatever) you can even swap the underlying database and nobody will notice.
A good rule of thumb is: if I change the implementation, do I need to ask/tell another team? If the answer is yes, that is not a microservice.
something like this
CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table;
not sure that the view is an internal structure because it will be exposed via API as it is.
SQL allows to swap the database at least if no specific SQL features are used.
Which proves my point: if I change a field type ( in the table ), I will have to change the view type. I need to change 2 services because one of them changed an implementation detail.
CREATE TABLE the_table (was_char CHAR);
CREATE VIEW table_read_api AS SELECT was_char FROM the_table;
a client consumes data via:
select was_char from table_read_api;
after the change
alter table the_table modify was_char int;
alter table the_table rename column was_char to now_int;
CREATE VIEW table_read_api AS SELECT TO_CHAR(now_int) as was_char FROM the_table;
the client uses the same query
select was_char from table_read_api;
the type of the column has not changed
it is still a char
only "internal implementation" is changed
And any benefit of a microservice owning it's own rDB is still, that schema changes aren't easily reversible. Specially when new, non predefined, data has been flowing in.
Stateless microservices are great, in the sense that you don't have to build multiple versions of APIs... but stateful microservices are just a PITA.
There's a third technical problem that microservices solve, and it's my favorite: isolation. With monoliths, you provide all of your secrets to the whole monolith, and a vulnerability in one module can access any secret available to any other module. Similarly, a bug that takes down one module takes down the whole process (and probably the whole app when you consider cascading failures). In most mainstream languages, every module (even the most unimportant, leaf node on the dependency tree) needs to be scoured for potential security or reliability issues because any module can bring the whole thing down.
This isn't solved at the language level in most mainstream languages. The Erlang family of languages generally address the reliability issue, but most languages punt on it altogether.
> The real reason that microservices may make sense is because they keep people honest around module boundaries.
Agreed. Microservices, like static type systems, are "rails". Most organizations have people who will take shortcuts in the name of expedience, and systems with rails disincentivize these shortcuts (importantly, they don't preclude them).
And for that matter it can (but not necessarily) limit how much damage a bad code release can do.
Of course you don't get those benefits for free just by using microservices, but impleminting those kind if boundaries in a monolith is a lot harder.
Microservices aren't inherently more secure.
Maybe it is possible with some low-level system calls, but at the very least it is a lot more difficult than using separate VMs or containers.
Separate process vaults, HSMs and other techniques of offloading security credentials - are the same for microservice architecture, as "monolith".
The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
How do give one module access to the vault/HSM without also giving any other code in the same process access? Even in the event of a security compromise. And that still doesn't address the problem of a vulnerability anywhere in the monolith potentially exposing any sensitive data in processes memory (such as user data including password).
> The implication that anything other than a microservice architecture must be exclusively an uber executable doing literally everything, is naive.
Ok, replace "microservices" with "service oriented architecture". I'm comparing specifically against a monolith where you have an "uber executable doing literally everything".
I doubt that anyone here is advocating for a system that it literally everything in one OS process. I bet even you would say that a process with a RDMBS connection is a monolith, even though it doesn't fit your definition.
As for modules - they can run as a separate process on the same machine, to isolate security critical elements.
Conversely, I doubt that any reasonable microservices architecture has every instance of a microservice in its own subnet with a firewall and strict network access permissions... including one off generated API keys.
Most, at best, use a static API key per microsoervice inside one large "secure" network... which leads me back to my point - they're not exactly easier to secure. Methods may differ a little bit, but the techniques are the same.
Then your example of a buffer overflow is going to be as bad for microservices, as for monolith.
Unless you're going to invest in a variety of systems, programming languages and OSes in your stack - your one buffer overflow, turns into buffer overflow on every other microservice... making your claim of "easier to isolate" a little bit delusional. (Classic example of Log4J bug, where monolith or microservices - once you're breached "game over")
It is about striking a balance. No reason should something be overly compounded or overly broken up.
I gave a talk [1] about scalability of Java systems on Kubernetes, and one of my recommendations is that Java-based systems - or any system on a runtime similar to the JVM, like CLR or even Go - should be scaled diagonally. Efficiency is the word.
While horizontal scaling can easily address most performance issues and load demand, in most cases it is the least efficient way for Java (and again, I risk saying .NET and Go), as these systems struggle with CPU throttling and garbage collectors.
In short, and exemplifying: one container with 2 CPUs and 2GB of RAM will allow the JVM to perform better, in general, than 2 containers with 1 CPU/1GB RAM each. That said, customers shouldn't be scaling horizontally to any amount more than what is adequately reasonable for resiliency, or unless the bottleneck is somewhere like disk access. For performance on the other hand, customers should be scaling vertically.
And Kubernetes VPA is already available. People just need to use it properly and smartly. If a Kubernetes admin believes a particular pod should double in number of replicas, the admin should consider: "would this microservice benefit even more from 1.5x more CPU/Memory than 2x more replicas?" and I bet to say that, in general, yes.
[1] https://www.youtube.com/watch?v=wApqCjHWF8Q
My default recommendation has always been to make instances "as big as possible, but no bigger". You may need 3 instances for redundancy, fault tolerance and graceful updates but past that you should probably scale up to near the size of your machine. There are obviously lots of complications and exceptions (for example maybe using most 3 instances uses 90% of your machines so you can't bin pack other processes there so it is better to use 4 instances at 70% as the machines will be used more efficiently) but bigger by default is generally a good option.
Exactly.
But I am seeing more and more of the opposite: make instances as small as possible, and scale out (not up) if/when needed.
Making data available to a thread on the same core has exceptionally high bandwidth. Just that alone will help with scaling.
If a team wants to try out a different language, or hosting model, or even just framework/tooling, those things can be really hard to do within the same codebase; much easier when your only contract is handling JSON requests. And if your whole organization is totally locked into a single stack, it's hard to keep evolving on some axes
(I'm generally against microservices, but this is one of the more compelling arguments I've heard for them, though it still wouldn't mean you need to eagerly break things up without a specific reason)
Not all engineers need to move around - DS and MLEs are generally useless on the frontend, and vice versa.
Microservices allow your permissions to be clear and precise. Your database passwords are only loaded into the process that uses them. You can reason about "If there's an RCE in this service, here's what it can do". Trying to tie that to monoliths is hard and ugly.
I would like someone to spell this out. It seems to me people are claiming that if a single binary serves some CPU-bound requests and some memory-bound requests, and you give it more memory, then the memory gets "wasted" on the CPU-bound part. Or if you give it more CPU, the CPU gets wasted on the memory-bound part. But this kind of assignment of resources to code paths seems to be a consequence of microservices. In a single computer, single binary situation resources should not get used up unless the workload actually wants to use them. A compute-heavy thread doesn't cost heap. A big heap doesn't slow down a compute-heavy thread. What am I missing?
My IDE should be able to easily traverse the call graph. My development environment should be simple to setup.
I’ve worked on microservices that required an insane amount of boilerplate to do simple things. Like 7 layers of controllers, clients, services, data services, etc, just to fetch a simple piece of data. And the developer experience of running dozens of services in a Kubernetes cluster running on my dev machine was awful.
Does anything like this exist?
I only dabbled many years ago but Erlang/OTP comes to mind.
And tRPC for TypeScript calls in browser and server.
It turns out that "transparent RPC" is basically a contradiction in terms. As soon as you start doing things across process boundaries, and even more so across network boundaries, it requires a very different approach for API design - something that's very cheap locally, like passing objects by reference, becomes expensive and full of footguns.
If you reduce the feature set to the point where it can be transparently mapped to either local or RPC - which is, basically, function calls processing and returning data organized into arrays & trees (but not graphs) - there's still the issue that RPC has so many more failure points that you have to handle that would never light up in local.
This is all still doable; I have my doubts about practical the end result would be, though.
Totally fine to support a limited set of primitives and collections passed by value, and require developers consider the failure modes.
In my experience HTTP+JSON or gRPC are usually lacking in at least one of those respects. If you constrain all the services to be the same language it becomes easier.
> https://github.com/NathanRSmith/lib-courier-js
Interestingly, we're pursuing a monorepo & multi-monolith setup for the next version of our platform. So lib-courier is no longer necessary to stitch it all together. It was fun while it lasted though. Once you understood the routing algo & code patterns, lots of stuff "just worked".
This experience has strongly impacted my view of microservices and for all personal projects I will develop in the future I will stick with a monolith until much later instead of starting with microservices.
How the hell... like... who decided to do Microservices in the first place if they didn't know this? This is such a rookie mistake. It's like somebody right out of high school just read on a blog the new way is "microservices" and then went ahead with it.
And both are way more common than they should be.
Microservices can have both design utility and simultaneously been a major fad.
Lurking reddit and HN you can watch development fashions come and go. It's really really profoundly hard to hold a sober conversation on splitting merits from limitations in the middle of the boom.
tl;dr: gartner_hype_cycle.png
In a start-up, having a team of young people will allow you to move fast, pivot and deliver. What you usually end up with though, is a tower built from spaghetti that's serving actual users. I see this again and again and then people get surprised that the software doesn't scale.
[1]: https://en.wikipedia.org/wiki/Second-system_effect
But... yeah. A colleague (25+ years) has to deal with this more than I do, but have hit it some over the years, and it happens more the older I get.
Often he or I may be 'right' but only because of existing constraints in a system. Someone with a different view may even be 'correct' in a nuts-and-bolts technical sense. Yes, A is 8% faster than B. But... we have an existing system in B that meets our current needs, and team who know B, have written it, and maintained it for a couple years. Switching tech stacks to something no one knows for an 8% network speed boost is not remotely helpful.
My experience was also that the young manager was very suspicious of the old fart. I've never had any problem working for managers younger than me; but some young managers don't know what to make of a team member that knows more than they do.
[0] No insult intended; almost everything I learned after the first 10 years, I learned from people younger than me.
Not trying to be harsh here, but not expecting an increase of network call in a system where each component is tied together with... network calls sounds a bit naive.
> We are now attempting to solve this with caches and doing batch requests
So you have built a complex and bottle-necked application for the sake of scalability, then having to add caching and batching just to make it perform? That sounds like working backwards.
Obviously, I have no clue on the scale of the project you are working on, but it sure sounds like you could have built it as a monolith in half of the time with orders of magnitude more performance.
Scalability is a feature, you can always add it in the future.
Yes, if you can't scale fast enough as you need to, it can hurt your business. Not being able to keep up with demand is a (luxury) problem that every business faces, not just in tech. They would often be called 'growing pains' in a business, and though they are bad, they rarely contribute to the failure of a company.
Starting a startup/service/platform with microservices before you even understand the bottlenecks/market fit/customers is usually not a good idea. You can come a very, very long way with a monolith before you hit performance and scalability limits. And once you do, you can always start breaking things up into smaller services for scalabity. Obviously you need to make sure you are scaling on time to keep up with demand.
'Nail it, then scale it', and 'premature optimization is the mother of all f-ups' are popular sayings for a reason.
> I was told ~1200 RPCs independently by several engineers at Twitter, which matches # of microservices. The ex-employee is wrong.
> Same app in US takes ~2 secs to refresh (too long), but ~20 secs in India, due to bad batching/verbose comms.
RPCs are on the server side. Why would they app take longer to refresh in India than in the US?
Some more explanation:https://twitter.com/mjg59/status/1592380440346001408
Let each microservice owner figure out how to achieve their latency+reliability SLA's in every location - whether replicating datastores, caching, being stateless, or proxying requests to a master location.
At my last job we created a whole fleet of microservices instead of a single modular project/repo. Some of them required non-trivial dependencies. Some executed pretty long-running tasks or jobs for which network latency is insignificant and will remain so by design. Some were few pages long, some consisted of similar-purpose modules with shared parts factored out. But there was no or little processes like “ah, I’ll just ask M8 and it will ask M11 and it will check auth and refer to a database. I.e. no calls as trivial as foo(bar(baz())) but done all over the infra.
(Because this is at the same time one of the defining elements of this architecture... and the first one to be opted out when you actually start using it "for real").
Lots of ways to solve a problem, though.
The primary reason for this is that PDFs can contain executable code and the common tools used to process them are full of unpatched CVEs.
Imagine if you are responsible, at runtime, for linking object files (.o) where each object is a just the compilation of a function.
Now why would anyone think this is a good idea (as a general solution)? Because in software organizations, the “linker’s” job is supposed to be done by the (“unnecessary weight”) software architect.
Microservices primarily serve as a patch for teams incapable of designing modular schemas. Because designing schemas is not entry level work and as we “know” s/e are “not as effective” after they pass 30 years of age. :)
> monolith
Unless monolith now means not-microservice, then be aware that there are a range of possible architectures between a monolith and microservices.
For this reason, I've been trying to push for building a monolithic app first, then splitting into components, and introducing libs for common functionality. Only when this is all done, you think about the communication patterns and discuss how to scale the app.
Most microservice shops I've been in have instead done the naïve thing; just come up with random functionally separate things and put them in different micro services; "voting service", "login service", "user service" etc. This can come with a very very high price. Not only in terms of network traffic, but also in debuggability, having a high amount of code duplication, and getting locked into the existing architecture, cementing the design and functionality.
And it is reassuring to hear that you seem to have success in avoiding these issues with a monolithic architecture, as I thought I was oldschool for starting to prefer monoliths again.
The main thing is that regardless of scaling, the app should always be able to run/debug/test locally in a monolithic thing.
Once people scale they seem to abandon the need to debug locally at their peril.
Scaling should just be a process of identifying hot function calls and when a flag is set, to execute a call as a network rpc instead.
[1] https://scholar.harvard.edu/waldo/publications/note-distribu...
I agree with sentiment of the paper in that it can never be fully transparent. When I say "it should just be a process of identifying hot function calls..." this is admittedly a little contrived - there would need to be a more thorough analysis.
The main point seems to be that partial failure is too difficult to handle without explicitly declaring some objects as "remote".
> Merging the models by making local computing follow the model of distributed computing would require major changes in implementation languages (or in how those languages are used) and make local computing far more complex than is otherwise necessary
Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
What comes to mind is the event loop. The fact that in Node.js, almost every function call is now an asynchronous promise due to the non-blocking paradigm coupled with the convenience of async/await syntax. These event loop runtimes provide a universal way to hook into asynchronous calls.
I would imagine that in 1994, envisioning the code added to the languages of the day needed to handle timeout and asynchronous invocations would have felt overwhelming.
Also thinking about ORMs and databases. It seems TOPLink for SmallTalk was the first, released in 1995 after this paper was. ORMs seem kind of like "transparent remoting", as they usually can run against an in-memory database or db on a local system, or over an TCP connection, which then needs to deal with latency and failure. And for any app talking to a DB with an ORM, means that the failure cases of the ORM must prevade throughout the stack.
> Given that the paper is from 1994, I'm wondering if there have been language changes that make this more achievable.
As the paper argues, it is not possible to make remote calls look like local ones. But it is trivially possible to make local calls look like remote ones! Use call-by-value parameter passing, and make all call asynchronous. The most successful solution I have heard of is Erlang's immutable-message-passing model.
It has been many years since I used it, but I think TOPLink Session and UnitOfWork interfaces threw exceptions signaling remote execution. This is like Erlang - you work with remote APIs; local execution is a special case.
I first wrote a program which ran on a web server about 25 years ago. In that time, computers have experienced about ten doublings of Moore's law, i.e. are now over a thousand times faster. Computers are very fast if you let them be.
Either way, one of my biggest pet peeves is the near-ubiquitous use of HTTP & JSON in microservice architectures. There's always going to be overhead in networked servies, but this is a place where binary protocols (especially ones like Cap'n Proto) really shine.
I wouldn't call that entirely unexpected. :-) It's a rather well-known issue:
(from https://grugbrain.dev)> especially since some of these services are not even in the same data center.
I think you need to answer why? If you can't put all of the services in one data center, then by definition you can't write a monolith either. If the monolith would happily run in one datacenter, then you should have all instances of your microservices in that one datacenter.
It surprised me that you would conclude that this is a problem with microservices. It's like if a particularly architect always punches you in the groin in every meeting, and you've concluded that architects are bad people, rather than this one architect is a bad person.
Why is the other service in another data center? Does it need to be in another data center? If it does, how will a monolith help?
When a critical vulnerability comes out for whatever language you are using, you now have to patch, test and deploy X apps/repos vs much fewer if they are consolidated repositories written modularly. Same can be said for library/framework upgrades, breaking changes in versions, deprecated features, taking advantage of new features, etc.
Keeping the definition of runtimes as modular as the code can be instrumental in keeping a bunch of related modules/features in one application/repository. One way is with k8s deployments and init params where the app starts specific modules which then lends itself to be scaled differently. I'm sure there are home-grown ways to do this too without k8s.
Why do we care about network latency, when we JUST established that microservices are about scaling large development teams? I have no problem with hackers ranting about slow, bloated and messy software architecture...but this is not the focus of discussion as presented in the article.
And then this conclusion:
> The key is to establish that common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be.
...so, like gRPC over HTTP? Last time I checked, gRPC is pretty well understood from an integration perspective. Much better than Enterprise Java Beans from the last century. Isn't this ironic? And where are the performance considerations for this backplane? Didn't we criticize microservices before because they have substandard performance?
> common architectural backplane with well-understood integration and communication conventions, whatever you want or need it to be
Regardless of the tech used to implement, this paradigm needs to be solved to have a good system. That backplane is not an implementation, but a set of understood guiderails for inter-module communication.
Even with gRPC, both sides need to know what to call and what to provide and expect in response. That's the "conventions" part. Having consistency is more important than the underlying tech. Just simple ReST over HTTP works just as well as gRPC.
Then why is the author even discussing microservices in the first place? Following your logic, they are an implementation detail just like modules.
Yes, the same often happens in microservices, but the extra complexity of a distributed system provides a slightly stronger nudge to decouple that means some teams at the margin do achieve something modular.
I’m something of a skeptic on modular monoliths until we as an industry adopt practices that encourage decoupling more than we currently do.
Yes, in theory they’re the same as microservices, without the distributed complexity, but in practice, microservices provide slightly better friction/incentives to decouple.
But in general,just write some damn code. Presumably you have to write code, because building a software engineering department is one of the most difficult things you can do in order to solve a business problem. Even with the smartest engineers in the world (which you don't have), whatever you ship is inevitably going to end up an overly complex, expensive, bug-riddled maintenance nightmare, no matter what you do. Once you've made the decision to write code, just write the damn code, and plan to replace it every 3-5 years, because it probably will be anyway.
The way I usually describe my preferred heuristic to decide between modules and microservices is:
If you need to deploy the different parts of your software individually, and there’s a cost of opportunity in simply adopting a release train approach, go for microservices.
Otherwise, isolated modules are enough in the vast majority of cases.