Heh, split databases is the thing I think is most problematic and the first thing I would eliminate in most microservices architectures. A huge fraction of the problems of microservices come from trying to split the data model along team structure / service domain rather than the true application / underlying business domain. It doesn't mean you shouldn't have multiple database, just the concept of splitting them arbitrarily along service lines is a huge cause of friction / impedance mismatch / overhead.
I actually like close to a full microservice architecture model, once you allow them all to share the database (possibly through a shared API layer).
What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
You should be able to spin up everything localy in a docker-compose container.
I don't want microservices; I want an executable. Memory is shared directly, and the IDE and compiler know about the whole system by virtue of it being integrated.
in my opinion "you need microservices" peaked around 2018-2019 ... does nowadays someone think that, apart from when you reach certain limits and specific contexts, they are a good idea?
We've removed/merged most of the unnecessary services. The ones left have operational needs to stay separate.
The current hell is x years of undisciplined (in terms of perf and cost) new ORM code being deployed (SQLAlchemy). We do an insane number of read queries per second relative to our usage.
I honestly think the newish devs we have hired don't understand SQL at all. They seem to think of it as some arcane low level thing people used in the 80s/90s.
I would really like to send this article out to all the developers in my small company (only 120+ people, about 40 dev & test) but the political path has been chosen and the new shiny tech has people entranced.
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
IMO, Engineering mindset is a huge challenge when it comes to 'do you do microservices'
And by that, I mean that I have at times seen and/or perhaps even personally used as a cudgel - "This thing has a specific contract and it is implicitly separate and it forces people to remember that if their change needs to touch other parts well then they have to communicate it". In the real world sometimes you need to partition software enough that engineers don't get too far out of the boundaries one way or another (i.e. changes inadvertently breaking something else because they were not focused enough)
I like goldilocks services, as big or as small as actually makes sense for your domain/resource considerations, usually no single http endpoint services in sight.
I feel like this has been beaten to death and this article isn't saying much new. As usual the answer is somewhere in the middle (what the article calls "miniservices"). Ultimately
1. Full-on microservices, i.e. one independent lambda per request type, is a good idea pretty much never. It's a meme that caught on because a few engineers at Netflix did it as a joke that nobody else was in on
2. Full-on monolith, i.e. every developer contributes to the same application code that gets deployed, does work, but you do eventually reach a breaking point as either the code ages and/or the team scales. The difficulty of upgrading core libraries like your ORM, monitoring/alerting, pandas/numpy, etc, or infrastructure like your Java or Python runtime, grows superlinearly with the amount of code, and everything being in one deployed artifact makes partial upgrades either extremely tricky or impossible depending on the language. On the operational and managerial side, deployments and ownership (i.e. "bug happened, who's responsible for fixing?") eventually get way too complex as your organization scales. These are solvable problems though, so it's the best approach if you have a less experienced team.
3. If you're implementing any sort of SoA without having done it before -- you will fuck it up. Maybe I'm just speaking as a cynical veteran now, but IMO lots of orgs have keen but relatively junior staff leading the charge for services and kubernetes and whatnot (for mostly selfish resume-driven development purposes, but that's a separate topic) and end up making critical mistakes. Usually some combination of: multiple services using a shared database; not thinking about API versioning; not properly separating the domains; using shared libraries that end up requiring synchronized upgrades.
There's a lot of service-oriented footguns that are much harder to unwind than mistakes made in a monolithic app, but it's really hard to beat SoA done well with respect to maintainability and operations, in my opinion.
You need multiple services whenever the scaling requirements of two components of your system are significantly different. That's pretty much it. These are often called micro services, but they don't have to actually be "micro"
That's the most nonsensical reason to adopt microservices imo.
Consider this: every API call (or function call) in your application has different scaling requirements. Every LOC in your application has different scaling requirements. What difference does it make whether you scale it all "together" as a monolith or separately? One step further, I'd argue it's better to scale everything together because the total breathing room available to any one function experiencing unusual load is higher than if you deployed everything separately. Not to mention intra- and inter-process comm being much cheaper than network calls.
The "correct" reasons for going microservices are exclusively human -- walling off too much complexity for one person or one team to grapple with. Some hypothetical big brain alien species would draw the line between microservices at completely different levels of complexity.
When people talk about scaling requirements they are not referring to minutiae like "this function needs X CPU per request and this other function needs Y CPU per request", they are talking about whether particular endpoints are primarily constrained by different things (i.e. CPU vs memory vs disk). This is important because if I need to scale up machines for one endpoint that requires X CPU but the same service has another endpoint requiring Y memory whereas my original service only needs Z memory and Y is significantly larger than Z then suddenly you have to pay a bunch of extra money to scale up your CPU-bound endpoint because you are co-hosting it with a memory-bound endpoint.
If all your endpoints just do some different logic and all hit a few redis endpoints, run a few postgres queries, and assemble results then keep them all together!!!
EDIT: my original post even included the phrase "significantly different" to describe when you should split a service!!! It's like you decided to have an argument with someone else you've met in your life, but directed it at me
I think I get your point: e.g. some part of your application absolutely needs the world's fastest NVMe as its local disk. Another part of your application just needs more cores from horizontal scaling. If you horiz scale to get more cores, you're also paying to put the world's fastest NVMe on each new machine.
Eh... I think you can achieve this in a simpler way with asymmetric scaling groups and smartly routing workloads to the right group. I feel like it's an ops/infra problem. There's no reason to constrain the underlying infra to be homogenous.
Operationally, it is very nice to be able to update one discrete function on its own in a patch cycle. You can try to persuade yourself you will pull it off with a modular monolith but the physical isolation of separate services provides guarantees that no amount of testing / review / good intentions can.
However, it's equally an argument for SOA as it is for microservices.
There are some other benefits like having different release cycles for core infrastructure that must never go down vs a service that greatly benefits from a fast pace of iteration.
Literally one function per service though is certainly overkill though unless you're pretty small and trying to avoid managing any servers for your application.
Another case I’ve seen is to separate a specific part of the system which has regulatory or compliance requirements that might be challenging to support for the rest of the larger system, eg HIPAA compliance, PCI compliance, etc.
I'm helping a company get out of legacy hell right now. And instead of saying we need microservices, let's start with just a service oriented architecture. That would be a huge step forward.
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
Service oriented architecture seems like a pretty good idea.
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
I'm curious, and the specific list of problems and pain points (if--big if!--everyone there agrees what they are) can help more clearly guide the decisions as to what the next architecture should look like--SoA, monolithic, and so on.
That's the right approach. This is what the article suggests:
>> For most systems, well-structured modular monoliths (for most common applications, including startups) or SOA (enterprises) deliver comparable scalability and resilience as microservices, without the distributed complexity tax. Alternatively, you may also consider well-sized services (macroservices, or what Gartner proposed as miniservices) instead of tons of microservices.
The other problem is that very very few people actually know how to design a microservice based architecture. I've worked with half a dozen different teams who claim they're building microservices, but when you look at the system it's just a giant distributed monolith. Most of them are people who worked in legacy code bases, and while they like the idea of microservices, they can't let go of those design patterns. So they do the exact same thing but just out everything behind network calls. Drives me absolutely fucking nuts
I found a different benefit to micro services — AI understands them and context matters. Monolithic app confuse ai where micro services enables them to be far more effective.
It's an interesting question how AI influences this. If it scales up the scope of what an individual engineer can do, and if the primary driver of microservice scope is Conway's law, then in theory microservices should get "fatter".
However I go the other way than you: I have found AI needs as much context as possible and that means it understands monoliths (or fatter architectures) better. At least, the agentic style approach where it has access to the whole git tree / source repository. I find things break down a lot when changes are needed across source repositories.
Another good use case for a microservice - if you are going to have to change the compute size for your monolith just to accommodate the new functionality.
I had an architect bemoan the suggestion we use a microservice, until he had to begrudgingly back down when he was told that the function we were talking about (Running a CLIP model) would mean attaching a GPU to every task instance.
There's one thing I've learned about microservices. If you've ever gone down the path of making them, failing and making them again until they all worked as they should with the desired 9's of uptime, then you'll only want to make them if it's really the right thing to make. It's not worth the effort otherwise.
So no I don't want microservices (again), but sometimes it's still the right thing.
52 comments
[ 4.3 ms ] story [ 55.6 ms ] threadLarge, shared database tables have been a huge issue in the last few jobs that I have had, and they are incredibly labor intensive to fix.
I actually like close to a full microservice architecture model, once you allow them all to share the database (possibly through a shared API layer).
What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
You should be able to spin up everything localy in a docker-compose container.
3 tier architecture proves time and time again to be robust for most workloads.
I don't want microservices; I want an executable. Memory is shared directly, and the IDE and compiler know about the whole system by virtue of it being integrated.
The current hell is x years of undisciplined (in terms of perf and cost) new ORM code being deployed (SQLAlchemy). We do an insane number of read queries per second relative to our usage.
I honestly think the newish devs we have hired don't understand SQL at all. They seem to think of it as some arcane low level thing people used in the 80s/90s.
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
And by that, I mean that I have at times seen and/or perhaps even personally used as a cudgel - "This thing has a specific contract and it is implicitly separate and it forces people to remember that if their change needs to touch other parts well then they have to communicate it". In the real world sometimes you need to partition software enough that engineers don't get too far out of the boundaries one way or another (i.e. changes inadvertently breaking something else because they were not focused enough)
1. Full-on microservices, i.e. one independent lambda per request type, is a good idea pretty much never. It's a meme that caught on because a few engineers at Netflix did it as a joke that nobody else was in on
2. Full-on monolith, i.e. every developer contributes to the same application code that gets deployed, does work, but you do eventually reach a breaking point as either the code ages and/or the team scales. The difficulty of upgrading core libraries like your ORM, monitoring/alerting, pandas/numpy, etc, or infrastructure like your Java or Python runtime, grows superlinearly with the amount of code, and everything being in one deployed artifact makes partial upgrades either extremely tricky or impossible depending on the language. On the operational and managerial side, deployments and ownership (i.e. "bug happened, who's responsible for fixing?") eventually get way too complex as your organization scales. These are solvable problems though, so it's the best approach if you have a less experienced team.
3. If you're implementing any sort of SoA without having done it before -- you will fuck it up. Maybe I'm just speaking as a cynical veteran now, but IMO lots of orgs have keen but relatively junior staff leading the charge for services and kubernetes and whatnot (for mostly selfish resume-driven development purposes, but that's a separate topic) and end up making critical mistakes. Usually some combination of: multiple services using a shared database; not thinking about API versioning; not properly separating the domains; using shared libraries that end up requiring synchronized upgrades.
There's a lot of service-oriented footguns that are much harder to unwind than mistakes made in a monolithic app, but it's really hard to beat SoA done well with respect to maintainability and operations, in my opinion.
Consider this: every API call (or function call) in your application has different scaling requirements. Every LOC in your application has different scaling requirements. What difference does it make whether you scale it all "together" as a monolith or separately? One step further, I'd argue it's better to scale everything together because the total breathing room available to any one function experiencing unusual load is higher than if you deployed everything separately. Not to mention intra- and inter-process comm being much cheaper than network calls.
The "correct" reasons for going microservices are exclusively human -- walling off too much complexity for one person or one team to grapple with. Some hypothetical big brain alien species would draw the line between microservices at completely different levels of complexity.
When people talk about scaling requirements they are not referring to minutiae like "this function needs X CPU per request and this other function needs Y CPU per request", they are talking about whether particular endpoints are primarily constrained by different things (i.e. CPU vs memory vs disk). This is important because if I need to scale up machines for one endpoint that requires X CPU but the same service has another endpoint requiring Y memory whereas my original service only needs Z memory and Y is significantly larger than Z then suddenly you have to pay a bunch of extra money to scale up your CPU-bound endpoint because you are co-hosting it with a memory-bound endpoint.
If all your endpoints just do some different logic and all hit a few redis endpoints, run a few postgres queries, and assemble results then keep them all together!!!
EDIT: my original post even included the phrase "significantly different" to describe when you should split a service!!! It's like you decided to have an argument with someone else you've met in your life, but directed it at me
Eh... I think you can achieve this in a simpler way with asymmetric scaling groups and smartly routing workloads to the right group. I feel like it's an ops/infra problem. There's no reason to constrain the underlying infra to be homogenous.
Operationally, it is very nice to be able to update one discrete function on its own in a patch cycle. You can try to persuade yourself you will pull it off with a modular monolith but the physical isolation of separate services provides guarantees that no amount of testing / review / good intentions can.
However, it's equally an argument for SOA as it is for microservices.
Literally one function per service though is certainly overkill though unless you're pretty small and trying to avoid managing any servers for your application.
(To clarify, I’m not disagreeing with you!)
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
[1] https://courses.cs.washington.edu/courses/cse452/23wi/papers... [2] corresponding HN thread from 2011 https://news.ycombinator.com/item?id=3101876
I'm curious, and the specific list of problems and pain points (if--big if!--everyone there agrees what they are) can help more clearly guide the decisions as to what the next architecture should look like--SoA, monolithic, and so on.
>> For most systems, well-structured modular monoliths (for most common applications, including startups) or SOA (enterprises) deliver comparable scalability and resilience as microservices, without the distributed complexity tax. Alternatively, you may also consider well-sized services (macroservices, or what Gartner proposed as miniservices) instead of tons of microservices.
And, now, SAAS is finally making the jump to the last position - hybrid/mini
However I go the other way than you: I have found AI needs as much context as possible and that means it understands monoliths (or fatter architectures) better. At least, the agentic style approach where it has access to the whole git tree / source repository. I find things break down a lot when changes are needed across source repositories.
I was stunned... He comes up with this stuff all the time. Thanks Matt.
I had an architect bemoan the suggestion we use a microservice, until he had to begrudgingly back down when he was told that the function we were talking about (Running a CLIP model) would mean attaching a GPU to every task instance.
I want just services.
So no I don't want microservices (again), but sometimes it's still the right thing.