I am not sure this is true. Complexity is a function of architecture. Scalability can be achieved by abstraction, it doesn't necessarily imply highly coupled architecture, in fact scalability benefits from decoupling as much as possible, which effectively reduces complexity.
If you have a simple job to do that fits in an AWS Lambda, why not deploy it that way, scalability is essentially free. But the real advantage is that by writing it as a Lambda you are forced to think of it in stateless terms. On the other hand if suddenly it needs to coordinate with 50 other Lambdas or services, then you have complexity -- usually scalability will suffer in this case, as things become more and more synchronous and interdependent.
> The monolith is composed of separate modules (modules which all run together in the same process).
It's of course great to have a modular architecture, but whether or not they run in the same process should be an implementation detail. Barriers should be explicit. By writing it all depending on local, synchronous, same-process logic, you are likely building in all sorts of implicit barriers that will become hidden dangers when suddenly you do need to scale. And by the way that's one of the reasons we think about scaling in advance, is that when the need comes, it comes quickly.
It's not that you should scale early. But if you're designing a system architecture, I think it's better to think about scaling, not because you need it, but because doing so forces you to modularize, decouple, and make synchronization barriers explicit. If done correctly, this will lead to a better, more robust system even when it's small.
Just like premature optimization -- it's better not to get caught up doing it too early, but you still want to design your system so that you'll be able to do it later when needed, because that time will come, and the opportunity to start over is not going to come as easily as you might imagine.
I've seen startups killed because of one or two "influential" programmers deciding they need to start architecturing the project for 1000TPS and 10K daily users, as "that's the proper way to build scalable software", while the project itself hasn't even found product-market fit yet and barely has users. Inevitably, the project needs to make a drastic change which now is so painful to do because it no longer fits the perfect vision the lead(s) had.
Cue programmers blaming the product team for "always changing their mind" as they discover what users actually need, and the product team blaming developers for being hesitant to do changes, and when programmers agree, it takes a long time to undo the perfect architecture they've spent weeks fine-tuning against some imaginary future user-base.
Something that does not scale to 10k users is likely so badly architected, it would be faster to iterate on it if it was more scalable hence better architected and more maintainable.
You simply can't get the software or support for a lot of smaller solutions. It can be sometimes easier to do the seemingly more difficult thing, and sometimes because all the money goes to those more difficult seeming technical problems and solutions.
I was part of a small team that built a $300M company on Ruby and MySQL that made every scaling mistake you can possibly make. This was also the right decision because it forced us to stay lean and focus on what we needed right now, as opposed to getting starry-eyed about what it was going to be like when we had 10 million users. At every order of magnitude, we had sudden emergencies where some new part of the system had become a bottleneck, and we scrambled like crazy to rearchitect things to accommodate. It was hard, and it was fun. And it was frugal. We eventually hit over 10 million users before I left, and I can’t say I regret the painful approach one bit.
On the flip side, I've seen a project fail because it was built on the unvalidated assumption that the naive architecture would scale to real world loads only to find that a modest real world workload was exceeding targets by a factor of 100X. You really do need technical leadership with good judgment and experience; we can't substitute it with facile "assume low scale" or "assume large scale" axioms.
Ugh, there is just something so satisfying about developer cynicism. It gives me that warm, fuzzy feeling.
I basically agree with most of what the author is saying here, and I think that my feeling is that most developers are at least aware that they should resist technical self-pleasure in pursuit of making sure the business/product they're attached to is actually performing. Are there really people out there who still reach for Meta-scale by default? Who start with microservices?
I don't buy the idea that people mainly reach for microservices for scalability or "pleasure" reasons though.
I personally reach for it to outsource some problems by using off the shelf solutions. I don't want to reinvent the wheel. And if everyone else is doing it in a certain way I want to do it in the same way to try to stand on the shoulders of giants and not reinvent everything.
I needed to build an internal admin console, not super-scalable, just a handful of business users to start. The SQL database it would access was on-premises, but might move to the cloud in future. Authorized users needed single sign-on to their Azure-based active directory accounts for login. I wanted to do tracing of user requests with OpenTelemetry or something like.
At this point in my career, why wouldn't I reach for microservices to supply the endpoints that my frontend calls out to? Microservices are straightforward to implement with NodeJS (or any other language, for that matter.) I get very straightforward tracing and Azure SSO support in NodeJS. For my admin console, I figured I would need one backend-for-frontend microservice that the frontend would connect to and a domain service for each domain that needed to be represented (with only one domain to start). We picked server technologies and frameworks that could easily port to the cloud.
So two microservices to implement a secure admin console from scratch, is that too many? I guess I lack the imagination to do the project differently. I do enjoy the "API First" approach and the way it lets me engage meaningfully with the business folks to come up with a design before we write any code. I like how it's easy to unit/functional test with microservices, very tidy.
Perhaps what makes a lot/most of microservice development so gross is misguided architectural and deployment goals. Like, having a server/cluster per deployed service is insane. I deploy all of my services monolithically until a service has some unique security or scaling needs that require it to separate from the others.
Similarly, it seems common for microservices teams to keep multiple git repos, one for each service. Why?! Some strange separation-of-concerns/purity ideals. Code reuse, testing, pull requests, and atomic releases suffer needless friction unless everything is kept in a monorepo, as the OP implied.
Also, when teams build microservices in such a way that services must call other services completely misses the point of services - that's just creating a distributed monolith (slow!)
I made a rule on my team that the only service type that can call another service is aggregation services like my backend-for-frontend which could launch downstream calls in parallel and aggregate the results for the caller. This made the architecture very flat with the minimum number of network hops and with as much parallelism as possible so it would stay performant. Domain services owned their data sources, no drama with backend data.
I see a lot of distributed monolith drama and abuse of NoSQL data sources giving microservices a bad reputation.
I don't get this scalability craze either. Computers are stupid fast these days and unless you are doing something silly, it's difficult to run into CPU speed limitations.
I've been running a SaaS for 10 years now. Initially on a single server, after a couple of years moved to a distributed database (RethinkDB) and a 3-server setup, not for "scalability" but to get redundancy and prevent data loss. Haven't felt a need for more servers yet. No microservices, no Kubernetes, no AWS, just plain bare-metal servers managed through ansible.
I guess things look different if you're using somebody else's money.
I ran a SaaS for 10 years. Two products. Profitable from day 1 as customers paid $500/month and it ran on a couple of EC2 instances as well as a small RDS database.
Another thing one has to consider is the market size and timeframe window of your SaaS. No sense in building for scalability if the business opportunity is only 100 customers and only for a few years.
It's not about scalability. It's about copying what the leaders in a space do, regardless of whether it makes sense or not. It's pervasive in most areas of life.
Honestly, in my experience, the only good reason to have microservices in a "software solution" is to be able to match 1 service -> 1 mantainer/team and have a big (read "nested", with multiple level of middle-managers) group of teams, each that may have different goals.
In this way it's very easy to "map" a manager/team to a "place" in the solution map, with very explicit and documented interactions between them
ThoughtWorks gathers this phenomenon under the term "envy": Web Scale envy [1] or Big Data envy [2] are two relevant blips on their technology radar. It is typically better to keep things simple.
Many startup business models have no chance of becoming profitable unless they reach a certain scale, but they might have less than 1% probability of reaching that scale. Making it scalable is easy work since it is deterministic, but growing customers is not.
Another perspective is that the defacto purpose of startups (and projects at random companies) may actually be work experience and rehearsal for the day the founders and friends get to interview at an actual FAANG.
I think the author's “dress for the job you want, not the job you have” nails it.
> Making it scalable is easy work since it is deterministic
I don't think that necessarily follows. Especially the language choice is almost impossible to change - look at Facebook, Dropbox, etc. Facebook ended up creating an entirely new language that only they use, because it was impossible to rewrite in another language.
Language choice (and probably database choice too) are essentially locked in from the start, and they do affect scaling.
Growing customers is probably harder, but I don't buy "do everything in hacky Bash scripts because you can fix it later". Nor do I think having solid foundations means you need to be less agile. Would Dropbox have been less successful if they wrote their backend in Typescript? I doubt it.
I think it was around 2015 when everything was basically AWS and Kubernetes
The turning point might have been Heroku? Prior to Heroku, I think people just assumed you deploy to a VPS. Heroku taught people to stop thinking about the production environment so much.
I think people were so inspired by it and wanted to mimic it for other languages. It got more people curios about AWS.
Ironically, while the point of Heroku was to make deployment easy and done with a single command, the modern deployment story on cloud infrastructure is so complicated most teams need to hold a one hour meeting with several developers "hands on deck" and going through a very manual process.
So it might seem counter intuitive to suggest that the trend was started by Heroku, because the result is the exact opposite of the inspiration.
I can relate - running a small hosting business. People come up with too complex solutions. They solve problems that they'd wish to have. For instance: HA setups are complex. If not done correctly, like in most cases, people don't gain the additional '9' from the SLA.
This piece is written with a pretty cliche dismissive tone that assumes that everything everyone else does is driven by cargo-culting if not outright ignorance. That people make these choices because they're just rushing to chase the latest trend.
They're just trying to be cool, you see.
Here's the thing, though: Almost every choice that leads to scalability also leads to reliability. These two patterns are effectively interchangeable. Having your infra costs be "$100 per month" (a claim that usually comes with a massive disclaimer, as an aside) but then falling over for a day because your DB server crashed is a really, really bad place to be.
> Almost every choice that leads to scalability also leads to reliability.
Yes, dealing with skew for every single change and hunting down bugs across network boundaries that could have been a function call is peak reliability.
I had a client with a system just like this. EBS, S3, RDS, Cognito, the lot. It cost $00s per month under almost no load, and was a maintenance nightmare - which was the real problem, not the cost, as it stopped working altogether eventually. A bit of hacking later, it all fits on a single VM that costs ~$10/month to run and is far easier to build, deploy and maintain.
> The first problem every startup solves is scalability. The first problem every startup should solve is “how do we have enough money to not go bust in two months”, but that’s a hard problem, whereas scalability is trivially solvable by reading a few engineering blogs, and anyway it’s not like anyone will ever call you out on it, since you’ll go bust in two months.
I laughed. I cried. Having a back full of microservices scars, I can attest that everything said here is true. Just build an effin monolith and get it done.
Break your code into modules/components that have a defined interface between them. That interface only passes data - not code with behaviour - and signal the method calls may fail to complete ( ie throw exceptions ).
ie the interface could be a network call in the future.
Allow easy swapping of interface implementations by passing them into constructors/ using factories or dependency injection frameworks if you must.
That's it - you can then start with everything in-process and the rapid development that allows, but if you need to you can add splitting into networked microservices - any complexity that arises from the network aspect is hidden behind the proxy, with the ultimate escape hatch of the exception.
Not sure I understand. What is a distributed monolith?
I'm not suggesting that the distributed bit is still coupled behind the scenes ( ie via a data backend that requires distributed transactions ) - the interaction is through the interface.
In the end you are always going to have code calling code - the key point is to assume these key calls are simply data passing, not behaviour passing, and that they can fail.
What else is need to make something network friendly? ( I'm suggesting that things like retries, load-balancing etc can be hidden as a detail in the network implementation - all you need to surface is succeed or fail ).
the swap from interface to network call is still non-trivial.
you get to have new problems that are qualitatively different from before like timeouts, which can break the adsumptions in the rest of your code about say, whether state was updated or not, and in what order. you also then get to deal with thundering herds and circuit breakers and so on.
Sure is more complex - but as I said key thing is to define those interfaces in a way that can be networked - you are just passing data not behaviour and the calls could fail to complete.
In terms of timing the call is synchronous and either succeeds or fails - the details like timeouts/ asynch underhood etc are hidden by the proxy - in the end the call succeeds or fails and if you surface that as a synchronous call you hide the underlying complexity from the caller.
A bit like opening a file and writing to it - most platform apis throw exceptions - and your code has to deal with it.
Just to be honest for a bit here... we also should be asking what kind of scale?
Quite a while ago, before containers were a thing at all, I did systems for some very large porn companies. They were doing streaming video at scale before most, and the only other people working on video at that scale were Youtube.
The general setup for the largest players in that space was haproxy in front of nginx in front of several PHP servers in front of a MySQL database that had one primary r/w with one read only replica. Storage (at that time) was usually done with glusterfs. This was scalable enough at the time for hundreds of thousands of concurrent users, though the video quality was quite a bit lower than what people expect today.
Today at AWS, it is easily possible for people to spend a multiple of the cost of that hardware setup every month for far less compute power and storage.
Around 2013 I was handling bursts up to thousands of requests per second for multi-megabyte file downloads with dynamic authentication using just PHP5, Apache2, and Haproxy, with single-node MySQL (or may have been MariaDB, by then?) as the database, and Redis for caching. On a single mid-range rented server. And Haproxy was only there for operational convenience, you could cut it out and it'd work just as well. No CDN. Rock solid.
My joke but not-actually-a-joke is that the Cloud is where you send a workload that's fast on your laptop, if you need it to be way slower. The performance of these fussy, over-complicated, hard-to-administer[1] systems is truly awful for the price.
[1] They're hypothetically simpler and easier to administer, but I've never seen this in the wild. If anything, we always seem to end up with more hours dedicated to care & feeding of this crap, and more glitchiness and failures, than we would with a handful of rented servers with maybe a CDN in front.
I've seen an application, a 95% CRUD application, which had about 100-1000 users across the UK, users who would only be using it from 9am-5:30pm, and at that - barely interacting with it. This was backed by literally the most sophisticated and complex architecture I have ever seen in my entire life.
There were 3 instances of cognito. RDS, DynamoDB and S3. The entire architecture diagram would only be legible on an A2 (heck, maybe even A1) page. And that was the high level diagram. The central A4 part of that diagram was a bunch of micro-services for handling different portions of this CRUD application.
This company could afford a system architect as well as a team of developers to work on this full time.
I was genuinely baffled, but this company was in an extremely lucrative industry, so I guess in this case it's fine to just take some of your profits and burn them.
Exactly.. it was a lot different when a typical server was 2-4 CPUs and costs more than a luxury car... today you get hundreds of simultaneous threads and upwards of a terabyte of ram for even less, not counting inflation.
You can go a very, very, very long way on 2-3 modern servers with a fast internet connection and a good backup strategy.
Even with a traditional RDBMS like MS-SQL/PostgreSQL, you aren't bottlenecked by the 1-2ghz cpu and spinning rust hard drives. You can easily get to millions of users for a typical site/app with a couple servers just for a read replica/redundancy. As much as I happen to like some of the ergonomics of Mongo from a developer standpoint, or appreciate the scale of Cassandra/.ScyllaDB or even Cockroach... it's just not always necessary early on, or ever.
I've historically been more than happy to reach for RabbitMQ or Redis when you need queueing or caching... but that's still so much simpler than where some microservice architectures have gone. And while I appreciate what Apollo and GraphQL bring to the table, it's over the top for the vast majority of applications.
I’m as likely to talk about human scale as hardware scale, and one of the big issues with human scale is what the consequences are of having the wrong team size in either direction.
When you reduce the man hours per customer you can get farther down your backlog. You can carve people off for new prospective business units. You can absorb the effects of a huge sale or bad press better because you aren’t trying to violate Brooks’ Law nor doing giant layoffs that screw your business numbers.
You have time for people to speculate on big features or more work on reducing the costs further. If you don’t tackle this work early you end up in the armed Queen Problem: running as fast as you can just to stay still.
I have been advocating similar setups to this for years, no one wants to hear it. I get downright puzzled looks from people who think you need moon-landing monitorability of every little service and database...
We have 100k paying users and most of them don't actively use our service, yet we have microservice slop with k8 and load balancers galore running in big tech cloud provider and document databases.
From our whole setup I can only think of _maybe_ one thing that should be split into a separate dedicated service, everything else could happily live in a monolith backed by a relational database and a caching layer and a storage solution.
Preach. Architects need to get back to base principles, stop reading so many blog posts, start writing more code, and actually try things for themselves. Almost every system I work on is architecturally obese and while it "could scale" it has never and will realistically never need to. The first thing I'd do to claw back performance (and productivity) in most instances is make the system less distributed.
One start up I worked at we had 2 Kubernetes clusters and a rat's nest of microservices for an internal tool that, had we been actually successful at delivering sufficient value would have been used by at most a 100 employees (and those would unlikely be concurrent). And this was an extremely highly valued company at the time.
Another place I worked at we were paying for 2 dev ops engineers (and those guys don't come cheap) to maintain our deployment cluster for 3 apps which each had a single customer (with a handful of users). This whole operation had like 20 people and an engineering team of 8.
I work at a place with 8 k8s clusters. We needed to evolve from generation 2 to generation 3 because of "manageability" or something. Gen 3 needed two clusters instead of one. Now we have 8 * (1 + 2) = 24 clusters.
Are they aware that namespaces exist? Surely you're getting more out of your clusters by having few clusters running many pods instead of many clusters running few pods?
At my job we also have some redundant clusters but that's because we're in the middle of a transition (really two transitions, the first of which was never completed), of the 10 clusters that fall under my responsibility 6 will hopefully be gone by the end of this year.
If you’ve ever been in a situation where you do suddenly face scale and have to rip apart a legacy monolith that was built without scale in mind, you’ll chuckle at this article. It’s extremely painful.
I'm not saying "you're never going to scale", I'm saying "you aren't going to scale right now". There's a happy medium between overengineering everything and making everything so shoddily that you can never grow.
Scale articles are too focused on architecture. What about business problems that come with scale. At a certain scale rare events are common many cases cease to be fixable by some random process that involves humans you have to handle a lot more business scenarios with your code.
I read this, and have the opposite experience. Your monolith will fester as developers step on each others toes. You aren’t solving for scalability, you’re solving for sovereignty. Giving other teams the ability to develop their own service without needing to conform to your archaic grey beard architecture restrictions and your lack of understanding what a pod is or how to get your logs from your cloud.
No, this whole article reads like someone who is crying that they no longer have their AS/200. Bye. The reason people use AWS and all those 3rd party is so they don’t have to reinvent the wheel which this author seems hell bent on.
Why are we using TCP when a Unix file is fine… why are we using databases when a directory and files is fine? Why are we scaling when we aren’t Google when my single machine can serve a webpage? Why am I getting paid to be an engineer while eschewing all the things that we have advanced over the last two decades?
Yeah, these are not the right questions. The real question should be: “Now that we have scale what are we gonna do with it?”
Comes down to knowing when to stop. You don’t really want to DIY your own orchestrator etc. So better off just using kubernetes. But then not going too far down that rabbit hole.
ie yes kubernetes but the simplest vanilla version of it you can manage
Simplest version of Kubernetes is zero Kubernetes. You can instead run your service using a process manager like PM2 or similar. I think even using Docker is overkill for a lot of small teams.
Frontend folks weren’t happy with how simple things were, so after seeing microservices, they invented microfrontends, and balance in complexity restored.
> The first problem every startup should solve is “how do we have enough money to not go bust in two months”, but that’s a hard problem, whereas scalability is trivially solvable by reading a few engineering blogs [...] Do you know what the difference between Google and your startup is? It’s definitely not scalability, you’ve solved that problem. It’s that Google has billions upon billions with which to pay for that scalability, which is really good because scalability is expensive.
Too true. Now that I've stepped into an "engineering leadership" role and spend as much time looking at finances as I do at code, I've formed the opinion that in 99.999% of cases, engineering problems are really business problems. If you could throw infinite time and money at the technical challenges, they'd no longer be challenging. But businesses, especially startups, don't have infinite (or even "some") money and time, so the challenge is doing the best engineering work you can, given time and budget constraints.
> The downsides [of the monolith approach]
I like the article's suggestion of using explicitly defined API boundaries between modules, and that's a good approach for a monolith. However one massive downside that cannot be ignored -- by having a single monolith you now have an implicit dependency on the same runtime working on all parts of your code. What I mean by this is, all your code is going to share the same Python version and same libraries (particularly true in Python, where it's not a common/well-supported use case to have multiple versions of library dependencies). This means that if you're working on Module A, and you realize you need a new feature from Pandas 2.x, but the rest of the code is on Pandas 1.x... well, you can't upgrade unless you go and fix Modules B, C, D ... Z to work with Pandas 2.
This won't be an issue at the start, but it's worth pointing out. Being forced to upgrade a core library or language runtime and finding out it's a multi-month disruptive project can be brutal.
> by having a single monolith you now have an implicit dependency on the same runtime working on all parts of your code
I think many people would not consider what I am going to suggest a monolith, but docker compose all hosted in the same server acts a lot like a monolith.
> particularly true in Python, where it's not a common/well-supported use case to have multiple versions of library dependencies
You can use virtual environments each with its python version and packages.
114 comments
[ 4.0 ms ] story [ 84.3 ms ] threadI am not sure this is true. Complexity is a function of architecture. Scalability can be achieved by abstraction, it doesn't necessarily imply highly coupled architecture, in fact scalability benefits from decoupling as much as possible, which effectively reduces complexity.
If you have a simple job to do that fits in an AWS Lambda, why not deploy it that way, scalability is essentially free. But the real advantage is that by writing it as a Lambda you are forced to think of it in stateless terms. On the other hand if suddenly it needs to coordinate with 50 other Lambdas or services, then you have complexity -- usually scalability will suffer in this case, as things become more and more synchronous and interdependent.
> The monolith is composed of separate modules (modules which all run together in the same process).
It's of course great to have a modular architecture, but whether or not they run in the same process should be an implementation detail. Barriers should be explicit. By writing it all depending on local, synchronous, same-process logic, you are likely building in all sorts of implicit barriers that will become hidden dangers when suddenly you do need to scale. And by the way that's one of the reasons we think about scaling in advance, is that when the need comes, it comes quickly.
It's not that you should scale early. But if you're designing a system architecture, I think it's better to think about scaling, not because you need it, but because doing so forces you to modularize, decouple, and make synchronization barriers explicit. If done correctly, this will lead to a better, more robust system even when it's small.
Just like premature optimization -- it's better not to get caught up doing it too early, but you still want to design your system so that you'll be able to do it later when needed, because that time will come, and the opportunity to start over is not going to come as easily as you might imagine.
Cue programmers blaming the product team for "always changing their mind" as they discover what users actually need, and the product team blaming developers for being hesitant to do changes, and when programmers agree, it takes a long time to undo the perfect architecture they've spent weeks fine-tuning against some imaginary future user-base.
I basically agree with most of what the author is saying here, and I think that my feeling is that most developers are at least aware that they should resist technical self-pleasure in pursuit of making sure the business/product they're attached to is actually performing. Are there really people out there who still reach for Meta-scale by default? Who start with microservices?
I personally reach for it to outsource some problems by using off the shelf solutions. I don't want to reinvent the wheel. And if everyone else is doing it in a certain way I want to do it in the same way to try to stand on the shoulders of giants and not reinvent everything.
But that's probably the wrong approach then...
At this point in my career, why wouldn't I reach for microservices to supply the endpoints that my frontend calls out to? Microservices are straightforward to implement with NodeJS (or any other language, for that matter.) I get very straightforward tracing and Azure SSO support in NodeJS. For my admin console, I figured I would need one backend-for-frontend microservice that the frontend would connect to and a domain service for each domain that needed to be represented (with only one domain to start). We picked server technologies and frameworks that could easily port to the cloud.
So two microservices to implement a secure admin console from scratch, is that too many? I guess I lack the imagination to do the project differently. I do enjoy the "API First" approach and the way it lets me engage meaningfully with the business folks to come up with a design before we write any code. I like how it's easy to unit/functional test with microservices, very tidy.
Perhaps what makes a lot/most of microservice development so gross is misguided architectural and deployment goals. Like, having a server/cluster per deployed service is insane. I deploy all of my services monolithically until a service has some unique security or scaling needs that require it to separate from the others.
Similarly, it seems common for microservices teams to keep multiple git repos, one for each service. Why?! Some strange separation-of-concerns/purity ideals. Code reuse, testing, pull requests, and atomic releases suffer needless friction unless everything is kept in a monorepo, as the OP implied.
Also, when teams build microservices in such a way that services must call other services completely misses the point of services - that's just creating a distributed monolith (slow!)
I made a rule on my team that the only service type that can call another service is aggregation services like my backend-for-frontend which could launch downstream calls in parallel and aggregate the results for the caller. This made the architecture very flat with the minimum number of network hops and with as much parallelism as possible so it would stay performant. Domain services owned their data sources, no drama with backend data.
I see a lot of distributed monolith drama and abuse of NoSQL data sources giving microservices a bad reputation.
I've been running a SaaS for 10 years now. Initially on a single server, after a couple of years moved to a distributed database (RethinkDB) and a 3-server setup, not for "scalability" but to get redundancy and prevent data loss. Haven't felt a need for more servers yet. No microservices, no Kubernetes, no AWS, just plain bare-metal servers managed through ansible.
I guess things look different if you're using somebody else's money.
Another thing one has to consider is the market size and timeframe window of your SaaS. No sense in building for scalability if the business opportunity is only 100 customers and only for a few years.
[1] https://www.thoughtworks.com/radar/techniques/high-performan...
[2] https://www.thoughtworks.com/radar/techniques/big-data-envy
Another perspective is that the defacto purpose of startups (and projects at random companies) may actually be work experience and rehearsal for the day the founders and friends get to interview at an actual FAANG.
I think the author's “dress for the job you want, not the job you have” nails it.
I don't think that necessarily follows. Especially the language choice is almost impossible to change - look at Facebook, Dropbox, etc. Facebook ended up creating an entirely new language that only they use, because it was impossible to rewrite in another language.
Language choice (and probably database choice too) are essentially locked in from the start, and they do affect scaling.
Growing customers is probably harder, but I don't buy "do everything in hacky Bash scripts because you can fix it later". Nor do I think having solid foundations means you need to be less agile. Would Dropbox have been less successful if they wrote their backend in Typescript? I doubt it.
The turning point might have been Heroku? Prior to Heroku, I think people just assumed you deploy to a VPS. Heroku taught people to stop thinking about the production environment so much.
I think people were so inspired by it and wanted to mimic it for other languages. It got more people curios about AWS.
Ironically, while the point of Heroku was to make deployment easy and done with a single command, the modern deployment story on cloud infrastructure is so complicated most teams need to hold a one hour meeting with several developers "hands on deck" and going through a very manual process.
So it might seem counter intuitive to suggest that the trend was started by Heroku, because the result is the exact opposite of the inspiration.
They're just trying to be cool, you see.
Here's the thing, though: Almost every choice that leads to scalability also leads to reliability. These two patterns are effectively interchangeable. Having your infra costs be "$100 per month" (a claim that usually comes with a massive disclaimer, as an aside) but then falling over for a day because your DB server crashed is a really, really bad place to be.
Yes, dealing with skew for every single change and hunting down bugs across network boundaries that could have been a function call is peak reliability.
- scaling vertically is cheaper to develop
- scaling horizontally gets you further.
What is correct for your situation depends on your human, financial and time resources.
I laughed. I cried. Having a back full of microservices scars, I can attest that everything said here is true. Just build an effin monolith and get it done.
Break your code into modules/components that have a defined interface between them. That interface only passes data - not code with behaviour - and signal the method calls may fail to complete ( ie throw exceptions ).
ie the interface could be a network call in the future.
Allow easy swapping of interface implementations by passing them into constructors/ using factories or dependency injection frameworks if you must.
That's it - you can then start with everything in-process and the rapid development that allows, but if you need to you can add splitting into networked microservices - any complexity that arises from the network aspect is hidden behind the proxy, with the ultimate escape hatch of the exception.
Have I missed something?
I'm not suggesting that the distributed bit is still coupled behind the scenes ( ie via a data backend that requires distributed transactions ) - the interaction is through the interface.
In the end you are always going to have code calling code - the key point is to assume these key calls are simply data passing, not behaviour passing, and that they can fail.
What else is need to make something network friendly? ( I'm suggesting that things like retries, load-balancing etc can be hidden as a detail in the network implementation - all you need to surface is succeed or fail ).
you get to have new problems that are qualitatively different from before like timeouts, which can break the adsumptions in the rest of your code about say, whether state was updated or not, and in what order. you also then get to deal with thundering herds and circuit breakers and so on.
In terms of timing the call is synchronous and either succeeds or fails - the details like timeouts/ asynch underhood etc are hidden by the proxy - in the end the call succeeds or fails and if you surface that as a synchronous call you hide the underlying complexity from the caller.
A bit like opening a file and writing to it - most platform apis throw exceptions - and your code has to deal with it.
Quite a while ago, before containers were a thing at all, I did systems for some very large porn companies. They were doing streaming video at scale before most, and the only other people working on video at that scale were Youtube.
The general setup for the largest players in that space was haproxy in front of nginx in front of several PHP servers in front of a MySQL database that had one primary r/w with one read only replica. Storage (at that time) was usually done with glusterfs. This was scalable enough at the time for hundreds of thousands of concurrent users, though the video quality was quite a bit lower than what people expect today.
Today at AWS, it is easily possible for people to spend a multiple of the cost of that hardware setup every month for far less compute power and storage.
My joke but not-actually-a-joke is that the Cloud is where you send a workload that's fast on your laptop, if you need it to be way slower. The performance of these fussy, over-complicated, hard-to-administer[1] systems is truly awful for the price.
[1] They're hypothetically simpler and easier to administer, but I've never seen this in the wild. If anything, we always seem to end up with more hours dedicated to care & feeding of this crap, and more glitchiness and failures, than we would with a handful of rented servers with maybe a CDN in front.
There were 3 instances of cognito. RDS, DynamoDB and S3. The entire architecture diagram would only be legible on an A2 (heck, maybe even A1) page. And that was the high level diagram. The central A4 part of that diagram was a bunch of micro-services for handling different portions of this CRUD application.
This company could afford a system architect as well as a team of developers to work on this full time.
I was genuinely baffled, but this company was in an extremely lucrative industry, so I guess in this case it's fine to just take some of your profits and burn them.
Yes, we can run twitter on a single server (https://thume.ca/2023/01/02/one-machine-twitter/) No, we do not want to run twitter on a single server
You can go a very, very, very long way on 2-3 modern servers with a fast internet connection and a good backup strategy.
Even with a traditional RDBMS like MS-SQL/PostgreSQL, you aren't bottlenecked by the 1-2ghz cpu and spinning rust hard drives. You can easily get to millions of users for a typical site/app with a couple servers just for a read replica/redundancy. As much as I happen to like some of the ergonomics of Mongo from a developer standpoint, or appreciate the scale of Cassandra/.ScyllaDB or even Cockroach... it's just not always necessary early on, or ever.
I've historically been more than happy to reach for RabbitMQ or Redis when you need queueing or caching... but that's still so much simpler than where some microservice architectures have gone. And while I appreciate what Apollo and GraphQL bring to the table, it's over the top for the vast majority of applications.
When you reduce the man hours per customer you can get farther down your backlog. You can carve people off for new prospective business units. You can absorb the effects of a huge sale or bad press better because you aren’t trying to violate Brooks’ Law nor doing giant layoffs that screw your business numbers.
You have time for people to speculate on big features or more work on reducing the costs further. If you don’t tackle this work early you end up in the armed Queen Problem: running as fast as you can just to stay still.
Now it seems things are swinging back the other direction and articles like "Use One Big Server" are getting re-discussed: https://news.ycombinator.com/item?id=45085029
("fashion is what goes out of fashion")
We have 100k paying users and most of them don't actively use our service, yet we have microservice slop with k8 and load balancers galore running in big tech cloud provider and document databases.
From our whole setup I can only think of _maybe_ one thing that should be split into a separate dedicated service, everything else could happily live in a monolith backed by a relational database and a caching layer and a storage solution.
One start up I worked at we had 2 Kubernetes clusters and a rat's nest of microservices for an internal tool that, had we been actually successful at delivering sufficient value would have been used by at most a 100 employees (and those would unlikely be concurrent). And this was an extremely highly valued company at the time.
Another place I worked at we were paying for 2 dev ops engineers (and those guys don't come cheap) to maintain our deployment cluster for 3 apps which each had a single customer (with a handful of users). This whole operation had like 20 people and an engineering team of 8.
Happy days.
At my job we also have some redundant clusters but that's because we're in the middle of a transition (really two transitions, the first of which was never completed), of the 10 clusters that fall under my responsibility 6 will hopefully be gone by the end of this year.
What if I use the cloud? I don't even know how many servers my database runs on. Nor do I care. It's liberating not having to think about it at all.
• https://benwheatley.github.io/blog/2025/02/26-14.04.07.html
• https://benwheatley.github.io/blog/2024/04/07-21.31.19.html
No, this whole article reads like someone who is crying that they no longer have their AS/200. Bye. The reason people use AWS and all those 3rd party is so they don’t have to reinvent the wheel which this author seems hell bent on.
Why are we using TCP when a Unix file is fine… why are we using databases when a directory and files is fine? Why are we scaling when we aren’t Google when my single machine can serve a webpage? Why am I getting paid to be an engineer while eschewing all the things that we have advanced over the last two decades?
Yeah, these are not the right questions. The real question should be: “Now that we have scale what are we gonna do with it?”
ie yes kubernetes but the simplest vanilla version of it you can manage
Too true. Now that I've stepped into an "engineering leadership" role and spend as much time looking at finances as I do at code, I've formed the opinion that in 99.999% of cases, engineering problems are really business problems. If you could throw infinite time and money at the technical challenges, they'd no longer be challenging. But businesses, especially startups, don't have infinite (or even "some") money and time, so the challenge is doing the best engineering work you can, given time and budget constraints.
> The downsides [of the monolith approach]
I like the article's suggestion of using explicitly defined API boundaries between modules, and that's a good approach for a monolith. However one massive downside that cannot be ignored -- by having a single monolith you now have an implicit dependency on the same runtime working on all parts of your code. What I mean by this is, all your code is going to share the same Python version and same libraries (particularly true in Python, where it's not a common/well-supported use case to have multiple versions of library dependencies). This means that if you're working on Module A, and you realize you need a new feature from Pandas 2.x, but the rest of the code is on Pandas 1.x... well, you can't upgrade unless you go and fix Modules B, C, D ... Z to work with Pandas 2.
This won't be an issue at the start, but it's worth pointing out. Being forced to upgrade a core library or language runtime and finding out it's a multi-month disruptive project can be brutal.
I think many people would not consider what I am going to suggest a monolith, but docker compose all hosted in the same server acts a lot like a monolith.
> particularly true in Python, where it's not a common/well-supported use case to have multiple versions of library dependencies
You can use virtual environments each with its python version and packages.