I feel like sometimes it’s a form of procrastination.
There are things we don’t want to do (talk to costumers, investors, legal, etc.), so instead we do the fun things (fun for engineers).
It’s a convenient arrangement because we can easily convince ourselves and others that we’re actually being productive (we’re not, we’re just spinning wheels).
What the hell have you built? Turns out a pretty straightforward service.
That diagram is just aws, programming language, database. For some reason hadoop I guess. And riak/openstack as redundant.
It just seems like pretty standard stuff with some seemingly small extra parts because that make me think that someone on the team was familiar with something like ruby, so they used that instead of using java.
"Why is Redis talking to MongoDB" It isn't.
"Why do you even use MongoDB" Because that's the only database there, and nosql schemaless solutions are faster to get started... because you don't have to specify a schema. It's not something I would ever choose, but there is a reason for it.
"Let's talk about scale" Let's not, because other than hadoop, these are all valid solutions for projects that don't prioritize scale. Things like a distributed system aren't just about technology, but also data design that aren't that difficult to do and are useful for reasons other thant performance.
"Your deployment strategy" Honestly, even 15 microservices and 8 databases (assuming that it's really 2 databases across multiple envs) aren't that bad. If they are small and can be put on one single server, they can be reproduced for dev/testing purposes without all the networking cruft that devops can spend their time dealing with.
Once you have a service that has users and costs actual money, while you don’t need to make it a spaghetti of 100 software products, you need a bit of redundancy at each layer — backend, frontend, databases, background jobs — so that you don’t end up in a catastrophic failure mode each time some piece of software decides to barf.
Oh my word Riak - I haven't seen that DB mentioned for years!
I totally get the point it makes. I remember many years ago we announced SocketStream at a HackerNews meet-up and it went straight to #1. The traffic was incredible but none of us were DevOps pros so I ended up restarting the Node.js process manually via SSH from a pub in London every time the Node.js process crashed.
If only I'd known about upstart on Ubuntu then I'd have saved some trouble for that night at least.
I think the other thing is worrying about SPOF and knowing how to respond if services go down for any reason (e.g. server runs out of disk space - perhaps log rotation hasn't been setup, or has a hardware failure of some kind, or the data center has an outage - I remember Linode would have a few in their London datacenter that just happened to occur at the worst possible time).
If you're building a side project I can see the appeal of not going overboard and setting up a Kubernetes cluster from the get-go, but when it is things that are more serious and critical (like digital infrastructure for supporting car services like remotely turning on climate controls in a car), then you design the system like your life depends on it.
Are you doing software for money? Because not having Kubernetes in the project will stop you from receiving money. Someone please create with one of these smart AI tools the ultimate killer app: Kubernetes+crypto+AI+blockchain+Angular+Redux+Azure (Working only in Chrome browser).
Really that's going way too far - you do NOT need Redis for caching. Just put it in Postgres. Why go to this much trouble to put people in their place for over engineering then concede "maybe Redis for caching" when this is absolutely something you can do in Postgres. The author clearly cannot stop their own inner desire for overengineering.
Thinking is scary. No one (among non-thinking colleagues) is going to criticize you for using de-facto standard services like kafka, mongo, redis, ecc... regardless of the nonsensical architecture you come up with.
Yes, I also put Redis in that list. You can cache and serve data structure in many other ways, for example replicate the individual features you need in you application instead of going the lazy route and another service to the mix. And don't get me started on Kafka... money thrown in the drain when a stupid grpc/whatever service would do.
Part of being an engineer is also selecting the minimum amount of components for your architecture and not being afraid of implementing something on your own if you only need 1 of 100s features that an existing product require.
I love the fact that the author "wrote" this page with massive CSS framework (tailwind) and some sort of Javascript framework, with a bundler and obfuscator - instead of a plain, simple HTML page. Well played! :-)
haha, right?!
I'm totally onboard with the author's philosophy, hence for websites: https://mastrojs.github.io – the simple web framework and site generator you could have built yourself.
As a practitioner we subconciously optimise for "beauty", in maths, physics or dev. Most hackers are self-motivated, by that beauty, not by 40k ork style functional design.
Ironic that the clicking those big buttons only causes a JS error to be logged to console with nothing else happening. That doesn't particularly lend to the authors credibility, although the advice of using simple architecture where possible is correct.
An improved CV, lets be honest most stuff is boring projects that could even be built with 1990's technology, distributed systems is not something that was invented yesterday.
However having in the CV any of those items from left side in the deployment strategy is way cooler than mentioning n-tier architecture, RPC (regardless how they are in the wire), any 1990's programming language, and so forth.
A side effect from how hiring works so badly in our industry, it isn't enough to know how master a knife to be a chef, it must be a specific brand of knife, otherwise the chef is not good enough for the kitchen.
The problem with doing things the sensible way (eschewing microservices and k8s when you work on projects that aren't hyperscale) is that you end up missing opportunities later on because recruiters will filter you because you can't meaningfully respond to the question about “how experienced you are with micro service architecture”. Granted I may have dodged a bullet by not joining a company with 50 engineers that claim to replicate Google's practices (most of which are here to make sure tens of thousands of engineers can work efficiently together), but still someone gets to pay the bill at the end of the month…
50 comments
[ 3.4 ms ] story [ 75.5 ms ] threadThere are things we don’t want to do (talk to costumers, investors, legal, etc.), so instead we do the fun things (fun for engineers).
It’s a convenient arrangement because we can easily convince ourselves and others that we’re actually being productive (we’re not, we’re just spinning wheels).
That diagram is just aws, programming language, database. For some reason hadoop I guess. And riak/openstack as redundant.
It just seems like pretty standard stuff with some seemingly small extra parts because that make me think that someone on the team was familiar with something like ruby, so they used that instead of using java.
"Why is Redis talking to MongoDB" It isn't.
"Why do you even use MongoDB" Because that's the only database there, and nosql schemaless solutions are faster to get started... because you don't have to specify a schema. It's not something I would ever choose, but there is a reason for it.
"Let's talk about scale" Let's not, because other than hadoop, these are all valid solutions for projects that don't prioritize scale. Things like a distributed system aren't just about technology, but also data design that aren't that difficult to do and are useful for reasons other thant performance.
"Your deployment strategy" Honestly, even 15 microservices and 8 databases (assuming that it's really 2 databases across multiple envs) aren't that bad. If they are small and can be put on one single server, they can be reproduced for dev/testing purposes without all the networking cruft that devops can spend their time dealing with.
Once you have a service that has users and costs actual money, while you don’t need to make it a spaghetti of 100 software products, you need a bit of redundancy at each layer — backend, frontend, databases, background jobs — so that you don’t end up in a catastrophic failure mode each time some piece of software decides to barf.
I totally get the point it makes. I remember many years ago we announced SocketStream at a HackerNews meet-up and it went straight to #1. The traffic was incredible but none of us were DevOps pros so I ended up restarting the Node.js process manually via SSH from a pub in London every time the Node.js process crashed.
If only I'd known about upstart on Ubuntu then I'd have saved some trouble for that night at least.
I think the other thing is worrying about SPOF and knowing how to respond if services go down for any reason (e.g. server runs out of disk space - perhaps log rotation hasn't been setup, or has a hardware failure of some kind, or the data center has an outage - I remember Linode would have a few in their London datacenter that just happened to occur at the worst possible time).
If you're building a side project I can see the appeal of not going overboard and setting up a Kubernetes cluster from the get-go, but when it is things that are more serious and critical (like digital infrastructure for supporting car services like remotely turning on climate controls in a car), then you design the system like your life depends on it.
Really that's going way too far - you do NOT need Redis for caching. Just put it in Postgres. Why go to this much trouble to put people in their place for over engineering then concede "maybe Redis for caching" when this is absolutely something you can do in Postgres. The author clearly cannot stop their own inner desire for overengineering.
Yes, I also put Redis in that list. You can cache and serve data structure in many other ways, for example replicate the individual features you need in you application instead of going the lazy route and another service to the mix. And don't get me started on Kafka... money thrown in the drain when a stupid grpc/whatever service would do.
Part of being an engineer is also selecting the minimum amount of components for your architecture and not being afraid of implementing something on your own if you only need 1 of 100s features that an existing product require.
However having in the CV any of those items from left side in the deployment strategy is way cooler than mentioning n-tier architecture, RPC (regardless how they are in the wire), any 1990's programming language, and so forth.
A side effect from how hiring works so badly in our industry, it isn't enough to know how master a knife to be a chef, it must be a specific brand of knife, otherwise the chef is not good enough for the kitchen.
https://nocommasql.com/