As an old(?) person who does everything by the book in C++, I couldn't really relate to it.
At least at my company it's hard to get hired to a programming position on soft skills alone. I've interviewed some very nice people who ended up not getting hired due to trouble whiteboarding.
As far as I can tell no one I work with is a an influencer on insta-twit. Some coworkers are better than others, but on the whole they all know the front end of a for loop from the back and are willing to learn.
And I end up spending way more time not writing code than writing it.
What's going on, it keeps switching between 'You are not Google. You ate not Netflix', and 'Developers are now measured in hits and views and it's not okay' or something - different articles on different sites even.
Some comments here seem to reference one, some the other...
I worked on the floor with two teams. One (ours) built our apps and infrastructure the correct way. JIT. Get it working, get it working correct, get it working fast. And given the scale, it did wonders.
The other team talked big and gave demos to higher ups and their higher ups and their higher ups. Web scale baby. And with that came the galactic infrastructure and all the buzzword needed in order to run it.
About 6 months later, they quit. They had spent their whole budget on astronaut architecture, that the didn't have any money or time left to build the apps that were supposed to run on said infrastructure.
"You are not FANG". But even if you are, JIT development.
Sorry. No, I wasn't referring to JIT Manufacturing...
It was merely a tongue-in-cheek to mean just build what's needed right now in front of you, then get it working at scale once there's an actual needs to scale. This is in contrast to starting from day one spending most of your time thinking of scale, then building everything so that it can infinitely scale (along with the complexities that come with it), only to find out that you have 8 users and your data can fit entirely in memory.
there's a vast middle ground between 'astronaut architecture' and 'common sense necessities', and knowing where a good cutoff point is is hard without experience.
I've met more than a few folks who consider web templating systems to be 'overkill', and have 0 understanding of the risk of xss. Same with sql/db escaping - "I just write the SQL and run it, using all those libraries is just a waste of time", etc.
Many of the projects I've come in to over the years were doing with a "JGID" mentality. And they did "get it done", "it" was just a steaming pile of crap when it was done. "Why is this taking weeks to do - the previous guy was so much faster?"
Had this one last year:
"This was never slow before when X worked on it, you've made changes what have you done? We need to call X back in to the project".
X just "got it done". And X was a db admin who was writing code. And X decided it would be good to have views join against other views which joined against other views, and have some queries which used those views run in triggers.
When X was on the project, and there were 40 users, it was fine. They hit 1000 users, and things were 'slow', so they upgraded to a larger EC2 instance. X left, and I came in, and several months later they hit ~30000 users (not active, just ... user account records). The system was dying with more than 5 active users, because of all the views joining other views on 30k+ records.
Unravelling that meant deciphering all the views, all the queries, all the code that touched all of it, and rebuilding a moderate portion, without tests, known 'good' data, or anyone on the project knowing what 'right' was - they just knew when things looked 'wrong' (or slow).
BUT... 18 months earlier, it was "JGID", and it got done. I'm still a bit perplexed why a professional DB admin thought views joining against multiple other views was a good approach.
We've all got horror stories, I'm sure, it's just that the "JGID" mentality is often preached by competent experts as a good approach, but picked up by beginners as the approach used by experts, and has bad consequences.
Out of curiosity - Which database is this? and is it really taboo to have views joining against multiple other views?
If the underlying views were performant- I'd assume the query optimizer would do the right thing(at least 90% of the time).
EDIT: I guess it depends - Just did more research and found this [1]. As long as the views don't do unnecessary heavy lifting or joining unnecessary tables, it should be fine.
postgres. I realize this is heresy to badmouth Postgres here, but it was Postgres (9.0 or 9.1 IIRC).
Had a query that selected from table X joining against view X which selected from view A, and view A joins against tables A,B and C, and table B also joins against a view which uses table A and C.
This was just bad. But trying to explain to non-tech people how bad it is, when "it used to work", is difficult. It used to work when you had 50-100 records. No one ever tested was this would be like with 30k records and 50 concurrent users all executing the same nested/circular view mess simultaneously.
But the fact that it was postgres is kind of beside the point. I don't know of any mainstream DB that would handle this well.
The short term fix was to do this large query once at the end of a process and cache the results; the set of queries in question were happening on a 'dashboard' view which everyone hit all the time. It would still cause problems with concurrency, because when 80 people would go through a process and get 'done' (think timed training exercises), the queries would still all be running more or less concurrently, and still cause timeouts, but it wasn't as frequent, because people tended to be staggered a bit more as they finished.
PostgreSQL has progressively reduced the number of optimisation boundaries that are encountered with views-on-views situations, but you can still wind up telling the database to churn and slosh a whole bunch of data from which you only cherry-pick a tiny portion.
I have definitely been guilty of failing to test how my schemata behave with large data sets.
Oh then there's ORMs. I've seen ActiveRecord spit out some frankly batshit insane queries that would stump a room of Einsteins. But somehow PostgreSQL picked it up, chopped it into a plan and got to work plowing through an incredibly wasteful and repetitious query.
> I have definitely been guilty of failing to test how my schemata behave with large data sets.
As have I. It's the dangerous part of the "get it done" approach. And there's no perfect approach - everything is a tradeoff. How much time do you spend dealing with situations that might never happen?
Experience does give you some grounding when making those tradeoff decisions. No, we don't need the architect the application to scale up/down to handle 25k concurrent users in 5 minutes; that's unlikely going to happen. Yes, we do need to spend the extra 2 hours installing and learning a templating system to avoid common XSS pitfalls.
I think SQL views are a bit of a code smell. They're really only useful in my opinion as a hack, where creating a rather gruesome abstraction within the database is easier than changing the query layer.
I'm currently working on a service written by a bunch of hipster developers who, wrote raw SQL in Python. And they wrote views, lots of really difficult to understand views that are self-joining (on JSON fields no less!). The performance is really unpredictable given the input sizes. It's the only time I'd call performance of a database chaotic because given an input size (beyond certain safe ranges) I have no idea what the performance is.
The problem is from my perspective a key bit of your application logic get hidden, you're then bound to migrations to change it.
> And they did "get it done", "it" was just a steaming pile of crap when it was done. "Why is this taking weeks to do - the previous guy was so much faster?"
When I mean JGID, I also meant not make it a steaming pile of shit :)
> When I mean JGID, I also meant not make it a steaming pile of shit :)
the tldr of my post is that people often don't know the difference. We also have grown a culture of people promoting "YAGNI", and its easier for some people to dismiss basic ideas with YAGNI.
Yeah, adding 5 language translations to your project on day 1 - YAGNI.
Storing passwords in plaintext? You need something more than that.
All the rules of thumb seem to have massive categories of exceptions where they don't work, and that makes me suspicious of applying new ones that I don't fully understand.
DRY up your code... unless the two code paths look similar but aren't the same.
YAGNI... unless "it" is a database backup, a load balancer, or test coverage of your signup flow.
KISS... unless your problem is complicated enough that a simple solution only implements half of what you need.
Microservices are great... if you've got more than one team and can support the operational overhead of SDN and service discovery. Monoliths are great... until you're pushing code to them every minute and they become impossible to refactor.
and yeah, you should be suspicious of doing anything that you don't understand. this is all knowledge work, and there's some implicit notion that when you make a decision you have some level of understanding. the bigger the potential impact, the more understanding you should have.
The first one seems to do the wrong thing because there is no mention of any stakeholder at all. Getting it working first is great though.
The second one seems to do the right thing by demoing to stakeholders first. Then, it goes on to make it web scale with buzzwords (which doesn't seem good).
The best combination seems to be: 1. demoing to higher up and 2. JIT (I'm not familiar with the word, but I get what you mean).
In the second case, because sprint velocity was on the up and the walls were covered full of bright colours, it looked like they were doing lots of work. The actually were doing lots of work, but none that the stakeholders cared about.
The second team emphasised the importants of scale and how they were going to solve that problem. The apps to run on top seemed to slip everyone's minds because ticket were being closed hitting KPIs.
When faced with building a simple web page, Team 1 spends a couple of days producing a single static html page and builds on it as necessary.
Team 2 spends 6 months showing upper management architecture diagrams and flashy mockups. They blow a million dollars on cloud hosting their load balanced web servers. They've got Kafka, ElasticSearch, triple redundant SQL servers, and a 12 person team to run it all. But when asked to present the actual page they announce that nobody on the team knows HTML and they just kind of assumed somebody else would do that part.
Non-technical managers cannot tell the difference between the two teams.
> A developer’s skill is measured in one parameter: how well he writes code.
While this could be a valid viewpoint, one thing I’d caution against is that this type of measurement inevitably leads to the commoditization of developers. Rating skill on a single point will lead to “you’re only worth what this metric says you are worth” and eventually to constant competition between developers instead of cooperation. At least that’s the experience I had in such positions during my younger years.
It's also wrong - if a developer can't communicate, interact well with team mates, sensibly translate requirements to program structure, discover requirements, pitch technological decisions to management, keep up on OSS tech that could prove useful, document well, review others code, etc etc... they're not at the top of their game.
Being able to write code well is just one of the skills needed in a good developer. It might be the main one, but without the rest it's not really enough.
Actually I’ve found that a lot of development is actually about image projection, or how good you make other people think you are. This is especially true with non-technical folks, who don’t know 1/2 of what developers do anyway.
Our office’s “rockstar” dev has his 3 widescreen monitors tilted vertically and he uses a split keyboard. He’s so quirky and leet.
Its all about execution and delivery. I'm pretty happy to have left the old days of corporate committee projects behind to do my own stuff. I'm also financially better off.
A related issue is also focusing so much on the tech (and often how well the product/service can scale) that you forget to do much marketing and end up delaying the product/service for months as a result. Or worse still, never actually finish building it at all.
Definitely follow the old get it working, then correct, then fast methodology, at least for projects without many billions of dollars in resources.
Having seen the inside of both good and bad product organizations, culture beats process any day of the week. Get the core values right, hire smart people and everything will fall into place.
A corollary is that cargo-culting well-developed processes generally means you’re copying from a company with a culture that couldn’t address those functions organically. So you may actually just be lifting an over-engineered process designed to address a cultural weakness at the original company.
I don’t think one needs to plan on being Facebook scale from the beginning, but I do think there is some consideration of scalability that needs to be considered from the start. Maybe you don’t need to think 10 steps ahead but you should at least have plans for 3 or 4 steps ahead of you don’t want to have to rearchitect everything from scratch in a year.
It might not be strictly necessary to containerize your app, for example, if you’re just running it on a single ec2 instance behind a load balancer, but it’s not a huge amount of work up front and there are some benefits and it means it’ll make it easier to move to ecs or k8s later if you need to.
You've missed the point. The argument is that eventhough aiming for FB scale is absurd, some considerations should be taken under consideration from the start. We're seeing YAGNI-style arguments being abused to sell the idea that just because it's absurd to think 10 steps ahead it's also absurd to think 2 or 3 steps ahead, which makes no sense at all.
Starting with two colos might be a good idea -- at least at the point where you start thinking of adding a second machine for data availability, it may as well be in another colo so you have a disaster recovery option.
Starting with partitioned data is a lot bigger jump. At most, I would say avoid building features that would require heroic efforts to implement in a partitioned world, but a single database can get huge these days, and maybe even huger by the time you need to scale. Chances are, 2TB of ram and 64 cores will fit your live DB needs; and if not, you'll know with time to make something else work
Start with a hosted DB service in the cloud. When you outgrow that then start thinking how to scale. 1st start sharding the data, then think about splitting a few services out if they are changing faster.
This concept we need to go full microservices multi region right from the get go is silly and wasted time / effort.
Until your idea sells and your market grows there is no reason to concern yourself with the logistics of scale.
Kubernetes, Machine Learning, Microservices, Service-Mesh, are the things that immediately come to (my) mind from such discussions with start-up dev-teams.
But somehow all of these are at least a few years old meanwhile. Could it be that the buzzword bingo we experienced in the last years has already peaked or why haven't we seen any new nonsense[1] (at least nothing that I'm aware that is not at least 24 months old)?
[1] it doesn't have to be nonsense but as the article suggests, not everyone is FAANG.
Machine learning may have been used and abused as a meaningless buzzword but kubernetes in particular and container orchestration in general is a well established and proven infrastructure technology. If you are not operating a monolith and you need to automatically deploy and scale your service then you have no technically-sound argument to not use this sort of infrastructure.
the question is if an early stage start-up with 4-5 devs where they typically already struggle to get basic CI in place ... and where 1 of the devs loves K8 and the others don't really know it (but don't object because it's nice to have it on the CV), whether they should really already focus on orchestration and scaling before they even have a MVP.
I'm not against k8 per se, but its complexity isn't something to ignore. It's all good if you have the people to run this but most places I know don't.
I think we have reached peak k8-hype last year when F. Campoy at GOTO suggested to wrap K8 around GO to make Go as resilient like Erlang/BEAM (see video below ...) https://www.youtube.com/watch?v=ScE9TnoWltA&t=1437s
as for ML I really think there are cool use-cases, but why would a pre-MVP start-up already be talking about hiring data scientists, ... when they don't even have any data (exception being they are an analytics start-up obviously).
> the question is if an early stage start-up with 4-5 devs where they typically already struggle to get basic CI in place ...
Nowadays it's super easy to setup a CICD pipeline. The basic settings and intro examples alone are enough to build and publish container images to a container registry.
> and where 1 of the devs loves K8 and the others don't really know it (but don't object because it's nice to have it on the CV), whether they should really already focus on orchestration and scaling before they even have a MVP.
That doesn't make any sense. Developing services to run on kubernetes is a software architecture problem, one which only affects some minot dev level details such as supporting env variable settings. Devs can be entirely oblivious to kubernetes and still be effective.
> I think we have reached peak k8-hype last year when F. Campoy
That's largely irrelevant and misses the whole point. Just because some people present some gimmicky take on a technology it doesn't mean the fundamentals are not good. Kubernetes allow distributed services to be deployed transparently across multiple providers (thus you are not held hostage by a seller), and allow services to automatically scale horizontally with demand and within resource limits gracefully degradate after that point. That doesn't go away if someone decides to reenact Inception with buzzworthy technologies.
Because we’ve railed on “programmers” for so long everyone wants to be a “software engineer” which more often than not leads to equally insane architectures. No you don’t need multi-region availability. No you don’t need to rewrite this in rust. No you don’t need a multi-master db setup. No you don’t need graphql.
I think this article is basically right, but I want to provide a personal perspective on this:
This is why I have found that I prefer working on a widely used product at one of these big companies. I find the most enjoyment in thinking through how to make products work acceptably for huge numbers of users, or how to pick the needles I need out of enormous data sets, or how to structure a codebase such that hundreds of people can contribute to it every day. I have found it very demoralizing in my career when I've been told not to think of user or data or codebase scaling considerations because YAGNI. I've lost a number of debates against just-get-it-done ethos, and mostly I think I was right to lose; I was wrong, YAGNI was right. But the rightness of YAGNI for those products didn't make me enjoy the work any more.
So I think the article is right that most companies don't have these scaling problems, but for me, I want to work for the ones that do. I think this is probably a common sentiment.
I feel the same way. I have always sought to work for such start-ups that are about to take off and do high-scale; where leverage is important. Leverage provided by code-bases where certain changes to the code makes others working on the same code base 10X more productive; leverage provided by org designs where changes to the org-design can make all the roles 10X more productive. Looking for these leverage enhancing opportunities and making it happen is a very satisfying and unique experience.
But there's always YAGNI of some kind in any situation and I'm good with the kind of YAGNI that these scaling situations have.
Yeah. Since working at a place that does have to think about problems in these terms, I find such YAGNI stuff much less compelling.
Guess what, some of us actually are gonna need it. And if you don’t begin with the future in mind, starting out with something “simple” can result in some very not-simple situations down the line.
There's a lot to be said about org design. Org design has to be right for the size of the company and the velocity you need to succeed. For example, when a start-up that is about to take off like a rocket ship, it needs to make a conscious change in org design to facilitate both strong foundation building as well as fast feature building with a layer of domain-specific platform/frameworks being curated to keep the leverage high and hence velocity high without blowing the headcount budget.
Ideally orgs should sow the seeds of this 3-layer org (and corresponding tech architecture) design from the beginning and consciously choose to scale the investments in those layers as needed in each growth spurt.
Usually people tend to shortchange this advice by thinking they can make the same 10 people do the work in all the layers. And that may be ok when you are small. But specialization in systems vs app-frameworks vs app feature development is highly important for velocity. Also your hiring loops and the "hiring bar" has to be different for these roles. And usually the hiring pipeline depth and churn rate is different for these roles. So, creating a structure where impedances are matched correctly between input and output, so to speak, is immensely helpful for achieving high productivity and cost efficiency.
57 comments
[ 2.5 ms ] story [ 119 ms ] thread[0] https://habr.com/en/post/456568/
At least at my company it's hard to get hired to a programming position on soft skills alone. I've interviewed some very nice people who ended up not getting hired due to trouble whiteboarding.
As far as I can tell no one I work with is a an influencer on insta-twit. Some coworkers are better than others, but on the whole they all know the front end of a for loop from the back and are willing to learn.
And I end up spending way more time not writing code than writing it.
Some comments here seem to reference one, some the other...
The other team talked big and gave demos to higher ups and their higher ups and their higher ups. Web scale baby. And with that came the galactic infrastructure and all the buzzword needed in order to run it.
About 6 months later, they quit. They had spent their whole budget on astronaut architecture, that the didn't have any money or time left to build the apps that were supposed to run on said infrastructure.
"You are not FANG". But even if you are, JIT development.
It was merely a tongue-in-cheek to mean just build what's needed right now in front of you, then get it working at scale once there's an actual needs to scale. This is in contrast to starting from day one spending most of your time thinking of scale, then building everything so that it can infinitely scale (along with the complexities that come with it), only to find out that you have 8 users and your data can fit entirely in memory.
Sometimes "make it so" from ST-TNG.
(Fwiw searching 'JIT development' does net first page results of people talking about JIT/lean manufacturing in a software development context.)
I've met more than a few folks who consider web templating systems to be 'overkill', and have 0 understanding of the risk of xss. Same with sql/db escaping - "I just write the SQL and run it, using all those libraries is just a waste of time", etc.
Many of the projects I've come in to over the years were doing with a "JGID" mentality. And they did "get it done", "it" was just a steaming pile of crap when it was done. "Why is this taking weeks to do - the previous guy was so much faster?"
Had this one last year:
"This was never slow before when X worked on it, you've made changes what have you done? We need to call X back in to the project".
X just "got it done". And X was a db admin who was writing code. And X decided it would be good to have views join against other views which joined against other views, and have some queries which used those views run in triggers.
When X was on the project, and there were 40 users, it was fine. They hit 1000 users, and things were 'slow', so they upgraded to a larger EC2 instance. X left, and I came in, and several months later they hit ~30000 users (not active, just ... user account records). The system was dying with more than 5 active users, because of all the views joining other views on 30k+ records.
Unravelling that meant deciphering all the views, all the queries, all the code that touched all of it, and rebuilding a moderate portion, without tests, known 'good' data, or anyone on the project knowing what 'right' was - they just knew when things looked 'wrong' (or slow).
BUT... 18 months earlier, it was "JGID", and it got done. I'm still a bit perplexed why a professional DB admin thought views joining against multiple other views was a good approach.
We've all got horror stories, I'm sure, it's just that the "JGID" mentality is often preached by competent experts as a good approach, but picked up by beginners as the approach used by experts, and has bad consequences.
If the underlying views were performant- I'd assume the query optimizer would do the right thing(at least 90% of the time).
EDIT: I guess it depends - Just did more research and found this [1]. As long as the views don't do unnecessary heavy lifting or joining unnecessary tables, it should be fine.
[1] https://dba.stackexchange.com/questions/151169/are-views-har...
Had a query that selected from table X joining against view X which selected from view A, and view A joins against tables A,B and C, and table B also joins against a view which uses table A and C.
This was just bad. But trying to explain to non-tech people how bad it is, when "it used to work", is difficult. It used to work when you had 50-100 records. No one ever tested was this would be like with 30k records and 50 concurrent users all executing the same nested/circular view mess simultaneously.
But the fact that it was postgres is kind of beside the point. I don't know of any mainstream DB that would handle this well.
The short term fix was to do this large query once at the end of a process and cache the results; the set of queries in question were happening on a 'dashboard' view which everyone hit all the time. It would still cause problems with concurrency, because when 80 people would go through a process and get 'done' (think timed training exercises), the queries would still all be running more or less concurrently, and still cause timeouts, but it wasn't as frequent, because people tended to be staggered a bit more as they finished.
I have definitely been guilty of failing to test how my schemata behave with large data sets.
Oh then there's ORMs. I've seen ActiveRecord spit out some frankly batshit insane queries that would stump a room of Einsteins. But somehow PostgreSQL picked it up, chopped it into a plan and got to work plowing through an incredibly wasteful and repetitious query.
As have I. It's the dangerous part of the "get it done" approach. And there's no perfect approach - everything is a tradeoff. How much time do you spend dealing with situations that might never happen?
Experience does give you some grounding when making those tradeoff decisions. No, we don't need the architect the application to scale up/down to handle 25k concurrent users in 5 minutes; that's unlikely going to happen. Yes, we do need to spend the extra 2 hours installing and learning a templating system to avoid common XSS pitfalls.
I'm currently working on a service written by a bunch of hipster developers who, wrote raw SQL in Python. And they wrote views, lots of really difficult to understand views that are self-joining (on JSON fields no less!). The performance is really unpredictable given the input sizes. It's the only time I'd call performance of a database chaotic because given an input size (beyond certain safe ranges) I have no idea what the performance is.
The problem is from my perspective a key bit of your application logic get hidden, you're then bound to migrations to change it.
HOLY TAMOLE! You might have my story beat there. That's one level I didn't have to deal with. My sympathies!
When I mean JGID, I also meant not make it a steaming pile of shit :)
the tldr of my post is that people often don't know the difference. We also have grown a culture of people promoting "YAGNI", and its easier for some people to dismiss basic ideas with YAGNI.
Yeah, adding 5 language translations to your project on day 1 - YAGNI.
Storing passwords in plaintext? You need something more than that.
DRY up your code... unless the two code paths look similar but aren't the same.
YAGNI... unless "it" is a database backup, a load balancer, or test coverage of your signup flow.
KISS... unless your problem is complicated enough that a simple solution only implements half of what you need.
Microservices are great... if you've got more than one team and can support the operational overhead of SDN and service discovery. Monoliths are great... until you're pushing code to them every minute and they become impossible to refactor.
etc.
What's needed right now, but that doesn't take a complete overhaul when it's time to start growing.
The first one seems to do the wrong thing because there is no mention of any stakeholder at all. Getting it working first is great though.
The second one seems to do the right thing by demoing to stakeholders first. Then, it goes on to make it web scale with buzzwords (which doesn't seem good).
The best combination seems to be: 1. demoing to higher up and 2. JIT (I'm not familiar with the word, but I get what you mean).
The second team emphasised the importants of scale and how they were going to solve that problem. The apps to run on top seemed to slip everyone's minds because ticket were being closed hitting KPIs.
Team 2 spends 6 months showing upper management architecture diagrams and flashy mockups. They blow a million dollars on cloud hosting their load balanced web servers. They've got Kafka, ElasticSearch, triple redundant SQL servers, and a 12 person team to run it all. But when asked to present the actual page they announce that nobody on the team knows HTML and they just kind of assumed somebody else would do that part.
Non-technical managers cannot tell the difference between the two teams.
While this could be a valid viewpoint, one thing I’d caution against is that this type of measurement inevitably leads to the commoditization of developers. Rating skill on a single point will lead to “you’re only worth what this metric says you are worth” and eventually to constant competition between developers instead of cooperation. At least that’s the experience I had in such positions during my younger years.
Being able to write code well is just one of the skills needed in a good developer. It might be the main one, but without the rest it's not really enough.
Our office’s “rockstar” dev has his 3 widescreen monitors tilted vertically and he uses a split keyboard. He’s so quirky and leet.
Definitely follow the old get it working, then correct, then fast methodology, at least for projects without many billions of dollars in resources.
And eventually realize there is no correct then fast after get it working, only more iterations forever.
A corollary is that cargo-culting well-developed processes generally means you’re copying from a company with a culture that couldn’t address those functions organically. So you may actually just be lifting an over-engineered process designed to address a cultural weakness at the original company.
It might not be strictly necessary to containerize your app, for example, if you’re just running it on a single ec2 instance behind a load balancer, but it’s not a huge amount of work up front and there are some benefits and it means it’ll make it easier to move to ecs or k8s later if you need to.
No one starts an application with the farm model and multiple colos.
Starting with partitioned data is a lot bigger jump. At most, I would say avoid building features that would require heroic efforts to implement in a partitioned world, but a single database can get huge these days, and maybe even huger by the time you need to scale. Chances are, 2TB of ram and 64 cores will fit your live DB needs; and if not, you'll know with time to make something else work
This concept we need to go full microservices multi region right from the get go is silly and wasted time / effort.
Until your idea sells and your market grows there is no reason to concern yourself with the logistics of scale.
You are Not Google (my favorite): https://blog.bradfieldcs.com/you-are-not-google-84912cf44afb
A piece from my Notes to a Young Software Engineer that likens engineering media to a bazaar, not a mirror of the industry:
Beware Engineering Media:
https://www.nemil.com/on-software-engineering/beware-enginee...
(Martin Fowler's YAGNI article is also great)
But somehow all of these are at least a few years old meanwhile. Could it be that the buzzword bingo we experienced in the last years has already peaked or why haven't we seen any new nonsense[1] (at least nothing that I'm aware that is not at least 24 months old)?
[1] it doesn't have to be nonsense but as the article suggests, not everyone is FAANG.
I'm not against k8 per se, but its complexity isn't something to ignore. It's all good if you have the people to run this but most places I know don't.
I think we have reached peak k8-hype last year when F. Campoy at GOTO suggested to wrap K8 around GO to make Go as resilient like Erlang/BEAM (see video below ...) https://www.youtube.com/watch?v=ScE9TnoWltA&t=1437s
as for ML I really think there are cool use-cases, but why would a pre-MVP start-up already be talking about hiring data scientists, ... when they don't even have any data (exception being they are an analytics start-up obviously).
this is how you get ants.
Nowadays it's super easy to setup a CICD pipeline. The basic settings and intro examples alone are enough to build and publish container images to a container registry.
> and where 1 of the devs loves K8 and the others don't really know it (but don't object because it's nice to have it on the CV), whether they should really already focus on orchestration and scaling before they even have a MVP.
That doesn't make any sense. Developing services to run on kubernetes is a software architecture problem, one which only affects some minot dev level details such as supporting env variable settings. Devs can be entirely oblivious to kubernetes and still be effective.
> I think we have reached peak k8-hype last year when F. Campoy
That's largely irrelevant and misses the whole point. Just because some people present some gimmicky take on a technology it doesn't mean the fundamentals are not good. Kubernetes allow distributed services to be deployed transparently across multiple providers (thus you are not held hostage by a seller), and allow services to automatically scale horizontally with demand and within resource limits gracefully degradate after that point. That doesn't go away if someone decides to reenact Inception with buzzworthy technologies.
This is why I have found that I prefer working on a widely used product at one of these big companies. I find the most enjoyment in thinking through how to make products work acceptably for huge numbers of users, or how to pick the needles I need out of enormous data sets, or how to structure a codebase such that hundreds of people can contribute to it every day. I have found it very demoralizing in my career when I've been told not to think of user or data or codebase scaling considerations because YAGNI. I've lost a number of debates against just-get-it-done ethos, and mostly I think I was right to lose; I was wrong, YAGNI was right. But the rightness of YAGNI for those products didn't make me enjoy the work any more.
So I think the article is right that most companies don't have these scaling problems, but for me, I want to work for the ones that do. I think this is probably a common sentiment.
But there's always YAGNI of some kind in any situation and I'm good with the kind of YAGNI that these scaling situations have.
Guess what, some of us actually are gonna need it. And if you don’t begin with the future in mind, starting out with something “simple” can result in some very not-simple situations down the line.
"Why try to be like others if you are condemned to being yourself" Link: https://mutterichbindumm.tumblr.com/post/92142897112/funeral...
Ideally orgs should sow the seeds of this 3-layer org (and corresponding tech architecture) design from the beginning and consciously choose to scale the investments in those layers as needed in each growth spurt.
Usually people tend to shortchange this advice by thinking they can make the same 10 people do the work in all the layers. And that may be ok when you are small. But specialization in systems vs app-frameworks vs app feature development is highly important for velocity. Also your hiring loops and the "hiring bar" has to be different for these roles. And usually the hiring pipeline depth and churn rate is different for these roles. So, creating a structure where impedances are matched correctly between input and output, so to speak, is immensely helpful for achieving high productivity and cost efficiency.