Ask HN: How to get experience deploying large scale services

10 points by it_learnses ↗ HN
I have 5+ years of .net development experience. All of it is for small to medium sized businesses (~ 100 users per min). I see a lot of senior dev postings requiring "experience deploying large scale services." I'd like to be qualified for such jobs within a couple of years. How do I acquire this kind of experience on the side?

3 comments

[ 3.1 ms ] story [ 15.7 ms ] thread
With some slightly questionable math you can turn 100 users/min into 7.2 million monthly visits. I guarantee you that's "large scale" enough. If you've actually got experience with deploying at your current job you should be fine, just apply. The only real way to learn is on the job.
"Large scale services" is a subjective number. If every request is 'heavy' at the backend the N of N req/s is lower. Lighter static content serving backends have a much higher N. Therefore, the engineering that goes behind optimizing these is different. IMHO, the N is not everything there is to a devOps engineer. In practical experience- a backend service i worked with had just 30 req/s at the front layer servers but generated 60M logs per day. As the logging guy i got to brag about larger Ns, while the front-layer work was more challenging.

1. Your 100 users/min may also be put across as 300 req/min.

2. Try startups that are going from 10 req/s to 100 req/s rather than directly approaching large, massively distributed backends.

3. To seriously qualify for these you also need the skill sets. Experience with centralized logging, multi area-zone data-centers, deploying at scale, configuration management, advanced version control workflows, deeper understanding of protocols, caching, awareness of security pitfalls, knowledge of GNU/Linux internals etc will certainly help.

4. Even in the absence of much production experience, you can significantly lower their concerns by demonstrating working knowlege in multiple of these above mentioned skill sets.

Also, take a look at https://github.com/cjbarber/ToolsOfTheTrade.

Find a company doing it at scale, join them and learn about how its done. If you can go to a Google or a Facebook then it's a great way to learn from a decade or more of their experience. Jet.com is a .Net shop looking for engineers. Might be worth checking out. Early days but I know they'll be doing something at scale.