Ask HN: Should I quit the field of software development?

155 points by sage76 ↗ HN
I have been in the field for a few years, mostly doing backend development.

I'll preface this by saying this field has never been my passion, just a profession I like.

I have been having an extremely difficult time with getting and clearing interviews. Screening rounds have become at least 2-3 hour long algorithm solving sessions which can range from dumb (requiring me to know some syntax that I am not allowed to look up) to extremely demanding.

System design rounds require me to solve problems that the top engineers in the world spend months on. One question I was asked was "How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale. In fact most of my work experience is kinda vanilla and boring, so I generally have nothing much to draw on or much to talk about.

Also, I'm not talking about FAANGs here. I never even applied to them. I apply to basic CRUD development roles, half of them entry level, and even there, the grilling has become intense, if I even get an interview.

I haven't even applied to jobs for weeks now, because in interviews I feel like I'm getting humiliated and I want to apologize and end them half-way.

I'm sorry if this is incoherent.

I'm sure the problem is me, since other people are doing fine. I need to draw a line and figure out some good indicators of when I should quit this industry, and anyone here who could give a few pointers on that, I would be thankful. I don't want to throw more time and resources into an industry where baseline expectations are something I can't/won't reach.

158 comments

[ 2.6 ms ] story [ 253 ms ] thread
I was in the same boat. I decided to try the research software engineer path (not scientific computing, they are different). The work here is reasonable: Build a maintainable piece of software, no need to scale for billion users or sth like that.

The downside is that the jobs are contract-based, as is every academic jobs.

So I'm thinking of switching to a more practical, less competitive driven field as well.

What are you passionate about? Find out what that is and see if they can use engineering at all. You’d be surprised at what you will learn to do the thing you love. We all have things about engineering that we don’t necessarily like but that isn’t any reason to give up. Keep trying, I hear the market is hard right now due to some type of pandemic.
I don't have recruiting experience in the field on either side in the past 5 years and likely am in a different country anyway, but from what you wrote I'd tend to believe that your selection of companies you applied to was a bit unlucky/wrong. Try smaller companies that aren't 100% IT focused so they have no pressure to hire rock stars to fix their issues with their 800 mediocre software engineers.
I don’t think I hate the field . I have my niche corner that I love doing. I tried to get out there and do something else and had the exact same burnout feeling you are having about tech interviews.

Taking a break from this all for now. I cannot afford to spend a few months monkeying around leetcode to work on stuff that I know I will never need in the job. I’d rather spend that time with my family.

Wouldn't the higher salary translate to retiring younger = more time with family?
I used to think so, when I was single and working more than I should. But, I have a one year old at home now, and I’d give anything to reduce stress elsewhere and spend more time with him. He is growing so fast right in front of our eyes, and I do not want to miss these moments in our lives.

Of course, I’m not talking about slacking to the job to use family time. I’d like to work in a low stress environment and shut work off when I need to. Interview prep demands me to spend a lot of hours on toy problems in leetcode which of course I can’t do at my work time. So, I decided to not do it, atleast now.

Sorry for going offtopic

"How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?"

Looks like the interviewer has atleast read the first chapter of "Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems"

If you read and understand the above book, I am reasonably sure that you can crack most of the system design interviews.

And never again need that knowledge on the job you apply for.
(comment deleted)
I would agree with you if we were talking about leetcode style data structure/algorithm questions, but system design is almost always relevant. I found 'Designing Data-Intensive Applications' to be a useful book beyond interviewing.

Even if you are not working on a product that requires supporting millions or billions of reads / writes, knowing what is overkill and what isn't, for your project's use case is still useful

Nah, you use similar knowledge all the time in lots of different systems in different fields. You’re not designing a production system in an interview scenario; you are demonstrating critical thinking, showing that you understand how to break down a problem and sketch out a solution, and can clearly and thoroughly explore the different constraints that need to be considered. These are general purpose skills.
Why do you think system design knowledge would not be used on the job? New systems are built all the time. Existing systems are extended/refactored all the time.

System design interviews IME are very applicable/productive actually. Especially ones that put you in a non-greenfield scenario.

Getting good at those requires studying topics that actually are relevant to the day to day job. The Kleppmann book referenced above is one of the bibles IMO. A software book that's actually worth the paper it's printed on.

Thought the same. He probably saw those JOIN vs Queue vs Hybrid designs and thought that he understood the entire system. Probably never even touched a system of that scale.
I feel ambivalent about this kind of question. You might argue that it's required general knowledge, but relying on having read any one particular book seems arbitrary and a "gotcha" question.

I could be missing something about US software culture - is this considered a must-read or "celebrity" book over there?

It's well-recommended in tech forum/enthusiast circles. It actually is a good book and worth reading if you're a backend dev.

But the % of devs in the US that have read it is probably below 2-3%. Most of us (devs, not HN users) don't read or do code after work.

It’s interesting how much of a bubble we can be in, that everyone’s coding/studying in their spare time. I do wonder what it actually looks like for the broader tech community.
Try applying for roles outside of tech companies if you haven't already. It can potentially be a lot less stressful while still being fulfilling and paying well.
I apply to insurance companies, banks etc, roles which are not glamorous.

Same deal there, in fact I just bombed another test. Had to do a whole bunch of things to a graph (4 part thing) in 45 minutes.

One look at the question and i knew I was toast.

Rule #1 of software industry: Never judge the work nature by the interview. Cracking interviews is a different skill which is completely different from what you need to solve real problems for real people.

If you quit now, it means the gatekeepers have won and I don’t think these gatekeepers as a good enough reason for someone to quit. As to when to know to quit: there will come a time when you will burn out. When writing code doesn’t excite you as much as it did before. When you find yourself in this zone for say 3 weeks in a row, take a break and try something else. There’s a fifty fifty chance you’ll comeback and about the same odds that you’ll find something else interesting too.

I wonder if you’re suffering with imposter syndrome - the feeling you’re not cut out for the job and everyone else is so much better. I see you’ve asked similar questions to this since 2016. It’s something most of us face in our careers and really sucks when it happens.

Look at some positives. You’ve been working in the field for a few years so you can clearly do the job. You’re applying during a pandemic and the market is tough right now. You’re clearly interested in tech or you probably wouldn’t be hanging around on HN on a Sunday, and this in my mind already puts you above many others!

So what can you do? Apply for jobs that don’t require algorithmic/systems interview tests. They do exist, I’ve worked for a couple! Or, game the system by studying interview questions (Crack the coding interview, competitive programming sites) so you’re more prepared. Of course you could also look for another job in a different industry, but I don’t think you really want to and are just a bit anxious with interviews. Finally, consider moving into SDET. A background as a developer makes the transition quick, you’re immediately competitive compared to those without dev backgrounds, and salaries are similar. They don’t tend to have the same interview questions.

> I see you’ve asked similar questions to this since 2016

That's good context. And to OP's point,

> I have been in the field for a few years...this field has never been my passion, just a profession I like.

I've been programming professionally for 10+ years, 20+ when you add in college and what I did growing up. That, and computers are a hobby of mine, and I definitely have a passion for them, and I know a lot of stuff beyond my day-to-day work. I've worked for multiple FAANGs, at other big name's you'd recognize, and startups you wouldn't. I get past phone screens probably 80%-90% of the time and get offers 33%-50% of the time.

There are a lot of people like me in the industry. We're not a majority, but it's hard to compete with us, it's not helpful to compare yourself to us, but it's hard because that's what interviewers compare you against.

> Apply for jobs that don’t require algorithmic/systems interview tests.

Yup. Companies also go easier on system design when someone has ~4 years of experience.

> Crack the coding interview

I can't remember which one of these I skimmed, but I didn't find any of it super helpful. I might be the wrong audience.

> competitive programming sites

Not the "competitive" ones, but if you haven't done 50 medium questions with some easy ones thrown in on HackerRank before doing a round of interviews, you haven't prepared. I'd actually avoid the hard questions because they usually involve a twist on one particular algorithm that will take a while to implement, so you'd never actually see it in an interview.

The important thing is to know your way around heaps, trees, tries, linked lists, graphs, etc.

I found this book helpful, but I read it specifically because I'm better at coding than algorithms. It's not great for knowing how to combine standard data structures to solve problems or how to traverse a tree.

http://mimoza.marmara.edu.tr/~msakalli/cse706_12/SkienaTheAl...

> We're not a majority, but it's hard to compete with us, it's not helpful to compare yourself to us, but it's hard because that's what interviewers compare you against.

This makes me so sad. I've prepped hard on leetcode for 6 years (since freshman year of college) and I haven't had much to show for it but a total compensation less than Stanford new grads despite two years of tenure. I wonder if there's any point of even living most days.

Don’t be so hard on yourself mate, you aren’t defined by your job or salary. We should try not to constantly compare ourselves to other people like this, there is always somebody better off or more successful.

Based on nothing but these two comments I’d much rather hang out with you than I would the commenter above you. So there’s one thing you win at.

Yo buddy, no need to compare yourself with others. You should compare yourself now with your past self six years ago: are you a better person now?

If you've spent 6 years doing leetcode and you're still bad at it, then perhaps realise that such problem solving games are not for everyone. Like sudoku. I prefer the word game where you find the word in a grid and I hate crosswords.

We're all different and that's what makes us unique and interesting.

> You should compare yourself now with your past self six years ago: are you a better person now?

Kinda? I had more potential then, I could have done Facebook after sophomore year and had my career set for life. Instead I failed that interview and my career has been treading water because I had to join Amazon out of undergrad.

Its apparent that I"m not good at these things...the problem is my happiness and financial security are related to them and I'm not doing well!

> I had to join Amazon out of undergrad.

You're suffering for a lack of perspective or gratitude. If you're a millennial, realize that you were probably told that you could do anything (and thusly everything), and the effort it would take was trivialized... Both are incorrect. You most likely can do 1 thing super well and it's going to be a shit ton of work if you want to compete against those equally competent.

In what way? I grew up Asian and went to a state school, I got the message that I was incapable of accomplishing anything from an early age!
> state school

5 of the top 10 CS programs are at state schools.

> I got the message that I was incapable of accomplishing anything from an early age

Apparently you took it to heart and can't recognize what you accomplished.

> a total compensation less than Stanford new grads despite two years of tenure.

> I failed that interview and my career has been treading water because I had to join Amazon out of undergrad.

WTF? You have an engineering(?) job at Amazon and are complaining you might have done better if you got the internship(?) at Facebook? And that with two years of experience, you make about as much as a new grad? Is this a humble brag?

You're probably in the upper 25% of tech for your experience and the upper 1% in the developed world for your age group.

Is the meat of your complaint that you'd be making more money if you did a stint at Facebook? Don't think it "sets your career for life." Big companies can even hurt carers long-term because you're not exposed to as much. Facebook on your resume will get you calls from Google recruiters and calls back from wherever you apply, but it won't inherently help you pass future interviews.

> I wonder if there's any point of even living most days.

This means you're valuing the wrongs things w/ the wrong weights. Compensation is to help you live more comfortably. If your pursuit of it is making you less comfortable (than the compensation received) you're doing it wrong.

It took me a long time to understand that interviewers are often less competent and giving them than they are at coding.

Typically an interview question should be hard enough to test the limits of any candidate. The goal is to see how someone thinks and how honest they are when faced with a technical challenge, not what they were able to memorize.

With that in mind, take heart if the interviews have been sometimes off the mark. Most interviewers were just pulled from some normal day to ask some questions and often unconsciously see it as an opportunity to show off a bit. They have the keys and you need them.

That said, if you do not feel passionate, I would recommend playing around with some other areas like frontend or ML at home, on your own time, and see if any of them click.

In addition, I mainly have run interviews as a 1st or 2nd round screener, and I suggest to continue with interview process even if someone kinda blows a question as long as it seems like they're not lying about their experience.

For interviewing experienced candidates I'm often trying to come up with a problem the team has faced and turn it into a series of questions. When it doesn't work, I feel like there's a 50:50 chance that I didn't phrase it well/didn't include enough info (but unless I happened to interview several candidates that week and we just don't have time) I am fine with letting a second screener or the whole team talk to a candidate.

But I guess OP is probably referring more to final interviews. But for me at least, phone screens really are ok to get things wrong, in case there is anyone out there with phone screen jitters.

I've seen no evidence that most interviewers would be any better at coding.
You'll have to, soon or late.

Sooner or later you'll start facing ageism. It is rampant. The interviewers will look younger, the interviews scarcer, the technology stack will move to a new shiny thing that is a rehash of ideas of previous fads of old, people will start using arguments like "fitting into the culture",...

I read somewhere a long ago that software developers have an half-life of 6 years. If you're long past that, better really start acquiring other skills.

I'm a 44 year old software engineer. I've done other things before, but I've gotten an offer every place I've interviewed.

This simply isn't true. It might be true in silicon valley or some other hipster magnets, but there's definitely a market for those of us who have been around for decades and been solving problems for decades.

I am based in the North East, also over 40, and my experience both of you are correct. Agism is absolutely real, but opportunities still exist.
What can be done about it?

I'm just starting out in my career, but I find the prospect of future me being devalued because he's a couple years older terrible, and I don't understand why others seem okay with it. Aging is going a second per second for everyone, isn't it?

The point is that there's really nothing to worry about. There are plenty of jobs for those of us not in our 20s.
Agism is illegal already, and yet we still deal with it.

As a young developer I watched great old developers, fantastic at their job and with super deep experience let go first.

My advice is make sure you save a lot. And be prepared to either retire early, switch careers, or start your own business/consult once you have over 10 years of experience. Connect with people, keep in touch, and make sure you stay current with new technologies.

As an outsider to the industry, I have always wondered what the situation is like in smaller companies or for starting your own business to serve local clients.

Part of the reason why I ask is that most of the places that I have worked for respect experience more than youth, and couldn't care less if the firms that they were contracting work out to employed experienced developers in their 20's or their 50's.

As an outsider to the industry, I have always wondered what the situation is like in smaller companies or for starting your own business to serve local clients.

Small companies, typically startups, are diverse. Some like hiring diamonds in the rough, which can include old developers. Other are obsessed with youth.

Starting your own business or consulting are standard and popular options for old developers.

As an industry insider, people complain as loudly as people from any industry. But just look at the actual unemployment and pay rates per industry. Few other industries are better.

Do you stay current?

I see resumes for similar aged people who know cold fusion and Delphi, and those that know JavaScript and Elixer.

I suspect interviews go very differently for the two groups.

I'm also a software engineer in my 40s who has not had trouble getting jobs. My answer is yes, I definitely keep current. Right now I'm working on Gatsby in TypeScript. 10 years ago it was Phonegap and native iOS. 20 years ago it was PHP and a bit of Mac software in Objective-C. By staying current I can negate the main objection that employers have to older developers, which is that our skills are outdated and we're unwilling to learn new things. It's more fun too!
Absolutely. C++, C, Java, Python, learning Rust now. Working on another MS just for fun.

I don't do web dev so I don't give a shit about javascript.

Yah it doesn't mean you need to know everything, but you have to know something current or I think life gets hard for you..
The interview process for software engineering has been broken for some time. Don’t give up.
Sure do something else for a while. In my experience, skills transfer well by experience. It's experience that counts.

Sounds like you're pushing a rock uphill and you need a change. Good luck.

I went through something similar, and ended up failing a bunch of coding interviews. Also I read David Graeber’s book on bull shit jobs, which was really cathartic for me.

I ended up quitting my job, selling my house, and now I’m looking for some property to buy so I can subsistence farm.

This is really awesome - but I feel like it's also an extreme that few are willing to take. One thing which is an argument for software development is that the relatively good salary should enable you to save up and take a sabbatical every 4-5 years. To me, that is absolutely worth doing this job, even if I don't always like it (also, there's also almost nothing I would be qualified to do, at the moment, to be fair).
For sure, and it’s definitely something I never would have planned or expected for myself. It was also partly influenced by COVID, and further compounded by pretty rough break up.

But the funny thing is, I really feel more comfortable and confident than when I was “succeeding”. I used to feel so guilty about a one line code fix taking weeks to deploy, or I used to feel so guilty and terrible for not paying attention during scrum and refinement and all those meetings. I always felt like I was working too hard and not hard enough.

Farming has this reputation of just interminable toil and physical labor, combined with high costs for agro chemicals. But then you read about farmers like Gabe Brown or Chris Trump, idk.

It only takes three nonlinearities to create chaotic, non predictable behavior.

I never had anything to do with farming but I can imagine its appeal. Living from the fat of the land and not caring about planning and preplanning and retrospectives and all that.

Best of luck!

Will look up Gabe Brown & Chris Trump.

About the three nonlinearities, I'm afraid I'm missing some context - never read up about chaos theory...

If nothing else I can imagine the hard work, hours and constant threat of bankruptcy of farming would make me very happy to get back behind that keyboard again!
> I always felt like I was working too hard and not hard enough.

Wow, you've succinctly described how I feel about my dev career spot on.

Not sure if its the exact same sentiment, but I've started to describe my work as "hard in all the wrong ways".
It's true that a lot of software jobs are in fact BS jobs. But is this a conversation that the industry is ready to have? Probably not.
I've read several stories that start exactly with what you're doing (quitting job, selling everything, moving...) where the author didn't stick to the plan but gained a lot from the journey anyway and ended up back in the workforce, except they were more productive and happier than before.

I think it takes a certain amount of reflection to reach a mental state where you become the person you want to be. That is crucial to avoid subversion of the self to the corporation. Once you get there, you can have a more symmetric relationship with your employer. Both parties end up better off. It's a shame that society often doesn't have the "speed bumps" which allow people to do this without going through radical ups and downs, but it is harder to change society than it is to change yourself. I hope you get what you need out of your journey.

Perhaps, but I doubt that I’ll ever be able to go back to a 9-5. Thank you for the kind words, they really do mean a lot.
I feel your pain.

One thing I would suggest is that programming/software engineering is a broad church. Not everything is large scale backend work. I personally get a lot of satisfaction out of frontend development, and the expectations there seem to be a lot less about hypothetical google-scale nonsense that will never happen in the role, and more "do you know the tech?". Despite what people say/think modern JavaScript/typescript is good to work with and in modern web apps that are 10s of thousands of lines of code there is often a surprising amount of actual "engineering" required.

That said, sometimes just knowing some of the basics might be enough compared to other candidates. E.g. for the tweets in 3 seconds thing, you could talk a lot about horizontal Vs vertical scale and what might mean etc etc and go from there rather than a "sorry I have no idea". You don't always need to be "correct" but just show you are aware of the hypothetical solutions to their hypothetical questions.

Good luck. Hang on in there. I appreciate this can be high stakes for tou, while posting an answer here in a comment is zero stakes.

With over 20 years experience in the industry, and some products I have worked on visible in SuperBowl advertisements, I can honestly say during interviews I can count to potato.

Most recently during a live coding challenge I forgot to you, you know, occasionally run and test the code I was working on, and had one hour to complete.

I can also anecdotally report code challenges seems to have become a lot more common in interviews recently.

And yet, we are still lucky to be in an industry with so many opportunities which pay so well. After several months of interviewing, and failing pretty much every code challenge, while being a programmer over 40, I finally got hired. And can happily report things are going well once again.

My advice is train interviewing. Train not just solving programming puzzles, but also doing it under the gun. That's not easy to train for real, but try to get as close as possible.

And there is no harm in keeping your options open regarding a career change. If not the interview process, but the work itself ever starts to really make you unhappy, it might be time to switch careers.

> I can also anecdotally report code challenges seems to have become a lot more common in interviews recently.

At least these are halfway representative of the job. It's a lot better than being told you're two inches tall, in a blender, and the blender's going to turn on in five seconds, what do you do.

Pleeeeease tell me you’ve actually been asked this question in an interview
No, but it's an apocryphal Google interview question. One of these days I'm actually going to weigh my head (another Google question).

I've gotten a logic riddle that involves scissors and rope, asking what my super power would be, and favorite movie. One that was at least based in math was how many ways are there for a knight to move from the lower left hand corner of a chess board to the upper right without moving back (no left or down moves). Not the DP-solution, though, a closed-form equation.

Get in a pool at the deep end with some floaties and some iron ankle weights. Add floaties and weights until you are at equilibrium at your neck line. Hold your breath and add some weights slowly to your hands until exactly when you start to sink. Drown, but include a request in your will to have them remove your head at autopsy. Have a friend, loved one, or coroner weigh it.
I fly out. If they're going to make a completely unreal scenario, I'll use a completely unreal solution :)
i think that is the correct answer, reverse the blades and then jump over them so the air current forces you out like one of those indoor skydiving places
Try to dismount the blade from the axle (they're designed to be easily separable, at least in the model I own) and just have the axle rotate and do nothing?
A bunch of people will tell you you just have impostor syndrome and 10x programmers are a myth and don’t believe everything you read about Silicon Valley and blah blah blah.

I dunno guys, maybe some of these people really should try to do something else! It is no shameful thing to leave the field of software development. We’ve had a huge influx of people and interest to our field in recent years, is it really so crazy to think that some of those people and some of that interest is not well-placed?

Were I to ask these questions in interview, I think I'd be looking for the candidate to have a bash, do some napkin math, come up with some contraints/failure modes and to communicate their assumptions well.

> How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?

In this example, I'd be looking for you to have fun, ask a bunch of questions and to make some sensible decisions based on the information you have.

To be honest though, it does seem strange to ask for a "basic CRUD development role". Admittedly, if I had an experienced candidate apply for an entry level position, I would be a little leery. Why do you think you're entry level despite a few years experience?

I'd maybe take a break from interviewing, and use the time you're currently spending on interview prep on building something as a personal project. If you enjoy it, then hang in there. If not, maybe you should stay at your current gig (you don't mention anything about this) and figure out what you do enjoy.

P.S. I'd also stay of Hacker News for a while. While I love it, I will admit that there's a higher degree of obsession here than in wider industry.

Sometimes, the interviewer will purposely give you a question beyond your reach. There are a few reasons.

- To zero in on what you already know.

- To see how you react in a situation where you don't have all the information.

What is your attitude when you are in this situation? Do you shut down? Walk out? Or are you able to form open ended questions to get the missing information?

In any type of job, you're always in situations where you don't have all the answers, and need to work with others with a positive attitude. I don't know how you are going to turn this around, but maybe looking at the situation from another point of view is the first step.

Just the fact that you are asking the questions tells me there is hope.

I ask questions beyond the job scope in interviews, but I when the applicant looks nervous, I tell them that's what I'm doing. I'm often pleasantly surprised at the answers and it's one of the more useful aspects of the interview for me.
I would much appreciate the info that that's what was happening. The context helps to calibrate correctly.
I'm sure it would make you less nervous (which is why I tell them), but would it actually help? Either the role is above your skill level, and it won't matter how you answer the questions, or it isn't and doing your best will be good enough.

Either way, there's no need to be nervous. IMO, you should focus on just answering the questions as best you can.

Helps as is in is this person looking for a specific bit of trivia that they'll write me off for not knowing or is it just an exercise to see how I reason my way through unfamiliar territory.

The fact there are so many hidden requirements you're trying to tease out is where the nervousness comes from.

One of my best interview experiences was when I was asked a few relatively simple questions, looked over a class of their code and describe what it does and some mistakes that might be in there, and went over a code sample I brought in.

Then he said, "Okay, I know you can do the job, now I'm just curious how much you really know."

And then the really hard, low level questions came out. I said I didn't know a few times, and struggled even when I sorta knew the answers, and he took it in stride and played the role of teacher and taught me a few things that I still remember today. I didn't feel pressure like I did in other interviews, despite being much harder questions than I get asked elsewhere.

If you do it that way, I'm cool with it. If you start right out the gate with the hard questions, and you make no indication that you're not expecting right answers, I have no choice but to assume I better get them right, or else (especially considering I haven't moved on to the next stage of interviews before for getting one question wrong, or saying I didn't know for one of the questions, or the HR person was looking for a particular keyword I didn't say, quite a few times).

Most interviews unfortunately have little to do with the actual work and require significant practice to do well.

> One question I was asked was "How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale.

If you're technically competent overall, you should be able to learn enough to make your way through this from a few hours of reading and watching youtube videos. That's what I did when I was in the same situation, and I passed the architecture interviews at multiple FANGs.

Same for the coding questions. You need to spend time reading interview books and doing leetcode.

> If you're technically competent overall, you should be able to learn enough to make your way through this from a few hours of reading and watching youtube videos.

I actually did read up a bit on it later. This may not be what twitter is doing right now, but in the videos i saw, this 3 second thing is not possible. You can batch process the tweets and use websockets or whatever to send them, or just let users get them when they log in to their accounts. That's what i read anyway.

I don't care to look much deeper unless I'm actually working at Twitter, and it seems that's my problem.

> I don't care to look much deeper unless I'm actually working at Twitter, and it seems that's my problem.

If you want to get a job, you probably just need to suck it up and spend some time doing things you don't want to do.

Spoken like someone sitting in the ivory tower of a FAANG.

> some time

Perhaps it was my fault in not explaining the amount of effort I did put in, but if the implication is i need to learn the intricacies of the architectures of every platform out there, then truly this is not the industry for me.

Awesome. Less competition out there :)
This comment comes across as emotionally charged. I recommend taking a step back, decompressing, and re-approaching the problem.

You should absolutely _not_ learn the intricacies of the the architectures of every platform out there. You should, however, think through some of the base architecture building blocks and what it looks like when they work together. You should be familiar with queues, databases, caching, basic tcp/network, observability (logging, metrics, tracing), and also basic application design (separation of concerns, testability, etc). Eventually, you will become familiar with different solutions for each of these components. Sprinkle in some reading about other companies architectures and you can start to see why they might have chosen what they did. Eventually, you can start talking about trade offs between different solutions. The biggest growth opportunities here would be discussions with folks who are more senior to you about these things and why they think they way they do about them.

As a former math teacher, it sounds like you are complaining about not knowing every problem's solution instead of learning the building blocks on how to solve the problem. As a silly example, some algebra students can only solve "3a + 2 = 11" because they have seen "number next to variable plus number equals a number, so I have to subtract the number without a variable from both sides, ...." and when they see "11 = 3a + 2", then they say they've never seen "a number equaling a number next to a variable plus another number, how are they supposed to memorize all these problems?!" Heck, some students only solve that problem by guessing different values of 'a' until they get the right answer. Other will start tossing operations at the wall until something sticks (this feels like most modern development honesty). However, students eventually "get it" and realize that they can solve "3(11 + 3a) - 3 + 66a = 16.6 + 113a/245" even though they've never seen that problem before.

The biggest hurdle for me to get the resume selected. How do people do that!?
It's easiest when the recruiter contacts you. So Linkedin profile, email on Github, etc.

Starting out is hard, the first job really is the hardest, and I doubt recruiters are looking that hard.

Once you have several recognizable, well-regarded companies on your resume (or Linkedin profile), even sending in your resume through an online submission will normally get you a callback. That, and you'll meet at these places, so the next time you're looking for a job, you check with friends to see if they're happy where they're at and if they'll refer you.

I'm starting out. How can I make recruiter contact me?
I'd like to give you a different view on the interview / question you mentioned. While it's certainly possible the interviewers are running the process badly (many are), pretend they have good intentions and answer to that. The tweet question is basically: can you think about performance.

Let's say you don't know anything about queues or distributed systems. You can say you never worked at that scale. You can also say what you think would be the number of followers (millions+), what would be the time needed for a write you a database (~millisecond each) and why that won't match 3 seconds, so you know how much time you have per follower. You can talk about the problems (disk persistence is slow, network is slow, keeping more in memory is better), that having many processes / machines working at the same time will likely be the way to go. You can try to pull the interviewer into discussion about it.

You don't need to know the answer (let's be honest, the interviewer didn't know it in details either) to actually talk about why the problem exists and what you understand about it.

> You can also say what you think would be the number of followers (millions+), what would be the time needed for a write you a database (~millisecond each) and why that won't match 3 seconds, so you know how much time you have per follower.

While we are on the topic I am really curious to know how they solve it actually.

Twitter has changed this approach few times I guess, earlier it used to be simply insert tweet into a collection of tweets, and then when you load use timeline, look up the people they follow and find/merge those tweets. But it's going to create a lots of load on systems. Another approach is to maintain a cache of user's timeline(mailbox of tweets), when user posts a tweet, lookup all the people who follow that user, and insert the tweet into each or their timeline cache. results have be pre-computed, so less load. Both approaches fails when you have folks with lots of followers, so may be they use a hybrid of these approaches. this is Discussed in detail in "Designing Data-Intensive applications" book.
> Both approaches fails when you have folks with lots of followers

The traditional mailbox architectural model can be improved via a few methods though to increase scale.

1. The entire tweet doesn’t need to be duplicated for each follower, only a lookup reference (“ID”) to the tweet.

2. Unlike in a traditional mailbox sense, each user’s timeline is a bounded collection. So rather than maintaining every tweet for every followed user in the timeline, it’s capped to the X most recent.

My guess would be lots of distributed queues, some resources dedicated to heavy lifting of few vs random noise of everyone else, and storage layer which allows lots of append traffic without blocking. Maybe some storage special casing like "randos get their tweets serialised on your timeline, but because you're following Taylor Swift we'll just query that common feed in parallel rather than copying it".

They've got lots of engineering blog content, there may be a better answer here https://blog.twitter.com/engineering/en_us/topics/infrastruc...

I don't work at twitter, but one thing you can know for sure is that the tweet needs to be distributed to multiple machines to meet this performance requirement. A single machine probably could probably send three million trivial messages in three seconds, but if you mix in any business logic it's not gonna happen.

What I'd do is have a shared highfanout queue. Once you have more than say 10k followers your tweets go to the high fanout cluster. You'd have hundreds or low thousands of machines, each of which serves a slice of consumers. When the tweet is sent, you write it to this queue. Each consumer is pulling from one of the shards. If you have a thousand workers that means each worker only needs to send a thousand messages in three seconds, which is very doable.

Only about 180k twitter users have more than 20k followers. If each users tweets every 200 seconds, which seems like a high estimate, then that implies a load of about 1kwps for this system, which seems doable, especially if you have a small intermediate layer of distributors which consolidates the write load.

That's just my sketch.

Right. Seems very doable to verbally sketch it if you have ever seen something like a fan out.

If you only worked in single Wordpress servers, I could see it a big leap.

To be fair, having worked in a twitter-like domain is a valid selection criterion for any internet company planning to handle lots of traffic. It is possible that working on wordpress servers would not prepare you very well for such a job. Being unable to even articulate a basic strategy for handling a problem like this would be something I would consider a yellow flag for a potential teammate. It would make me question whether the person would be able to help me in my work.
If someone's resume was only wordpress sites, and they somehow made it into a technical interview for a team that managed twitter-like systems, I'd consider that a critical failure of the selection process. As an interviewer, I would try and make that experience as painless as possible for the candidate, and if not totally shocked by their abilities, have very direct conversations about the shortcomings of the selection process with whoever brought them in in the first place.
> I'd consider that a critical failure of the selection process.

Me too. Unfortunately, I have no control over recruiters or the questionable candidates they sometimes choose to send in. I just have to do my best to ascertain whether the person is a good potential teammate.

Yes, and my point is to not take that critical failure out on the candidate.
I wouldn't "take it out" on the candidate. I always treat my candidates with due respect and courtesy. I would no-hire them, though, if they were not a match for the team.
Follow up question for a little clarification. Would you write the tweet into the queue once or append a message to the queue for each follower containing the tweet id?

Seems to me you’d need to do the latter right? That way you ensure you process each follower at least once, but each worker doesn’t need to be aware of others it’s just pulling messages from the queue.

I don't think so. Maybe for the active users, but maybe not even then. Most twitter users are inactive and so you wouldn't want to fan that kind of data out so widely. You'd either have to pay enormous gc costs or enormous storage costs for someone like Donald Trump (0.5 GB per tweet assuming 64 bit tweet IDs). But I think I would just have a sharded in memory cache of recent tweets for the big dogs and if someone loads their feed you would do lookups into the sharded db for each subscription. You'd then cache the feed for the user so you don't have to do that lookup very often.

If you were going to write, you'd write into the cache, rather than persisting anything to disk. That's my guess at least.

I can’t speak to how it’s actually solved, but I can give a sketch from my experience. The game is generally to (1) consider denormalizing data and (2) rethink when you have the computer do work.

Denormalizing data refers to moving away from a model where you store one and only one copy of each ‘tweet’ in something like a relational database, to a model where you might actually store a separate copy of each tweet for each follower. That is kind of an extreme example of denormalization, but it’s a good way to illustrate how you could make it near-instant for any user to load their twitter homepage. If you stored the interesting tweets of every person I follow in a table just for me, you would make ‘reads’ (loading my homepage) incredibly cheap, but writes (someone with a lot of followers tweeting) very expensive.

Those trade offs exist everywhere in a system like this, which is (2), you get to decide when you do work. If celebrities with a million followers tweet an average of once a second, but people load their feeds a hundred thousand times a second, it is entirely acceptable to do 10000x more work for a celebrity tweet posting. This is actually the same concept behind using indexes in relational databases, but done more explicitly.

My personal answer to this question would probably start somewhat space-innefficient. I would take each tweet and put it into an event processing queue which writes a reference to it into each followers feed. This sounds nasty, but it scales with the number of writes, not reads, and we have less writes, and it scales linearly with the follower count of the writer. I would then improve efficiency by thinking about dormant accounts, caching, and maybe doing a bit more work on read.

The other requirement worth thinking about is the 3 seconds. How many followers are going to be watching for that 3 seconds to matter. And how much slack can be in that? Could 1.5 million followers be sleeping and not need the tweet for a few hours or only when they next sync.
If it's a pub/sub model, you wouldn't care at all about how many were watching right? You'd just publish.

And for something like Twitter, you'd probably Publish but then also log to some kind of "Notifications" store, so if a user did care but was not actively watching, on their next subscription they'd receive the messages they'd missed.

The plot twist in this system is blocking, muting, and locked accounts, which make denormalized timelines a lot harder. You basically need a very fast side system that can be queried for visibility, especially because eventually consistent privacy features (i.e., running a "account became locked" feature on a similar queue to "fan out tweets" in your model) are not treated lightly by customers.
One trick to problems like this is to solve it differently for celebrities, head queries, etc., than for less commonly accessed data. Essentially have two different systems and combine the data because designing a single system that meets both scale requirements is harder.
You also don't need to know the answer in order to ask lots of clarifying questions. From the question alone, without an greater understanding of the landscape, I suspect the question is unsolvable.
I really enjoyed reading: Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems Book by Martin Kleppmann

Can't recommend it enough and it works through a lot of this talking exactly about twitter.

I work at scale that is similar to twitter, just different concerns. Billions and billions of events where the results are expected in seconds or less. This is a beautiful example of how to ace a design interview. While we prefer folks with experience at scale, we are happy to bring on candidates lacking that experience when they can break down a problem like op did. I recently hired a dev who did not know about not trusting the network and duplicate delivery of packets and who was running into these issues at her own work (but was not senior enough to be "allowed to work on it"). However, she did break down our similar interview question. Even though she did not "ace" it, we can see her potential. Hired.
"I am thinking of ending things." :D

You shouldn't rely on capitalist strangers's thoughts about your situation... (It's terrible what some ppl are doing... wake up...)

Find a normal job to survive... while you try to make something cool in other side with your hacking skill...

So your first mistake I believe is to think the interviewer is just testing you on a narrow exercise.

Approach it differently: most of the time they are rooting for you! Hey you might be their future colleague. That in mind, focus on tackling the exercise showing how you think about it. That's mainly what they want to see and if you are transparent with them about the fact you don't know well this layer so it might be a little bit handwavy it is fine, manage expectations and when you guessed or rebuilt something existing right it might even play in your advantage.

On the passion side of it, I have to be blunt, it is a self fulfilling prophecy here. If you are not interested by the field and related fields, the lack of curiosity will impede your growth. You won't get the next interesting wave, you'll stay on mainly boring things and this will reinforce your feeling that the field is boring. Computer engineering is a non stop learning endeavor.

On HN there have been numerous interview horror stories that I just didn't understand. Recently though, I've been through one, so I think I understand what you have been through a bit better.

Interviews are done by people, trying to size you up. Even though their question would seem to be humilitating you can still try to make the best of them. As others have already posted, getting through the interview is a skill on itself and I have seen interviewees that passed full score and have been terrible to work with and vice versa.

The real question is: is this the only profession you care about? If so, it's time to learn to game the interview. If not, do feel free to give a shot at the other profession. It may benefit from your experience as a software developer.

As many have posted on HN before, interdisciplinary employees have higher employment value.

Anyway, good luck with whatever you decide.

The goal behind these questions is not for you to find the right answer, unless the interview is about a specific topic where you should know the right answer.

The goal is to have information on your thought process, how you solve problems, and how you communicate.

- Assumptions you have

- Your ability to clarify ambiguities

- Your interviewing skills: with coworkers, clients, reports, managers, applicants, you spend a lot of time interviewing people to extract meaning, facts, context, information.

- How you frame a problem

- How you behave in unfamiliar waters: you will build things you can't find the answer to on Stack Overflow or in a blog post.

- Your awareness of tradeoffs: do you make decisions being aware of the tradeoffs involved, do you have magical thinking, or cult of the tool/framework.

>"How do you make sure that a celebrity's tweet reaches all of her followers in less than 3 seconds?". I had no idea, I have never handled that kinda scale.

Would you not like to tackle that problem and work on systems that handle that kind of scale? One argument against this is "who cares, they're not twitter so why are they talking about hypotheticals. They should interview me on what I'm going to do on the job". This is very valid, but there also is a counterpoint to this: this might stem from the desire of the interviewer not to be biased. If they quizz you on a problem they have been working on for the last year, they'll probably get frustrated by the fact you can't find the right answer to the questions. Or worse, they'd be impressed if you happen to find the exact solution they have converged to. You'll appear to be much much "smarter" than you are, and the expectations will be set high since you found the answer in a few minutes rather than a year. This is dangerous.

>I'm sorry if this is incoherent.

It's very coherent and understandable.

>I haven't even applied to jobs for weeks now, because in interviews I feel like I'm getting humiliated and I want to apologize and end them half-way.

We need to reframe this. If you put yourself in the shoes of whoever is interviewing you, they are trying to vet you because they care about their team, and they want to hire someone who will raise the bar of the team. From another perspective: if you were already in the team, wouldn't you want whoever is in charge of hiring to be an effective gatekeeper? Would you want a person who is impressed by an applicant spitting a few buzzwords and frameworks? That applicant then lands the job, gets assigned to your team, and becomes the reason you apprehend going to work? They may have been burned by past "bad" hires and have decided to tighten it up. People holding a team back, undermining decisions, commiserating while never explicitly voicing their concerns. This is a problem that must be prevented from happening, or dealt with swiftly if it happens.

Yes, they're not FAANGs but it's precisely that! They may be a small team in which you as a person are a large percentage of the workforce. If you are one in a team of 10, you represent 10% of the workforce. If you were at a FAANG, you'd represent a much smaller addition.

Furthermore, if it is a small team, it will hopefully grow, and you will take on more responsibilities, and you'll eventually hire people. They need to be very, very, disciplined about hiring people who will shape the future of the company.

It can be frustrating if you look at it from your perspective of someone who has a lot to offer and feeling humiliated, but that feeling is a very mutable reality once you factor in these different perspectives. Maybe even enjoy the interview, enjoy interacting with smart people, ask questions to learn new things, discover things you didn't know were a thing, and be glad for that team they have someone shielding them. You can ask them for feedback.

It is a fit that didn't ha...