The situation is not going to improve as long as business stakeholders and engineering managers (some closer to MBAs than actual engineers) think of software engineers as construction workers. They think we are fungible, they don't understand the craft of programming etc, and have very short term mindsets. Took me a while but then I realised that I needed to interview my prospective employers as much as they were interviewing me, and to just ignore those not worth working for.
We are fungible. Most software development is not rocket science. If any of us got hit by a bus tomorrow HR would send our next of kin “thoughts and prayers”, a flower and tell them how to collect our life insurance and then immediately open a req and get flooded with hundreds of probably good enough replacements.
The OP thinks that candidates spending a lot of time on applications is OK, as long as the company shows respect by spending a lot of time themselves. I think this is mistaken - I care about how much time I have to spend, and am a lot less concerned about how much time the company takes.
There's a trade-off: if a company spends more time / requires more effort on an interview process, they can get a better signal on the candidate's abilities, but then they'll lose out on candidates who are unwilling / unable to commit this time. This might just be a hard trade-off in recruiting.
I think it's reasonable to make a donation on behalf or provide an honorarium if someone makes it far enough into the process. Something like a $250 gift card or equivalent.
Not enough to make it worth farming interviews for compensation, but enough to show that the company appreciates that you spent 2-4 hours working on their take home.
> most interview questions have very little to do with day-to-day responsibilities. all good software engineers are generalist and live coding does not select for generalists
If I had a dollar for every time I heard this (flawed) argument, I’d be rich and would no longer have to sell ads on my Hacker News comments. I’m going to get hate for this unpopular opinion but here we go.
So often, “But Leetcode isn’t like REAL programming” is the siren song of the programmer who probably overestimates their coding skills and experience.
Yes, I hate to say it - live coding is actually one of the best signals you can get on a candidate’s seniority and ability to program a computer (and more importantly, their core computer science skills). A good interviewer is trained to know how to probe your CS knowledge during this, and will watch how you structure code, break down problems, debug, and think about testing. They will even ask you to make changes to see how coachable you are and what you might be like to work with. It’s not about inverting a binary tree while sharing your screen, it’s about showing me how you solve a problem, then translate what’s in your head to code.
Take home exercises provide little to no signal, and screen out people who have families (who wouldn’t bother with a 4 hour take home exercise after work). I don’t want to see how you Google, I want to see how you think.
These candidates always want some version of, “But trust me, bro! Hire me: I’m a senior engineer, I don’t remember how to Leetcode! I’m good, I promise!” But what they won’t admit to themselves is that a good senior engineer is able to do all the things a junior can do PLUS all the things a senior can do.
It’s not perfect, but I won’t hire anyone that can’t pass a live coding round.
This comment brought to you by Poppi. Poppi: it’s soda for people who are silly enough to believe soda can be healthy.
> It’s not perfect, but I won’t hire anyone that can’t pass a live coding round
I'd like to add two points to this:
First, I like that you said "live coding" rather than leet code. The floor for live coding should be super low, with a high ceiling and lots of flexibility. That allows you to say, nope, they didn't pass the floor level, easy binary decision, no hire. Pick a fun toy to build in 90 minutes and the high ceiling + flexibility will yield tons of signal from applicants.
Second, I see live coding sessions like this as a positive sign from potential employers. It lets me know that my future colleagues will have some baseline level of competence. If you've worked on a team that didn't do live coding, and you've had to carry water for someone who can't actually do the day-to-day technical "hard skill" work of software engineering, you probably feel the same way. Never again.
I start with a simple problem with several answers of varying difficulty and literally instruct the candidates to code the first correct solution they can think of.
Nested for loops, brute force, it’s absolutely fine.
I just want to see you translate what’s in your head to code. I want to see you structure code, use data structures, etc. Then we will talk through optimizations, refactoring, testing, edge cases, etc later.
You want even the under qualified candidates to feel like they got some pieces right and had a fair shot.
As you start probing deeper, the top candidates stand out. They answer your questions about the differences between data structures, how you would test this piece, how you would monitor that piece in production, how you could parallelize it, etc.
The primary problem with hiring is that developers are a single status with not performance benchmark. The solution is to segment need by capability.
Let’s face the reality that most developers will never be able to write original software and just put text to screen using a tool or framework. Don’t call these people engineers. These people are the assembly line of software. Measure them according to desired patterns. They are copy/paste but smarter than data entry and understand some of the restrictions in place. Expectations are low and compatibility and replacement are the key business values.
Next are the people who test software, the QA. We expect more from these people and then work them harder for less money at a lower level of reputation.
Next are the people who evaluate software. These people are closer to engineers. These people include accessibility, security, and performance experts. These people are more like a combination of QA and senior developers. Evaluate these people on these criteria: written essay, technical knowledge, force them to measure things in real time and see how they perform.
Next are the people who actually write software applications. Let’s call these people solution delivery. These people are similar to junior architects and actually build things. These people should be evaluated only on the basis of organizational capabilities above that of the engineers that measure things.
Finally are the software owners. These people resemble a combination of project management and junior architects. They must have the experience to know how to build original software, like the junior architects, but also a planning vision to push though demands from competing stakeholders. There is busy savvy to this comes from a solid engineering planning vision plus superior communication skills most lesser software people never honed. Think of these people as senior principals with real authority. Evaluate these people on their delivery experience, using numbers, and reputation.
I don't want to give away my secrets, because this has actually worked really well for me and I'm afraid that I'll lose my edge as an employer - however I have a very small neck of the woods and HN seems very US-centric so I think I'll be ok.
What has worked for me, honestly, is being directly involved with my hiring pipeline and having conversations.
It seems like common sense, but there's a lot of reasons not to do this and people will make good arguments to prevent it. "What about bias", "your time is more important" etc;
However, bias is an unfortunate consequence of selecting for value fit anyway and I can't think of a more important task than selecting the members that will be the future of the company.
I've had some positions that were open for a weekend where I got 400 applicants, and yes, it was daunting to go through and give each of them an honest shot, but you know: I had to do it. What's the alternative? I might miss a fantastic candidate because someone didn't understand what I actually need.
Evaluating programmers and "devops" people is just insanely hard, technologies are mostly fungible. If you can write one C-like language you can learn the others in about a month, but what can't be taught is what your values are, if you think in a systemic way, if you're easy to work with and respect others.
So, my solution is to have a conversation, challenge what they know, see how they react when challenged, see how they react when they reach the end of their knowledge and see what they're most proud of and if they get excited by it.
No gotchas, no esoteric internal handshake, just: are you defensive? Are you curious? Are you passionate? Can you communicate effectively and are you intelligent.
If you hit those, you can do anything.
"How do you even know who to interview?"
This is a hard question, for me there's not a lot of candidates that are physically located in my region, so those go through as long as they have something on the CV that looks relevant. For others it's a combination of: would it be easy for them to move, have they worked remote (and can do it in a region where I have a tax entity) and how strong of a fit to the role is the CV, lots of experience in games would be what I expect since I work in video games - but if you're going for a backend programming role then: what have you built and what do you list as your responsibility to achieve it?
With this mindset I managed to build a high performing, high trust team that executed very well on (literally) impossible demands. If the ownership of the company was better that team would have easily been world class.
We also exceeded dunbars second (clan) number with the size of the team, so it wasn't intrinsic to small teams (80+).
As much as I dislike leetcode style interviews, if I fail one of those, I learn what I can and move on.
Failing a take-home is an entirely different thing. It's a huge loss in time and mental energy.
I've only done 3 of those in my career and only because the projects sounded interesting. 1 of those 3 resulted in a job offer which I can now confidently say in hindsight was the worst job in my career (...so far!).
I'm now leaning towards just filtering out companies that do take-homes because it signals to me that they don't care about their candidate's time and how a company treats its candidates is usually a good indicator of how they treat their employees.
I've had the complete opposite experience, and feel the complete opposite way. What is there to learn from failing a leetcode? It feels like luck of the draw - I didn't study that specific problem type and so failed. Also, there is an up front cost of several months to cover and study a wide array of leetcode problems.
With a take home I can demonstrate how I would perform at work. I can sit on it, think things over in my head, come up with an attack plan and execute it. I can demonstrate how I think about problems and my own value more clearly. Using a take home as a test is indicative to me that a company cares a bit more about its hiring pipeline and is being careful not to put candidates under arbitrary pressures.
Would you rather do 10 take-homes or 10 leetcode questions?
Either way, when you fail, chances are that you will not get any meaningful feedback other than "we have decided to move forward with other candidates".
If you had done a take-home, how could you know where you went wrong?
If you had done a leetcode question, you can look up the question after the interview and usually learn from your mistakes.
With leetcode you usually don't need the interviewer's feedback to improve. You don't even need the interview. And after a certain point you won't need that much time to prepare.
I feel the complete and total opposite. With leetcode-esque ones it's just a luck of the draw that you can conjure up the memory for whatever it is they're asking you to do. The decent ones are the ones that at least are somewhat realistic, but the truly terrible ones are those where you have to remember how to do an algo of some sort but you have no access to outside tools of any kind. I've never come out of a leetcode and felt like I learned anything or could've done something better, not to mention the artificially crafted high stress environment. I feel like they also literally never reflect anything even approaching the actual work, either, and basically test your memory more than anything else. There's a reason there's a billion books out there about how to "crack" the leetcode style interviews.
On the other hand, with takehomes I get to approach a project as if it were my own little hobby thing. I get to approach it in ways that are comfortable for me to work, with my music on, in my own editor, on my own setup, with no time pressure (or at least very light time pressure, just like during the actual job). I always use it as an opportunity to try out something new as well, so I'm also learning in the process even if I don't get the job. And in my experience even when rejected, I've always gotten detailed feedback for areas of improvement, which has never happened in leetcode interviews for me.
What if it's an appropriately scoped take-home assignment? 1-2 hours max. I would say that's fair and gives people with interview anxiety more of a chance.
The problem is that the more we go in that direction, the more hiring in this industry rewards gamifying and grinding, and that bleeds into the profession as a whole. And at that point it’s just not worth working there
At my first employer circa 2005 we had a simple 1-2 hour interview and a 90 day probationary period. Respected people's time, gave everyone a fair chance to prove themselves. I don't know why it's not more common in the industry.
Part of what lead to it I think is we hired largely straight out of college and doing a 9-hour interview with someone with little experience is a waste of time.
It worked great. In my five years there we only had a couple people not make it past the probationary period.
I don't mean to be harsh, but as an engineer you owe it to yourself to try and get better at interviewing. Does it suck? Yeah absolutely. Is it an annoying process? Definitely. But even if you have an easy and stable job things can change quickly at any company. You're only closing doors on yourself.
If you get nervous, the main thing you can do is more interviews. My personal anxiety peaks right before the start time, luckily my bathroom is next door to my office! But after doing dozens of interviews I settle in once it gets rolling.
If you hate leetcode, well just get good at it. Yeah it is kinda dumb but it is straightforward to practice. And there is a lot more to a leetcode interview than knowing tricks - you need to communicate well.
Take homes? Yeah they are time consuming. If you really need a job do them, otherwise pass on the company!
Overall as a candidate you really need to try and go one level up on selling yourself - not just why you are a great candidate (which you are of course!), but why you would succeed at this role in particular.
I disagree. I know when I get questions right, or can't do them. In my experience, either I screwed up a question and legit didn't get the job, or I was fine and the interview process is just a bit random. Sometimes it's the interviewer's fault for asking bad questions. Most commonly it's: "how do you light a fire?" "with a match" "but you don't have any matches" "a lighter" "no lighter either" "flint" "no flint" "stick and bow" "all the wood around you is wet" "I give up" "you use a magnifying glass of course!"
You can actually ask for feedback and usually they'll give it. In the EU they have to tell you what they wrote about you in their internal system under GDPR (or at least my company's lawyers believed that)!
The one time I got really great feedback unprompted from a hiring guy it was basically "they were disappointed you didn't know X" and I was like "weird, I do know X!". Very interesting to see it from the other side but I don't think it was very actionable.
I would probably adjust my original post to say you cant better without useful feedback.
I always ask for feedback after an interview, its almost always useless. And conflicting between company interviews.
Its pretty rare for me to totally bomb an interview to the point i don't know the question. Usually the situation is I answered the coding question, handled the follow ups, wrote working code and still get the feedback that the coding portion didn't go well. How do you use that feedback? Useless imo.
So glad I changed industries after my first s/w job out of uni wanted 2 hrs unpaid overtime per day and the boss ran the office like he was lord of the manner, swearing and blowing up whenever it suited him. If I had the skills these people want for the price they're willing to pay I'd just start my own business and reap the rewards myself. Most people aren't worth working for. Perhaps the pool of employers is far larger in the US which makes it easier to shop around for a good one.
> interviews should: be applicable. reflect the actual job duties
No, it doesn't matter that much what task the candidate is doing in the interview. It matters what the interviewer is looking at. I'm sure plenty of interviewers don't understand this, and I think this is often missed when people debate about Leetcode interviews - including in this article.
> most interview questions have very little to do with day-to-day responsibilities
You're missing what the interview questions are. Yes, one part of the interview is "here is a puzzle, can you solve it?", but many of the other questions should be things like "explain why this doesn't work", "why did you start with this approach?" and "are you sure that is the best name for that function?"
Leetcode interviews are perfectly "applicable", as long as the interviewer is using them as a convenient frame to see how you think, communicate, and write/read/edit code and isn't trying purely to assess your skill at quickly solving leetcode puzzles.
> cannot distinguish a senior programmer from a marketer using chatGPT
This is empirically false, because companies haven't suddenly lost all their hiring signal since ChatGPT came out. But if a company has this problem, they suck at interviewing.
(I admit the style of interviewing I'm describing has its own problems, and in particular doesn't address what I think is the biggest issue: the fact you're partly testing people's ability to (appear to) perform under acute pressure.)
>you're partly testing people's ability to (appear to) perform under acute pressure
You are also testing people's endurance. I once did an on-site interview with Google, and it was a solid 6 hours of leet-style puzzles, one after the other.
> but many of the other questions should be things like "explain why this doesn't work", "why did you start with this approach?" and "are you sure that is the best name for that function?"
This is important and something more interviewers should do. The blind adherence to leetcode doesn't tell you much, especially if you're silent during the interview instead of having a short back-and-forth every 15 minutes or so. The problem solving process is more important than the problem solved.
The problem with leetcode is time. If you know a good solution to a puzzle, or can instantly come up with it, you have the time to write it down, make it work, discuss, improve, etc. If you spend extra 10 minutes on a wrong approach, you're very strained, and if you spend 15 minutes, you're toast.
But knowing the good solution by heart is not the point! For that reason, when I ran code interviews, I often gave hints to the candidate, or sometimes just stated that a particular algorithm is a good solution, and it's usually implemented using this and that. ("Use BFS. You'll need a queue.")
This is where the demand for realistic conditions kicks in: at work, you're not going to invent an algorithm, you'll ask the machine what works in a situation like yours, consider, and choose. Doing otherwise would be imprudent.
> I admit the style of interviewing I'm describing has its own problems
We need to be clear, ALL styles of interviewing has problems. There's no global optima for problems like this. It's important to say this because people will point out problems as reasons not to switch while ignoring the existing ones. The answer entirely depends on "what are you optimizing for?" Be precise
Also worth pointing out, the style you describe is the traditional engineering interview. Leetcode type problems were initially done this way. It is also easy to make an effective system worse by trying to improve it. This often happens by trying to apply hard metrics to noisy problems. You don't improve your accuracy, you are just fitting a certain type of noise better (and encouraging Goodhart's Law).
Everyone calls them Interviews but they are not really interviews. They are exams.
Oral exams, live coding exams, system architecture exams, take-home exams, behavioral examinations, code review exams, extended essay writing exams, case study exams, sample work trials.
You can't be a real professional if you have to take exams in every job change.
In serious professions, people take exams early in their careers for being certified. Sometimes they take additional exams to renew their certificates. And that's all.
They don't take exams from random people in random companies that know nothing about evaluating knowledge. They take official, standardized exams prepared by professional testers/educators.
Engineering jobs can't be standardized. Engineering and required skills and knowledge is too broad for that.
An interview is not an exam. It's a meeting. The interviewer asks questions to learn about the candidate. The interviewer may ask some questions to learn about the company and the position. That's all. That's the universal definition of a job interview. All the other things are additional tests and exams.
Do they need to do those exams for better selection? Probably not. Their "hiring process"es are not backed by any science. Then why are they doing that? They have to filter somehow. If there are 1 to 100s ratio of candidates for each position, they need to filter hard. Exams are the standard method for ranking and filtering.
> In serious professions, people take exams early in their careers for being certified. Sometimes they take additional exams to renew their certificates. And that's all.
The field of programming emerged from mathematics, not engineering unfortunately. So we lack any useful certification processes.
Indeed, and ridiculous exams at that. Imagine being at school and they had a several question exam from anything across a four year degree. No information on what area to focus on, because there’s no focus on purpose.
Quarter of a million dollars in a suitcase, loaded gun, and a copper ticking clock on the table next to you. Picture of your kids that need new school clothes.
Miss one question and you fail. Must also be confident, friendly, and not too old while doing it.
—> “I can’t find anyone that can pass my simple test, they must all be frauds.”
This is not how proper exams work, and for good reason.
I agree with the article. Discussing previously created code and do code reviews live covers a lot of ground. Whereas live coding is just meaningless for the stated reasons.
But a 9-hours interview process seems just too much... I think you will only ever know a candidates true fit once you start actually working together and 2-3 short sessions with someone is enough to get that go/no go feeling.
You can't hire without taking risks.
Where I live, you usually have a 3-months probation time in which both sides can quit within a 7 days period... so the risk is manageable.
Just feel a candidates fit and then go for it... and adjust when necessary. Don't overthink it.
Not sure if they're still doing it, but GitLab does the code review interview, and I too really liked it.
Before the interview, you clone the repo and get the app running on your machine.
For the first half of the interview, you review a pull request in real time. There's a mix of obvious and non-obvious callouts. And the second half, you actually implement your suggestions.
Honestly the code review portion alone is a great indicator of a dev's experience and soft skills.
If I can get a person talking about tradeoffs - be it speaking to a past project, or a hypothetical I give them - then I think I can tell who the serious developers are fairly quickly.
> companies often over-index on crystallized knowledge over fluid intelligence.
another way to say this: focus on aptitude. in my hiring funnel, this is a core tenet. you need to be able to capture polyglots and systems thinkers. its still pretty hard to design a process that balances this all very well. combine that with an absolute glut of applicants and you have a very challenging problem.
I won't forget the in-person interview round where I coded a frontend visualization for a data graph (tracking global shipping), then fielded a post-work general interview round from the whole company (~10 ppl) about specifics and "choices" made during a rush to finish. I ended up not going due to comp, but they were acquired months later. Life is funny.
It's crazy, I've been trying to hire an architect to build a house for me, but I don't trust them (they're all liars) so I asked them to draw up blueprints in front of me and describe everything they are doing in great detail. I need to understand what they are doing. This building is important and I don't want a bad hire. They should be able to do this in 10-15 minutes. If they knew what they were doing, this request wouldn't be an issue.
Then I was trying to get someone to do my taxes. So I've been asking every applicant to do my taxes from last year so I can see if they really know what they are doing. I mean sure, they've done taxes for years, but these are my taxes. I've even tried giving them math puzzles around asset depreciation, but people just keep hanging up the phone.
And then, I wanted to add a wing added to my house and I've been trying to get these entitled contractors to come build a shed in my backyard so I can see if they really know how to use nail guns. I've heard people are just big'ol lairs lately, and I need to see how they work. I mean, sure these guys have built many houses in the past, but we have high standards here and only hire A players.
I've only been getting horrible candidates! No one is worth hiring! There is a huge shortage of qualified people.
2. At home coding challenge. Candidates can pick one from 2 coding challenges. But we try to keep them engaging and fun, but still complex in details. Sometimes people don’t want to invest this time. That’s acceptable, but in this case you have to show os some of your work from the past, so we can discuss this.
3. Interview with 2 engineers from the team. We are doing coding challenges as a base for this interview. It’s just a way to get people talking with each other on technology and how they work.
4. Make an offer or say no to the candidate. Everyone involved in this process from our side has a veto right. So if one person says no - it’s a No.
5. Send contract to the candidate, if they accept the offer. This is the first time in the process where HR is involved. Everything else before was done by the Dev Team.
I think this is the most important part, show respect by taking care of the process by yourself and communicating with the candidate.
60 comments
[ 2.8 ms ] story [ 91.4 ms ] threadMost software people do web front end or web back end or CRUD. Most of the rest do apps, whether mobile or desktop. What's non-fungible about us?
There's a trade-off: if a company spends more time / requires more effort on an interview process, they can get a better signal on the candidate's abilities, but then they'll lose out on candidates who are unwilling / unable to commit this time. This might just be a hard trade-off in recruiting.
Not enough to make it worth farming interviews for compensation, but enough to show that the company appreciates that you spent 2-4 hours working on their take home.
If I had a dollar for every time I heard this (flawed) argument, I’d be rich and would no longer have to sell ads on my Hacker News comments. I’m going to get hate for this unpopular opinion but here we go.
So often, “But Leetcode isn’t like REAL programming” is the siren song of the programmer who probably overestimates their coding skills and experience.
Yes, I hate to say it - live coding is actually one of the best signals you can get on a candidate’s seniority and ability to program a computer (and more importantly, their core computer science skills). A good interviewer is trained to know how to probe your CS knowledge during this, and will watch how you structure code, break down problems, debug, and think about testing. They will even ask you to make changes to see how coachable you are and what you might be like to work with. It’s not about inverting a binary tree while sharing your screen, it’s about showing me how you solve a problem, then translate what’s in your head to code.
Take home exercises provide little to no signal, and screen out people who have families (who wouldn’t bother with a 4 hour take home exercise after work). I don’t want to see how you Google, I want to see how you think.
These candidates always want some version of, “But trust me, bro! Hire me: I’m a senior engineer, I don’t remember how to Leetcode! I’m good, I promise!” But what they won’t admit to themselves is that a good senior engineer is able to do all the things a junior can do PLUS all the things a senior can do.
It’s not perfect, but I won’t hire anyone that can’t pass a live coding round.
This comment brought to you by Poppi. Poppi: it’s soda for people who are silly enough to believe soda can be healthy.
I'd like to add two points to this:
First, I like that you said "live coding" rather than leet code. The floor for live coding should be super low, with a high ceiling and lots of flexibility. That allows you to say, nope, they didn't pass the floor level, easy binary decision, no hire. Pick a fun toy to build in 90 minutes and the high ceiling + flexibility will yield tons of signal from applicants.
Second, I see live coding sessions like this as a positive sign from potential employers. It lets me know that my future colleagues will have some baseline level of competence. If you've worked on a team that didn't do live coding, and you've had to carry water for someone who can't actually do the day-to-day technical "hard skill" work of software engineering, you probably feel the same way. Never again.
I start with a simple problem with several answers of varying difficulty and literally instruct the candidates to code the first correct solution they can think of.
Nested for loops, brute force, it’s absolutely fine.
I just want to see you translate what’s in your head to code. I want to see you structure code, use data structures, etc. Then we will talk through optimizations, refactoring, testing, edge cases, etc later.
You want even the under qualified candidates to feel like they got some pieces right and had a fair shot.
As you start probing deeper, the top candidates stand out. They answer your questions about the differences between data structures, how you would test this piece, how you would monitor that piece in production, how you could parallelize it, etc.
Let’s face the reality that most developers will never be able to write original software and just put text to screen using a tool or framework. Don’t call these people engineers. These people are the assembly line of software. Measure them according to desired patterns. They are copy/paste but smarter than data entry and understand some of the restrictions in place. Expectations are low and compatibility and replacement are the key business values.
Next are the people who test software, the QA. We expect more from these people and then work them harder for less money at a lower level of reputation.
Next are the people who evaluate software. These people are closer to engineers. These people include accessibility, security, and performance experts. These people are more like a combination of QA and senior developers. Evaluate these people on these criteria: written essay, technical knowledge, force them to measure things in real time and see how they perform.
Next are the people who actually write software applications. Let’s call these people solution delivery. These people are similar to junior architects and actually build things. These people should be evaluated only on the basis of organizational capabilities above that of the engineers that measure things.
Finally are the software owners. These people resemble a combination of project management and junior architects. They must have the experience to know how to build original software, like the junior architects, but also a planning vision to push though demands from competing stakeholders. There is busy savvy to this comes from a solid engineering planning vision plus superior communication skills most lesser software people never honed. Think of these people as senior principals with real authority. Evaluate these people on their delivery experience, using numbers, and reputation.
What has worked for me, honestly, is being directly involved with my hiring pipeline and having conversations.
It seems like common sense, but there's a lot of reasons not to do this and people will make good arguments to prevent it. "What about bias", "your time is more important" etc;
However, bias is an unfortunate consequence of selecting for value fit anyway and I can't think of a more important task than selecting the members that will be the future of the company.
I've had some positions that were open for a weekend where I got 400 applicants, and yes, it was daunting to go through and give each of them an honest shot, but you know: I had to do it. What's the alternative? I might miss a fantastic candidate because someone didn't understand what I actually need.
Evaluating programmers and "devops" people is just insanely hard, technologies are mostly fungible. If you can write one C-like language you can learn the others in about a month, but what can't be taught is what your values are, if you think in a systemic way, if you're easy to work with and respect others.
So, my solution is to have a conversation, challenge what they know, see how they react when challenged, see how they react when they reach the end of their knowledge and see what they're most proud of and if they get excited by it.
No gotchas, no esoteric internal handshake, just: are you defensive? Are you curious? Are you passionate? Can you communicate effectively and are you intelligent.
If you hit those, you can do anything.
"How do you even know who to interview?"
This is a hard question, for me there's not a lot of candidates that are physically located in my region, so those go through as long as they have something on the CV that looks relevant. For others it's a combination of: would it be easy for them to move, have they worked remote (and can do it in a region where I have a tax entity) and how strong of a fit to the role is the CV, lots of experience in games would be what I expect since I work in video games - but if you're going for a backend programming role then: what have you built and what do you list as your responsibility to achieve it?
With this mindset I managed to build a high performing, high trust team that executed very well on (literally) impossible demands. If the ownership of the company was better that team would have easily been world class.
We also exceeded dunbars second (clan) number with the size of the team, so it wasn't intrinsic to small teams (80+).
Failing a take-home is an entirely different thing. It's a huge loss in time and mental energy.
I've only done 3 of those in my career and only because the projects sounded interesting. 1 of those 3 resulted in a job offer which I can now confidently say in hindsight was the worst job in my career (...so far!).
I'm now leaning towards just filtering out companies that do take-homes because it signals to me that they don't care about their candidate's time and how a company treats its candidates is usually a good indicator of how they treat their employees.
With a take home I can demonstrate how I would perform at work. I can sit on it, think things over in my head, come up with an attack plan and execute it. I can demonstrate how I think about problems and my own value more clearly. Using a take home as a test is indicative to me that a company cares a bit more about its hiring pipeline and is being careful not to put candidates under arbitrary pressures.
Either way, when you fail, chances are that you will not get any meaningful feedback other than "we have decided to move forward with other candidates".
If you had done a take-home, how could you know where you went wrong?
If you had done a leetcode question, you can look up the question after the interview and usually learn from your mistakes.
With leetcode you usually don't need the interviewer's feedback to improve. You don't even need the interview. And after a certain point you won't need that much time to prepare.
On the other hand, with takehomes I get to approach a project as if it were my own little hobby thing. I get to approach it in ways that are comfortable for me to work, with my music on, in my own editor, on my own setup, with no time pressure (or at least very light time pressure, just like during the actual job). I always use it as an opportunity to try out something new as well, so I'm also learning in the process even if I don't get the job. And in my experience even when rejected, I've always gotten detailed feedback for areas of improvement, which has never happened in leetcode interviews for me.
Part of what lead to it I think is we hired largely straight out of college and doing a 9-hour interview with someone with little experience is a waste of time.
It worked great. In my five years there we only had a couple people not make it past the probationary period.
If you get nervous, the main thing you can do is more interviews. My personal anxiety peaks right before the start time, luckily my bathroom is next door to my office! But after doing dozens of interviews I settle in once it gets rolling.
If you hate leetcode, well just get good at it. Yeah it is kinda dumb but it is straightforward to practice. And there is a lot more to a leetcode interview than knowing tricks - you need to communicate well.
Take homes? Yeah they are time consuming. If you really need a job do them, otherwise pass on the company!
Overall as a candidate you really need to try and go one level up on selling yourself - not just why you are a great candidate (which you are of course!), but why you would succeed at this role in particular.
You can actually ask for feedback and usually they'll give it. In the EU they have to tell you what they wrote about you in their internal system under GDPR (or at least my company's lawyers believed that)!
The one time I got really great feedback unprompted from a hiring guy it was basically "they were disappointed you didn't know X" and I was like "weird, I do know X!". Very interesting to see it from the other side but I don't think it was very actionable.
I always ask for feedback after an interview, its almost always useless. And conflicting between company interviews.
Its pretty rare for me to totally bomb an interview to the point i don't know the question. Usually the situation is I answered the coding question, handled the follow ups, wrote working code and still get the feedback that the coding portion didn't go well. How do you use that feedback? Useless imo.
Why would you do this given the expected average tenure is probably like 18 months to 2 years?
No, it doesn't matter that much what task the candidate is doing in the interview. It matters what the interviewer is looking at. I'm sure plenty of interviewers don't understand this, and I think this is often missed when people debate about Leetcode interviews - including in this article.
> most interview questions have very little to do with day-to-day responsibilities
You're missing what the interview questions are. Yes, one part of the interview is "here is a puzzle, can you solve it?", but many of the other questions should be things like "explain why this doesn't work", "why did you start with this approach?" and "are you sure that is the best name for that function?"
Leetcode interviews are perfectly "applicable", as long as the interviewer is using them as a convenient frame to see how you think, communicate, and write/read/edit code and isn't trying purely to assess your skill at quickly solving leetcode puzzles.
> cannot distinguish a senior programmer from a marketer using chatGPT
This is empirically false, because companies haven't suddenly lost all their hiring signal since ChatGPT came out. But if a company has this problem, they suck at interviewing.
(I admit the style of interviewing I'm describing has its own problems, and in particular doesn't address what I think is the biggest issue: the fact you're partly testing people's ability to (appear to) perform under acute pressure.)
You are also testing people's endurance. I once did an on-site interview with Google, and it was a solid 6 hours of leet-style puzzles, one after the other.
This is important and something more interviewers should do. The blind adherence to leetcode doesn't tell you much, especially if you're silent during the interview instead of having a short back-and-forth every 15 minutes or so. The problem solving process is more important than the problem solved.
But knowing the good solution by heart is not the point! For that reason, when I ran code interviews, I often gave hints to the candidate, or sometimes just stated that a particular algorithm is a good solution, and it's usually implemented using this and that. ("Use BFS. You'll need a queue.")
This is where the demand for realistic conditions kicks in: at work, you're not going to invent an algorithm, you'll ask the machine what works in a situation like yours, consider, and choose. Doing otherwise would be imprudent.
Also worth pointing out, the style you describe is the traditional engineering interview. Leetcode type problems were initially done this way. It is also easy to make an effective system worse by trying to improve it. This often happens by trying to apply hard metrics to noisy problems. You don't improve your accuracy, you are just fitting a certain type of noise better (and encouraging Goodhart's Law).
Oral exams, live coding exams, system architecture exams, take-home exams, behavioral examinations, code review exams, extended essay writing exams, case study exams, sample work trials.
You can't be a real professional if you have to take exams in every job change.
In serious professions, people take exams early in their careers for being certified. Sometimes they take additional exams to renew their certificates. And that's all.
They don't take exams from random people in random companies that know nothing about evaluating knowledge. They take official, standardized exams prepared by professional testers/educators.
Engineering jobs can't be standardized. Engineering and required skills and knowledge is too broad for that.
An interview is not an exam. It's a meeting. The interviewer asks questions to learn about the candidate. The interviewer may ask some questions to learn about the company and the position. That's all. That's the universal definition of a job interview. All the other things are additional tests and exams.
Do they need to do those exams for better selection? Probably not. Their "hiring process"es are not backed by any science. Then why are they doing that? They have to filter somehow. If there are 1 to 100s ratio of candidates for each position, they need to filter hard. Exams are the standard method for ranking and filtering.
But we are professional engineers, not students.
The field of programming emerged from mathematics, not engineering unfortunately. So we lack any useful certification processes.
Most devs are doing work that more closely resembles pipe fitting or carpentry than any engineering discipline.
Quarter of a million dollars in a suitcase, loaded gun, and a copper ticking clock on the table next to you. Picture of your kids that need new school clothes.
Miss one question and you fail. Must also be confident, friendly, and not too old while doing it.
—> “I can’t find anyone that can pass my simple test, they must all be frauds.”
This is not how proper exams work, and for good reason.
But a 9-hours interview process seems just too much... I think you will only ever know a candidates true fit once you start actually working together and 2-3 short sessions with someone is enough to get that go/no go feeling.
You can't hire without taking risks.
Where I live, you usually have a 3-months probation time in which both sides can quit within a 7 days period... so the risk is manageable.
Just feel a candidates fit and then go for it... and adjust when necessary. Don't overthink it.
Before the interview, you clone the repo and get the app running on your machine.
For the first half of the interview, you review a pull request in real time. There's a mix of obvious and non-obvious callouts. And the second half, you actually implement your suggestions.
Honestly the code review portion alone is a great indicator of a dev's experience and soft skills.
I'd love to hear if anyone has critiques of this style.
It helped that it was 90 days contract to hire.
Eventually, enough time passed that the talent pool grew considerably and most people are baseline competent.
Consequently, I now find that respect and time efficiency matter a lot more.
another way to say this: focus on aptitude. in my hiring funnel, this is a core tenet. you need to be able to capture polyglots and systems thinkers. its still pretty hard to design a process that balances this all very well. combine that with an absolute glut of applicants and you have a very challenging problem.
I won't forget the in-person interview round where I coded a frontend visualization for a data graph (tracking global shipping), then fielded a post-work general interview round from the whole company (~10 ppl) about specifics and "choices" made during a rush to finish. I ended up not going due to comp, but they were acquired months later. Life is funny.
Then I was trying to get someone to do my taxes. So I've been asking every applicant to do my taxes from last year so I can see if they really know what they are doing. I mean sure, they've done taxes for years, but these are my taxes. I've even tried giving them math puzzles around asset depreciation, but people just keep hanging up the phone.
And then, I wanted to add a wing added to my house and I've been trying to get these entitled contractors to come build a shed in my backyard so I can see if they really know how to use nail guns. I've heard people are just big'ol lairs lately, and I need to see how they work. I mean, sure these guys have built many houses in the past, but we have high standards here and only hire A players.
I've only been getting horrible candidates! No one is worth hiring! There is a huge shortage of qualified people.
If only there was a way to fix this.
The worst ones were the leetcode interviews I couldn’t pass
1. 60 minutes interview with the CTO.
2. At home coding challenge. Candidates can pick one from 2 coding challenges. But we try to keep them engaging and fun, but still complex in details. Sometimes people don’t want to invest this time. That’s acceptable, but in this case you have to show os some of your work from the past, so we can discuss this.
3. Interview with 2 engineers from the team. We are doing coding challenges as a base for this interview. It’s just a way to get people talking with each other on technology and how they work.
4. Make an offer or say no to the candidate. Everyone involved in this process from our side has a veto right. So if one person says no - it’s a No.
5. Send contract to the candidate, if they accept the offer. This is the first time in the process where HR is involved. Everything else before was done by the Dev Team.
I think this is the most important part, show respect by taking care of the process by yourself and communicating with the candidate.
This can spark so many interesting discussions, from syntax, architecture, cs, product etc
It is literally the job that engineers do 99% of their time yet we don't interview on this.