I think there's also another element that the author is missing and that is likability. I've never experienced this, but I've heard about people being put through grueling interview processes merely to weed them out because they were deemed "not a good culture fit".
I think at least half of all interviews are a popularity contest, and the other half are your qualifications.
Jm2c but interviews tell you absolutely none, nothing, about what kind of a professional the candidate is.
I have no clue whether he'll care and help or pretend to work and drag everybody else down.
There's a huge number of incredibly capable developers who could pass any interview but then spend days playing video games and sabotaging projects and teams.
I really don't believe in technical interviews, I'd rather base the relationship on trust, if you tell me you're good/experienced at X I trust you to be. If it was bs you'll be shown the door with ease.
Instead many companies make it insanely hard to get you hired, but also incredibly hard to cut you out even if you're impact is a very net negative.
Cargo culting, and the fear of bad hires. Everyone wants "the best engineers". Probably 1% of tech firms work with deep tech, where boundaries are pushed, and every percentage of improvement or degradation can result in non-trivial gains or losses. The rest can do just fine with normal engineers, and do interviews like the rest of the professional world.
Valid points, but I think the most obvious reason is that (at least the recognizable) employers are swamped with applications that all look great on paper, which likely got much worse with the rise of good LLMs. Compared with similar high paying carreers, like medicine (multi-year residency) or high finance (solving math and probability problems on the spot), the hiring process for software engineers isn't especially gruesome.
A major issue the author missed is it is much harder to fire a non-performer in the tech industry today.
It takes 2 quarters (ie. 6 months) to go from recognizing a problem employee to firing said employee.
This makes the risk of hiring the wrong candidate significant as a hiring manager, because a bad hire reflects badly on you and eats up your budget thus preventing a backfill.
On top of that, firing individual employees can lead to litigation risk (even if frivolous), thus requiring hiring managers to go through the extreme song and dance of the PIP process and documented reprimands in order to provide counsel if litigated.
Companies keep ignoring the historical interview point. I have been to a few occasions when companies needed the exact thing i built in the past (e.g. migration to clickhouse), but chose to put me into a random take home related to a different technology (e.g. some bigquery assignment) and eventually reject me. Go off script and ask me details about the project which might solve your hands, why do i need to talk about something else?
Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool.
Suppose 90% of candidates are excellent and 10% are terrible. If the excellent 90% only need to interview at one company, whereas the bad 10% need to interview at 20 companies, then only 0.9/(0.1*20+0.9)=31% of interviews will be with qualified candidates. To retierate: almost 70% of interviews will be with terrible candidates, even though 90% of people applying for jobs are excellent.
Because the cost of a bad hire is so consequential, the interview process is not designed to efficiently handle a minority of qualified candidates, but rather efficiently weed out a majority of horrible candidates. It is therefore a terrible process for the people actually qualified to pass it.
In my experience, technical interviews are not really useful past the very basic "Can this candidate actually write a conditional and has the slightest clue about programming?". Ability to solve hard leetcode-like problems under time pressure in a stressful environment doesn't meaningfully translate to "will be a great contributor to the team work on the kind of problem we have".
Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time.
Most of my time in interview nowadays is spent understanding what the candidate has done before, explaining to them what we do and asking open questions to see how they would approach our issues and how they link them to their experience. If it seems to fit, we hire. My country standard contract offers a fairly long probation period for new hire and we don't hesite about parting with people when it's not working after a quarter. We are very explicit about this policy.
The alternative I prefer isn't any of the given ones, but a scaled interview. I start with something that you can solve with a Python one-liner, if you know what you're doing, but if you solve that instantly I've got a series of questions that will scale up until we're eventually doing the task using untrusted user input, outputting into a security-sensitive environment, under heavy load that we want to be cheap, in a highly-available environment, etc. etc. I also tell the applicant up front that this is the plan, because if you don't do that, it feels like you're jerking them around by constantly changing the requirements. I've also got a number of directions it can go depending on the applicant, and depending on what they tell me about their past experience.
The other nice thing about this is your interviewee generally doesn't leave feeling like a failure; it's not like I have three questions and you can get them all wrong. Unfortunately there are some people who end up spinning on the very easiest question for the entire session, and, uh, well... I can only do so much, if you really don't know anything about programming at all. This is at least the exception, though.
I have not had to do an interview in the age of practical AI yet, though. In person I don't think I'd have to change much, I've always interviewed with a policy of "I'm not worried about whether the string split command takes its parameters in this or that order, I just want to know you know it exists" and I can basically serve as an AI in the same way I was already being the API reference. Remotely, I'm not sure what I'd do yet.
Interviews allow just a little time for strangers to attempt to sync up and get a sense of each other.
Actual jobs offer much more time for people to learn to sync and communicate.
Also, on both sides of the interview table, people have varying strengths when it comes to short and long term communication skills. Plenty of interviewers are not good at interviewing, just as plenty of candidates are not good at their side of the process.
In short, interviews are a very poor approach to choosing who belongs long term.
Ultimately, regardless of the interview process used, every job is ultimately a long term “try and see” interview. You only know how someone fits by trying them for a while.
I know it is hearsay, but it seems many times interviews try to get free work from candidates. Here, have a look at this real problem, see if you can solve it in whatever the interview time frame is.
When there are too many candidates and not so many jobs, and absolutely no regulation, employers are free to exploit applicants without consequences. Employers will do this even when there is no actual job being offered, so they can use the availability of applicants to apply downward pressure on the salaries of the workers they already have.
Interviews have been extensively studied by phd, figuring out what works and what doesn't. I have yet to see anyone write a blog or comment that shows even an awareness that this research exists, much less what it in it. (I know this research exists, but I'll admit to not knowing how to find it. My company insists the interview process they make me follow is based on it)
Invert a binary tree, find the optimal path in an n-dimensional space of real numbers given by a function, design a horizontally scalable build system, get grilled by at least two people on your leadership skills... then, if you're extremely lucky, get hired to configure the button to be blue and have round corners. And get laid off in 9 months.
(Not saying this happened to me but it's a common story I've heard in the last few years)
I agree with the author that it is hard to assess someone's skill if you have a list of 100 people to interview and you know nothing about them. The bigger the stack of applications the easier it is to treat them like data and not people.
I know plenty of software developers who write, maintain, or contribute to OSS libraries; they write blog posts, give talks at conferences/meetups, and make videos in their spare time (or sometimes as part of their work). I've rarely walked into an interview where the hiring manager or the technical interviewer hadn't just read my name off my resume as I joined the meeting. Get to know the candidate's work before assuming they know nothing!
Maybe people involved in the hiring process should be given more time to properly research a candidate. Relying on ATS' and putting the burden on applicant's to do the work of proving themselves is causing a lot of folks to burnout just trying to get a job.
I tended to do the "Historical Interview" one. In my case, it worked well.
I think take-home tests are probably damn near worthless, these days. People will just feed the assignment into an LLM, and return the results.
I am not a fan of LeetCode, because I don't want to waste time, studying stuff that won't have any relevance to what needs to get done. I like to keep my dance card full, and wasting precious time to make someone else happy, isn't my idea of a good time.
That said, I understand why they are there. I just feel that they aren't really something that I'd use to judge senior-level talent; which was what I used to hire.
The 100m / 10k analogy is fantastic, but the reality is even worse.
Let’s say you expect a candidate to stay for average five years. That’s about 1200 working days. If you get to interview them for 1 day, then that day is 8.3m to the 10k. You get to watch them run for 8.3m and decide how well they can run the 10k.
> Asking a candidate to solve some problem which lets a candidate work in a slightly less high stakes environment. but requires candidates to do extra work, taking longer than just the interview.
I refuse a lot of these. I've put a lot of time into many of them, just to be ghosted.
Typically, when the take-home comes with a laundry list of frameworks and 3rd party libraries, I walk away. Coming up to speed with all of them is too time consuming for the probability of getting ghosted.
It takes a lot of work on the interviewers side to come up with a good take-home. It requires discipline to reject the temptation to just throw in a laundry list of requirements.
Is there any other industry with such a disconnect between education and job where the candidates need to do another exam when they try to join the job?
22 comments
[ 3.6 ms ] story [ 51.4 ms ] threadI think at least half of all interviews are a popularity contest, and the other half are your qualifications.
I have no clue whether he'll care and help or pretend to work and drag everybody else down.
There's a huge number of incredibly capable developers who could pass any interview but then spend days playing video games and sabotaging projects and teams.
I really don't believe in technical interviews, I'd rather base the relationship on trust, if you tell me you're good/experienced at X I trust you to be. If it was bs you'll be shown the door with ease.
Instead many companies make it insanely hard to get you hired, but also incredibly hard to cut you out even if you're impact is a very net negative.
It takes 2 quarters (ie. 6 months) to go from recognizing a problem employee to firing said employee.
This makes the risk of hiring the wrong candidate significant as a hiring manager, because a bad hire reflects badly on you and eats up your budget thus preventing a backfill.
On top of that, firing individual employees can lead to litigation risk (even if frivolous), thus requiring hiring managers to go through the extreme song and dance of the PIP process and documented reprimands in order to provide counsel if litigated.
Because the vast majority of job interviews are with terrible candidates, even if the majority of candidates are excellent. This apparent paradox has a simple explanation: excellent candidates selectively apply to a few companies and get interviews/offers at almost all of them. On the other hand, terrible candidates are rejected at every step of the hiring process, and have to constantly reenter the interview pool.
Suppose 90% of candidates are excellent and 10% are terrible. If the excellent 90% only need to interview at one company, whereas the bad 10% need to interview at 20 companies, then only 0.9/(0.1*20+0.9)=31% of interviews will be with qualified candidates. To retierate: almost 70% of interviews will be with terrible candidates, even though 90% of people applying for jobs are excellent.
Because the cost of a bad hire is so consequential, the interview process is not designed to efficiently handle a minority of qualified candidates, but rather efficiently weed out a majority of horrible candidates. It is therefore a terrible process for the people actually qualified to pass it.
https://en.m.wikipedia.org/wiki/Simpson's_paradox
Our best hires are nearly always coming from the network of a team member or people we contracted with and decided to hire full time.
Most of my time in interview nowadays is spent understanding what the candidate has done before, explaining to them what we do and asking open questions to see how they would approach our issues and how they link them to their experience. If it seems to fit, we hire. My country standard contract offers a fairly long probation period for new hire and we don't hesite about parting with people when it's not working after a quarter. We are very explicit about this policy.
The other nice thing about this is your interviewee generally doesn't leave feeling like a failure; it's not like I have three questions and you can get them all wrong. Unfortunately there are some people who end up spinning on the very easiest question for the entire session, and, uh, well... I can only do so much, if you really don't know anything about programming at all. This is at least the exception, though.
I have not had to do an interview in the age of practical AI yet, though. In person I don't think I'd have to change much, I've always interviewed with a policy of "I'm not worried about whether the string split command takes its parameters in this or that order, I just want to know you know it exists" and I can basically serve as an AI in the same way I was already being the API reference. Remotely, I'm not sure what I'd do yet.
Actual jobs offer much more time for people to learn to sync and communicate.
Also, on both sides of the interview table, people have varying strengths when it comes to short and long term communication skills. Plenty of interviewers are not good at interviewing, just as plenty of candidates are not good at their side of the process.
In short, interviews are a very poor approach to choosing who belongs long term.
Ultimately, regardless of the interview process used, every job is ultimately a long term “try and see” interview. You only know how someone fits by trying them for a while.
When there are too many candidates and not so many jobs, and absolutely no regulation, employers are free to exploit applicants without consequences. Employers will do this even when there is no actual job being offered, so they can use the availability of applicants to apply downward pressure on the salaries of the workers they already have.
(Not saying this happened to me but it's a common story I've heard in the last few years)
I agree with the author that it is hard to assess someone's skill if you have a list of 100 people to interview and you know nothing about them. The bigger the stack of applications the easier it is to treat them like data and not people.
I know plenty of software developers who write, maintain, or contribute to OSS libraries; they write blog posts, give talks at conferences/meetups, and make videos in their spare time (or sometimes as part of their work). I've rarely walked into an interview where the hiring manager or the technical interviewer hadn't just read my name off my resume as I joined the meeting. Get to know the candidate's work before assuming they know nothing!
Maybe people involved in the hiring process should be given more time to properly research a candidate. Relying on ATS' and putting the burden on applicant's to do the work of proving themselves is causing a lot of folks to burnout just trying to get a job.
I tended to do the "Historical Interview" one. In my case, it worked well.
I think take-home tests are probably damn near worthless, these days. People will just feed the assignment into an LLM, and return the results.
I am not a fan of LeetCode, because I don't want to waste time, studying stuff that won't have any relevance to what needs to get done. I like to keep my dance card full, and wasting precious time to make someone else happy, isn't my idea of a good time.
That said, I understand why they are there. I just feel that they aren't really something that I'd use to judge senior-level talent; which was what I used to hire.
I refuse a lot of these. I've put a lot of time into many of them, just to be ghosted.
Typically, when the take-home comes with a laundry list of frameworks and 3rd party libraries, I walk away. Coming up to speed with all of them is too time consuming for the probability of getting ghosted.
It takes a lot of work on the interviewers side to come up with a good take-home. It requires discipline to reject the temptation to just throw in a laundry list of requirements.