Since there is a goodly number of other companies that mimic the process, the rejectee can apply the prep to the next job. Not saying that’s socially optimal, just a specific possible value.
So your solution is take home coding. How is doing an assignment for every company I apply for better than practising coding questions that can be used in interviews at any company?
At Aserto we recognize this problem and address it by offering candidates a choice. They can either take a take-home assignment, or opt for a whiteboard style interview question. This approach lets us meet candidates where they're most comfortable and removes a degree of bias from our interview results.
I think that's a great approach - do you have any interest in sharing the overall interview process?
I run a job board for companies that don't do LeetCode interviews and could get Aserto added to it!
I was an engineering hiring manager for several small startups. I never asked for coding on a whiteboard. I asked some work style questions and then concentrated on a collaborative exercise where I laid out a problem and important applicable technology. Then I'd answer any Google-ish questions in realtime. This technique found good collaborators and also which people were good problem solvers. Never made a bad hire from this method. I only wish I had discouraged others from "code on the whiteboard" type bs.
Used a similar process: ask an open ended problem with no "correct" solution that the interviewers (us) might be looking for, and also weren't domain specific. The same question could be asked of MEs, EEs, software developers, you name it -- they could just talk about the part of the problem that mattered to them.
An example: one we used a few times was "we decided we want to build a drone for the police. So if a bank is being robbed the cop who arrived first could take it out and it will help solve the crime." Some people would ask a bunch of constraints up front; others would just dive in and ask questions as they went. And they'd think about the problem "so we want to chase the getaway car long enough until a regular copter can get airborne, so let's assume X MPH for Y minutes minimum". Someone else thought "small is probably better because perhaps it can look into the bank when it would be dangerous for a person to try, and we have to assume a lot of them will be destroyed, so cost matters." etc etc.
Another great one is "Jucero (this is just after they collapsed) has inspired us to make a bluetooth-only coffee machine". My favorite candidate started with "even with BT you'll really need a control panel for reasons XXX and YYY. And BT generally won't add much, if anything, but if product management insists, let's think up some halfway plausible use cases..." and then ended up in a discussion of the protocol and how to think about making it robust against failure."
These were always fun, and told us a bit about how the candidate thinks, without the pressure of doing it "right". Often the solutions were excessively elaborate, or missed elements a real product would need, but so what? They were "first drafts" in a situation with some pressure. And who cares about remembering the precise arguments to a function or anything that needed looking up.
And the length of these things helped strain out bullshitters. One candidate immediately said "the best way to solve this is to use an XXX. The other interviewer and I looked at each other excitedly. That was the best way and nobody else had suggested it. The candidate went on to ignore XXX, even when we asked questions in the hope of sending the discussion back that way. It's as if they had said "oh, well for this you'll need an adjustable wing" and then drew a wheeled vehicle that could not fly.
This is how it's done. An engineer just walking through a problem 1-on-1 with a candidate can reveal more information about a candidate's skill in just 15 minutes than any number of leetcode or annoying, useless take home exercises.
True, but that's expensive. Leetcode-style quizzes make the most sense if you just want to pick a bunch of promising candidates out of a large amount of applicants, all with minimal effort.
A lot of leetcode type interviews also skew towards people who can keep talking out loud while writing code. I can’t remember the last time I was programming and constantly talking out loud. I like to be quiet, focus and then write code. Preparing for interviews felt like I was a code monkey preparing for a special dance.
The other day I got a PDF with prep materials for Facebook interview. It's a few pages of pure insanity, e.g. "In your tech screen, you’ll be asked to solve one or two problems in under 35 minutes. Practice coding solutions to medium and hard problems in less than 15 minutes each to help you be ready for the constraints during the
interview.", "If your tech screen is in person, you'll use a whiteboard"
Who can solve a "medium to hard" problem in 15 minutes, on a whiteboard, under stress, twice in a row? Who are they looking for? Leetcode psychos?
It's known for quite some years that this interview approach produces _those_. By induction, those who have FAANG lines in their resumes are, unfortunately, suspicious for this reason alone. All else being equal, non-FAANG person would get preference on the interview, and seeing somebody applying to FAANG causes both frowns and sighs. One should be really desperate for that these days.
Which system design books does it suggest reading? Last time I interviewed I did poorly on the system design interview because it was all about a scalable web app while my experience is mostly in smallish devices.
Interviewing is one of the unsolved problems in computer science. I think take-home assignments have their own tradeoffs such as time investment by the engineer or the (I'm assuming) increased chance of cheating. You also still run into the question of what a standard take-home assignment should look like, and how to prevent take-homes from becoming a game in the exact same way leetcode has become a game.
Yeah, you pay someone to take the assignment for you. Which, by the way, there's already a market out there where people are paying others to interview in their place (deceiving the company). I think video interviews are less cheatable than unsupervised assignments but still cheatable since bad actors can hire people to impersonate them. Anyways, whether or not these would be genuine significant risks in your hiring pipeline is pretty company-dependent.
As a hiring manager, I find talking in detail about work (school or industry) a candidate has done is a lot more illuminating than a coding problem. The trick is that it takes more preparation and focus from the interviewer because they need to know how to ask the right questions, they must know how to guide the conversation in real time, they must understand details about entire project in a few minutes, etc.
how many of the questions asked in the interview actually apply to their jobs? if you know of an example where it's really helpful to write a sorting algorithm correctly in 15 minutes, could you share it? :D
I ramped up to do a bunch of interviews last year after some covid layoffs and, I'd rather the format stayed in its current form. It isn't great, but I can study one style of interviews and apply that across the board to all the companies I interview for. The system interviews are easy, it's just talking about side projects or work I do for a living anyway. After ramping up and successfully landed a role last year I do a couple of leetcode questions every few days to keep my skills sharp.
I have been burnt doing so many take home exercises, getting ghosted, zero feedback declines that I am drawing a line under the sand and not doing them anymore.
The alternative is prepping for many different interview types, sometimes full of people blindly happy for candidates to burn a weekend in homes of getting a callback, other-times full of unprepared, unmotivated interviewers who pull crappy nonsensical tests out their ass. No thanks. While the leetcode style isn't perfect at least it's reasonably standardized and consistent. I'll stick with that thanks.
We ask candidates to prepare a 20min presentation about one or two interesting projects they worked on. The whole dev team (8 people) is present and we all discuss it in a friendly, genuinely interested manner on eye-level, trying to understand what the candidate has build by asking design choices or more details on certain aspects, how error paths were handled, which trade off considered, how was the collaboration in his old team etc. Typically goes for 45min. The candidate is encouraged to ask the team any questions about how we work, what our tools and processes are etc.
We get to know candidate and the candidate us.
60-75 min is all it takes.
Afterwards we sit together as a team and share our impression, could we see working with them, did anybody see red flags, etc. Maybe 15-30min conversation. Then we reach a go-nogo decision right there and now. No coding questions, no brainteasers or similar.
Haven't seen a candidate yet that could bullshit their way through the presentation and crowd consensus decision.
Personally, as an engineer at a highly desirable company, I would hate this process, and I imagine your candidates hate it too. And if your point of view during hiring comes from a place of looking for candidates trying to 'bullshit their way through' (your words), I wouldn't feel comfortable working there.
"bullshitting" was indeed a bad choice of words on my part and it's not how we see our candidates nor it's reflective of our rational why we choose this interview style. It was a mere reference to another post further above.
We try to make the candidates as comfortable as can be in a stressful interview scenario by having them talk us through some of their previous work. If that does not do it, i do not know what is. If you feel comfortable to code while others watch, good on you. I personally am not that kind of person.
We try to be mindful of their time by having a single interview with minimal mandatory preparation time and same day feedback.
We want to give them a chance to see how we as a team interact.
I finished my onsites with a FAANG company. Recruiter comes back to me and says every interviewer gave you the green light but the hiring committee wants another interview with these specific questions asked.
By this point I had already spent 8 hours with them. Told them to beat it and stop wasting my time.
It's ridiculous how disrespectful these companies are to our time.
What I find utterly sad when I read this is that the rest of the world, no let me rephrase it: the actual world, both intuitively knows this process is wrong and operates recruitment differently.
Engineers of the world don't work at FAANG companies, just a very small subset of them. Engineers who want to work there are like teenagers who were told they must study at an ivy league if they want to something with their life. It's a small club, it is not selective at all, it is just not a club that a lot of people would care to join.
What I find more interesting to observe in this post is the recruiter himself. How he was led to believe an awful community was "better" until he realized he had to escape from it, and now he is teaching something that almost every engineer in the world already knows.
So, yes, I'll upvote this one but probably not for the reason I'm expected to :)
Sometimes I think that FAANGs real power comes from their nontechnical hires. They pay top dollar for their CxOs, market strategists, domain experts, etc. With this kind of firepower, they can get along with any engineers. They just need to fill seats. The high salaries for software engineers are just to shackle their engineers with golden handcuffs so they'll stick around long-term, and not spread any secondhand strategic vision to competitors.
51 comments
[ 3.1 ms ] story [ 50.2 ms ] threadThere's no shortage of engineers who are preparing months and months on end, solving hundreds or thousands of leetcode problems.
There are coaching institutes, and an entire industry devoted to cracking these interviews.
The financial reward is worth it for most people.
So in essence, the only value in failing at such an interview is knowing what to prepare to clear it later.
Typo. "I was not testing their knowledge..."
Great article.
An example: one we used a few times was "we decided we want to build a drone for the police. So if a bank is being robbed the cop who arrived first could take it out and it will help solve the crime." Some people would ask a bunch of constraints up front; others would just dive in and ask questions as they went. And they'd think about the problem "so we want to chase the getaway car long enough until a regular copter can get airborne, so let's assume X MPH for Y minutes minimum". Someone else thought "small is probably better because perhaps it can look into the bank when it would be dangerous for a person to try, and we have to assume a lot of them will be destroyed, so cost matters." etc etc.
Another great one is "Jucero (this is just after they collapsed) has inspired us to make a bluetooth-only coffee machine". My favorite candidate started with "even with BT you'll really need a control panel for reasons XXX and YYY. And BT generally won't add much, if anything, but if product management insists, let's think up some halfway plausible use cases..." and then ended up in a discussion of the protocol and how to think about making it robust against failure."
These were always fun, and told us a bit about how the candidate thinks, without the pressure of doing it "right". Often the solutions were excessively elaborate, or missed elements a real product would need, but so what? They were "first drafts" in a situation with some pressure. And who cares about remembering the precise arguments to a function or anything that needed looking up.
And the length of these things helped strain out bullshitters. One candidate immediately said "the best way to solve this is to use an XXX. The other interviewer and I looked at each other excitedly. That was the best way and nobody else had suggested it. The candidate went on to ignore XXX, even when we asked questions in the hope of sending the discussion back that way. It's as if they had said "oh, well for this you'll need an adjustable wing" and then drew a wheeled vehicle that could not fly.
Who can solve a "medium to hard" problem in 15 minutes, on a whiteboard, under stress, twice in a row? Who are they looking for? Leetcode psychos?
Yes :) .
It's known for quite some years that this interview approach produces _those_. By induction, those who have FAANG lines in their resumes are, unfortunately, suspicious for this reason alone. All else being equal, non-FAANG person would get preference on the interview, and seeing somebody applying to FAANG causes both frowns and sighs. One should be really desperate for that these days.
:(
Haha this needs to be said with Zuck's poker face and without a smiley, just "Yes".
Even the slightest problem solving one of those will be a sureshot ding.
I find going into the technical details of projects on the candidates CV quickly sorts them out.
I have been burnt doing so many take home exercises, getting ghosted, zero feedback declines that I am drawing a line under the sand and not doing them anymore.
The alternative is prepping for many different interview types, sometimes full of people blindly happy for candidates to burn a weekend in homes of getting a callback, other-times full of unprepared, unmotivated interviewers who pull crappy nonsensical tests out their ass. No thanks. While the leetcode style isn't perfect at least it's reasonably standardized and consistent. I'll stick with that thanks.
We get to know candidate and the candidate us.
60-75 min is all it takes.
Afterwards we sit together as a team and share our impression, could we see working with them, did anybody see red flags, etc. Maybe 15-30min conversation. Then we reach a go-nogo decision right there and now. No coding questions, no brainteasers or similar. Haven't seen a candidate yet that could bullshit their way through the presentation and crowd consensus decision.
We try to make the candidates as comfortable as can be in a stressful interview scenario by having them talk us through some of their previous work. If that does not do it, i do not know what is. If you feel comfortable to code while others watch, good on you. I personally am not that kind of person.
We try to be mindful of their time by having a single interview with minimal mandatory preparation time and same day feedback.
We want to give them a chance to see how we as a team interact.
By this point I had already spent 8 hours with them. Told them to beat it and stop wasting my time.
It's ridiculous how disrespectful these companies are to our time.
Engineers of the world don't work at FAANG companies, just a very small subset of them. Engineers who want to work there are like teenagers who were told they must study at an ivy league if they want to something with their life. It's a small club, it is not selective at all, it is just not a club that a lot of people would care to join.
What I find more interesting to observe in this post is the recruiter himself. How he was led to believe an awful community was "better" until he realized he had to escape from it, and now he is teaching something that almost every engineer in the world already knows.
So, yes, I'll upvote this one but probably not for the reason I'm expected to :)