Good ideas! I have been doing a bit of hiring lately, and wish I had read this before I started. Sounds like a really scalable way of doing a first selection.
It is completely scalable. I did it. The second round was an estimate for a more significant 'paid' task that advanced the company's goals (i.e. real work). The third round was completion and delivery of the task (which paid just enough to keep qualified candidates interested). My time was more valuable than the cost of having the few interested candidates that got that far do the various useful bits of work.
At the end I did a few long long interviews and really got to know who I was going to be working with and they me.
This is actually a fairly clever and atypical approach to hiring. I agree that the meaning of "hacking" is often stretched, and maybe this is one of those cases, but it's hardly the worst.
(However, like the Microsoft/Google brainteaser interviews, I could see these little extraneous tasks getting pretty annoying if they became widespread.)
"Can they write a little code? Do they have enough experience to be able to deploy code somewhere? (An[sic] fairly good test we've found of whether or not somebody is a "hacker". Hackers have playgrounds.)"
I guess you could assume a negative correlation there (no playground == not a 'hacker'), but I doubt there's an equally strong positive correlation. I know plenty of people with a VPS or in-home sandbox server whom I wouldn't put in the 'hacker' category (myself included).
You should not hire people that do any of the things you ask. They are giving their valuable services away for free - since your company exists to make money, your employees giving their time so freely is likely to affect your profitability.
People thinking about applying - Reconsider. The company is too cheap to pay for good work - they extract what they need from interviews. Its likely that you'll never discuss pay with them since you're not going to get hired once they have their answers.
(the details)
Getting work done in an interview is a poor practice since the candidate does not know if the work relates to an actual problem or is fictional.
If an actual problem, you are asking for the candidate's time and could end up using the candidate's approach - this puts you into a conflict of interest: you have a candidate that just gave you what you need and you are in a position to hire.
If the problem is fictional, why bother? How does it help?
Besides, candidates are smart too: it's likely they hacking the interview process too. Not sure where that leaves anyone, but good luck with that.
Possible solution: Pay the candidates for the time they spent working on your problems in the interview.
Though I doubt this is a problem in interviews. But asking for a solution even before the interview will skew your distribution of appliciants, like you point out.
Not sure if this is a bad thing --- you want to hire people for whom solving your problems is easy.
It is a problem for freelancers - people always ask for free solutions to real problems and various ways. Candidate employees are in the same situation, but during the worst possible time - the interview.
As you said, you want to hire people for whom solving your problems is easy. Keep in mind that your problems are not unique and probably already have a solution, so you really need someone that.....woops...almost gave it away ;)
we're talking about 10 minutes of coding here, not actually building some core functionality for a company that could be extracted from the interview process
startup are by definition cheap, so if you want to work for one you better come prepared for that. working in a small team is like adding someone to the family, so knowing how you think and solve problems - and what your level of initiative is - is crucial
Using some test like this to rate applicants probably only points out a weakness in communication skills, but is better than just slamming through random resumes. A well-written cover letter should easily answer the first two (and listed as most important to twilio in the article as the folks building the team) points listed.
I think it is also entirely NOT scalable. What if you get 1000 submissions? In my area of the world (Research Triangle Park in NC) there is a company that famously gets hundreds to a thousand of resume and cover letter submissions per SW dev opening, especially when the job market is tight. One healthcare software startup I knew about was getting so many resumes for open positions that they developed a policy of trashing resumes received after 9AM on any given day. Just to keep the piles manageable. How can that be effective?
But maybe a bulk of the applicants to the positions mentioned in the article clearly just shot a resume at a job listing with no further effort demonstrated? Can we get some numbers on how many applicants just sent a resume versus attempting some of the "extra" work you asked for?
I've thought for some time that the best thing an applicant to a job could do would be to provide a portfolio of code (or website or demo) that clearly shows finished projects and results they have achieved previously. But that could be a pretty intense process on the employer side - how do you review even 50 of these portfolios in a meaningful way if each one is even small code samples (say 2K lines of code)? Then the employer and candidate could sit down and talk through the design choices and schedule trade-offs made on that code.
Plus, using previously written code completely punts on the idea that you might prefer a really smart hacker who is inexperienced in some technology instead of poor developer who has spent a career in that technology churning out mediocre work. My guess is that the hackers who would care enough to actually have a code portfolio out there for employers to see are probably a cut above the average software dev type anyway.
Maybe this is my startup - given many portfolios of hacker's work, how do we make it effectively searchable by employers who need to find good matches? There has to be a good web app out there for this - recruiters get big percentages on candidate placements, so there is clearly some potential market. I don't have any data to back up this supposition, but don't we think that recruiter candidate placements can be pretty closely approximated with some data mining given a portfolio of work and simple heuristics?
An additional problem is that many hires are made via personal networks - it's just easier to say, oh yeah, that person worked with our friends who were really happy with his/her work on that project, we should think about bringing him/her in.
The problem of hiring and finding great candidates has been bubbling in my head. But I haven't had any blinding flashes of inspiration for really great solutions - any thoughts out there?
15 comments
[ 1476 ms ] story [ 1068 ms ] threadTwilio Core Engineer: http://www.linkedin.com/jobs?viewJob=&jobId=743817&f...
At the end I did a few long long interviews and really got to know who I was going to be working with and they me.
(However, like the Microsoft/Google brainteaser interviews, I could see these little extraneous tasks getting pretty annoying if they became widespread.)
http://www.phys.lsu.edu/faculty/gonzalez/Teaching/Phys7221/v...
David Jones tries to create an unrideable bicycle (and nearly fails).
I guess you could assume a negative correlation there (no playground == not a 'hacker'), but I doubt there's an equally strong positive correlation. I know plenty of people with a VPS or in-home sandbox server whom I wouldn't put in the 'hacker' category (myself included).
You should not hire people that do any of the things you ask. They are giving their valuable services away for free - since your company exists to make money, your employees giving their time so freely is likely to affect your profitability.
People thinking about applying - Reconsider. The company is too cheap to pay for good work - they extract what they need from interviews. Its likely that you'll never discuss pay with them since you're not going to get hired once they have their answers.
(the details)
Getting work done in an interview is a poor practice since the candidate does not know if the work relates to an actual problem or is fictional.
If an actual problem, you are asking for the candidate's time and could end up using the candidate's approach - this puts you into a conflict of interest: you have a candidate that just gave you what you need and you are in a position to hire.
If the problem is fictional, why bother? How does it help?
Besides, candidates are smart too: it's likely they hacking the interview process too. Not sure where that leaves anyone, but good luck with that.
[want a solution? hire me ;) ]
Though I doubt this is a problem in interviews. But asking for a solution even before the interview will skew your distribution of appliciants, like you point out.
Not sure if this is a bad thing --- you want to hire people for whom solving your problems is easy.
As you said, you want to hire people for whom solving your problems is easy. Keep in mind that your problems are not unique and probably already have a solution, so you really need someone that.....woops...almost gave it away ;)
startup are by definition cheap, so if you want to work for one you better come prepared for that. working in a small team is like adding someone to the family, so knowing how you think and solve problems - and what your level of initiative is - is crucial
I think it is also entirely NOT scalable. What if you get 1000 submissions? In my area of the world (Research Triangle Park in NC) there is a company that famously gets hundreds to a thousand of resume and cover letter submissions per SW dev opening, especially when the job market is tight. One healthcare software startup I knew about was getting so many resumes for open positions that they developed a policy of trashing resumes received after 9AM on any given day. Just to keep the piles manageable. How can that be effective?
But maybe a bulk of the applicants to the positions mentioned in the article clearly just shot a resume at a job listing with no further effort demonstrated? Can we get some numbers on how many applicants just sent a resume versus attempting some of the "extra" work you asked for?
I've thought for some time that the best thing an applicant to a job could do would be to provide a portfolio of code (or website or demo) that clearly shows finished projects and results they have achieved previously. But that could be a pretty intense process on the employer side - how do you review even 50 of these portfolios in a meaningful way if each one is even small code samples (say 2K lines of code)? Then the employer and candidate could sit down and talk through the design choices and schedule trade-offs made on that code.
Plus, using previously written code completely punts on the idea that you might prefer a really smart hacker who is inexperienced in some technology instead of poor developer who has spent a career in that technology churning out mediocre work. My guess is that the hackers who would care enough to actually have a code portfolio out there for employers to see are probably a cut above the average software dev type anyway.
Maybe this is my startup - given many portfolios of hacker's work, how do we make it effectively searchable by employers who need to find good matches? There has to be a good web app out there for this - recruiters get big percentages on candidate placements, so there is clearly some potential market. I don't have any data to back up this supposition, but don't we think that recruiter candidate placements can be pretty closely approximated with some data mining given a portfolio of work and simple heuristics?
An additional problem is that many hires are made via personal networks - it's just easier to say, oh yeah, that person worked with our friends who were really happy with his/her work on that project, we should think about bringing him/her in.
The problem of hiring and finding great candidates has been bubbling in my head. But I haven't had any blinding flashes of inspiration for really great solutions - any thoughts out there?