I can see where most of these are coming from. Estimation, consulting/sales, personality, management & leadership. The penguin in the sombrero is harder to pin down. Are they trying to see how you analyse novel situations? As in, "Well, it's wearing a sombrero but comes from the antarctic, so it's likely been on holiday. Since this office isn't in Mexico or the antarctic it's probably on a travelling holiday, which means its young and doesn't have any money. Since it's a construction company, best case is it wants a job as a labourer. Worst case, it wants to use the phone to call its parents and ask for a bail out." I know, call me Sherlock. I expect I've missed the point though. I really would like to know what other people have to say about this question.
> The penguin in the sombrero is harder to pin down.
My response would be, "If a penguin in a sombrero walks through the door and starts talking, the most plausible reason for why he's here is that you've spiked my coffee with LSD."
Penguin in a sombrero? It sounds to me like one of the smaller penguins from Happy Feet, so any line one of them might say. For instance, Hey, where did all the girls go?
My guess is that they want to learn your sense of humor, to know whether you'll fit with the team.
I am pretty sure (and given this is HN I'm sure a googler will be along soon to confirm/deny this) that Google have a policy of no longer asking those kind of brain-teasers for their software engineering roles. They may still do so for PMs, etc.
Certainly when I had my Google on-site for an engineering role I was spared the infamous questions that crop up in articles like this from time to time.
I've seen variations of #18 before but more involved versions where I've had to write pseudo code to place clock hands based on the current time from a timer.
I've also seen various variations of the Fermi questions before, though maybe not exactly these ones. Like half of the questions listed there fall into that model: Estimate how many XYZ are there in ABC?
Of the others, “How would you direct someone else on how to cook an omelet?” seems pretty reasonable in terms of sussing out communication skills, but a lot of the rest are just insanely stupid questions, IMO.
#18 is kind of interesting, because there's two possible answers. For a modern analogue clock where the hands move in fixed increments, rather than smoothly, the answer is simply 5 minutes. For older ones it's 25/6 minutes.
I like #12, just because it begs to be answered with "perfectly".
#18 actually seems like a real-world problem a programmer might face. Besides, there are coldly logical steps you can take to solve the question, and a closed, correct answer, unlike most of the open-ended waffly answers elsewhere in the list.
I don't really ask these kinds of questions, but I do kind of like them.. the giant puzzle/estimation types. Can someone who asks these regularly give some feedback on how well you think they work? I'm curious how much you end up learning from them.
That said, there are some doozies in this list. For example:
"On a scale of one to ten, rate me as an interviewer".
Um, okay but only if the next question is to rate you as an asshole.
In my experience these "Fermi questions" aren't that useful, the primary thing they tell you is whether the person has previously been exposed to the concept of "Fermi questions" or not.
I guess if the person being interviewed is 30+ and working in software development and hasn't been exposed to Fermi questions then maybe that's a red flag, but other than that I've seen no indication that answering these types of questions well correlates to real world performance.
I once interviewed at a place that had an interesting spin. They gave me three fermi questions, which I had 5 minutes total to estimate:
1. Mass of the earth in kg
2. US GDP
3. Number of carriages on the London Underground
I then had to give a 95% confidence interval for each one (i.e. an interval that I was 95% sure the true answer lay in) and then asked, for each estimate, which bet I wanted to take:
1. Bet that the answer is inside the interval at odds of 10 to 1 on.
2. Bet that the answer is outside the interval at odds of 30 to 1.
There's a lot of work that has to be done there, and potential for a very interesting discussion (this was for a trading/research role, rather than a software developer role).
Which would be more impressive: that I long ago memorized that the Earth's mass is about ~5.98E24 kg or that I could estimate it knowing Kepler's third law (T-squared = C * R-cubed) and orbital information about the Moon(28d@250,000miles), Earth (365d@1AU), and Mercury (88d@0.4AU), where 1AU=93million miles. Or that I could cross-check it by knowing the circumference of 25,000 miles to get the approximate volume, and knowing that the density is around 5tons/cubic meter? Or I could use the orbital numbers to estimate G then knowing that Earth's surface gravity is 9.81m/s/s then I can also get M_earth.
All of those involve memorized numbers, and most require physics knowledge. But I have a feeling that the more complicated ones would impress them more, and make them more likely to hire me. If that's the case, then my strategy is not to give the answer which I know, but to exaggerate the difficulty by hiding my actual knowledge.
GDP? It's about the size of the debt, which is something like 7 trillion, so I would have estimated $10 trillion. (Actually, the debt is $12, and the GDP is $15.) I might have double-checked my answer because I think the budget is about $4 trillion, and 20% of GDP giving $20 trillion. (Actually 3.54 trillion and 22.6% GDP).
How is this memorized knowledge useful? Especially since I don't know how the GDP is actually calculated? Nor the difference between GNP and GDP. So this mostly tests to see if I've paid attention to the economic news. It's almost like asking me if I know the hottest temperature ever recorded in Death Valley.
And as for the London Underground - too many variables for me to even estimate. Turns out to be 4134, and that what I thought was part of the Underground was actually rail services. If forced, I think would have said: 20 rail lines? (Actually, 11.) At rush hour, 10 cars separated by 5 minutes distance, for a total of about 90 minutes per line? Which gives 3600 cars. I probably would have rounded up to 4,000 for spares, and then been amazed at how close I was. (Ahh, I forgot the factor of two because trains are going in both directions. And I think the inter-train spacing is less than 5 minutes on some of the lines.)
But this only works because I've been to London and made those observations. Without that experience, I would be off by an order of magnitude.
So there you go - this proves that I have a degree in physics, that I follow the news, and that I've visited London. It helps with the job you applied to because ... ?
(And if you say it's because I can reason about these sorts of problems, then, see "have a degree in physics" and add "and also math.")
Almost all of the interesting stuff comes after you've made your estimates (although the process of getting to the estimates is not completely uninformative).
If the candidate performs well on this question, then they have shown:
1. They have the ability to reason about problems in a potentially new domain.
2. They have the ability to pare down a problem to what is important and what can be neglected.
3. They can make good order-of-magnitude estimates (very important for an equities trader).
4. They are aware of the typical size of their errors, or can estimate them.
5. The are aware of the typical sources of error.
6. They understand the notion of a confidence interval and can apply it to real-world situations.
7. They can reason about bets with different characteristics (e.g. different expected value, different skew characteristics) and choose a good portfolio of bets.
I think that knowing how GDP is calculated is useful information for someone working in finance. Especially someone who is going to have to trade through economic announcements. Would you hire a trader who didn't follow economic news?
PS re "shows you've visited London" - the interview was in London. I assume if it had been in New York, they would have asked about the New York metro.
I passed the first test on rote memorization. The second was nearly the same. Only the third would have given information about my ability to estimate, which isn't enough to give real information about those 7 bullet points. I remain doubtful about the utility of these tests.
There is nothing, for example, which probes my experience with statistics.
Here's a more in-depth test. We run an medical experiment on lab rats. The protocol says that we will experiment on 20 rats, and that a mortality rate above 30% is too high so we must stop the experiment. After 5 rats, 2 have died. Should we stop the experiment now? Why or why not?
One could look at that and say that a 2/5=40%>30% so of course it should be stopped - it's higher than the allowed rate.
A better response is to point out that if 10% is acceptable, then there's a 0.9^3 * 0.1^2 * C(5,3) = 7% chance that this is an expected statistical fluctuation, and that the experimental protocols should have given some guideline on how certain one needs to be in knowing if the mortality rate was too high.
An even better response would be to point out that this means the first three rats lived, and it was only rats 4 and 5 which died, as otherwise the experiment would have stopped earlier under this protocol. In that case the possible experimental states are: 1) first rat died, so the experiment stopped = p. 2) first lived, second died, stop = p. 3) first lived, second lived, third died, stop = p. 4) first, second, third lived, fourth died, fifth lived = (1-p). and 5) first, second, third lived, fourth and fifth died = p giving (assuming I recall how to do this correctly) p+p+p+(1-p)+p = 1.0 or an estimated mortality rate of p=0.3.
In other words, this experiment is allowed to keep on going under the given protocol.
But this sort of modeling awareness is not present in the three tests you mentioned, despite being directly relevant, I assume, to what a trader does.
Yes, this kind of statistical modelling is certainly very relevant. I don't want to give the impression that this was the only question I was asked, or that it covers everything you need to know about a candidate. The entire question and discussion took about 15 minutes. I was asked questions of a similar kind to the one you propose in the other 11.75 hours of interviews.
I watched a once-coworker give a 30 minute lecture to high school students on how to do these giant puzzle/estimation types. His trick was to take the geometric mean of reasonable min/max bounds at different stages of the estimation.
Example question: How many flights occur in America in a day?
You can start from the bottom, with an estimate of the number of people who fly, then the number that can fit in a plane, et cetera. Alternatively, you can start from the top at the number of airports and their flight capacity. With a bunch of sqrt(minbound*maxbound) and reasonable guessing, the students (via groupthink) ended up with two roughly matching estimates that were within an order of magnitude.
“What’s your favorite song? Perform it for us now.”
I'd have to go with "Take this job and shove it". There's nothing worse than places that mandate you be "fun" and "offbeat". If I am fun/offbeat, all you as an interviewer have to do is make me comfortable enough to show it. If you can't do that, why do I want to work for you?
These aren't just wacky, they are stupid. "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten. Little or no relation to software development.
Here's what we do. Interested in feedback, please!
Via the phone:
* A few background questions (why are you looking for a job, what sounds interesting about what we are doing) (20-30 minutes)
* A few programming challenges (of increasing difficulty) that involve stuff the candiate is not expected to know (some algorithmic, most problem-solving type thing), and has to solve it via Google and Stack Overflow or whatnot, to make sure they can work outside of their comfort zone (I've had a lot of people freeze up at "Write a program to get Time from outside of the system", for example) (couple of hours til lunchtime)
* Then pair programming for the rest of the day to make sure we can all get along with this person (after lunch, and next day if possible)
If reasonable, we put a new hire on a 1 month contract before we make a final offer (we give a tentative salary offer at day one so they know approximately what it'll be but cannot judge equity grants until we have more time with them).
> "How many ping-pong balls can fit in a VW bus" is just about the worst question I've ever gotten.
The point of questions like this is "Does this guy have any common sense?" Some of my least favorite hires were people who were whip-sharp at the details of their chosen programming language, but then when hired, would go off and do crazy and impractical stuff with it.
Whether or not the question accomplishes a good evaluation of someone's common sense is up for debate, but unless you're hiring drones, you do have to ask some questions that get a broader picture of the way the person thinks.
(BTW, your "pair programming for the rest of the day" is a great way of getting that. But pair programming doesn't/wouldn't work at any place I've ever been, for a variety of reasons.)
I actually have never asked an estimation question, but I understand the motivation for asking this kind of question.
I'm curious--what do you ask in interviews to get at a candidate's common sense? It's not very easy to do.
Or do you think you don't need to assess common sense at all? Maybe it's not, but like I said, a lot of the hires I later regretted came down to something like that.
> Or do you think you don't need to assess common sense at all?
shrug If we're talking developers... The ability to drive to the location on time, answer questions without going blank are good indicators of common sense for me. Extracurricular activities, anything from volunteering at an animal shelter, to making civil war models to home brewing beer, are huge in my book because they are a good indicator of a grounded individual. That mixed with some pair-programming should give more than enough information. YMMV.
I think the "real world, open-book" problems have to be crafted well, but I think they give a good sense of "Common sense". They are set up that you can absolutely prematurely optimize or go down really crazy paths (and I've seen them happen).
You don't need another developer to pair program to get that - essentially you set up a clean computer with their editor on it and your testing/interview environment, and they can hack away while you watch.
But yes, if there's some way to get a candidate to pair program on actual, real code with real developers, it's invaluable. Just make sure they sign a contract stating they are giving you the rights to the code, and that they are adequately compensated for their time at some reasonable hourly rate.
I see a little value in "Fermi problems" like ping pongs in a VW bus. I use one like it on a candidate's second call (first call with an engineer; takes place after our on-site recruiter has screened tehem). The point is to see how they handle a relatively intractable problem. The only real wrong answer is either a refusal to come up with an answer or an off-the-cuff guess with no explanation. If they at least try to extrapolate an answer from reasonable assumptions, that's fine.
To your process, my thoughts:
- Open-book challenges like you describe (in which you can use stack overflow and whatnot) are awesome. However, why do you have them do stuff they're not expected to know? From what I've read, the only thing hiring studies seem to agree on is that the questions which best predict a candidate's long-term success are those that simulate what the candidate will actually be working on. Getting them to learn outside their comfort zone is good, but maybe you'd be better off doing that by finding problems you've actually had in the course of your work that are outside of their comfort zone.
- Short-term contracts are tricky. On one hand, I think they're the best way to test how well a candidate will work out. On the other hand, switching jobs is a lot of work and stress. If I had a choice between two pretty good companies, one of which was a 1-month contract and one of which was a normal job, I'd definitely take the latter. If I had to move to get your job, it'd make rejecting such a proposal even more of a no-brainer. If it works out for you, that's great, but I'd be really worried about losing excellent candidates.
- It seems like your remote screening is lighter than what I'm used to. At my current gig, we send candidates a challenge to be completed in their own time (3-9 hours of work)- we've found it's a great prelude to the phone screen with an engineer. It's pretty much a code sample for which we have an excellent reference (all the previous submissions). We've found it improves the quality of candidates that get to our on-site.
Re: open book, what I meant was "not expected to know" means something that they can't do blindfolded. For a Rails person making a contrived app from scratch doesn't tell me anything other than they could type `rails foo` and run a few generators.
What I mean is that, I put together a few challenges with APIs we work with (or gems we've written, that sort of thing) and have them solve a problem that will require them to use their innate ability, drive, and Google/StackOverflow/etc. to solve the problem.
It's not something that they would be able to do without the internet. Does that make sense?
The contract thing is tough, although at least in NYC the market is so tight that there's really no risk of unemployment, but yeah, we wouldn't do relo + contract. We'd start them off as remote and see if that worked. If so, then relo; it's risky risky for both parties (especially in NYC) that it's kinda crazy to expect either party to agree to it.
The candidate has no idea what NYC is like, and the organization may be reluctant to fire a bad hire who just isn't fitting in because they feel bad. At least with a remote contract, no one has to move and we can see if we both like each other enough to get married. Then would come the offer + relo., or if it's not a good fit, end the contract (with 2 weeks notice at least).
Finally, I'd say the initial screen is a bit light. We used to have about a 1-2 hr. technical Q&A followed by Fizzbuzz-style questions, but that had a low false-positive rate so we got rid of it.
That is to say, the background work I did (googling them, stack overflow, github, twitter, their resume, etc.) was a pretty good filter at keeping out the obviously underqualified.
However, I'm considering putting it back as it helped give me a more nuanced picture of the candidates that I end up doing as part of the contract, and it might help to have some first-impressions ahead of time (but I am always worried about false first impressions clouding my judgement).
The minute hand is at 300 degrees and the hour hand is at 354 degrees, so the correct answer is 54 degrees (subject to me not having messed up the arithmetic).
You're totally right, of course. I originally wrote 55 degrees (using exactly your argument) then looked at an actual clock and edited my post. Overthinking...
tech companies asking these silly questions doesn't make them top employers. they should better be focusing on the domain instead of wasting time on nonsense stuffs. it happens only in bay area. crap!
M$: 1.Define diaper. As in adult, children, paper, cloth, etc. 2.So it doesnt fall down the hole it is covering.
Amazon: Depends on how much PHP I have to write.
Google: Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
Zillow: Define success.
Expedia: Which language version for the T9 keypad? In what language do the words
have to be found?
“If you were to get rid of one state in the US, which would it be and why?
Define "rid".
“How many cows are in Canada?”
Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
“How many quarters would you need to reach the height of the Empire State building?”
Define the specific quarter used for the exercise. Also, tell me the temperature
at which the exercise will be done.
“A penguin walks through that door right now wearing a sombrero. What does he say and why is he here?
What type of penguin, and sombrero?
“What songs best describes your work ethic?”
What genres are allowed?
“Jeff Bezos walks into your office and says you can have a million dollars to launch your best entrepreneurial idea. What is it?”
Define best.
“What do you think about when you are alone in your car?”
Is the car ON? Am I driving somewhere? If so, Where?
“How would you rate your memory?”
Which type of memory (short/mid/long-term?)?
“Name 3 previous Nobel Prize Winners.”
Can it be for the same Nobel Prize?
“Can you say: ’Peter Pepper Picked a Pickled Pepper’ and cross-sell a washing machine at the same time?”
When do you want the Washing Machine delivered?
“If we came to your house for dinner, what would you prepare for us?”
Are you allergic to any type of food?
“How would people communicate in a perfect world?”
Define perfect world.
“How do you make a tuna sandwich?”
Which type of tuna sandwhich?
“My wife and I are going on vacation, where would you recommend?”
Are you going together?
“You are a head chef at a restaurant and your team has been selected to be on Iron Chef. How do you prepare your team for the competition and how do you leverage the competition for your restaurant?
When is the competition?
“Estimate how many windows are in New York.”
Are we refering to regular windows or computing devices running the operating system Windows? Plus,
define Window. Is it an opening in a wall? Does it need to have a glass plane?
“What’s your favorite song? Perform it for us now.”
Define favorite (as in top 10 ranking, overall, etc.). Define perform (sing, sing+dance, dance, mime, etc.).
“Calculate the angle of two clock pointers when time is 11:50.”
Of course it matters. The ask to make the calculation when its 11:50. I need to know if I'm free to be able to go and calculate it at exactly 11:50. I would not want to do the calculation at 11:49 or 11:51, because then I could be missing another calculation appointment.
Using when implies that I do the calculation at the exact point in space-time, and not calculate the angle of the clock pointers at the moment the clock marks 11:50.
But I always show up for the weekly lets sing happy birthday to those who had their birthday this week. And sometimes I'm the one who brings the cake. That's the definition of fun[1].
>Are we refering to regular windows or computing devices running the operating system Windows? Plus, define Window. Is it an opening in a wall? Does it need to have a glass plane?
>“How would you direct someone else on how to cook an omelet?”
>Are they disabled in any way?
>“If we came to your house for dinner, what would you prepare for us?”
>Are you allergic to any type of food?
>“My wife and I are going on vacation, where would you recommend?”
>Are you going together?
I actually quite liked these answers, and I would think positively of a candidate who asked them (assuming I would ask the questions in the first place, which is unlikely).
While "Define this that and the other ad nauseum" is probably overly pedantic, the above do illustrate an ability and propensity both for analysis as well as even perhaps empathy - both, IMO, meaningful skills when developing a product.
Its all in good humor, but it shows that I've been involved in product development/marketing for a while. I'm particularly proud of the "my wife and I are going on vacation" answer, because I was actually asked that one in real life by a friend. I replied the same way. His response was to laugh histerically. :)
59 comments
[ 5.5 ms ] story [ 138 ms ] threadMy response would be, "If a penguin in a sombrero walks through the door and starts talking, the most plausible reason for why he's here is that you've spiked my coffee with LSD."
He says "deslizarse". He's there because he is my power animal.
My guess is that they want to learn your sense of humor, to know whether you'll fit with the team.
Certainly when I had my Google on-site for an engineering role I was spared the infamous questions that crop up in articles like this from time to time.
I've also seen various variations of the Fermi questions before, though maybe not exactly these ones. Like half of the questions listed there fall into that model: Estimate how many XYZ are there in ABC?
Of the others, “How would you direct someone else on how to cook an omelet?” seems pretty reasonable in terms of sussing out communication skills, but a lot of the rest are just insanely stupid questions, IMO.
I like #12, just because it begs to be answered with "perfectly".
That said, there are some doozies in this list. For example: "On a scale of one to ten, rate me as an interviewer".
Um, okay but only if the next question is to rate you as an asshole.
I guess if the person being interviewed is 30+ and working in software development and hasn't been exposed to Fermi questions then maybe that's a red flag, but other than that I've seen no indication that answering these types of questions well correlates to real world performance.
1. Mass of the earth in kg 2. US GDP 3. Number of carriages on the London Underground
I then had to give a 95% confidence interval for each one (i.e. an interval that I was 95% sure the true answer lay in) and then asked, for each estimate, which bet I wanted to take:
1. Bet that the answer is inside the interval at odds of 10 to 1 on. 2. Bet that the answer is outside the interval at odds of 30 to 1.
There's a lot of work that has to be done there, and potential for a very interesting discussion (this was for a trading/research role, rather than a software developer role).
All of those involve memorized numbers, and most require physics knowledge. But I have a feeling that the more complicated ones would impress them more, and make them more likely to hire me. If that's the case, then my strategy is not to give the answer which I know, but to exaggerate the difficulty by hiding my actual knowledge.
GDP? It's about the size of the debt, which is something like 7 trillion, so I would have estimated $10 trillion. (Actually, the debt is $12, and the GDP is $15.) I might have double-checked my answer because I think the budget is about $4 trillion, and 20% of GDP giving $20 trillion. (Actually 3.54 trillion and 22.6% GDP).
How is this memorized knowledge useful? Especially since I don't know how the GDP is actually calculated? Nor the difference between GNP and GDP. So this mostly tests to see if I've paid attention to the economic news. It's almost like asking me if I know the hottest temperature ever recorded in Death Valley.
And as for the London Underground - too many variables for me to even estimate. Turns out to be 4134, and that what I thought was part of the Underground was actually rail services. If forced, I think would have said: 20 rail lines? (Actually, 11.) At rush hour, 10 cars separated by 5 minutes distance, for a total of about 90 minutes per line? Which gives 3600 cars. I probably would have rounded up to 4,000 for spares, and then been amazed at how close I was. (Ahh, I forgot the factor of two because trains are going in both directions. And I think the inter-train spacing is less than 5 minutes on some of the lines.)
But this only works because I've been to London and made those observations. Without that experience, I would be off by an order of magnitude.
So there you go - this proves that I have a degree in physics, that I follow the news, and that I've visited London. It helps with the job you applied to because ... ?
(And if you say it's because I can reason about these sorts of problems, then, see "have a degree in physics" and add "and also math.")
If the candidate performs well on this question, then they have shown:
1. They have the ability to reason about problems in a potentially new domain.
2. They have the ability to pare down a problem to what is important and what can be neglected.
3. They can make good order-of-magnitude estimates (very important for an equities trader).
4. They are aware of the typical size of their errors, or can estimate them.
5. The are aware of the typical sources of error.
6. They understand the notion of a confidence interval and can apply it to real-world situations.
7. They can reason about bets with different characteristics (e.g. different expected value, different skew characteristics) and choose a good portfolio of bets.
I think that knowing how GDP is calculated is useful information for someone working in finance. Especially someone who is going to have to trade through economic announcements. Would you hire a trader who didn't follow economic news?
PS re "shows you've visited London" - the interview was in London. I assume if it had been in New York, they would have asked about the New York metro.
There is nothing, for example, which probes my experience with statistics.
Here's a more in-depth test. We run an medical experiment on lab rats. The protocol says that we will experiment on 20 rats, and that a mortality rate above 30% is too high so we must stop the experiment. After 5 rats, 2 have died. Should we stop the experiment now? Why or why not?
One could look at that and say that a 2/5=40%>30% so of course it should be stopped - it's higher than the allowed rate.
A better response is to point out that if 10% is acceptable, then there's a 0.9^3 * 0.1^2 * C(5,3) = 7% chance that this is an expected statistical fluctuation, and that the experimental protocols should have given some guideline on how certain one needs to be in knowing if the mortality rate was too high.
An even better response would be to point out that this means the first three rats lived, and it was only rats 4 and 5 which died, as otherwise the experiment would have stopped earlier under this protocol. In that case the possible experimental states are: 1) first rat died, so the experiment stopped = p. 2) first lived, second died, stop = p. 3) first lived, second lived, third died, stop = p. 4) first, second, third lived, fourth died, fifth lived = (1-p). and 5) first, second, third lived, fourth and fifth died = p giving (assuming I recall how to do this correctly) p+p+p+(1-p)+p = 1.0 or an estimated mortality rate of p=0.3.
In other words, this experiment is allowed to keep on going under the given protocol.
But this sort of modeling awareness is not present in the three tests you mentioned, despite being directly relevant, I assume, to what a trader does.
I watched a once-coworker give a 30 minute lecture to high school students on how to do these giant puzzle/estimation types. His trick was to take the geometric mean of reasonable min/max bounds at different stages of the estimation.
Example question: How many flights occur in America in a day?
You can start from the bottom, with an estimate of the number of people who fly, then the number that can fit in a plane, et cetera. Alternatively, you can start from the top at the number of airports and their flight capacity. With a bunch of sqrt(minbound*maxbound) and reasonable guessing, the students (via groupthink) ended up with two roughly matching estimates that were within an order of magnitude.
I'd have to go with "Take this job and shove it". There's nothing worse than places that mandate you be "fun" and "offbeat". If I am fun/offbeat, all you as an interviewer have to do is make me comfortable enough to show it. If you can't do that, why do I want to work for you?
If http://www.glassdoor.com/Interview/LivingSocial-Interview-RV... are to bee believed this company are wasting a lot of peoples time.
The correct response to this question is to say "Could I please hold your pen?", and wink slyly.
I consider it stealing as much as I consider using the toilet paper they provide for me stealing.
Here's what we do. Interested in feedback, please!
Via the phone:
* A few background questions (why are you looking for a job, what sounds interesting about what we are doing) (20-30 minutes)
* A bunch of topgrading questions http://www.smarttopgrading.com/ (maybe an hour)
In person:
* A few programming challenges (of increasing difficulty) that involve stuff the candiate is not expected to know (some algorithmic, most problem-solving type thing), and has to solve it via Google and Stack Overflow or whatnot, to make sure they can work outside of their comfort zone (I've had a lot of people freeze up at "Write a program to get Time from outside of the system", for example) (couple of hours til lunchtime)
* Then pair programming for the rest of the day to make sure we can all get along with this person (after lunch, and next day if possible)
If reasonable, we put a new hire on a 1 month contract before we make a final offer (we give a tentative salary offer at day one so they know approximately what it'll be but cannot judge equity grants until we have more time with them).
:D
The point of questions like this is "Does this guy have any common sense?" Some of my least favorite hires were people who were whip-sharp at the details of their chosen programming language, but then when hired, would go off and do crazy and impractical stuff with it.
Whether or not the question accomplishes a good evaluation of someone's common sense is up for debate, but unless you're hiring drones, you do have to ask some questions that get a broader picture of the way the person thinks.
(BTW, your "pair programming for the rest of the day" is a great way of getting that. But pair programming doesn't/wouldn't work at any place I've ever been, for a variety of reasons.)
GROOOOOAAAAN
Maybe twenty years ago, and maybe if you were the individual to think up the question.
Honestly I'm so goddamned bored with these unoriginal blither-blather questions I consider them a strike against the interviewer and the company.
I'm curious--what do you ask in interviews to get at a candidate's common sense? It's not very easy to do.
Or do you think you don't need to assess common sense at all? Maybe it's not, but like I said, a lot of the hires I later regretted came down to something like that.
shrug If we're talking developers... The ability to drive to the location on time, answer questions without going blank are good indicators of common sense for me. Extracurricular activities, anything from volunteering at an animal shelter, to making civil war models to home brewing beer, are huge in my book because they are a good indicator of a grounded individual. That mixed with some pair-programming should give more than enough information. YMMV.
You don't need another developer to pair program to get that - essentially you set up a clean computer with their editor on it and your testing/interview environment, and they can hack away while you watch.
But yes, if there's some way to get a candidate to pair program on actual, real code with real developers, it's invaluable. Just make sure they sign a contract stating they are giving you the rights to the code, and that they are adequately compensated for their time at some reasonable hourly rate.
To your process, my thoughts:
- Open-book challenges like you describe (in which you can use stack overflow and whatnot) are awesome. However, why do you have them do stuff they're not expected to know? From what I've read, the only thing hiring studies seem to agree on is that the questions which best predict a candidate's long-term success are those that simulate what the candidate will actually be working on. Getting them to learn outside their comfort zone is good, but maybe you'd be better off doing that by finding problems you've actually had in the course of your work that are outside of their comfort zone.
- Short-term contracts are tricky. On one hand, I think they're the best way to test how well a candidate will work out. On the other hand, switching jobs is a lot of work and stress. If I had a choice between two pretty good companies, one of which was a 1-month contract and one of which was a normal job, I'd definitely take the latter. If I had to move to get your job, it'd make rejecting such a proposal even more of a no-brainer. If it works out for you, that's great, but I'd be really worried about losing excellent candidates.
- It seems like your remote screening is lighter than what I'm used to. At my current gig, we send candidates a challenge to be completed in their own time (3-9 hours of work)- we've found it's a great prelude to the phone screen with an engineer. It's pretty much a code sample for which we have an excellent reference (all the previous submissions). We've found it improves the quality of candidates that get to our on-site.
What I mean is that, I put together a few challenges with APIs we work with (or gems we've written, that sort of thing) and have them solve a problem that will require them to use their innate ability, drive, and Google/StackOverflow/etc. to solve the problem.
It's not something that they would be able to do without the internet. Does that make sense?
The contract thing is tough, although at least in NYC the market is so tight that there's really no risk of unemployment, but yeah, we wouldn't do relo + contract. We'd start them off as remote and see if that worked. If so, then relo; it's risky risky for both parties (especially in NYC) that it's kinda crazy to expect either party to agree to it.
The candidate has no idea what NYC is like, and the organization may be reluctant to fire a bad hire who just isn't fitting in because they feel bad. At least with a remote contract, no one has to move and we can see if we both like each other enough to get married. Then would come the offer + relo., or if it's not a good fit, end the contract (with 2 weeks notice at least).
Finally, I'd say the initial screen is a bit light. We used to have about a 1-2 hr. technical Q&A followed by Fizzbuzz-style questions, but that had a low false-positive rate so we got rid of it.
That is to say, the background work I did (googling them, stack overflow, github, twitter, their resume, etc.) was a pretty good filter at keeping out the obviously underqualified.
However, I'm considering putting it back as it helped give me a more nuanced picture of the candidates that I end up doing as part of the contract, and it might help to have some first-impressions ahead of time (but I am always worried about false first impressions clouding my judgement).
Sorry what was that again? I forgot what you asked
Go see an ENT specialist.
I know the correct answer is supposed to be 0, but the hour hand moves closer to hour+1 as minutes approach 60. I guess I would ask for a compass.
Another fave is: “How many quarters would you need to reach the height of the Empire State building?”
My answer would be: "What's the price of admission?"
30 degrees between hours. 10 is at 300 degrees, 11 is 330 degrees. 50 minutes into the hour is 5/6 of 30 or 25. 330+25 == 355.
Amazon: Depends on how much PHP I have to write.
Google: Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
Zillow: Define success.
Expedia: Which language version for the T9 keypad? In what language do the words have to be found?
“If you were to get rid of one state in the US, which would it be and why?
Define "rid".
“How many cows are in Canada?”
Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
“How many quarters would you need to reach the height of the Empire State building?”
Define the specific quarter used for the exercise. Also, tell me the temperature at which the exercise will be done.
“A penguin walks through that door right now wearing a sombrero. What does he say and why is he here?
What type of penguin, and sombrero?
“What songs best describes your work ethic?”
What genres are allowed?
“Jeff Bezos walks into your office and says you can have a million dollars to launch your best entrepreneurial idea. What is it?”
Define best.
“What do you think about when you are alone in your car?”
Is the car ON? Am I driving somewhere? If so, Where?
“How would you rate your memory?”
Which type of memory (short/mid/long-term?)?
“Name 3 previous Nobel Prize Winners.”
Can it be for the same Nobel Prize?
“Can you say: ’Peter Pepper Picked a Pickled Pepper’ and cross-sell a washing machine at the same time?”
When do you want the Washing Machine delivered?
“If we came to your house for dinner, what would you prepare for us?”
Are you allergic to any type of food?
“How would people communicate in a perfect world?”
Define perfect world.
“How do you make a tuna sandwich?”
Which type of tuna sandwhich?
“My wife and I are going on vacation, where would you recommend?”
Are you going together?
“You are a head chef at a restaurant and your team has been selected to be on Iron Chef. How do you prepare your team for the competition and how do you leverage the competition for your restaurant?
When is the competition?
“Estimate how many windows are in New York.”
Are we refering to regular windows or computing devices running the operating system Windows? Plus, define Window. Is it an opening in a wall? Does it need to have a glass plane?
“What’s your favorite song? Perform it for us now.” Define favorite (as in top 10 ranking, overall, etc.). Define perform (sing, sing+dance, dance, mime, etc.).
“Calculate the angle of two clock pointers when time is 11:50.”
AM or PM?
“Have you ever stolen a pen from work?”
Define stolen.
“Pick two celebrities to be your parents.”
Define celebrities.
“What kitchen utensil would you be?”
Define utensil (mechanical, electric, electronic, etc). Define kitchen (commercial, American, European, etc.).
“If you had turned you cell phone to silent, and it rang really loudly despite it being on silent, what would you tell me?”
Which cellphone (brand, type, OS, etc)?
“On a scale from one to ten, rate me as an interviewer.”
Does the scale go from best to worst or otherwise?
“If you could be anyone else, who would it be?”
Define be.
“How would you direct someone else on how to cook an omelet?”
Are they disabled in any way?
It doesn't matter.
Using when implies that I do the calculation at the exact point in space-time, and not calculate the angle of the clock pointers at the moment the clock marks 11:50.
:)
I know you're joking, and there is value to getting the interviewer to clarify the question and confirm assumptions.
But some of your responses ("define be") are nitpicky to the point of "this guy wouldn't be any fun to work with".
But I always show up for the weekly lets sing happy birthday to those who had their birthday this week. And sometimes I'm the one who brings the cake. That's the definition of fun[1].
[1] In corporate America.
It definitely doesn't mean "fun only during non-work bullshit".
>Are we refering to regular windows or computing devices running the operating system Windows? Plus, define Window. Is it an opening in a wall? Does it need to have a glass plane?
>“How would you direct someone else on how to cook an omelet?”
>Are they disabled in any way?
>“If we came to your house for dinner, what would you prepare for us?”
>Are you allergic to any type of food?
>“My wife and I are going on vacation, where would you recommend?”
>Are you going together?
I actually quite liked these answers, and I would think positively of a candidate who asked them (assuming I would ask the questions in the first place, which is unlikely).
While "Define this that and the other ad nauseum" is probably overly pedantic, the above do illustrate an ability and propensity both for analysis as well as even perhaps empathy - both, IMO, meaningful skills when developing a product.
"One too many, if you make me work on J2EE."