Am I the only one who thinks that this kind of design question is the new "how many gas stations are there in Manhattan?"
In both cases, the point isn't to get the right answer, but (allegedly) to see how the person thinks. With the estimation question, the trick is to make up some plausible-ish numbers and then multiply and/or add them to get a plausible-ish result. If someone's seen one of those before, they'll nail it for sure. If not, it's a crapshoot. In theory, you're measuring whether or not someone's able to reason well enough to spontaneously estimate something off the top of their head, but comparing the number of people who've seen those question before vs. the number of people who can answer that type of question without having ever seen it before, what you're really filtering for is people who have heard of or seen Fermi questions.
A while back, someone's interviewing experience got posted to HN and they mentioned that they got asked to design a URL shortening service maybe five times. They failed it the first couple of times and got progressively better each time. I doubt the interviewers meant to measure whether or not this person had done enough interviews to catch on to the style of questions that are currently trendy, but that's what they actually measured.
I think the most common objection to this is that the point of these questions is to drill down and figure out how the person really thinks, but in empirical studies on interviewing, they find the actual evaluation of chatty questions like this is heavily influenced by all kinds of biases. Techniques that have more clear-cut evaluation criteria, like work sample tests, and even completely non-technical interviews like behavioral interviews end up being better filters. Even then, the filtering isn't "good" (IIRC, the last time I read one of those studies, work sample tests score the best, and had a correlation of around .5 with an ideal filter), but it's better.
For these kinds of questions, even if you haven't seen the specific question before, there's all sorts of interview gamesmanship that helps tremendously. One thing in the blog post, not spending an hour on requirements gathering, is an example of that. In real life, if you're going to write an application from scratch, spending more than an hour on requirements gathering is perfectly reasonable for a lot of problem domains. But if you're playing the interview game, you know that you have, at most, an hour to sketch out the entire problem, so you have to cut the requirements gathering phase short (but not too short). The post mentions that this gets at real skills. That's true. It does. The problem is that everyone who's seen this kind of question five times is going to be good enough at the interview gamesmanship that they don't need to have good real skills to breeze through the "don't spend too long talking about requirements" sub-filter.
The difference here is that the Monopoly question is asking the candidate to reason about how to solve the problem with a computer program. The author admits that there can be biases, but isn't that the case even with work sample tests? Someone could have a novel solution to a problem, and the reviewer could say "oh, that's too clever to maintain."
The paradox is that the best programmers tend to be the worst at interviewing (because they usually get hired quickly) and the worst programmers gradually get better at interviewing (after doing lots of them).
I think it's a fine question. You are asked to design a program that would do something sufficiently complex and ill specified, which is something you will encounter on the job.
I am not saying that you shouldn't test the coding abilities of your candidate, but it looks to me a good way to see how he/she reasons about the various components of the system and asks for requirements.
I stumbled across it doing a Google search for "interview question elevator design"...I haven't been asked in a technical interview about how I would implement the logic for an elevator controller, but I heard about the question second-hand at a college job fair...and since then, I still think about that question. In fact, I think I've thought about it almost every time I've ever waited for an elevator in New York (that, and the story of the poor woman who was crushed while stepping into an upward-shooting elevator in Midtown a few years ago)...so...4 times a day, times 6 years...and I still haven't come up with enough optimal algorithms to cover all the edge cases and scenarios that I come across :)
The Monopoly question that the OP mentions is good, but I think is a bit problematic since not everyone has played Monopoly. And not everyone is familiar even with the standard rules. But everyone's been in an elevator before, and virtually no one thinks about how complicated an elevator's operation might be: it's one of those incredible, life-changing inventions that you take for granted after the first time you've pressed a button. So I think it's great material for an open-ended technical question.
The single-elevator logic is convoluted enough: If the elevator is going from bottom floor to 10th floor, obviously it should stop if someone on the 5th floor signals their intention to go up. But what if, after the 1st floor door closes and the elevator starts moving, a 2nd-floor person wants to go up? What's the cutoff in last-second button pushes to prevent the elevator from too abruptly stopping?
And of course, the more complicated logic is if someone on the 5th floor press Down while the elevator is heading up. Then you have to implement a queue system. But simple FIFO may not be efficient...if the elevator is moving up, and someone on the 5th floor presses Down, and then, someone on the 7th floor also presses Down...it would seem that the elevator should stop at the 7th floor first, on its way down.
And then things get really complicated with even just one more elevator, particularly with race conditions. If both elevators are moving up, and a 5th floor signal for "Down" is followed by a 7th floor signal for "Down"...ideally, the first elevator to finish its Up job will take the 7th floor...Should it also take the 5th floor Down job? Or let the other elevator handle that? And should that decision change if the 7th floor person signals their intention to go to the 4th floor, whereas the 5th floor person most likely just wants to go to the bottom floor?
Beyond the interruption/queuing logic, there's a lot of interesting discussion about how the elevators should be positioned during idle time. Presumably, they should return to ground floor in the mornings, and stay near the top (or at least midway) during the beginning of lunch, and the end of the day.
What I like about the elevator question is that elevator algorithms are their own science. But a lot of the complexity can be realized just by taking the time to think about all the scenarios you've encountered as an elevator user. And so given the typical technical interview situation, it's a good test of the candidate's reflection, ability to ask questions of the requirements, the ability to apply heuristics for good non-optimal solutions, and the ability to not let edge-cases dominate the thinking.
This seems roughly the same as the monopoly question. Both are seemingly more about enumerating all the edge cases than anything to do with software design or implementation. In 20-30 minutes what do you generally expect to get out of asking this? The benefit to me of elevator over monopoly is that there are inherent issues of safety that dovetail nicely into fault avoidance and recovery and into broader system design and testing.
I lived in a building (6 floors) where the elevator appeared to be always in one of four states:
Door opened, going to floor n, waiting for input inside elevator, waiting for input from anybody.
It only remembers a single floor - where it's going now. Once it goes into the input state, it's a race among everybody who wants to take the elevator to push the buttons.
I was never quite sure whether the 'waiting for input inside elevator' actually existed, because every once in a while the elevator would get summoned by someone else before I could give it a destination when inside.
I guess this is a bit off-topic, it's just to say that even very simple logic can solve the problem, in a possibly infuriating way. ;)
Maybe everyone has been in an elevator, but not everyone has daily experience with them. And not all elevators have lots of contention. I'd say it was several years ago I last waited for an occupied elevator.
"What I like about the elevator question is that elevator algorithms are their own science."
Are they? I think they have analogies in task schedulers. You have things like fairness guarantees (maximizing number of trips made or people moved is not ideal, as that could keep someone waiting for an elevator for weeks), livelock (elevators being stuck on short trips between floors 1 and 2 with people waiting forever on floor 55), elevator/CPU affinity (if we know it is highly likely that the person on floor 14 wants to go to floor 7, we might not let the elevator going past it to floor 3 stop at 14, but schedule the elevator now a few floors higher that we knew will stop at 7 stop there instead, but if we are fairly sure he wants to go to floor 6, where the elevator is going anyways, we might want the first elevator to stop there)
For the "ground floor in the morning" thing, task schedulers can switch behavior, too, for example by detecting the difference between batch-like and GUI-like programs. Extreme example (probably far-fetched): in embedded systems, the task scheduler could even know, say, when Wall Street opens, and make sure the cron task that should run then is paged in when it does.
This question gets a lot simpler once you realize how they were originally implemented and many still are today: state machines. There is no history, no FIFO, the only thing that exists is the now. It's lot easier to answer, "I'm going up and am on the 3rd floor, someone is waiting on the 4th floor, do I start slowing down?" Then it is to answer, "someone pushed up on 4th and someone pushed down on 7th.... etc.".
This is also why many elevators perform very poorly under heavy load. You want to skip floors when the elevator is already full.
Every time I use the lifts in the building I work in I think of this question. There's only 7 floors, yet the software appears to be rubbish.
The idle time question is an interesting/important one, arguably the more important for elevators for relatively low utilisation, where the capacity is there, but you wait for a lift as they are all elsewhere.
In my building it seems that not only do the elevators not have sensible 'resting points' at peak times of the day, it seems that it also doesn't weight requests at all, so you get a massive number of people waiting at the ground floor, as only one lift is sent at a time.
And even if there are multiple elevators waiting at the ground floor, they will only open one at a time, once another lift has left.
I've come across the elevator design question in one of my graduate courses. I remember that one way to solve the elevator problem is to use Petri Nets (from the exotic world of graphs, math-modelling, and reliability engineering).
> What's the cutoff in last-second button pushes to prevent the elevator from too abruptly stopping?
A good starting point might be to have a stopping distance table for whatver the definition of "graceful stopping" is. Then, the elevator stops iff at the time the button is pressed, the stopping distance is less than the distance from the top of the elevator to the top of the floor, i.e. it doesn't need to backtrack.
Oh, God, elevators. One of the hats I wear is "Facility Integration Technical Lead" and integrating our robot with the various elevator systems of buildings is one of the most painful things about deployment.
There's a whole introductory text about this problem space, "The Vertical Transportation Handbook"[1]. If you can get your hands on a copy of it, you'll probably be there for a while. :)
The problem with this question is that if you've asked someone who's played Monopoly a lot, also the old DOS version, and the latest EA versions - he/she might stick to concrete implementations and go in details how it could be done (since it's out there).
I think that would still be valuable though- If you (the interviewer) had done your homework, you should be able to recognize these concrete implementations, and ask the interviewee to elaborate on how they might improve the design. Best case the interviewee starts innovating a bit, and you get a glimpse into his/her design process. Worst case it becomes clear that the interviewee can only follow patterns that he/she is already familiar with.
You would probably also get the opposite problem: I bet some people (particularly international candidates) have little to no experience with Monopoly, and the question would be unfair to them. If the goal is to start with their vague understanding of the game and see what gaps they fill in and what assumptions they make, then it doesn't work very well if you have to explain the game from scratch. It also takes quite a bit of time to explain the game if someone hasn't seen it before, which doesn't feel like a very efficient use of interview time.
I once got a similar question, but with Battleship. I don't really enjoy these high-level design things, they seem silly and irrelevant to me, but I can see how someone with different interests would like it. (That particular place made me an offer, I'm not doing sour grapes.)
So what do you enjoy? "Implement a binary search algorithm for the following list in 10 minutes"? I find CS pop quiz-type questions to be next to worthless in assessing candidates.
+1 so many CS graduate interviews involve these sorts of questions, where, just like many CS exams, the people who can memorise the answers just reel them off as they're usually from a subset of about 20 problems. I know a fair few terrible programmers on £40k+ (GBP) jobs straight out of school, because they memorised this subset of problems.
Are they irrelevant because you don't do design or because you think design is self evident? Working on a large scale project with technical debt will convince you of how important design is.
All it's convinced me of is that most people don't think about decoupling, structure and reuse... The older I get the more I tend to favor functional composition over more typical OO implementations.
If you practice ALL of these questions for at least TWO passes: https://oj.leetcode.com/problems/ , you can get a software engineer role in at least one of the famous tech companies, e.g., facebook, linkedin, google, amazon, microsoft, ...
Nowadays, very very few companies innovate in recruiting.
I've interviewed with most "hot" companies, and I can find most interview questions (>90%) from https://oj.leetcode.com/problems/, exact same question.
I mean, yes, but if you can write passable code in two of [C++, Java, Python] for 170 problems of varying levels of complexity, I don't think it's much of a stretch to say you're a good programmer and could probably get a SE job at a company I've heard of before.
I remember being interviewed in the DC area, where I was asked a question involving subway systems and routes. Being a mid-western lad completely unfamiliar with subways (and mass transit in general), I had a lot of trouble visualizing it. I would imagine that people who rode mass transit on a regular basis wouldn't have as much difficulty. Got a job offer anyway, though.
I generally went with a question my friend told me, "what was the last non-work, non-school program you wrote?". The theory being that a person who has used programming for something personal was probably a better candidate since they saw personal value in their profession.
In a good week I work 50 hours. In a bad week, 80 or 90 (or more). In my spare time I have family commitments, play guitar, draw, paint, hike, ride my mountain bike, and travel. Ask me that question and I'll probably find a way to wind up politely and leave.
I love this question and use it as part of my interview deck. It is only part of the deck, however.
This question is a great positive indicator, but a terrible negative indicator.
That is to say, a person who works on a lot of stuff in their spare time is very likely to be someone I'd like to hire. The reverse isn't true, which is why there are other questions in the deck. I realize that there are great developers who don't have a well populated github profile and dozens of side projects.
If I had a candidate that had trouble with the question, I'd expect them to say so. I'm a pretty reasonable person. I would not expect them to try and exit the interview early with little to no indication why.
What if you have side projects that make money, i.e. are "work"? Is that discounted by "non-work, non-school" or by non-work do you mean "not your day job"?
I imagine many would be hesitant to mention such experience, but I think it would be a great thing to hear about in an interview.
If you're interviewing an parallel universe's patio11-equivalent, and they say, "I have been running FlashCardCreator.com since 2001...", and goes on to explain neat things about what she's learned from it, it sounds like that should be a big green flag. Not only is such a person likely to have lots of cross-domain insight, but they clearly have the mojo to deliver working code.
If you're worried that they might decide that their work-outside-yours is going to cannibalize their time, they probably have an answer for why you don't need to worry much about that. (After all, they __did__ apply.) I would be surprised if you had to worry about them leaving any more than you would any other developer.
It's a completely open-ended question and I word it differently depending on how the interview is going. You can demonstrate passion for technology even if what you talked about was a one-off utility that you wrote for your job.
I want to see if your eyes light up when you talk about it.
I phrase it so it is the "not your day job" version. In the more general sense a better phrasing is "something that wasn't required of you by others". Since I wrapped it in other questions, I got the idea across. If it made money, cool. Just so there was no one saying "you need to do this".
I'm sure there are many here who would agree with you, but let me offer an alternative viewpoint. You said:
> Ask me that question and I'll probably
> find a way to wind up politely and leave.
The implication is that rather than opening a dialog with someone to (a) find out why they think that question is relevant, and (b) offer reasons why you'd be a good fit despite not working with code outside your job, you'd simply leave?
For me, as a potential employer, that would be a good reason not to want to hire you. It makes you sound like you would not be an easy person to work with. It sounds like if someone came to you with an unreasonable software requirement you would simply shut them down without trying to find a way to deliver something that finds a good balance between what they've asked for, and what can be done.
But perhaps that's not the impression you intended. If not, could you expand further?
You have a very good perspective on what an employer would love to hear from someone, regardless of whether they code for fun on the side.
However, as an employee, such questions might be highly correlated with past companies that are looking for people who do not have family commitments, or who are willing to do unpaid extra work at home "for fun". As an interviewee, it makes me wonder, even if only fleetingly, "Will I be discriminated against for not being (young+single)?". (Are we even allowed to ask about marital status or age?)
Even if such a thing was never your intention, the interviewee might have had prior bad experiences with those who ask such a question, and view cutting the interview short as a way to ensure they don't get mistreated later.
In that case, just as an interviewer looks for indications beyond a single question, so should an interviewee. Cutting an interview short because of one indicator seems short-sighted. Surely better
would be to use it as one indicator, and then start looking to see if there are more.
But I understand the point point of view, I'm just trying to give some balance, and point out that the black/white attitude this suggests is sometimes a clear indication that you are not cut out for a particular position.
My initial answer may have been a bit short, but it wasn't actually binary; my response would depend on, among other things, how the question was asked, as well as other indicators. As a mid-career professional, however, that question's a sign that I'm probably not going to fit into the "culture". To be honest, I've been through my share of interviews and I only remember getting a similar question once (it was related to technical blogging/participation on tech sites like StackOverflow or similar). In that case they made it clear that it wasn't a necessity but it was definitely nice to have. I went through the first interview and was asked back for another, but I dropped out of the process for unrelated reasons.
Regardless, if it makes me not want to work for you, and you don't want to hire me because of my reaction to the question, then my likely response would be win/win, right? :)
Maybe, although the other aspects of your response - the reasons why you don't program outside of work - make it clear that I would definitely want to know more, and it's completely plausible that you would be a good fit. But we wouldn't discover that if you simply wound things up and walked out. I find it hard to believe that you are active here and would not be an excellent contributor - the walking out is at odds with that.
Then again, maybe you would not be a good fit. My comment is simply that based on just one question you are actively avoiding finding out.
> Ask me that question and I'll probably find a way to wind up politely and leave.
I'm always confused when I see this sentiment. I only ever see it on HN, and only when someone feels they've been asked a question below their station or that is inherently insulting.
Neither - just an indicator that we may not be a good match and I'll act accordingly. I have a limited amount of time, and when I'm searching I'm pretty ruthless about controlling the number of interviews I take.
Yes, but it's not like you're going to spend that time saved doing an interview with someone else. I mean you've got the time scheduled, I just think it's rude of either party to cut it short because you're not 100% in love with every aspect of the job (or candidate). The interview is a process, go through it and collect your thoughts and assumptions at the end.
So, you've never once did any program for your own benefit? Sent in a patch for some project? Programmed something because you thought it would be neat or scratched an itch? I know plenty of carpenters who put in long hours but have used their skills to build a chair or table for themselves.
I've never used this as an only question, but I have got better results from people who have used their skills to solve some problem for themselves, but that isn't unique to programming.
I used to think this way, but one of the best programmers I work with basically refuses to even touch a computer outside of work. And yet he's extremely good at what he does, and you can figure that out within like 15 seconds of talking to him about a hard problem. As far as heuristics go I don't think this is a good one.
Your friend is an exception. I've met a few, too, but the majority of top-tier programmers I know program outside of work. I'm not talking about "hackathons" and shit like that, or github profiles--usually just personal projects for practical or amusement (often both) purposes.
Heuristics are, unfortunately, quite valuable in hiring scenarios, especially for start-ups.
It's also just much more fun to work with people who love programming, and an easy way to tell (even if it's susceptible to a few false negatives like your friend) is to find out if they program in their own time.
It probably depends on where you work. Most of my colleagues are amazing developers, but very very few of them actually do side projects at home. Most of us wish we had more time, and would like to do this or that as a side project - but there isn't enough time in each day to do everything you want to do.
The only exception to this was a guy who did amazing work on side projects, but the work he did for his paying job suffered significantly...
>It's also just much more fun to work with people who love programming, and an easy way to tell (even if it's susceptible to a few false negatives like your friend) is to find out if they program in their own time.
The thing is though, you can love programming and find that 40 hours a week of it is more than enough. I think what you'd find with this heuristic is that it tends to favor young people. It's a lot easier to be enthusiastic about some side project when you're 20 than when you're 40 and have a lot of responsibilities.
I usually just shorten that to "what was the best program you wrote?". I've found that there were alot of people and fresh students which put alot of quality work in their school / work projects and those are worth hearing about too.
i HATE this question but its becoming more and more commonplace. I work hard, programming 40-70 hours a week, I solve all sorts of in work problems with ideas that I've come up with to make my daily work life easier. I barely touch a computer outside of the office. Does that make me a bad candidate for all programming jobs now?
The question is a litmus test for whether or not you're passionate about programming. That may sound unfair ("Why do I need to be passionate about this? It's just a job...") but the field we work in is constantly changing and the only way anyone has a hope in hell of staying abreast is if they really love it. So if you manage to keep tabs on the industry through your colleagues and you're excited about what you're working on at work, then the personal projects are much less important, but you need to demonstrate that.
It's not. If you have kids there's just no time to start a personal project next to a full time job. Having a side project just tells your employer you have a lot of spare time.
On the other hand, you can code dozens of personal projects every week and be terrible at programming. When I had the time for many personal projects, I was a much less capable developer than I am now. I run circles around that guy now.
So I'm not sure why the person without personal projects needs to demonstrate that they're keeping tabs on the industry any more than someone who does.
Seems like you want to ensure X by testing for Y and mistaking correlation with causation.
> So if you manage to keep tabs on the industry through your colleagues and you're excited about what you're working on at work, then the personal projects are much less important, but you need to demonstrate that.
I disagree. If passion is an important quality to you, then, as the interviewer, you need to ask questions that reveal whether or not the candidate has that quality. It's absurd to ask about personal projects as a proxy for asking "are you passionate about programming" and then to expect the candidate to guess your true intentions and answer accordingly.
I think you hit on a much better question! Just ask "are you passionate about programming?" and then ask them to defend their answer. If they code arduino robots in their spare time or simply love solving people's problems at work they have to convince you either way. That's so much better than trying to gauge their passion with a presumptive question.
The problem with phrasing the question that way is that it's way too open-ended (and likely to make the candidate regurgitate the same BS from their cover letter, which is not the point). An interviewer's job is to make it as easy as possible for the candidate to prove themselves. If they don't have side projects, that's fine--there are other questions that will hopefully provide similar information (ie. "What project have you worked on that was the most satisfying for you personally?"). But the side project question has a high signal/noise ratio.
No. I mean, if that's the case then obviously just walk away but when I say I want passionate people I mean I want people who view programming as a super-power and are still excited by that idea.
that is just funny. Programming is same as other things, some people excel some people suck at it. Nothing super power there. Sure it helps a lot if you do what you like but passion is too strong word for that.
And as other comment, in my experience when they ask for passion it equates free work , meaning more hours and same pay.
You're not the only one. Once upon a time I enjoyed working on personal projects outside of work, but a combination of factors have killed this.
First, I have a family now, and spending time with them is more important to me than any job. (And any company that has a problem with this is not a good fit.)
Second, I've been lucky enough to find jobs that are interesting and intellectually stimulating. My first programming jobs were kinda boring, so I had extra creative energy and learning and building things was an outlet.
Third, owning a home always involves upkeep and fixit projects. These can be fun and are generally easier get my kids to involved in than teaching them to program (going back to #1).
I think the first problem is that you're working more than 40 hours a week already... I'd ask how much of that is coding vs. meetings vs. other time.
I'm in the office 40-45 hours a week typically, I spend 20-30% of my time just thinking about the problems I am solving, or stuff like this, or following github projects, or other technology reading. I probably do about 4-5 hours of actual programming a day at work. My output during that time tends to match or exceed most of my peers in both quality and quantity.
The problem is getting people to understand more hours isn't the same as more productive output. Spending a couple hours thinking through a problem (or even prototyping) will usually save you more time down the road. This is true from simple data modeling to working through how to fit a system together.
As to the original question.. it is far from the only indicator of a person's own skill and drive, but it is an indicator. Especially in more junior level candidates. There's also the fact that the technology in use changes rapidly... and do you take the time to learn it... either out of your in-office day, or outside.
Programming is like any other job, in that programming is a tool that you use. I like this question, as it determines what the candidate uses the tools of their trade for. The answer may be that you do not program in your free time, and that may be just what the interviewer is looking for: someone that knows programing is a tool for making money, nothing more. Conversely, they may not like the answer, as it does not show the person thinks that programming is a tool to be used in the same way as a lawnmower; to more easily achieve an objective. In the end, the question, response, and counter response reveal a lot about the people and the situation. I'll add, in the end, the only thing that you, as an interviewer, should be concerned with is: how will this person make me money? Programming ethos, passion, and skill all add to this, but culture fit, propensity to post to HN at work, and family constraints all fit into this seemingly simple question of how to make dollars.
In my personal experience I have found that the people that put stock in this sort of question believe in the "coder" mystique and that someone eating and breathing coding is somehow more competent than someone that treats computer programming as a professional career.
It is not uncommon for people to feel this about a number of professions such as chefs, bakers, graphic artists, etc.
However, in my experience I prefer to judge an applicant on professional credentials rather than their free time because while it may tell me quite a bit about them it tells me little about their capabilities that I couldn't find out in better ways.
I beg to differ, Monopoly can be done using 'simplistic' object design. An array is the monopoly map, a user is an object and the bank is a singleton. Done.
supsep, you are a great candidate but unfortunately you don't seem that great of a fit for our Abstract Enterprise Java culture. I am sure there are a lot of other companies who adhere to your mindset!
I'm not sure why this got down-voted, but I thought the same thing while reading the article. If you store the game state as a giant hash-map (as I would in Clojure), then problems of "is-a" and "has-a" don't really exist. Especially considering that almost all squares are singletons anyway. So the game state can look as simple as:
I've actually heard, "you can't afford me" in an interview before. (It was not technically an interview, but a meeting between my boss and someone he wanted to recruit.)
I think my boss (CEO of a mid-sized public company) took it as a challenge and gave him a very large offer -- which was politely refused.
I don't have been to coding interviews recently but is this what it is like these days? How long do these questions generally take to complete? When I hear white boards, UML diagrams and OO-design I am thinking about investing at least a few hours.
Back in my day there were no interviews like these. You just dropped your CV, had a chat with the guy and if it clicked you got a contract with a month on (paid) trial. When I hear things like this I am thinking about all the people who are investing an (unpaid) day (sometimes even recurring interviews, what!) solving questionnaires with no real context during the job.
Paid trials also work (for example, I was a contract-to-hire at my current job).
But many people don't want to leave the security of their current job for a temporary contract, so you would need to spend more time vetting those candidates.
New Zealand has this law, in which new jobs have a 3 month 'trial period'. (unless specified otherwise in the contract). So if someone's hired and it's clear they cannot code or interact well with the team they can be let go as long as it's within that 3 month period.
Personally... I don't like it. But I do understand the logic, and how it helps businesses from making expensive mistakes in regards to hiring.
I just spent a few months interviewing developers in all career stages. I ask a problem in a similar vein that's designed to get both of us up in front of the whiteboard working out the design of a problem together. In those 20+ interviews I've seen all kinds of answers to the problem, some bad, some ok, and some pretty good. I can learn a ton about you as a developer with this question. And almost without fail the candidate ends the interview with something along the lines of "that was a fun problem to work on".
I combine that problem with developng a simple end to end feature together and the combination of that with a high level design problem turns out to be a really good indicator of how someone actually performs on the job.
I would much rather hear about their past projects and have them clearly convey design decisions that they made before then have them work on a "on the fly" question. Everybody has off days, not many people have whole off years or careers. Just because I can't design Monopoly in one hour on a certain given day doesn't mean I do not have the relevant experience in my past, whether it be professional or as an undergrad.
It's not as though this is the only question by which the interviewee will be appraised. This is one more tool in the kit alongside asking about previous projects, interesting classes, career goals, overcoming tough situations, etc.
Whether or not questions like these are effective, the point of them is to engage with someone in a way that illustrates how they tackle complex, loosely defined issues: the majority of engineering problems in a nutshell. In most modern tech companies, good communication and not being afraid to ask questions are absolutely essential. Someone who will take an incomplete list of requirements and draw up an entirely incomplete product without questions or discussion is as much a liability as someone who takes the requirements and does nothing, paralyzed by indecision and not knowing what to do.
And oftentimes, candidates aren't perfect but you're going to hire them anyway. Going through this process can illustrate how you would expect to interact with the candidate and where the initial rough spots will be as the candidate begins working with the team.
From another point of view, candidates are usually more than prepared to talk about anything on their resume. In some cases, this is equivalent to listening to someone give a prepared speech. Asking them free-form questions like the above can be a good way to understand how the candidate reacts in situations where they aren't fully prepared. For some positions this is irrelevant, but for others it's incredibly important to have some measure of poise and thoughtfulness when engaging with someone who doesn't necessarily agree with everything you say and is questioning your thought process.
In short, these kinds of questions aren't primarily testing your design and programming abilities, but your ability to communicate and reason with another person in a technical context.
You missed my point. People are prepared to talk about their resume, but when you dig in to their past experience and specifics that is where you get to the interesting discussions.
Case and point:
Tell me about a design pattern you used on X project?
A singleton? Ok, why did you use that?
and a long discussion about testability and architecture ensues with questions from both sides. That is WAY more like the communication that happens on the job than a contrived problem which in many cases is disconnected from real life constraints.
So I tried a new approach inspired by this kind of interview question. In a nutshell: it aims to simulate the interactions you'll have with the candidate when they receive their first project, without the time burn of a full pair programming day.
1) determine the system the candidate will design. It should be an internal system or an internal oriented problem, not an external thing like Monopoly or URL shortening services.
2) stack your interview team with a mixture of people who can contribute different thoughts to this problem. Engineering, UI, marketing, management, etc.
3) first interviewer of the day introduces the topic and helps the candidate understand some of the internal lingo and bootstrapping questions they'll have right off the bat. The first interviewer is pretty crucial because the candidate has to be prepared to write things down and carry information across each interview.
4) all subsequent interviewers come in and ask two questions:
* What problem were you given to solve?
* I'm an expert in technology / business area X. How can I help?
By the end of the day when they're having their likely optional interviews with more senior people, they should look disheveled but pretty happy, because now they're pitching to senior management an idea they collaborated on with a wide variety of team members.
They should have a whiteboard full of stuff, stacks of paper, whatever. You should get a good idea of how they organize themselves under pressure.
The interview team should have a good feel for how the candidate leveraged them for answers. The interviewers should have done a lot of talking! There should also be clear evidence of the candidate getting better towards the end.
At the conclusion they're tasked with taking a step back to the higher level to explain the idea to management - regardless as to whether management is technical or not, they should be expected to walk away understanding the big picture. Having non-technical "business" people contribute to the interview is critical, since they help the candidate understand some of the human requirements.
I could go on and on, I'm pretty excited about how this technique has worked so far. The key element is deliberately having the candidate carry knowledge between interview teams. I'm sure I'm not the first person to think of this :)
What company do you work for, out of curiosity? I absolutely love this idea; I would be extremely impressed if I showed up for an interview and this happened.
Thank you, that sounds like a brilliant technique!
One question: Do you let candidates know ahead of time what the interview process will consist of? That sounds like an important step that many companies leave out - whatever interview process they use.
I think if I knew before arriving that this how it would work, I'd be a lot more excited about the interview and better prepared too.
Yes, the candidate should know in advance if there is a particularly structured interview technique being used, for sure.
Because it also requires a lot of mental commitment from the interview team as we'll as the candidate, I'd recommend using this technique as the final hurdle. I don't think you want to phone screen someone, then just toss them into this process. You need to have them cross some basic technical gate (like FizzBuzz, a take-home coding project, whatever), and have them meet a handful of people first, just to make sure they pass those early smoke tests.
In fact I think this technique works best when you want to seal the deal with a very strong candidate who is entertaining several options. Interviews should always be a two-way process, and if a candidate walks away from your interviews thinking "I had real trouble getting my point across to those people, I would never work there" then that represents success for the interview process.
The strongest candidates tend to be the ones least influenced by money and more interested in the problems, the people they will be working with, the culture and the environment of the organization. So having your interview process structured around revealing that gives you the best possible chance of matching your team with people excited to be there.
Conversely if you're a candidate, even if the company you're applying with doesn't seem to have thought too deeply about this stuff, don't despair. Hiring is one of the hardest things to do and also tends to be done by the seat of the pants (Amazon seems like a notable exception) You can impose your own structure by asking the right questions and making special requests of the recruiter. Tell them you want a chance to write code with someone! Or that you want to mock up a UI that solves a problem they have. Or you want to see what their strongest engineer thinks of your design for this service you've put together.
It doesn't matter if you're desperate for the job, or whether you wish they'd stop calling you ... this kind of stuff makes you stand out in the crowd as proactive and motivated by the right reasons.
This is probably doable if you're interviewing 1 candidate a week (or one in 2 weeks). Most tech companies run through several candidates a week; and finding engineers/designers/managers who will play along with your question will be very difficult. I've been to interviews where they didn't know I was coming that day; where interviewers failed to turn up; where wrong interviewers showed up; etc. etc.
I know where you're coming from there. It depends on the company. I think hiring is the most important thing a company does, so it's really critical to get everyone wanting to participate. I'd suggest that most push back folks get is because people generally dislike having to interview - they are tired of meeting lots of B grade candidates and even a few hours a week feels like a drag.
Having said that, there are ways to offload. There's a biotech startup (I'd have to dig for their name, I remember they did all their coding in mathematica) that described their hiring process to me. Hopefully I'm not messing up the details too badly.
I recall one guy who went through it said it was way more stressful than his final year university exams. They tested candidates, all candidates, on physics, math, chemistry, biology, computer science, and more. Candidates were given projects in each area, and a week to complete all of them.
It was deliberately structured so that there was no possible way that anyone could humanly complete the entire set of tests, much less while also working a full time job. This was deliberate, so that they could observe how candidates chose to prioritize. They also wanted to give people more than they could do to stimulate a "by any means necessary" approach they knew would be required in the actual job.
Finally, it also caused a lot of people to say "erm, no thanks. That's a ridiculous amount of work", which they rationalized as being a good test for candidates who saw a mountain of work and then baulked.
Not sure if I'd use a technique like this myself, but I found the idea interesting!
Love how every time a post concerning interviews pops-up on hackernews everyone gets defensive. Like all things I think the nature of the interview should reflect the role the developer will have when hired. If the developer won't be in charge of developing an entirely new system or project on his own it seems overarching to ask this type of question. At the end of the day the core attributes that matter are intelligence, ability to learn, experience and motivation. In this case, this question really only tests experience.
I ask a JavaScript developer to debug a small problem and then come up with functional approach to solve it. Makes for good step by step exploration. Each step can be expressed in single English sentence and then coded directly into JavaScript. Much more precise than designing OO hierarchy.
Designing a system architecture (Monopoly question) and solving a practical coding problem are different skillsets, so ideally you would ask the interviewee both types of questions.
99 comments
[ 2.5 ms ] story [ 161 ms ] threadIn both cases, the point isn't to get the right answer, but (allegedly) to see how the person thinks. With the estimation question, the trick is to make up some plausible-ish numbers and then multiply and/or add them to get a plausible-ish result. If someone's seen one of those before, they'll nail it for sure. If not, it's a crapshoot. In theory, you're measuring whether or not someone's able to reason well enough to spontaneously estimate something off the top of their head, but comparing the number of people who've seen those question before vs. the number of people who can answer that type of question without having ever seen it before, what you're really filtering for is people who have heard of or seen Fermi questions.
A while back, someone's interviewing experience got posted to HN and they mentioned that they got asked to design a URL shortening service maybe five times. They failed it the first couple of times and got progressively better each time. I doubt the interviewers meant to measure whether or not this person had done enough interviews to catch on to the style of questions that are currently trendy, but that's what they actually measured.
I think the most common objection to this is that the point of these questions is to drill down and figure out how the person really thinks, but in empirical studies on interviewing, they find the actual evaluation of chatty questions like this is heavily influenced by all kinds of biases. Techniques that have more clear-cut evaluation criteria, like work sample tests, and even completely non-technical interviews like behavioral interviews end up being better filters. Even then, the filtering isn't "good" (IIRC, the last time I read one of those studies, work sample tests score the best, and had a correlation of around .5 with an ideal filter), but it's better.
For these kinds of questions, even if you haven't seen the specific question before, there's all sorts of interview gamesmanship that helps tremendously. One thing in the blog post, not spending an hour on requirements gathering, is an example of that. In real life, if you're going to write an application from scratch, spending more than an hour on requirements gathering is perfectly reasonable for a lot of problem domains. But if you're playing the interview game, you know that you have, at most, an hour to sketch out the entire problem, so you have to cut the requirements gathering phase short (but not too short). The post mentions that this gets at real skills. That's true. It does. The problem is that everyone who's seen this kind of question five times is going to be good enough at the interview gamesmanship that they don't need to have good real skills to breeze through the "don't spend too long talking about requirements" sub-filter.
I am not saying that you shouldn't test the coding abilities of your candidate, but it looks to me a good way to see how he/she reasons about the various components of the system and asks for requirements.
I stumbled across it doing a Google search for "interview question elevator design"...I haven't been asked in a technical interview about how I would implement the logic for an elevator controller, but I heard about the question second-hand at a college job fair...and since then, I still think about that question. In fact, I think I've thought about it almost every time I've ever waited for an elevator in New York (that, and the story of the poor woman who was crushed while stepping into an upward-shooting elevator in Midtown a few years ago)...so...4 times a day, times 6 years...and I still haven't come up with enough optimal algorithms to cover all the edge cases and scenarios that I come across :)
The Monopoly question that the OP mentions is good, but I think is a bit problematic since not everyone has played Monopoly. And not everyone is familiar even with the standard rules. But everyone's been in an elevator before, and virtually no one thinks about how complicated an elevator's operation might be: it's one of those incredible, life-changing inventions that you take for granted after the first time you've pressed a button. So I think it's great material for an open-ended technical question.
The single-elevator logic is convoluted enough: If the elevator is going from bottom floor to 10th floor, obviously it should stop if someone on the 5th floor signals their intention to go up. But what if, after the 1st floor door closes and the elevator starts moving, a 2nd-floor person wants to go up? What's the cutoff in last-second button pushes to prevent the elevator from too abruptly stopping?
And of course, the more complicated logic is if someone on the 5th floor press Down while the elevator is heading up. Then you have to implement a queue system. But simple FIFO may not be efficient...if the elevator is moving up, and someone on the 5th floor presses Down, and then, someone on the 7th floor also presses Down...it would seem that the elevator should stop at the 7th floor first, on its way down.
And then things get really complicated with even just one more elevator, particularly with race conditions. If both elevators are moving up, and a 5th floor signal for "Down" is followed by a 7th floor signal for "Down"...ideally, the first elevator to finish its Up job will take the 7th floor...Should it also take the 5th floor Down job? Or let the other elevator handle that? And should that decision change if the 7th floor person signals their intention to go to the 4th floor, whereas the 5th floor person most likely just wants to go to the bottom floor?
Beyond the interruption/queuing logic, there's a lot of interesting discussion about how the elevators should be positioned during idle time. Presumably, they should return to ground floor in the mornings, and stay near the top (or at least midway) during the beginning of lunch, and the end of the day.
What I like about the elevator question is that elevator algorithms are their own science. But a lot of the complexity can be realized just by taking the time to think about all the scenarios you've encountered as an elevator user. And so given the typical technical interview situation, it's a good test of the candidate's reflection, ability to ask questions of the requirements, the ability to apply heuristics for good non-optimal solutions, and the ability to not let edge-cases dominate the thinking.
I was never quite sure whether the 'waiting for input inside elevator' actually existed, because every once in a while the elevator would get summoned by someone else before I could give it a destination when inside.
I guess this is a bit off-topic, it's just to say that even very simple logic can solve the problem, in a possibly infuriating way. ;)
Are they? I think they have analogies in task schedulers. You have things like fairness guarantees (maximizing number of trips made or people moved is not ideal, as that could keep someone waiting for an elevator for weeks), livelock (elevators being stuck on short trips between floors 1 and 2 with people waiting forever on floor 55), elevator/CPU affinity (if we know it is highly likely that the person on floor 14 wants to go to floor 7, we might not let the elevator going past it to floor 3 stop at 14, but schedule the elevator now a few floors higher that we knew will stop at 7 stop there instead, but if we are fairly sure he wants to go to floor 6, where the elevator is going anyways, we might want the first elevator to stop there)
For the "ground floor in the morning" thing, task schedulers can switch behavior, too, for example by detecting the difference between batch-like and GUI-like programs. Extreme example (probably far-fetched): in embedded systems, the task scheduler could even know, say, when Wall Street opens, and make sure the cron task that should run then is paged in when it does.
This is also why many elevators perform very poorly under heavy load. You want to skip floors when the elevator is already full.
The idle time question is an interesting/important one, arguably the more important for elevators for relatively low utilisation, where the capacity is there, but you wait for a lift as they are all elsewhere.
In my building it seems that not only do the elevators not have sensible 'resting points' at peak times of the day, it seems that it also doesn't weight requests at all, so you get a massive number of people waiting at the ground floor, as only one lift is sent at a time.
And even if there are multiple elevators waiting at the ground floor, they will only open one at a time, once another lift has left.
http://en.wikipedia.org/wiki/Petri_net Simulation: https://www.youtube.com/watch?v=59jW9dngt_c
A good starting point might be to have a stopping distance table for whatver the definition of "graceful stopping" is. Then, the elevator stops iff at the time the button is pressed, the stopping distance is less than the distance from the top of the elevator to the top of the floor, i.e. it doesn't need to backtrack.
[1] http://www.worldcat.org/title/vertical-transportation-handbo...
http://www.abandonia.com/en/games/895/Monopoly+Deluxe.html
http://www.textmodegames.com/download/monopoly.html
And here complete list:
http://en.wikipedia.org/wiki/Monopoly_(video_games)
http://www.mobygames.com/search/quick?q=monopoly
Nowadays, very very few companies innovate in recruiting.
I've interviewed with most "hot" companies, and I can find most interview questions (>90%) from https://oj.leetcode.com/problems/, exact same question.
This question is a great positive indicator, but a terrible negative indicator.
That is to say, a person who works on a lot of stuff in their spare time is very likely to be someone I'd like to hire. The reverse isn't true, which is why there are other questions in the deck. I realize that there are great developers who don't have a well populated github profile and dozens of side projects.
If I had a candidate that had trouble with the question, I'd expect them to say so. I'm a pretty reasonable person. I would not expect them to try and exit the interview early with little to no indication why.
If you're interviewing an parallel universe's patio11-equivalent, and they say, "I have been running FlashCardCreator.com since 2001...", and goes on to explain neat things about what she's learned from it, it sounds like that should be a big green flag. Not only is such a person likely to have lots of cross-domain insight, but they clearly have the mojo to deliver working code.
If you're worried that they might decide that their work-outside-yours is going to cannibalize their time, they probably have an answer for why you don't need to worry much about that. (After all, they __did__ apply.) I would be surprised if you had to worry about them leaving any more than you would any other developer.
I want to see if your eyes light up when you talk about it.
I couldn't imagine working that often and not enjoying it.
For me, as a potential employer, that would be a good reason not to want to hire you. It makes you sound like you would not be an easy person to work with. It sounds like if someone came to you with an unreasonable software requirement you would simply shut them down without trying to find a way to deliver something that finds a good balance between what they've asked for, and what can be done.
But perhaps that's not the impression you intended. If not, could you expand further?
However, as an employee, such questions might be highly correlated with past companies that are looking for people who do not have family commitments, or who are willing to do unpaid extra work at home "for fun". As an interviewee, it makes me wonder, even if only fleetingly, "Will I be discriminated against for not being (young+single)?". (Are we even allowed to ask about marital status or age?)
Even if such a thing was never your intention, the interviewee might have had prior bad experiences with those who ask such a question, and view cutting the interview short as a way to ensure they don't get mistreated later.
But I understand the point point of view, I'm just trying to give some balance, and point out that the black/white attitude this suggests is sometimes a clear indication that you are not cut out for a particular position.
Regardless, if it makes me not want to work for you, and you don't want to hire me because of my reaction to the question, then my likely response would be win/win, right? :)
Then again, maybe you would not be a good fit. My comment is simply that based on just one question you are actively avoiding finding out.
I'm always confused when I see this sentiment. I only ever see it on HN, and only when someone feels they've been asked a question below their station or that is inherently insulting.
I've never used this as an only question, but I have got better results from people who have used their skills to solve some problem for themselves, but that isn't unique to programming.
Heuristics are, unfortunately, quite valuable in hiring scenarios, especially for start-ups.
It's also just much more fun to work with people who love programming, and an easy way to tell (even if it's susceptible to a few false negatives like your friend) is to find out if they program in their own time.
The only exception to this was a guy who did amazing work on side projects, but the work he did for his paying job suffered significantly...
The thing is though, you can love programming and find that 40 hours a week of it is more than enough. I think what you'd find with this heuristic is that it tends to favor young people. It's a lot easier to be enthusiastic about some side project when you're 20 than when you're 40 and have a lot of responsibilities.
So I'm not sure why the person without personal projects needs to demonstrate that they're keeping tabs on the industry any more than someone who does.
Seems like you want to ensure X by testing for Y and mistaking correlation with causation.
I disagree. If passion is an important quality to you, then, as the interviewer, you need to ask questions that reveal whether or not the candidate has that quality. It's absurd to ask about personal projects as a proxy for asking "are you passionate about programming" and then to expect the candidate to guess your true intentions and answer accordingly.
First, I have a family now, and spending time with them is more important to me than any job. (And any company that has a problem with this is not a good fit.)
Second, I've been lucky enough to find jobs that are interesting and intellectually stimulating. My first programming jobs were kinda boring, so I had extra creative energy and learning and building things was an outlet.
Third, owning a home always involves upkeep and fixit projects. These can be fun and are generally easier get my kids to involved in than teaching them to program (going back to #1).
I'm in the office 40-45 hours a week typically, I spend 20-30% of my time just thinking about the problems I am solving, or stuff like this, or following github projects, or other technology reading. I probably do about 4-5 hours of actual programming a day at work. My output during that time tends to match or exceed most of my peers in both quality and quantity.
The problem is getting people to understand more hours isn't the same as more productive output. Spending a couple hours thinking through a problem (or even prototyping) will usually save you more time down the road. This is true from simple data modeling to working through how to fit a system together.
As to the original question.. it is far from the only indicator of a person's own skill and drive, but it is an indicator. Especially in more junior level candidates. There's also the fact that the technology in use changes rapidly... and do you take the time to learn it... either out of your in-office day, or outside.
It is not uncommon for people to feel this about a number of professions such as chefs, bakers, graphic artists, etc.
However, in my experience I prefer to judge an applicant on professional credentials rather than their free time because while it may tell me quite a bit about them it tells me little about their capabilities that I couldn't find out in better ways.
{:unowned-properties #{:baltic} :players [{:owns #{:atlantic}}] :property-states {:atlantic {:hotels 0 :houses 2}}}
Etc....data driven design...wins every time.
True-est, and it's the perfect filter.
I think my boss (CEO of a mid-sized public company) took it as a challenge and gave him a very large offer -- which was politely refused.
Back in my day there were no interviews like these. You just dropped your CV, had a chat with the guy and if it clicked you got a contract with a month on (paid) trial. When I hear things like this I am thinking about all the people who are investing an (unpaid) day (sometimes even recurring interviews, what!) solving questionnaires with no real context during the job.
Is it really this bad?
But many people don't want to leave the security of their current job for a temporary contract, so you would need to spend more time vetting those candidates.
Personally... I don't like it. But I do understand the logic, and how it helps businesses from making expensive mistakes in regards to hiring.
I combine that problem with developng a simple end to end feature together and the combination of that with a high level design problem turns out to be a really good indicator of how someone actually performs on the job.
Whether or not questions like these are effective, the point of them is to engage with someone in a way that illustrates how they tackle complex, loosely defined issues: the majority of engineering problems in a nutshell. In most modern tech companies, good communication and not being afraid to ask questions are absolutely essential. Someone who will take an incomplete list of requirements and draw up an entirely incomplete product without questions or discussion is as much a liability as someone who takes the requirements and does nothing, paralyzed by indecision and not knowing what to do. And oftentimes, candidates aren't perfect but you're going to hire them anyway. Going through this process can illustrate how you would expect to interact with the candidate and where the initial rough spots will be as the candidate begins working with the team.
From another point of view, candidates are usually more than prepared to talk about anything on their resume. In some cases, this is equivalent to listening to someone give a prepared speech. Asking them free-form questions like the above can be a good way to understand how the candidate reacts in situations where they aren't fully prepared. For some positions this is irrelevant, but for others it's incredibly important to have some measure of poise and thoughtfulness when engaging with someone who doesn't necessarily agree with everything you say and is questioning your thought process.
In short, these kinds of questions aren't primarily testing your design and programming abilities, but your ability to communicate and reason with another person in a technical context.
Case and point:
Tell me about a design pattern you used on X project?
A singleton? Ok, why did you use that?
and a long discussion about testability and architecture ensues with questions from both sides. That is WAY more like the communication that happens on the job than a contrived problem which in many cases is disconnected from real life constraints.
actual blog: http://www.gamasutra.com/blogs/MarkMennell/20140929/226628/M...
1) determine the system the candidate will design. It should be an internal system or an internal oriented problem, not an external thing like Monopoly or URL shortening services.
2) stack your interview team with a mixture of people who can contribute different thoughts to this problem. Engineering, UI, marketing, management, etc.
3) first interviewer of the day introduces the topic and helps the candidate understand some of the internal lingo and bootstrapping questions they'll have right off the bat. The first interviewer is pretty crucial because the candidate has to be prepared to write things down and carry information across each interview.
4) all subsequent interviewers come in and ask two questions:
* What problem were you given to solve?
* I'm an expert in technology / business area X. How can I help?
By the end of the day when they're having their likely optional interviews with more senior people, they should look disheveled but pretty happy, because now they're pitching to senior management an idea they collaborated on with a wide variety of team members.
They should have a whiteboard full of stuff, stacks of paper, whatever. You should get a good idea of how they organize themselves under pressure.
The interview team should have a good feel for how the candidate leveraged them for answers. The interviewers should have done a lot of talking! There should also be clear evidence of the candidate getting better towards the end.
At the conclusion they're tasked with taking a step back to the higher level to explain the idea to management - regardless as to whether management is technical or not, they should be expected to walk away understanding the big picture. Having non-technical "business" people contribute to the interview is critical, since they help the candidate understand some of the human requirements.
I could go on and on, I'm pretty excited about how this technique has worked so far. The key element is deliberately having the candidate carry knowledge between interview teams. I'm sure I'm not the first person to think of this :)
EDIT: styling
One question: Do you let candidates know ahead of time what the interview process will consist of? That sounds like an important step that many companies leave out - whatever interview process they use.
I think if I knew before arriving that this how it would work, I'd be a lot more excited about the interview and better prepared too.
Because it also requires a lot of mental commitment from the interview team as we'll as the candidate, I'd recommend using this technique as the final hurdle. I don't think you want to phone screen someone, then just toss them into this process. You need to have them cross some basic technical gate (like FizzBuzz, a take-home coding project, whatever), and have them meet a handful of people first, just to make sure they pass those early smoke tests.
In fact I think this technique works best when you want to seal the deal with a very strong candidate who is entertaining several options. Interviews should always be a two-way process, and if a candidate walks away from your interviews thinking "I had real trouble getting my point across to those people, I would never work there" then that represents success for the interview process.
The strongest candidates tend to be the ones least influenced by money and more interested in the problems, the people they will be working with, the culture and the environment of the organization. So having your interview process structured around revealing that gives you the best possible chance of matching your team with people excited to be there.
Conversely if you're a candidate, even if the company you're applying with doesn't seem to have thought too deeply about this stuff, don't despair. Hiring is one of the hardest things to do and also tends to be done by the seat of the pants (Amazon seems like a notable exception) You can impose your own structure by asking the right questions and making special requests of the recruiter. Tell them you want a chance to write code with someone! Or that you want to mock up a UI that solves a problem they have. Or you want to see what their strongest engineer thinks of your design for this service you've put together.
It doesn't matter if you're desperate for the job, or whether you wish they'd stop calling you ... this kind of stuff makes you stand out in the crowd as proactive and motivated by the right reasons.
Basically, interviewing is a clusterfuck. :-)
Having said that, there are ways to offload. There's a biotech startup (I'd have to dig for their name, I remember they did all their coding in mathematica) that described their hiring process to me. Hopefully I'm not messing up the details too badly.
I recall one guy who went through it said it was way more stressful than his final year university exams. They tested candidates, all candidates, on physics, math, chemistry, biology, computer science, and more. Candidates were given projects in each area, and a week to complete all of them.
It was deliberately structured so that there was no possible way that anyone could humanly complete the entire set of tests, much less while also working a full time job. This was deliberate, so that they could observe how candidates chose to prioritize. They also wanted to give people more than they could do to stimulate a "by any means necessary" approach they knew would be required in the actual job.
Finally, it also caused a lot of people to say "erm, no thanks. That's a ridiculous amount of work", which they rationalized as being a good test for candidates who saw a mountain of work and then baulked.
Not sure if I'd use a technique like this myself, but I found the idea interesting!
http://bahmutov.calepin.co/functional-javascript-interview-q...