Ask HN: How do you identify a great software developer in an interview?

Try to give a (not too outlandish) problem that they cannot prepare for. People memorize algorithms or can adapt their mathematical knowledge, but when given an obscure or abstract scenario, you can get a good idea of how their mind works. Encourage them to take their time and think out loud so you can understand their process.

I would add, work together with them on a problem that is completely out of their comfort zone and/or background. See what kind of questions they ask, how fast they recognize a dead-end, and whether they learn from what you are trying to explain. If you have a hard time teaching them something new in an interview, probably it's not a good fit.

The answer is obvious and right before your eyes: you can't.

This problem has been examined again and again. The only way to find people with sufficient skill in any field is to test them (formal graded test, not idiosyncratic problems).

No one has ever demonstrated a reliable technique for selecting "great" software developers (or great scientists, or great engineers, &etc).

I find that a 15-minute chat about a related topic that's not on their CV (i.e. something tangential to their "career") is a great way to sniff out hackers. Being a hacker myself, I have a very sensitive bullshit-o-meter and fairly varied interests, and I can quickly figure out whether that person is a "career programmer" who will end up disappointing, or someone who's got real passion for the job.

Add to that an intelligence filter (well, I have an affinity for smart people too), and maybe an informal chat about some past projects that they've worked on if still in doubt, and I think I can recognise good programmers even on a text chat system.

In fact, I spotted our last hire on IRC and was about 80% sure that he was someone we wanted to hire, after just 30 minutes of chatting.

Someone already posted the link to my article on how to recognise a good programmer - that breaks down the process I follow (though it's more internalised by now).

Sit down for a week with each candidate, perhaps with all candidates, in a hackathon. Solving a real world problem. Then you can see who is worth what, their strong points, their weaknesses. It costs you time, but it's --in my experience-- worth every minute spent.

I interviewed someone recently, someone much older than me, and it was a strange experience. We talked through his career, early on he had done some really interesting things, attacked some problems from unexpected angles and won big. But as we progressed through his career, it seemed to fade. Gradually all that early fire was crushed out of him and he became a 9-5er. So I thought, do I hire him and maybe try to turn him back into what he was? Or is it too late? In the end he was rejected by the other interviewer anyway, but it was a tough decision, and I could have forced it if I really believed in him. But it just goes to show, an interview can open up more questions about a person than it answers, and there's just no way to tell without actually working with them for some time.

I would probably give them a problem that involves processing/handling a large copious amount of data and compare their strategies.

Usually the great programmers come up with clean, elegant, efficient and scalable solutions.

Set up a competition geeks would be interested in, then try to hire the bests (for your definition of bests).

I'd give a real world problem that is quite large, for example: Develop a school management system which synchronises across several computers.

I'd ask for 2-3 possible ways of designing the architecture. Then just drill down to the details, ask about the technology they would use, how they would go about implementing the algorithmic parts of things, the networking part of things, etc.

You'll get a good sense of how the person works on a high level as well as on a more detailed level. You'd also find out if the person is current on available technologies.

Algorithms are pointless to know, very rarely does one use that knowledge as a real life programmer. I know how quicksort works, but I've hardly ever _needed_ to know, and I've NEVER had to implement it.

I ask questions about two things that seem great predictors. The first is interest in music. I'm not sure why but people who can read music and play any instrument well tend to make great programmers. I have no explanation for this. The second question is what the highest level math course they had was and what they found hard about it. This one reflects a personal bias. Knowing the answer for myself gives me a rough measure of how hard the person can think and their ability to handle abstractions.

Obvious answer: You probably can't identify the good ones without actually testing them.

I worked at an oil services company where I fairly regularly interviewed candidates. I was 180 degrees wrong on every interview I did. The bad ones turned out good, and the good ones turned out bad.

I would not, knowing what I know now, bother with anything else except a chat with some managers, and a some pseudo-code testing of the kinds of problems I'd expect a computer science grad to be able to solve.

This opinion is backed up by a basic training course for some new hires I gave recently, where my ranking basically flipped around from my first impressions, and the guys with actual real ability didn't really float to the top until after about 10 or 12 hours of training / evaluation. However, this could be shortened easily into an interview format.

There's a lot of people who hide their light under a bushel, and a lot of people who talk a good game, but are basically just warm bodies.

Despite how hard you try, you can't get this 100% right. This is how I approached hiring:

1. Weed out the truly incompetent. A very simple programming exercise is sufficient. Keep in mind that the goal of the exercise is not to identify a great programmer, but rather to identify those who can't program at all.

2. Set expectations. During the interviewing process, try your best to work out what you would expect from the individual if hired. Would you expect them to churn out a ton of quality code? Do you expect them to be great at determining business needs and writing less code, but code with a high impact? Is this going to be your go to guy for technology X? Do you expect your junior person to learn Y and Z so they can start contributing after N months?

3. Communicate your expectations clearly. Make sure the candidate knows what's expected of him or her. Towards the end of the interview discuss how they feel about those expectations. Ask how they plan on going about meeting them.

4. After hiring: If you get it right great, if not, let the person go sooner than later. You do yourself and the individual a great disservice if you try to turn them into something they are not. Don't try to turn a lump of coal into a diamond. It's a trap. It doesn't work. They will be unhappy and resentful if you try. Nobody likes to be fired (or do the firing) but you have to tell yourself it's best that they get back out there and find the job that they can be happy and successful at.

I think its obvious someone fresh out of college will have "excellent algorithmic thinking" since that's how they spent the last 3/4 years. So in that case you should be spending more time during interviews looking at the more likely areas of weakness, which I'd sum up as "real-world experience." If you're interviewing experienced candidates, then you can generally hazard a guess that their areas of weakness are elsewhere (lack of passion, knowledge and experience that hasn't kept up with the state of the art, etc).

Interviews are all about probing. They shouldn't be a set of rote questions that you apply to every candidate regardless of background. You should make a guess at what the candidate's weaknesses are, and then push on them, push on them some more. This will also help you learn how well they deal with adversity.

Of course, before this, testing is a good way to determine what a candidate's strengths and weaknesses are.

As a total aside, I've found that US Customs and Border Patrol (the ones who do "secondary inspections") are about the best interviewers out there. They probe so much you end up feeling a little violated.

Setup a dozen seemingly random questions that will help you know what personality the person has.

I've seen people do great, well-commented code at interview, and then after I recommend them, they turn into mediocre 9-5 coders.

People always misrepresent at interviews; I can't think of a way around this.

I honestly think too much emphasis is placed on fixed questions and a particular style of interviewing. I've had a huge amount of success with improvised interviews that have both conversational elements drawn from the candidate's resume and some kind of problem solving task drawn from the type of work they will be doing.