Ask HN: What is a better approach to interviewing?
I want to make the interview experience better for applicants but have no control over the format or time. There is a lot of criticism of the algo centric approach and I'd like to change that within the confines given to me.
292 comments
[ 3.8 ms ] story [ 286 ms ] threadThere are many awesome people you can hire that you wouldn't hang out after work, and that's OK - you don't have to be friends with all your coworkers, you just have to be able to work together.
Instead, identify qualities & behaviors that your company values or are exhibited by successful employees and ask for examples of the candidate demonstrating them.
Yes people in teams should like each other because otherwise you will have a bunch of unmotivated pixel pushers as developers that only think when the time runs out.
In all of the companies were soft skills did not matter and I worked for people were coming and going.
No you don't have to be friends outside of work but usually you are still spending 8 hours 5 days a week with these people and maybe for some people it is okay to not like the other people in the company but from my experience this leads to no good in the long run.
Instead, I think we should attempt to really dig down into elements of "culture" and identify the ones that are relevant to the job vs those that are not. For example, some relevant elements might be:
These are all important topics to address in order to ensure that the potential hire will be able to work effectively in the position, and that they won't be miserable.And there are plenty of things that are _not_ important:
All of these things, on both lists, could fall under the nebulous "culture" umbrella. So let's stop using that word and start talking about specifics.I'd also note that for some of these things I expect the candidate to ask _me_ about them if that's something they care about. In fact, those sorts of questions are a very positive signal to me. It shows that the candidate has some insight into their own work and what makes them happy and productive. I'm very happy to find out that this isn't a good fit _before_ they've started working with me!
> * How disputes are settled.
Suppose you pose to the candidate a situation in which they are in the right, and ask them how they would proceed. They respond by giving the impression that they wouldn't easily compromise on their correct point of view. Is this positive or negative?
For many interviewers the answer will unconsciously depend on the candidate's gender. If the candidate is a man it's positive because they are "assertive" and "stick to their guns", whereas if they're a woman it's negative because they're "pushy" and "unwilling to listen to other points of view".
This should, ideally, depend on your team and your organization, right? There's no one right answer here.
I agree that there is an opportunity for bias here, but that goes for pretty much any question you could ask. You could ask them about programming languages they've used and find out they've used a lot of different languages. And the bias response would be to think that for men this means they're inquisitive and curious, but for women this means they're flighty and unable to stick with anything.
One thing that I think can help is to talk with the hiring team in advance about what you hope to get out of each question. For example, in asking about how you'd handle a technical dispute, you could focus on whether they would go directly to their teammates or manager before going to the next level up in the management chain. And on the flip side, you may want to make sure that their answer isn't "if I'm right I will never let it go because it's important to be right." By focusing on a few key points you're looking for, it may (I hope) limit the potential for bias.
But it's legitimate to ask about non-technical soft skills in an interview. Those are a _huge_ part of doing any job that involves interacting with other people. And conversely, there are people who are hugely destructive to a team or organization because of the lack of such skills. I don't think the answer to bias is to simply avoid any topic where bias could come into play.
Corporate culture is not about being similar to your coworkers, it is exactly what you said - company values and ways of working together. Don't get tripped up by the word "culture" - it has multiple meanings.
Rather than saying how you are constrained, what freedom do you have?
45 minutes - do code review of this extremely buggy code like you'd review a coworker's code
45 minutes - presentation on a technical topic to one or more team members
45 minutes - behavioral. Let's talk about failures, conflicts in your career. Who are your role models? What do they do well? Leadership abilities, career progression
45 minutes - troubleshooting. Here's a docker container. Make the service inside it work
Also number 1 (writing simple API matching a spec) - this doesn't require prep if it's using language that I am using currently / have been using in my daily job for a long time.
But I have found often been required to write code in languages that are on my CV but I haven't used them in 5+ years.
Also talking about past failures, career history. I can barely remember in detail 2-3 years so I would need to spend a little time to just be prepared about a random very specific question about a project I worked on 5 years ago that might come up during interview because it would silly if I didn't remember (which I wouldn't if you ask me randomly about something I did 5 years ago).
Small things like these add up, obviously the more you want to get a certain job the more time you would spend preparing, it's only natural. If you are only looking around you might just wing it since you don't care much.
I am not criticising your system, it seems pretty sensible actually btw.
Am interested to learn more. Are there open positions?
That works best for programmers with entry to mid-level. In Senior level you need to focus more on problem solving.
You want someone that can easily talk about the pros & cons of using different data structures or algorithms to solve problems. Who can move up & down layers of abstraction without getting confused & lost. Who can hide complexity to make clean, readable, maintainable code.
Many people, especially junior/less experienced engineers, have only memorized solutions to problems. They don't have a strong toolbox of solutions and the experience to know how to apply them (and evaluate the options).
A good problem solver is methodical - they don't just guess at random data structures, they identify what the requirements are of the problem and then choose the appropriate data structure (or compose one) which meets those requirements. They'll have a process, which may be different than your process, that they use to break down problems into smaller, solvable pieces.
It sounds to me like we are forgetting all the other important things.
Like how easy it will be to replace that data structure if maybe the requirements changes? Do we really need to optimize this thing fully now? Or maybe for the next 5 years to simple solution will be more then enough? And so many other questions.
A lot people need to think longer then 1 minute to come up with a good solution or maybe you need to try or maybe you need to learn something?
Isn't that what we actually do? How often did we think that framework X is the best but after a while it was not that great? Don't we need to play around with an API to understand it better? Or let me read about that and tomorrow we can talk about because last time I was doing something with binary trees was in university because in the real world I don't need it that often because yeah we have other important things to do.
There's like 5 basic data structures. Choosing the right one based on desired functionality and explaining why you picked it, that's just table stakes.
I'm not paying you six figures to write software professionally if you can't tell me when to choose a linked list over an array, and apply that knowledge to solve a problem.
The right data structure does not only depend on performance. I can write you the most performant code in the world if you want to but it will take 5 years. Oh we only have 1 month? then sorry for now an Array will do.
In most cases you pay someone for the skill to learn new things fast and stay motivated. Why? Because in good companies you want to hold on to the people for a long time and this people need to educate themselves. Why? Because even the basics depend on the context they are in and languages, frameworks and runtime and other stuff will change over time. 20 Years ago memory space was a problem? Do we even think about memory now? Not really unless your working on some embedded systems.
1. Those are the building blocks of software, and
2. They don't stop being useful when you move from domain A to domain B
What I said was that "movement" was a building block of life and your response was "no, cause if you do polka-STYLE movement while trying to lay bricks, you'll FAIL".
Right, but what they both have in common is movement. The fact that you felt the need to add style is exactly why I'm correct.
uh, not really.
In my world majority of performance issues comes from N+1 queries, I don't need people who can do crazy things with trees. I need people who write testable code, actually write tests, care about security and take responsibility for their code/refactor.
Of course you can argue that N+1 query fits algorithms part, but very often algos mean leetcode
But, since "you guys" always talk about data structures then I'd want to ask - how often do you use something fancier than literal basics?
https://www.youtube.com/watch?v=YQs6IC-vgmo
I've had plenty of candidates who don't know what data structure to use in that example, and they just start randomly listing things...tree, hash, linked list, etc until they stumble on the correct answer.
Lots of people say this however I've never seen anyone come up with evidence that their team actually is excellent.
My ideal interview questions are as follows:
* No trick question. Reaching a solution shouldn't involve obscure knowledge or an "aha!" moment. * Only basic data structures and algorithms, the kind that you will realistically use in most day jobs. * Start with a very basic problem, and ask follow up questions with increasing complexity. * Extend the problem in different dimension to probe the candidate in different axis. For example, ask the candidate to design an API for the code, or to run it at massive scale.
It's hard to come up with problems that cover all these points, but you can get close. I find these kind of questions allow me to assess candidate without making the process unfair or overly stressful for them.
Hire them, give them a week to catch up, and their off; or wait months to find a perfect match? Any developer with a few years experience knows how to pick up the next piece of technology, the same way everybody else does: read docs, search blogs and forums, ask StackOverflow, etc.
Now if all you find is Perl guy, that is another problem.
You can find all my past articles[1], and I'm happy to find you specific ones. I think, given that you can't switch away from algorithmic interviews, at least make the interviews more collaborative, define your expectations up front, and generally find ways to help the candidate show their strengths.
[0] https://hiringfor.tech/2020/02/10/false-positives-and-false-...
[1] https://hiringfor.tech/archive.html
Of course, this requires knowing what skills & knowledge are needed for the role - you may need to review the job description or push on the hiring manager to quantify them.
When asking problem solving or coding questions, work with the candidate as if you were two teammates solving the problem together - don't be adversarial, don't play "gotcha!", don't try to show off or prove yourself smarter.
If you have an awesome candidate, they will be smarter/more skilled/more experienced/more knowledgeable than you in some technical areas. That's a good thing!
Writing some algorithm No why should I? If I need an algorithm that must be fast I will look up a peace of code that is used and tested by other people.
No I will not live google in front of you watching me. How is this a real world scenario? How came up with this?
Like stupid questions like: How many log entries should a service have each hour? Yes this was a real question in a job interview.
IT Job interviews are broken in so many ways.
I refuse to live code or have this you now have X minutes to solve a problem stuff. Again this is not how I work and also I have never seen people work like that in the real world. Only in shitty teams with shitty organization and where they were thinking that they were agile because the sprint ends in 5 minutes...
Here is the only way of interview I tolerate these days: 1) (optional) Have a call with HR
2) Talk with people from the team in an open way. Talk about real challenges and how the applicant can help in the challenges of the team. Check if the applicant will fit your team! Good team chemistry eats technical knowledge by 1000x. If you only take people because they know some algorithm you will most likely end up with something in Germany they call "fachidioten". They will solve the problem on a technical level but will bring with them 1000 other social, product and long term problems with them.
3) If you still not convinced that this person is good enough for your team then. Give that person a small challenge he/she can do at at any time she/he wants.
4) Let the applicant send you the solution and if you think it is okay then talk about the solution of the applicant and see how he/she reacts to criticism. Again a team that has good social skills and likes each other will eat every team were you have rock stars that think they are the best.
volume if you need to hire a lot of people? volume of applicants?
I agree that the in person time pressured performative problem solving & coding engineering interview process is often a poor approximation of the work.
I had quite a few “aha” moments where I was “ah wait, I totally didn’t do that, have I?”. Didn’t get the job, however neither did I like them.
We had one case were you could clearly see that the person had no clue what he is talking about it.
Its quit easy to see.
Just ask them to change something small.
If at least two people who worked closely with someone like them enough to give a referral, they are probably worth hiring.
When hiring what works for me is looking at past experience. If someone says they are a cpp performance guy, that creates an expectation of what they should be able to chat about. If you're making it up, you will run out of stuff to say, and I'm patient enough to let people keep talking. Hopefully I'll learn something too, it happens often with top pros. But basically if you're experienced in a field it should be hard to come up with anything where you draw a total blank. If that's the case you're pretty good in my book, the main differentiator after that is whether you've done exactly what the firm needs or just something similar.
With entry level people it's hard. That's why they typically make less money, it's an Akerlof lemon cut due to the being a lot of people who just aren't going to be good at it. This isn't even necessarily a coding thing, but general life maturity. We had a guy who couldn't wake up on time to get to work, and he couldn't phone in when he was late either. I reckon there's no way to know about this kind of attitude problem until you hire them. This is also why you can expect a relatively high jump in pay after your first few years, it's proven that you aren't one of the lemons. The distribution is a large mass of hard to differentiate professionals, plus some duds of near zero value.
So for entry level, your best bet is simply finding someone you think is smart. You'll end up falling back to the same thing as everyone else who is hiring: the prestige of the uni they went to, and some pot luck questions to make sure they picked up something. However that's really not enough to know what you want to know, which is whether they are able and willing to learn what you need.
If there's one thing I don't want to do, it's work by assignment. I'm not a school kid, I'm a professional software developer. I'll tell you what needs to be done. You can inform me of the goals you have for the company, and I'll make sure that happens.
What I'm saying is that despite all my successes, you would consider me a 'dud'.
Of course, it doesn't mean the opposite holds. They could be interested in something but not end up being interested in the job.
Most articles complain about the status quo but don't offer any practical advice. When they do, it is some approach that may not be an option on most cases - e.g. 40 minute interview followed by a 90 day probation period.
Live coding interview can be pretty tough for candidates, but we try to make a lit bit less difficult by:
- Giving them time and resources to prepare.
- Letting them use their preferred language and IDE/editor.
But since you have no control over format or time, maybe try to:
- Use reasonable problems - to assess skills people will actually need on the job. In our case it usually boils down to using comodity structures like a dictionary or map to do simple tasks efficiently.
- Use problems that start simple but can be extended. This is good because when candidates finish the first part, they get a boost in confidence. It is also nice because if they freeze, you can help them finish the current "level" and still have material to assess other skills.
- Set them up for success during interview:
Making these little tweaks helped us to diminish the pain of this kind of interview.Some links/references:
[1]: https://lawler.io/scrivings/erics-guide-to-hiring-software-d...
[2]: https://medium.com/@alexallain/what-ive-learned-interviewing...
[3]: https://www.holloway.com/g/technical-recruiting-hiring/about
1. Hire based on the resume, gut check on the interview. Is it plausible that they played the role that they said they did? If you were a major contributor to an RPC framework in C, but you can't walk a linked list... Maybe your resume is too inflated. This means that the questions are basic because their purpose is not to rank candidates but to ensure the validity of the resume.
2. Focus on reading over writing. I would rather a candidate who can read code someone else wrote, articulate what it does, and maybe plan for how to extend it than someone who can pretend to invent something new.
#2 I have mixed feelings about. it may make sense if you’re after an expert on that language but may evaluate wrongly candidates that would be great reading code once they get just a bit more familiar with and get the overall architecture. In many companies, you’d be forbidden from sharing actual code with non employees, so the excerpt shown in the interview would have to be purposely crafted and loose value. I propose a more interesting exercise could be to examine a random piece of opensource that Both the interviewer and interviewee are unfamiliar with and trying to get a good discussion from it, but it’s a risky move leaving the interviewer exposed.
How would you design/write xyz
What's wrong with this code [code] (bug, security issue, bad code)
What do you think about
My "FizzBuzz" coding question that I ask on phone screens is "Write a function that takes a string and returns a list containing the most frequent character." It has a few clarifications that are needed and I expect the person to ask about (What if there is no most frequent character like in an empty string (return an empty list), multiple characters equally frequent (each equally frequent character should be in the returned list once), etc) and has some basic logic requirements.
I am continuously astounded when people can't do this. I encourage them to use the language they're most comfortable with. I give them time to think about the problem, encourage them to ask questions etc. If they seem stuck I'll ask them to walk me through what they're thinking about, etc.
I've had multiple people with master's degrees in CS or with years of professional programming experience get stumped by this problem and it just makes zero sense to me. I took undergraduate CS courses and this would've been an easy problem in those classes. I was a TA in an undergraduate intro to programming course and I would've expected all of my students to be able to solve this problem in similar circumstances.
It seems to me like someone with a Master's degree in math has come in and I'm asking them to "solve for x" in the kind of equation you might give to an eighth grader and they're unable to do it.
Unfortunately people tire me very quickly, I have a limited budget of people time per-day and interviews frequently blow that budget early in the day yet continue into the afternoon (talking particulary onsites for bay area companies here).
I would say 9/10 I manage fine but there have been occasions especially when I have been in between jobs and taking lots of interviews when I have just bombed because a particular company stacked all the code heavy interviews late into the panel.
Separately from that speaking as a hiring manager for several roles I don't think coding on demand is a good metric. I don't expect people to do it once hired so it's not useful as a measurement of their fitness for the role either.
Design heavy interview questions on the other hand I find very useful. They generally are less specific and speak more to experience, knowledge and taste. Things that are very relevant in an engineers real-world productivity. They are very punishing vs code on-demand interviews for new grads though so if your pipeline expects to take a lot of new grads you may need a different approach.
However in my experience questions like "How would you architect a replacement of Amazons S3 service with similar durability" or "Given a smart grid reporting metrics from every home in the US how would you store this data for analysing both long-term trends and intra-day anomalies" yield by far the best signal to noise ratio.
This isn't obvious at all - I've interviewed many people that have faked their GitHubs or were just completely incompetent at software development despite their amazing GH, OSS profiles or CVs.
People lie, a lot. I'm sorry, but having "a profile" isn't a useful signal for an interviewer :(
> However in my experience questions like "How would you architect a replacement of Amazons S3 service with similar durability" or "Given a smart grid reporting metrics from every home in the US how would you store this data for analysing both long-term trends and intra-day anomalies" yield by far the best signal to noise ratio.
Agreed, in my experience these questions give by far the best signal for a good coworker in general. Sadly they're not appropriate to interview junior developers.
So yeah, it might not be useful for recruiters/sources that don't understand the technical side deep enough to evaluate but it 100% sure be good signal for an interviewer if they know the fields and projects in question you are hiring for (they probably should...).
(I was also a TA in a course that did these sorts of trivial challenges, and going through that experience really made me take a hard look at how I perceived those 1st year students)
But the thing about interviews is that you always want to heavily bias towards false negatives. A single bad hire is just astronomically costly. I think it’s fair to say that many smaller companies have literally been bankrupted by one incompetent person. When it comes to engineering positions, this is especially true.
I’d rather nuke 20 good candidates having a bad day, then risk letting one bad engineer through the door. That’s why I’m skeptical that the interview process can even be “fixed”.
Fundamentally it’s a mis-alignment of incentives. Employers have very strong to rule out candidates at even the slightest hint of incompetence. But for the interviewee, being judged so quickly, harshly and unfairly on a such a personal level makes them feel vulnerable and hurt.
First, I would feel like if I have to ask questions if you then I’m missing something obvious you want me to be able to deduce. And if I take too long I’d fail your test.
So I’d jump in merrily and work up a quick and dirty version before even realizing that the output has ambiguities. Hopefully at that point I’d ask how you want them resolved, but maybe I decide you want to see me to make my own decisions and I’d come up with my own way to handle them.
I had just received offers from two FAANGs, and I was flown into the Bay Area to interview at a third. One of the interviews was mostly behavioral, with 20 mins at the end for an algorithms problem that I had done before on LeetCode. I did so badly that after 20 minutes the only thing on the whiteboard was the problem statement. I still remember the interviewer taking a picture of it, and me kind of cringing because of how bad it looked. I didn't get an offer, unsurprisingly.
Because they disqualified themselves from hiring me. What I remember from that phone screen was getting mentally _TIRED_, sighing, and then lying about not being able to do it.
Seriously, don't fucking ask me to describe code or algorithms over the phone. The number of times I've had someone do some variation of this over my lifetime is such that I don't want to have you as a co-worker or a manager.
Do you know why I have this attitude? Because the number of times I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed, is enough that I've decided if you think this is the right way to go about hiring, you're doing it wrong and I don't want to be in that hellhole.
I imagine if I was hiring someone to play in a band or sing in a choir it would probably be a good idea to ask them to play or sing a bit before extending an offer. If I was hiring someone to work at plumbing company, I'd probably ask them a "How would you fix..." type question or two.
Saying that these questions are forbidden just because you are sometimes tired of doing them strikes me as unreasonable. While, of course, you get to decide what signals would make a company a bad fit for you - e.g. if I asked you my question you might think my company would be a bad fit for you, the interviewer decides which answers are a bad fit for the company. I would definitely take an answer like this as indicating a bad fit for the company.
This line from your comment is interesting to me though: "I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed". I frequently ask questions where I expect the candidate to want to clarifications or challenge what I'm saying. That's part of my job, and I'm trying to test for it the best way I know how. If you're happy with your job interviewing performance, then more power to you, but if you aren't - you may consider changing how you think about instances where the interviewer is wrong. Perhaps they are presenting opportunities for you to engage with them in a productive way, and even if they aren't doing so intentionally, maybe they are doing so unintentionally.
You know what's unreasonable? Interviewing a bricklayer and asking them to describe how they lay bricks over the phone. And this is why your comparison to singing is hugely flawed. The phone is literally designed for carrying and hearing voices, not for more complex things.
You act as if it's literally impossible to figure out if someone can code unless you're trying to do it over the phone.
> This line from your comment is interesting to me though: "I've seen people actively be wrong in what they're looking for, as in technically wrong and therefore I failed". I frequently ask questions where I expect the candidate to want to clarifications or challenge what I'm saying.
I once had someone ask me what _type_ was returned from a controller in asp.net MVC, and when I gave the technically correct response this person told me I was wrong, a view is returned, then immediately ended the call. The number of times I've seen stuff like this happen is way too high.
But more than that, I want to address the attitude in that last point. This is a _HUGE_ problem with interviewers. They often come to conclusions they ought not come to based upon the circumstances. And the defenseless interviewee's are left trying to suss out hidden requirements like "expects them to ask clarifying questions". Interviewee's come in and have to try and navigate these byzantine, hidden requirements. And the question is, why do interviewers have hidden requirements like this? Because they know if they're _HONEST_ and communicate what they're looking for, the candidate will give it to them. That by itself should be a wake up call that interview's are hostile and un-real.
I'm just in a position in my career that I have the leverage to refuse to play those games. If you do something like that, you fundamentally have no clue how to actually hire people, and I refuse to bring any value to you or your company.
But asking a candidate for 30-50 lines of code is completely normal.
The fact that the internet exists and the interviewer could have used an online utility doesn't change my responsibility in that equation, that's just nuts.
There's only one (not counting ties), so you want a single-item list? Or if 's' is the most frequent, because there are 5 of them, do you want [s, s, s, s, s]? Or since the list just needs to contain the most frequent character, seems like [r, s, t] would be correct as well. Now that I think about it, just returning the original string would have to contain the most frequent character. Also, do capital and lowercase of the same letter count as different characters?
Just seems like a rather unclear directive.
I think of it as the FizzBuzz version of getting requirements for the thing you're supposed to be doing.
Expect professionalism from both parts.
I would expect a skilled programmer to be be able to make intelligent inferences about the intended behavior in the face of ambiguous problem statements. FWIW, I would probably explain my reasoning as I went -- would you interrupt a candidate to say you'd rather see a different behavior (ex: my prototype would return a list with an empty string, but maybe you'd rather an empty list), or just silently judge them? Also, it look me about 2 minutes to produce a reasonable prototype -- maybe if the challenge were harder I'd expect someone to ask more questions up front, but in this case it's trivial to iterate.
If the only thing you are testing for is the instinct to clarify unclear requirements, you'll surprise people who are used to the leetcode-style, thereby getting false negatives. It's just as much a trick question then, just in a different dimension.
I generally take a look at their resume and then do some research about the tools they've used in advance. Then, during the interview, I ask about what they like/don't like/find interesting about those tools. The goal is absolutely not to gotcha them, but instead to find out what they're interested in in that space. If I ask a question that it becomes clear they've lied/fabricated about on their resume, I say something to the effect of "No worries" and change the subject.
Depending on the role, you need more info than just what languages/frameworks they've used. For more senior roles, or roles that involve architecture/cloud functionality, I'd ask about how they've built systems in the past. If they call out AWS, I ask about what resources they've used, how, and why. If you've written down DynamoDB but cannot speak intelligently about access patterns or secondary indexing, it's kinda clear that you just used a system someone else defined. Whether or not that's a problem depends on what role they're applying for. If they can speak intelligently about how they got to a specific DynamoDB structure, they probably are being honest enough about their experience. Note, it needs to be clear that the candidate is not speaking in the abstract, but about things they've actually done. Googling stuff is easy, finding the weird parts of tech in practice is hard.
Ultimately I want them to feel comfortable enough to get chatty about development. Usually I find out enough about their skills while they're chatting - I think most would be surprised to find how clearly you can understand a person's abilities without directly asking about them. You just kinda have to spend some time up front learning pros/cons/common pitfalls of the tech on their resume.
As an interviewer, I don't really care if a candidate happened to know the answer to an algorithmic puzzle or has a deep knowledge of academic minutia. I want to know three things: can this person reliably and meaningfully solve technical problems, can they communicate their findings to the team, and finally – do they strive to learn the details or do they just skim/superficially solve problems? Obviously expertise in a job-relevant domain is great too - but I'd consider it a bonus; the difference between a senior and a junior/mid-level hire.
The first two are part of a quiz we give candidates, it has a couple basic questions on it that simply require them to logically think through a problem they likely haven't seen before and provide feedback. I don't even care if the answers are correct, but they should be able to explain their thought process and reasoning clearly in a couple followup questions. Bonus points to candidates that ask for feedback on their answer.
For the last point, during the interview I get the candidate to describe a project at a recent workplace, once they have given the description, I'll ask questions to see if they evaluate and learn from their previous work.
I am constantly shocked at how many candidates I interview that fail one or more parts of this spectacularly (especially senior level candidates), to the point where I question whether evaluating candidates these ways is unfair, but also knowing that if someone doesn't do this they likely are going to fail when given a problem without an obvious answer.
Provide candidates a small sample "project" of 50-100 lines, a Flask webapp perhaps. Work with them to get it running on whatever interview machine they're using. Have them extend it with some fairly trivial feature, around FizzBuzz complexity but with a bit of a design component. These questions should be carefully designed and standardized across interviews.
Key point: completely strip out identifying information from their solution and pass it to someone else for grading. The person "giving" the question is a proctor, not a judge, and they're officially on the candidate's side. Possibly add a side dish of talky conceptual, design, and behavioral questions, with as much blinding in the assessment of the answers as possible, though that's harder if it's a dialogue... Hmm.
A good interview is fun. You get a nice bite-sized problem to solve with someone to bounce ideas off of, and maybe learn something in the process. I'd suggest almost a game level-design approach to coding problems. Make interviewing fun again.
My favorite interview question I ever had was to design an elevator system. There's no right answer. It wasn't even mostly code, it was mostly talking. But every so often, they'd say "write that as a method" and let me imagine whatever api I wanted. I had so much fun doing that.
https://play.elevatorsaga.com/
It would be an interesting idea just to have a candidate go to the site and screenshare to watch them do some coding.
As an interviewee, get a strict proctor that doesn't want to give out information? Well you're not going to appear as competent as the interviewee who got the proctor that wanted the interview to be fun
I would say it's still an improvement over leaving the whole thing up to the whims it the interviewer.
It had a few benefits:
First, this is much closer to the type of work that we'd ask a new employee to do, as compared to the typical industry interview question: "did you pay attention in computer science? You did? Now prove it by solving this problem that's a thinly-veiled algorithm exercise. Using a computer? Oh heck no, you're going to use a marker! There's no trick to it, we don't ask interview questions that have tricks, but if it leaks externally, it's 100% useless and we cannot ever ask it again"
On that topic, who cares if the premise leaks? If the code somehow got out, we'd just write another one.
We get to hear them talk, out loud, about their expectations and how the app's organization differs from them. This isn't interesting by itself, but the candidate gets a chance to articulate their experiences and how well they notice patterns. For more senior engineers, you get to see how quickly they can pattern match and explain the structure of the client and server, and explain how it could evolve under different circumstances. "You're in a startup that just got a ton of VC based on this hackathon project. Your friends hire you and give you this codebase. How do you spend your next 2 months?"
Also, you'd be surprised how many people disqualify themselves on attitude. People will read sections of the toy app and say out loud, "who the hell wrote this? This is terrible." And you'll ask them, "What do you mean by that?" and they'll rant about some section of it, and you're like, "wow, I really don't want to work with this person." If these engineers can't even keep it together during an EXERCISE whose PREMISE is that the code doesn't work well and can be improved, it's a negative signal that they're going to demonstrate any empathy when they're working with you or any of your colleagues over the years.
For senior engineers I would say it's all about design/architectural problems and talking generally about concepts and taste to gauge fit.
Mid-level engineers I think the take-home is king. A scoped problem, 1-2 hours at most that can be followed up with an interview talking about an extending their solution is pretty bulletproof. You get to understand what tools that are comfortable with, practices around source-control, testing etc that you would expect from an experienced but not overly senior engineer.
Finally for entry level candidates I don't really know. Personally I hire entry level people on gut feel. Algorithms and datastructures just tell me if they a) went to university and b) paid attention. Seeing as I did neither of those things and still write IMO decent software that seems like a hypocritical measure. So I stick with how I feel about them, their level of enthusiasm, how well they researched the position, what stuff they have played with etc.
At the end of it the biggest burning question they had was "where are the unit tests". I was like ... you see this circle here with this label inside it? This was critical to the entire system and therefore had unit tests surrounding it.
didn't get the job, they seemed kind of annoyed that I didn't get into great detail about things like unit tests and instead talked mostly about the architectural problems I solved.
I wish companies actually valued that level of knowledge.
The company itself wasn't actually solving complex problems and therefore the developers involved couldn't actually understand the level of complexity I was able to deal with.
I get the feeling a lot of developers believe dealing with state in a react app is a complex problem, but I've been doing this stuff for 25+ years and left that behind as complex long ago. I know that's probably going to come across as arrogant, but it's how I feel about the entire thing. I see people all the time squabbling online over very specific source code related things as if they're important.
It was a VPS automated deployment system that could deploy both windows and linux to various datacenters around the world while also installing some 50+ pieces of software (as configured).
All excuses for why this won't work can be answered with: then make a better test.
As a species, across all cultures, we tell ourselves that the human we're after -- the student, the employee, the promotee -- can't possibly be selected by test alone.
When in fact, if we can't write a test to select the qualified humans, then either we're too lazy to write one, or, more likely, we actually want to leave plenty of room for human bias to do the actual selecting.
And this is ok because we have a special power: we can judge the value of every human, and its future likelihood for success, with a single conversation. If we weren't in a tech company, we could make a very good living reading palms.
We never hire people who don't pass our wonderfully fuzzy exams, so we have no evidence that we're selecting the best people or not. No worries, though, our palm reading is very, very accurate.
The way we look at it is like this. We make a test no one can pass. We always have one more question or one more "level" that there's not enough time for. Because when all candidates fail, we have to fall back on our palm reading, which is just how we like it.
Power and privilege are precious resources to us. We give them out to those most likely to reciprocate. That's what we're poking for with our "culture fit".
In the future, students of our culture will look back on our hiring practices and say, "It was illegal to hire based on race, age, sex, and a million other things, but not beauty??? They didn't start with beauty? And they never realized that beauty needed to be in the mix? I don't understand."
But we understand. It makes perfect sense.
Creating such an exam is an unsolved problem.
Why can't we write a test to see if she can code in Python at an appropriate level of expertise without mixing it in with algorithm gotchas? What are the algorithms she needs to know? Why can't we provide her with a list of those algorithms and show her how we will test for mastery of them?
>Why can't we provide her with a list of those algorithms and show her how we will test for mastery of them?
That's exactly what is done, it's called leetcode.com, recruiters send you a link to it and tell you to study. I don't see how anything you're describing isn't covered by the existing whiteboard interview dynamic.
leetcode is an open-ended ocean of knowledge you have study, it's not a test.
So it's all standard questions that have answers you can study for ahead of time.
leetcode as used by FAANG is a hurdle that you can't ever be sure you actually cleared. it's not a test that returns a pass fail score.
The same reasons whiteboard interviews are problematic will apply to any test done at scale.
If you were a team lead tomorrow, how would you go about hiring your coworkers?
Honestly, that should tell you everything you need to know about:
a) What kind of technical skills they value (i.e. will work to improve in themselves if necessary)
b) Their interaction style with coworkers.
Sure, this interview can probably be gamed, but IMO a lot less than your typical "reverse a linked list" style interview.
It allows your company to learn, to get a feel for the person's ability to communicate, and to stay away from stupid things like tests.
It's also easily the best interviewing process I've ever experienced.
---
edit: Since it appears that multiple people have taken this to represent an hour+ long presentation, let me clarify here.
A better way to think about this would be a long-form discussion about a topic. The initial "talk" would be 5-10 minutes, with a Q&A afterwards to ask for clarifying questions.
I've been working in this field for over 25 years. I spent roughly 30 minutes of prep for the interview and it was easily the best interview process I've ever experienced.
If someone is spending 5+ hours on this then they don't really know or have opinions about any of the topics and that doesn't seem right for common technical topics.
As far as I'm concerned, this is you trying your hardest to have the most negative view on something that's mostly innocuous that you can. You're making it sound as if we would then take them into the next room and beat them until their morale improved.
If this is your reaction to someone stating their initial view of your proposal and their perceived issues with it then okay I guess. Not sure why you posted something on a discussion forum when that's how you react to any negativity about it. Anyway, don't see much point in talking with you.
So with that being said.
If you read back over the conversation you'll see that when the initial negative statement was made I realized how presentation could have been misinterpreted as "powerpoint presentation" rather than simply a long form discussion and therefore I responded with a clarification.
Up to this point, all reasonable.
You could have chosen to respond with something along the lines of "If the time commitment was too onerous I'd probably opt out", or some variation thereof, at which point I would have further clarified.
Instead you declared, with authority, that only the ones spending an onerous amount of time on it would do well. This was a completely unreasonable response with undue negativity and unfairness.
At which point I responded that I've actually gone through this process and it only took 30 minutes. I then pointed out the unreasonableness and unfairness you were exhibiting.
The thing is this. After having declared that I've gone through it myself and it only took 30 minutes, you had no authority to fall back on, so you went meta and started with an attack on my character.
So here we are ... doing the meta thing.
I will not be responding to you further, and it's probably best that you don't respond to me further as well.
I agree with the other commenter, it baffles me that we think that asking someone to write code, when the majority of their job will be writing code, is somehow an unfair interview process so the solution is to have them give only hours of powerpoint presentations.
I've come across many a smart person who would abhor the idea of listening to a presentation, let alone give one. Give me a document they'll say, and they will go through it at depth and collaborate amazingly over text. A mix of both is required. You don't want everyone to be giving presentations all day, just as you don't want no one to be attending any presentations. Diversity is essential, and it's more effective to create structures that allow different types of people to work with one another than to mould people into personalities.
It's akin to saying that because some people have anxiety issues, we should completely do away with interviews and just let people randomly walk in and do work.
Because that's the only way to potentially avoid the criticism you gave here, only that's also not true because locale comes into play. "You didn't create a building every 500 feet that people can walk into, so now you're disenfranchising people without cars!".
My point here is that this is not a useful criticism. Come up with a useful grievance, one that's actually actionable, and then maybe we can talk.
Not every personality difference can be reduced to anxiety. That is precisely the kind of closed world-view I was talking about that can form within homogeneous groups.
You act as if the people listening aren't aware of this.
What is it you believe exactly, that if person A sounded nervous they wouldn't be hired?
The entire point is to get a conversation started, and the following Q&A allows them to do something everyone needs to do in a job, which is to answer questions.
No interview process is perfect. You can literally give me a description of any interview process and I can start finding problems with it.
Just as you can literally give me a description of any software solution and I can start finding problems with it.
That you can find problems with it doesn't make it useless or harmful. Please come up with better criticisms.
I can do presentations, however you are asking me to hold a free 30 minute lecture teaching your employees about something. I'd rather not, basically every company gets excited to hire me anyway and I'd rather spend a few hours doing some comfortable coding over spending a few hours preparing a presentation.
Technically speaking, if we're in an interview to "optimize for the right thing", then we would just have them come in and type.
But we all know that's also optimizing for "the wrong thing" despite it being what we do physically day in and day out.
This is because you _CAN'T_ "optimize for the right thing" in an interview. It's not possible and this needs to be acknowledged.
Therefore you go for something else. Anyone who is a software developer can listen to a 5-10 minute presentation from another developer and start forming general, broad opinions. They can then start asking questions afterwards to help clarify those opinions.
But the Q&A also has another purpose. Way too often interviewers will come to conclusions that don't follow (non-sequitur). I once had someone tell me they didn't feel I would be a team player because I told them as long as I had headphones, open office would be ok. The main difference I've found between business people and technical people is that business people will ask clarifying questions. The Q&A sets the expectation for the _INTERVIEWER_ to ask clarifying questions.
If you can’t boil your work down into accessible presentations and adapt them for technical or non-technical audiences, you won’t get anywhere.
Nobody will agree to help you, allow your team to grow, give you budget for tools you need, make compromises with you around integration requirements or shared design patterns, unless you can frequently convert your software work into highly professional presentations.
I’d argue that competent presenting and writing skills are of equal, or even greater, importance than software skills or expert knowledge, whether for a research software job, software contractor job, SWE within tech / ecommerce / banking etc industries - across the board there are pretty much zero types of software engineering jobs in which software skills are actually the most important thing. Presenting and writing are frequently much more important.
To be clear, explaining your work is a part of every developers job. But top notch presentation skills don't need to be a part of it. Developers can talk to other developers in sub-optimal ways that are still effective. But when your hiring process has an hour long presentation as a part of it, you are optimizing for the thing that is very likely not the most important feature of the role. The point is to structure your hiring process such that it optimizes for the right things for the role.
But also, I really really dislike when others talk about "optimizing" people. It's not possible to optimize your hiring process for anything useful. Due to this, there should be a lot more empathy in the hiring process than their actually is because everyone wants to make it robot-like by "optimizing" something.
This isn't true in my experience, there are plenty of well paid jobs where you don't have to talk to non-technical people. For example, lets say you are developing infrastructure at Google, how often do you think you talk to non-engineers? Never, everyone in your management chain will be engineers, your product managers are ex-engineers, your users are engineers etc. This is true for directors managing lots of people as well as well.
And even when you develop services much of it is just getting the technical parts right so you can work there as well at Google with very little time spent talking to non-technical people, so senior positions there are mostly about technical stuff.
You could argue that Google and other big companies are a special case, however if we exclude low paid jobs where you just earn half of what you'd do at Google or Facebook then pure technical work like this is a very significant fraction of the market. I'm not sure why I'd work on my presentations skills so I could get hired by a company paying me less.
I can speak from experience since I help run the site reliability incident response team in my (large ecommerce) company.
How often do rank and file SRE ICs need to speak with non-technical staff about projects? All the time.
They give tech talks and council presentations, they contribute work to quarterly roadmap project pitches, and they give business scorecard presentations to walk higher level managers through the breakdown of costs and benefits, why we should pay money for a new cloud tool, what time & effort some org-wide planned migration or upgrade is going to require.
Let alone basic stuff like writing effective tickets / issues / RFCs / PR descriptions, postmortems, etc., and walking teammates through work artifacts daily.
For every two hours of software work, I’d expect 1 hour of “wider audience communication” work - and that is for deeply technical ICs in roles like SRE. The closer you get to product engineering, the more the ratio shifts towards communication.
I can tell you this translates specifically to Google & Facebook - because most of the SRE colleagues in my company came from Google & Facebook, and attest to that being the case there too.
Our group hires people in the software+hardware+mechatronics space, and we look for people that bring diverse skills we don’t yet cover well. The kind of person we want can teach us something new in 30 minutes, and a good project for that could be backyard hackery or university research. Either is good. I totally love the format. (Then again, I am a “Talk is cheap. Show me a working (even if janky) robot.” kind of person.)
Related, a former BioMed engineer coworker said Medtronic used a similar process - start the interview day with a presentation.
I've done a lot of both traditional interviews (presentation on research or previous work followed by 1:1) and "tech" interviews (coding challenges and whiteboarding) from both sides. Tech-style interviews evaluate a much narrower swath of skills are are much more easily gamed.
I feel that a presentation evaluates broader things: how well do you communicate and how comfortable are you discussing and answering questions about a topic you know well.
It's a better indicator of job performance, i.m.o.
Some developers are more comfortable with communicating via 1:1 conversations, design documents, proof of concept PRs, etc.
I've done a lot of public speaking and presenting, but am aware I am in the minority of developers. I've also managed a lot of people and many of them have struggled with self confidence and presenting to large groups. This has not been a reflection on their performance as an engineer.
To explain, college curriculum's require atleast 1 public speaking class. These classes are infamously difficult for a lot of people who find it scary to talk in front of a class of 30+ students. But society as a whole considered it to be the young persons responsibility to overcome this fear.
Nowadays, it's far more likely that people are going to make arguments such as "the class shouldn't be required" or some other thing to try and help the young person avoid the scary public speaking.
My point here is that yes, it's true there are going to be some who struggle with presenting to 5-6 people. That's on them. If they can't talk to 5-6 people they're going to find something as simple as a daily standup to be scary as well, should we just get rid of it? (Actually, I hate standups and think they're a waste of time, but you get the point).
It's also not clear to me why you characterized this as a "large group", but I'll give you the benefit of the doubt and say maybe you're used to interviews where 20-30 people get involved. That's certainly not what I'm suggesting.
And finally,
I can't imagine a young developer not being able to have a 5-10 minute talk in front of others about something like, say "What is memory safety in rust and why should it be used for new projects" when we're hiring for someone to work in rust.
A more senior person would obviously get more generic options. The one I personally spoke about was roughly "What can we as a company be doing to empower software developers to create the best software possible".
* in a formal setting,
* to people you don't know,
* with a high stakes outcome (performance determines whether you get the job)
Is not neither a skill that developers need to have, nor is creating a "better experience for applicants". This is in no way analogous to giving an update at standup.
I've had applicants physically shake during interviews when I'm doing my best to create a comfortable environment. Some people just get really nervous. And when hired those same people have become comfortable quickly and performed well in their day jobs.