VP of Engineering at Eaze, previously CTO at Getable and engineer at Yammer.
No, not at all. Whiteboard interviews test one thing
well: How well does a candidate code on a whiteboard.
Engineers on my team never have to code on a whiteboard
(whiteboards are really bad at running code), why would I
make candidates do something that I don't ask the
engineers already on my team to do?
This comes close, but I think the real issue that most of these reasons aren't hitting on is that software development is typically done asynchronously. Whether it's a whiteboard or a computer connected to a projector or TV or a live peer coding session it doesn't matter: if done in the interview, it puts the candidate on the spot in a way that they rarely (if ever) will be on the job. Developers typically are able to take time by themselves or with trusted/known colleagues to solve a problem. There are very few opportunities to get over performance anxiety related to coding in front of others that we don't know.
Minor kvetch: quoting the way you did it makes long lines very difficult to read. I usually quote like this:
> I am a quote
This makes the quotation doubly-distinct, and still readable on mobile devices.
===
To the actual point I wanted to make: plenty of engineers at companies code on a whiteboard. They schedule a meeting, grab a room, and talk things out, while making notes, diagrams and sometimes even actually writing out some code on a whiteboard.
I hope nobody asks people to write compilable code on a whiteboard, but asking a person to talk through a problem and jot down some pseudocode doesn't seem like an awful thing to me. Communication is a pretty critical part of our job.
“Jotting down some code on a whiteboard” with a colleague is a much lower stress scenario than deriving and writing down, under a time limit, some DP algorithm you might not have used since college, and proving its correctness and run time to some stranger who may be barely out of college themselves.
One particular expectation when whiteboard coding for an interview is that there is little/no downtime, you must be making progress or talking about your thoughts. Incidentally, 50% of my thought process then becomes "How do I talk about what I'm thinking about? Oh no, I'm only thinking about talking about what I'm thinking about..."
I'm working on a large-ish project right now, and into one of the more tricky bits literally today. And you know what? I take breaks. I go read something on the internet or post a comment, or I go for a walk, or I go do one of the smaller housekeeping tickets I keep in my queue to give my brain a rest from continually focusing on the hard parts of the problem.
Most of the time their technology stack with be vastly different than anything else you've worked with. There will be other people you will need to discuss and coordinate with. Articulating your thoughts and expressing yourself clearly and concisely is more important.
Vast majority of the technical problems have been solved and its delivery a solution to a business problem that is the most important aspect.
For this reason I don't do living coding sessions. People have asked me `why` which I politely say the answer is no. They seem to like the boundaries I set for myself and my project's more than some sort of white board.
Yes, I've been turned down for jobs over the white board but I've had more positive results than negative.
"Interviews are necessarily more stressful than typical work. That's not just a property of whiteboard coding."
My sister, a pediatric ER nurse would disagree with you. The interviews are largely behavioral and are a breeze. No dummy is wheeled in with head trauma, random "new" diseases aren't invented and asked to be treated, etc. The simple fact of the matter is if hospitals hired in the same way, they would have no staff.
I would argue that a piece of paper showing that someone attended school and graduated and is licensed to be an ER nurse carries much more weight than a piece of paper showing that I'm allegedly qualified to be a software engineer.
I know it's totally anecdotal, but we've all heard horror stories about candidates who couldn't write a for loop; some of us have witnessed these things first hand. And yes, phone screens should be filtering out those sorts of candidates long before they start sweating with a dry-erase marker in their hand, but, well.
How likely is an applicant for a nursing job to flat out not know how to stitch up a wound, or to take a patient's pulse and temperature?
Again, the op stated these things are put there to "show how people react to stress" -- my post was a response to that, there is no re-creation of a stressful environment in the situations I stated.
"I know it's totally anecdotal, but we've all heard horror stories about candidates who couldn't write a for loop; some of us have witnessed these things first hand. And yes, phone screens should be filtering out those sorts of candidates long before they start sweating with a dry-erase marker in their hand, but, well."
Yes, it is a failure of how you have set up your hiring pipeline which you are now band-aiding. The majority of those folks can be screened by one look at a resume or in the first 30 seconds of the phone interview. Other hiring managers in our company repeatedly had this problem until we got them to focus of the right candidate qualities and ask the appropriate questions.
Take for example your local symphony orchestra; they have the same problem where people with visions of "making it" show up not being able to play at all. Want to audition? Send us an audition tape and a check. When you show up for the audition, you'll play a selection from these pieces and be asked to sight read this music. It is ironic that for an industry that is in a sense so subjective that the gates in the hiring process are more concrete.
To make an analogy, the software world is akin to:
Interviewer: "I see you are interviewing for the 1rst chair violin. The 3rd chair tuba player is really into experimental music, he would like to transpose Vivaldi's Four Seasons into a new scale with 12.5 notes per octave with a slight progressive jazz leaning. Oh, and since we all know there is pressure in performing in front of an audience, you have 30 seconds to think before the 2nd chair begins to throw rotten food at you. Reaction to stress and all you know...here is your tuba."
"But I don't play tuba...Are you asking me to play tuba? Am I going to be playing this nutcase's new music as part of our program?"
The analogy to white-board interviews for hiring a musician is:
"Here's a couple of pages of unfamiliar sheet music that a second-year student should be able to play. You have an hour to figure out how to muddle through it on an instrument of your choice."
People hiring musicians don't do that, because they instead prefer to give candidates 16 bars of complex sheet music, and expect them to play it perfectly during the audition.
The programming equivalent would be to give someone a hard take-home problem, let them stew on it, bring them into the interview, and ask them to type in their solution, from memory, into a text file, on a keyboard with a broken Backspace key. That they will then compile, run, and compare the result of to that of the other 60 candidates auditioning for the role.
Are you sure you want to do auditions, instead of interviews?
> It is ironic that for an industry that is in a sense so subjective that the gates in the hiring process are more concrete.
That's because there's fifty thousand correct ways to solve a trivial programming problem, but only 'one' way to correctly play second violin in Vivaldi Four Seasons.
Music is a subjective art. Playing music is a mechanical process. My iPod can play music. My iPod can't implement a sorting algorithm.
That's because she's already went through all that stress and bullshit and skills testing in med school and residency. If any clown could call themselves a nurse, and would apply to ER nurse jobs, it would take all of ten seconds before nurses would have to do whiteboard triage interviews.
I have no idea what the candidate did in their CS undergrad. Maybe they cribbed all their work from their roommate. Maybe they went to a party school. Maybe they spent the last 4 years as a 'Senior Developer'
at FooCorp copying files from hard drives to floppy disks, and posting a few paragraphs a day on the company's WordPress install. Maybe they are an Architecture Astronaut who can talk for six hours about how great Haskell is at doing multi-manifold monadic trivariable entaglement, but has no idea how to do any real work.
Or maybe they spent the last decade building Bigtable and MapReduce, and Spanner, and TensorFlow at Google. I'm not an expert on Bigtable, or MapReduce, or Spanner, or TensorFlow, though - and I can't definitively, in 60 minutes, tell if the person I'm talking to is bullshitting me. I can't tell if they actually did any of that work, or they coasted. I can't tell if the complicated problem they are describing to me is actually hard, or if they are embellishing it. Even if I felt confident that I could make that conclusion, my opinion would be incredibly colored by personal biases.
Oh, I should check their GitHub, you say? Well, guess what - Jeff Dean - the guy who did spend the last decade building Bigtable and Mapreduce, and Spanner, and TensorFlow - doesn't have a GitHub account. Presumably because he has better things to do with his free time, then work on OSS.
Oh, I should hire fast and fire fast? Don't get me started on why that doesn't work...
At least a test of triage skills would be a real world skills test. The equivalent for most whiteboard software engineer interviews would be a quiz on cellular biology.
Hospitals also hire people who went already through an even more rigorous screening process than even exists in our field just to become eligible to be nurses and doctors.
Once you're an RN or an MD with all the appropriate qualifications, assuming you're not outright faking them, they can be reasonably confident that you are, in fact, a competent nurse or physician and move on from there.
We don't have that, and so we have to spend a lot of time making sure that a prospective candidate even has the basic skills and qualifications of a software engineer.
As an embedded systems engineer, I disagree. I have never been asked to design a circuit, or even pseudocode firmware, on a whiteboard. In the electronics manufacturing industry, interviews consist of talking, mostly about projects I've worked on before and projects they've done before. Occasionally someone will give me the same written test they give technicians, but those tend to be pretty easy.
I don't think people should play CS-quizshow during whiteboard interviews. Being asked to implement a breadth-first search, or whatever, is kind of annoying - especially since there's effectively one correct answer. At best, you're testing my memory and whether I read the right interview book.
I had to do that at my last interview. I got through it ok-ish. The more interesting question was a more general system design question - how would I architect a system to do X, under constraints Y and Z. What would I do if a new constraint came up? Ok, now how would I make it more resilient? There, the whiteboards is just a tool I can use, not the primary focus.
Surely not BFS. That's something you can figure out from first principles and knowledge of some data structures like stacks and queues, but I'd expect knowledge of those data structures anyway.
I don’t care if candidates can figure out BFS under intense pressure in 30-60 minutes. That is nobody’s job anywhere ever. I want to know if someone can think creatively, if they’re a pain in the ass to work with, if they’re thoughtful and careful or if they shoot from the hip. Admittedly, these are hard to measure (which is one reason why I would support some kind of mild licensure, but that’s a different subject), but being able to derive BFS does not correlate with competence or ability. Maybe if you’re doing router hacking or something, I guess, but specific algorithms are so little of most developers’ careers.
Honestly, we all might as well read chicken entrails because that would be about as predictive as our current interview practices.
> some DP algorithm you might not have used since college, and proving its correctness and run time to some stranger who may be barely out of college themselves.
I'm not opposed to white-boarding, but to reinforce this point I'll note that I would have done much better on some of these problems when I was just out of college than I would today, because they were all fresh in my mind.
I assure you I wasn't a better programmer or employee then.
"Whiteboard interview" typically does not mean "sketch a general design of a system", or the other things whiteboards get actually used for on the job. It means "here's an algorithm challenge, solve it by writing the code on the whiteboard, and we'll flunk you if you make any mistakes".
Not always a whiteboard, either; one of the best engineers I ever knew flunked a Google interview because of a situation where he had to write a bash script and read it over the phone to the interviewer, who apparently didn't transcribe it correctly on the other end. But "they only hire the best", you know, and their practices are rock-solid and proven.
Oof, I've had that experience during a technical phone screen with a different "hires only the best" company. I was asked to write (over the phone) a trivial statistical algorithm and started to describe the algorithm: "Function F returns a double and has two parameters, pointer to the start of the double array P and integer N for length of array." Apparently on the other end of the line was a human compiler that kept rejecting my input and preferred "double F open parens double star P comma int n close parens"!
Thanks! I really enjoy coming up with the themed minigames. The next one planned is a Kanban board minigame ... think of a cross between Tetris and the boardgame Patchwork.
Ha! I did that for a year...and thinking about doing it again. I mean, selling my home and moving to say, costa rica, i could stretch my dollar, dev my heart out, and watch all the sunsets my heart desires
I don't doubt lots of companies do this, but I skeptical about the coding on a whiteboard. They talk in person, sketch out diagrams, highlight important aspects - why? because these things are all easier on a fluid analog medium like a whiteboard or piece of paper.
Actual Code? Not a chance. Portability, Durability, Efficiency - It's the worst of all worlds.
I've worked at quite a lot of companies. Many of us will, occasionally, grab a room / whiteboard to go over something but 99% of the time it's a sort of flow or UX type thing.
I've _never_ seen someone, or have asked to participate in, some sort of whiteboard coding thing. Even with pseudo code. That just doesn't seem helpful to me, at all.
Agreed; when there is code talking, we share screen on a projector/flatscreen and do it that way. How is writing code on a whiteboard good for anyone? Cumbersome and not reusable.
Thank you for the quote pointed. I will remember that for next time! As to you’re point, I think others have covered it well, but I will add my agreement that communicating in an environment where you are comfortable and familiar with the topics ahead of time and won’t have your career prospects put in the line is much easier. Some people like myself simply can’t prepare for an infinite number of possible whiteboard scenarios and look competent.
There's some logic to that. It's the one of the same reasons we have high stakes exams in school: to show what the individual can do under pressure. In an interview setting you can't simulate what it's like to be at the end of a high-intensity sprint, but you can ask the candidate to answer a tough question on a whiteboard and (ideally) get a notion of how they work when the going gets tough.
tremendous respect to air traffic controllers. i cannot imagine the level of professionalism and responsibility these people must have! no amount of monetary compensation is enough, in my opinion.
The pressure of a whiteboard interview has nothing in common with the pressure of a short deadline. It's like testing people for Tour de France by putting them on a unicycle.
And what is it measuring exactly? In real life you don’t get the score. You either did it or not. Nobody cares about how you did it, only answer matters.
And interviews are not pressuring, they are simple. No hard desisions, no anxiety whether it is going to work. You in and 5 hours later you out.
Lame.
I guess it depends on the job, but for outage response, being able to understand a problem, and update or write new code in a reasonable amount of time is kind of a useful skill. Discussing it on a board with others can be useful, if the code isn't complex. Of course, that also depends on the environment; if you're in an environment can't actually push any code, even emergency fixes, quickly, then it doesn't help a lot.
If the existing search algorithm is the cause of the outage, maybe. If I a clever search algorithm will help me filter the data so I can reduce the scope of the outage, yes.
> Whiteboard interviews test one thing well: How well does a candidate code on a whiteboard.
What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
I hate this concept that if it's not typing code into an editor then it's not "real work".
I absolutely communicate with my coworkers using a whiteboard and pseudocode. I reject the idea that being put on the spot is necessarily "artificial". To the contrary, I think that the number of engineering jobs where you can assume you'll never be put on the spot or have to communicate complex ideas verbally or visually is relatively low.
Now if this particular skill isn't interesting to an employer and they'd rather spend the time talking about some other candidate capability, that's a whole different story.
The point is that the interview style in question overemphasizes on-the-spot responses. How is prowess in a game of "gotcha" an important indicator of engineering skill?
I tend to agree with (what I think is) the sentiment of some of the respondents in the article: if your whiteboard interview is a game of gotcha, then the problem is your interviewer, not the whiteboard.
I think the complaint that programmers pretty much never have to think about code under pressure is a fair criticism. The whiteboard really puts you on the spot. But if it's your company and your hiring managers are playing a game of gotcha on the whiteboard, you need better interviewers.
State dependent learning is a bit of a factor too. I can count on one hand the number of times I’ve presented anything on a whiteboard at work, yet just about every company asks these questions during interviews. A screenshare would be a better representation of on-the-spot problem solving, if that’s what you’re testing, because I’m at least in an environment I typically work in.
Whiteboard coding interviews are one thing that I have to go out of my way to practice for and get better at over time when interviewing (ie over the course of a few failed interviews). I don’t feel more skilled or smarter by the end of the set of interviews, but I inevitably do better at these kinds of problems. Should the interview be testing my competency at day-to-day work skills or how well I’ve practiced my interview skills, because whiteboard coding problems only achieve the latter.
> A screenshare would be a better representation of on-the-spot problem solving, if that’s what you’re testing, because I’m at least in an environment I typically work in.
I do coding interviews in the following manner:
I select an interesting problem from a book. I personally complete the problem, measuring what was challenging and the time it took for me to complete the problem. I check the textbook solution, making notes of how my solution differs from the textbook's. If I like the problem, and if it fits with the development role, I invite the candidate to bring a development laptop, I give them a hard time limit of twice the time it took me to complete it, and have them share their screen while they attempt to complete it.
Afterwards we test their solution, I ask them some questions about it. I grade based on code taste, the number of hints they need, and their completion time. All candidates for a position are given the same question. Performance in coding interview is about two third's of the overall candidate evaluation. The ability to sit down and write good code in a time constraint manner is a very important part of being a developer.
This would be a good method if your goal is to hire a carbon copy of yourself.
First, your definition of "interesting problem" probably includes a lot of personal biases. What you find interesting probably touches stuff you've had experience with, but just because you've done work on say, customized implementations of binary search, doesn't mean it's a good baseline for all developers everywhere. Maybe they've worked on different domains or solved different problems and can't relate to questions you personally find "interesting".
It'd be much fairer to select a boring question that involves standard stuff that everyone has to face at some point in their careers, like string manipulation. And even then, it just involve normal transformations ('look for and remove special characters') and not wacky, HackerRank-style rules ('no two bs after cs but before as').
I mostly agree with this approach. I think that you absolutely should be measuring the "boring stuff" because that's 90% of the work by number of hours spent.
On the other hand, I think there needs to be some proxy for "how well does the interviewee navigate complexity?" Because a programmer who does it well is far more valuable than a programmer who doesn't, and there's high variance on this between programmers and between fields of expertise.
The systems design interview sort of gets at it, and whiteboard interviewing often tries to get at it. I feel like I've yet to settle on an approach I'm happy with.
> Whiteboard coding interviews are one thing that I have to go out of my way to practice for and get better at over time when interviewing (ie over the course of a few failed interviews). I don’t feel more skilled or smarter by the end of the set of interviews, but I inevitably do better at these kinds of problems.
You have a good point, I noticed this about myself as well. However, I don't think whiteboard interviews are special in that regard. I noticed that I got noticeably better at the "sit down and code me a simple web API endpoint" interview, and I got noticeably better at the systems design interview, and so on.
I do think that whiteboard interviews tend to lean very algorithm-heavy, which doesn't reflect the role's actual day to day. And practicing algorithm interviews really does feel like an inefficient use of time unless your role truly demands algorithmic proficiency (like if you're a graphics dev).
> How is prowess in a game of "gotcha" an important indicator of engineering skill?
It's not; I couldn't agree more.
In real life when using engineering "skill", not only am I not anxious/stressed by trying to write on a whiteboard, but I'm using my laptop, with my editor, as well as access to man pages, docs, and google to look things up.
In addition to the overemphasis of performance responses, I think "how much do you know from memory with 10 seconds of thinking" vs "how much do you know with 10 seconds of google" is useless. The only fairly complex datastructures I can perfectly describe from memory are the ones which I used most recently...
It's bizarre to me that you think that whiteboard coding tests "communication", in any way. It's not like a presentation, or anything. It's one-sided combat where someone with a secret tries to get someone who doesn't know the secret to regurgitate the secret, on the spot, while pretending that s/he didn't memorize the secret in advance while cramming a great big "Cracking the Programmer Secret" book to prepare for the entire silly exercise in Kabuki theater.
If you want to test communication skills amongst programmers, I dunno...ask them to write something in coherent English. Or here's a crazy thought: ask them to document some code. I guarantee that 80% of working "rockstar coders" will fail (but not before whining, crankily, about how unfair it is that you would actually make them do such a useless thing, since, y'know...never actually documents anything IRL, dude).
Programmers like whiteboarding because it lets them believe that interviews can be reduced to purely objective functions, and because GooAmaSoftBook does it. They're too scared to deviate from the pack, lest people think they aren't as "elite" as everyone else.
Indeed. Or give a talk, or write a design doc, or...so many better options, all of which could be balanced out across a day of interviews.
Think how much you might actually learn about a candidate if your interview process replaced a day of solid whiteboarding with a code review, some pair programming, a session of documenting someone else's code, behavioral interviews, etc. It's almost as if you'd be treating them like a...person!
> It's bizarre to me that you think that whiteboard coding tests "communication", in any way. It's not like a presentation, or anything.
I feel like we're probably at an impasse if I can't convince you that a whiteboard is a decent medium to communicate ideas around datastructures and algorithms, but I appreciate your point of view.
I can only say my experience, which I hope you will take into account as one anecdote. I don't read books about "cracking the coding interview" or do leetcode or hackerrank. I left school 14 years ago or so my stash of cs trivia/secrets/gotcha isn't particularly full. I've done whiteboard interviews where I come up with at best a naive solution.
And yet I've received offers for fairly senior engineering positions at Amazon and Twitter and (hopefully tomorrow) from Google. Most of the whiteboard interview isn't even around the code, although that's a small part. Most of it is analyzing the problem, discussing constraints, discussing tradeoffs, walking through data structure manipulations, drawing arrows and boxes, that kind of stuff. Some code, maybe 30% of the interview. I just keep having this experience where nobody wants to play the gotcha game, they want to know how well I can communicate while solving a problem and they think they get that information out of the interview (I agree with them).
That experience makes it hard for me to understand a viewpoint that believes that whiteboard interviews are about memorizing secrets in advance.
Have you considered that your experience interviewing at these places might have been atypical?
Every set of interview prep material I've seen from Facebook, Amazon and Google recruiters asking me to interview in the last 3 months has included links to things suggesting "cracking the coding interview" is a good start and pointing to similar docs otherwise.
> I guarantee that 80% of working "rockstar coders" will fail (but not before whining, crankily, about how unfair it is that you would actually make them do such a useless thing, since, y'know...never actually documents anything IRL, dude).
Are you sure you're not just replacing one stereotype with another here? Do you think it's fair to say that 80% of people who do well at whiteboard coding value documenting their software so little that they'd be fail at it while disparaging the concept?
I think that was hyperbole for emphasis, not a strong generalization as you imply. I've met and worked with plenty of programers up and down the scale of ability who think "code is self documenting."
There are plenty of things one can gauge candidates on via the whiteboard. System design, application design, psuedocoding implementation (or real coding, if necessary - it should be natural flowing from psuedocode to implementation though), problem solving with visuals to denote intuition...the possibilities are vast.
There seems to be a bunch of ingenuous assertions here that is characteristic of someone who doesn't think one should have to prove oneself. I'm at a FAANG and we don't even ask anything crazy hard for algorithms/data structures like Google or Facebook does, but our interviews still manage to be intense for technical and soft skills - they will challenge even strong candidates and one cannot pass the interview without displaying strong technical acumen or communication skills.
There is so much more to technical assessment than just coding. A whiteboard is just a standard medium for communicating ideas in a collaborative manner that still works the best for any sort of deep collaborative design when compared to any other medium so far.
> What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
Perhaps it would be more effective to have the candidate whiteboard a concept that they are already familiar with, be it a high-level engineering principle or a system/solution they have built in the past. Attempting to solve a problem you have just been presented with AND communicating the solution effectively is a big ask.
That's an excellent idea. I'm in no way saying the existing method is perfect. Just that some of the things it tests around communication and being put on the spot and analyzing a problem in a way that is understandable to the rest of the room is actually a really good engineering skill. There can be other great ways to measure those skills.
I know my opinion is unpopular, but I sometimes do think I've figured out something other people miss. I think when it comes to whiteboard inteviews, candidates are often playing the wrong game. They think it's about gotcha questions and they think they fail because they didn't leetcode hard enough. I don't think that's true, they are just trying to game the thing that's easy to measure.
In my experience with the "terrible" FANG companies, it's not about gotcha questions. I get the offers even though I don't often find a non-naive solution. The people I'm in the room with really do want to see my thought process and they really do want me to communicate the tradeoffs with them. People don't fail the gotcha questions because they don't know trivia or because they forgot a detail from their CS classes. They fail the whiteboard interview because they see an unfamiliar question and say: "I don't know that trivia" instead of drawing out possible solutions and having a conversation with their interviewers.
Thanks for elaborating so well on your initial premise, I think you bring up some good points about how this kind of interactive problem-solving can be an effective tool available to the interviewer. For better and/or worse, there's a reason why it's so prevalent now and controversial.
I'm inclined to describe it as a sort of interrogation technique, you're offering a stimulus (the problem) and aggregating a number of reactions to form an opinion, and open up new lines of questioning. Very delicate work.
I think problems arise when unskilled interviewers present an excessively obtuse problem to the candidate, then read far too much into their responses (this approach is also easily "hacked" by those who've memorised common problems of this ilk). To stick with my interrogation analogy, it's the equivalent of screaming in the suspect's face, noticing that they gulp before shifting in their seat and scratching their face, then deciding that "they must be lying".
They think it's about gotcha questions and they think they fail because they didn't leetcode hard enough.
But ... if you read some of the feedback from interviewers at "those companies" that rely on these interviews, they say the reason they failed someone is exactly because they "didn't leetcode hard enough". It is manifested as:
"Well other candidates got the same solution as you, they just did it 10 minutes faster"
or
"You missed an edge case, even though your core algorithm was correct".
This is a huge issue with these interviews. It's all too easy for interviewers to evaluate candidates based on how fast, correct, neat or "complete" you answer was. It's easy and takes no time.
> "You missed an edge case, even though your core algorithm was correct".
That may have been a reason why I have failed some interviews but I feel like most interviewers are actually good about this and will say something along the lines of "what about this input?" where my code does not work and then I have that "oh shit, that won't work for that" and then I fix it.
Agreed, thats a disingenuous point. "It's not about getting the right answer but the way you think". I've never found that to be true. If you don't get to the right answer, you're gone. If they planned to ask two questions and you only got through one, you're gone no matter how you "thought" about it. A huge part of this is Leetcode practice. If you can't solve most algorithm questions on whiteboard in less than an hour (because you haven't practiced) then you won't pass any interviews.
I try to start there, but most candidates act shockingly uninterested in going into details of their past work. I've directly asked about what the interesting parts of a project were to them and gotten things like "I had to learn ES6 and I hadn't used much JS recently" without much elaboration able to be teased out.
If you can't give me good examples from your past, I'm going to throw my own questions at you. I tend not to do coding on the whiteboard, though. Lots more boxes and lines and schema and system interaction-y stuff.
That's an awesome idea -- as someone who rabidly hates whiteboard interviews, to the extent that I'm looking at the responses in this article as a note of who to consider applying with next, I would love the challenge of "explain a complex concept you're already familiar with". I've never gotten that in an interview before.
What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
This is incredibly common, and you tell the candidate in advance that they will be expected to give an n-minute presentation on a subject of their choosing (need not even be tech) so they can prepare.
Tell me this: when did a mob last ambush you at your desk and demand that you invert a red-black tree on a whiteboard that they happen to have with them right now? I'm guessing that never happens where you work. If it does then fair enough :-)
>I reject the idea that being put on the spot is necessarily "artificial". To the contrary, I think that the number of engineering jobs where you can assume you'll never be put on the spot or have to communicate complex ideas verbally or visually is relatively low.
I think it's safe to say that being put on the spot in front of people you don't know who are judging you is quite different from being put on the spot in a team meeting with someone from product who will watch your demo of a potential new project/idea/concept.
We do whiteboard problems, BUT we also emphasize beforehand that the interviewers in the room are there to work through the problem with them. This is meant as more of a conversation, we're less concerned about getting to the "right, fastest, or most optimized" answer. Not a hard problem, no mindgames, no syntax rules, and no complicated pre-known algorithm work other than an if and a loop. Let's just talk about some pseudocode.
We've never rejected somebody for having a "bad" answer, because they were able to at least talk about what they're doing. We have had people flat out refuse to even try, and that seems like a pretty big red flag.
That sounds like a much better way of handling a whiteboard problem.
In my most recent on-site interviews, my interview would be in a room with full floor to ceiling glass windows, the next interviewer would come in, ask one or two personal questions, and then either cut me off after a couple minutes or just simply say "we're going to move on to the whiteboard now, do this". Meanwhile people are walking by staring into the room, looking at the whiteboard, etc.
Which is just too rigid, IMO.
When I interview a candidate, I want to know that they'll be able to work together with myself and my current teammates as a team. So I would rather do something similar to what you described.
Every time an interviewer has told me something like this, they then nitpick syntax and appear to be primarily concerned with "does my whiteboard code compile" sorts of problems. And getting stuck / asking for help feels like I get docked for getting stuck. Same with less optimized. So it's hard to trust such an explanation - clearly my interview would be better if I came up with the perfectly optimized solution, or gave a lecture on the options rather than trying to discuss options.
So it seems to me entirely reasonable to still be nervous.
Mind you, I don't have a better answer, but I don't think just explaining that suboptimal is ok and it's a discussion really helps.
This is generally the same experience I have had. I'll even talk through the assumptions (e.g. "Can we assume I know how to do argument checking?") and then after I've done the heavy-lifting of the problem-solving I'm now discussing how I didn't do something I brought up as an assumption.
Also, generally, if I'm whiteboarding in front of my coworkers/team/etc. I'm discussing something I've thought about for more than the few moments I have to digest the question while standing at the board during an interview.
> Can we assume I know how to do argument checking?
I would probably question you at some point about a particular value of an argument and almost all candidates will get points for thinking out load that of course a NULL would get it crashing and that a simple check could stop it. Then it leads to discussion about how it should be handled. Sometimes a NULL means the program should cast an exception, other times it should just return without any side effect.
IMO the hate for whiteboard questions is really a symptom of poor interview technique.
Maybe I was just lucky, but I interviewed with google and dropbox recently and both companies gave me laptops to type in, did not ask me once about if my code compiled, didn't comment on my style, nor did I have to perfectly know the standard library (for google, they told me to just guess at the random library functions and I also forgot some of the functions names for a set. For dropbox, one of the guys re-explained to me semaphore functions and how they worked since I forgot since it had been so long since I used them in college). I passed both.
Granted, I will give you that optimized code has definitely been important. One of those interviews even had me use a bit array (at least I think it was that) to efficiently a store a list of booleans but at least they gave me the relevant function calls to use an already implemented one.
All these discussions conflate a related, but separate problem with tech interviewing: most tech interviewers don't have much experience interviewing, and have never had good coaching on interviewing, and as a result mostly suck at it.
Picking on whiteboard code for not compiling is not good interview technique. But it's also a common enough failing that it's hard to say a bad interviewer in that regard means a bad workspace, unfortunately.
Bad workplaces are so common though that it’s a perfectly good heuristic to guess that if a place asks you to do whiteboard questions it’s a bad engineering workplace. You’re not missing much by passing, and the heuristic probably won’t be wrong much.
Is it even disputed anymore that the experience of working at the big tech companies is abjectly awful? I mean, you gotta work somewhere and they “are fine” and you might as well try to get paid well, but that’s a far cry from a workplace that stands out as good.
The only place I’ve worked that was “good” instead of “treats people poorly but where else are they going to go” was a small, boutique financial company.
Small companies that are randomly led by non-dummies are the best, but are exceedingly rare. Out of the vast majority comprising the other stuff, it’s mostly all so miserable that you’re doing yourself a favor to skip it unless the pay is some dramatic increase for you or you otherwise have some emergency requiring you to take some job.
Of course it’s disputed. There are lots of happy employees of tech giants. Any company with thousands and thousands of employees is going to have a wide variety of teams of varying qualities.
(And yeah, the pay makes a difference. Not a lot of places where you can make a few hundred thousand dollars a year.)
I work for a large tech company and don’t know of any company like that with a large number of employees happy with their company.
(I’m making a distinction between the idea that a job “is fine” where one “is happy” merely because the culture is at least not worse than elsewhere while the pay is better vs actually feeling positively about one’s company’s culture and corporate behavior.
For example, my friends who work at Facebook are “happy” with their jobs, mostly because of pay and because they know if they switch to other companies, politics, corporate misbehavior, etc., will just continue. But these same people tell me frequently how sad, upset, soulless, disappointed they regularly feel because of their employer.
That feeling I think is extremely widespread in tech, almost all employers. That’s the only part in my viewpoint that matters for whether someone “is happy” at their job, and it’s that type of bad culture I think can be easily flagged by little stuff, like bad whiteboard interviews.)
I blame the modern attempt to turn software developers into assembly-line workers. With sprints and tasks and momentum and a million tiny cuts that take the innovation out and replace it with anonymous criticism and process over product.
I largely agree. Modern companies are set up to cargo cult lots of process and pay lip-service to innovation while taking extremely risk-averse and conservative approaches to practically everything they do.
Nobody has any idea what’s going on; nobody has clarity. And when someone does have clarity, middle management is only interested if they can set it up like a battery to harvest credit from. If they can’t, their incentives are usually more aligned with intentionally abusing process to “manage out” innovative people.
I wouldn't go as far to say whiteboard interviews == bad company. I will evaluate the company if a whiteboard interview is run poorly however and consider it a bullet dodged.
going through / having just gone through this process this is where i've basically ended up as well... crappy interview experiences are universal enough that they don't disqualify a company automatically (though obviously i'm not terribly fond of them :-).
the one thing i'd add is that i do take particularly thoughtful / empathetic interview processes, interviewers who actually know and understand the question (rarer than you'd think), etc. as a fairly strong signal, as those sorts of things don't just happen by chance-- they are the byproduct of a culture of thoughtfulness (and thoughtful people tend to be thoughtful about most things).
I caught an interviewer red handed once by handing him the marker and asking him to show me. He tried and failed and then his coworker tried to fix his mistake and failed.
By the time we get to the whiteboard problem we've already assessed technical skills as well as we can. The whiteboard problem is to test their fit on the team. It has worked well for us and certainly weeded out some people who could have been problematic culturally. Is that a "failing?" Maybe we lost some more technically skilled individuals that way, but our team isn't exactly lacking because of it.
> clearly my interview would be better if I came up with the perfectly optimized solution, or gave a lecture on the options rather than trying to discuss options.
Right. And a competency based interview goes better if you have a perfect example for every question. But it isn't an auto-fail if some of your answers are mediocre
Personally though - I'd rather hire someone who is able to properly communicate their thought process over an imperfect solution, than someone who was unable to discuss/explain the perfect solution they scrawled down.
When I give interviews candidates often say they don't remember some API. I tell them I don't care and they should make something up and I'll figure out what they are doing from context. The only time this burns candidates is when they clearly don't understand an ADT well enough to give it a sane interface.
As a candidate, to me, that sounds a lot like, "we're going to do the same things we would normally do in a technical interview, but we're going to evaluate you based on random, arbitrary, subjective criteria that you will never know about rather than your actual level of skill."
I'll bite: yes, if there's rapport and the candidate doesn't feel caught off-guard or needlessly pressured. If it's whiteboarding like you'd whiteboard while on the job with your coworkers, that's a good idea that can tell you a lot about a candidate.
Unfortunately, in my experience, that doesn't seem to happen very often. A lot of people seem to treat it as an adversarial process or expect, for lack of a better description, a "performance" over contrived problems.
I've had my share of the second kind, which were without exception some of the most unpleasant interviews I've ever gone on.
> We do whiteboard problems, BUT we also emphasize beforehand that the interviewers in the room are there to work through the problem with them. This is meant as more of a conversation, we're less concerned about getting to the "right, fastest, or most optimized" answer. Not a hard problem, no mindgames, no syntax rules, and no complicated pre-known algorithm work other than an if and a loop. Let's just talk about some pseudocode.
We do the same thing as well. As long as the candidate can explain their abstractions in pseudo code we are good. We also keep to a 1-1 between interviewer and candidate so that the candidate isn’t intimidated.
I’ve interviewed at Facebook as well and although the interview was not successful I have never been nitpicked about syntax and compilabiltiy/runnability of my whiteboard solutions. I came off with a very healthy appreciation of the way they try to engage the candidate and reduce stress.
You are asking the candidate to compartmentalize their thoughts to solve a toy problem during a period of time when they are attempting to put various nascent clues about your organization together in order to form a better understanding and to identify how they can provide value / fit into the flow.
Fizz-buzz tests have their place, but I think an onsite interview is past that point. In general I have found that white board exercises don't allow me to either understand or convey how I can apply my expertise to provide value for the organization.
Agreed. For us this is a communications/team fit exercise and not a technical exercise. It has done a good job of filtering out bad fits, like the guy who refused to talk to the women on our team regarding anything technical.
> We do whiteboard problems, BUT we also emphasize beforehand that the interviewers in the room are there to work through the problem with them.
Except at the end of the day this is an interview, you are judging their answers and evaluating their performance. I have been part of white boarding sessions like this and it does not reduce anxiety and still feels completely removed from what I actually do on a daily basis.
Not judging their answers. The only thing we're judging is their ability to communicate in a group setting similar to what happens on the team on a regular basis.
We at least on a weekly basis somebody has some small problem they need help working through, it generally ends up being diagraming or pseudo-coding to figure out the best approach. If somebody can only participate by hiding in a corner typing out code, then the process worked for both sides.
I highly doubt a normal engineer is going to be giving a demo to their team with high stakes in the first three or so months. I understand if they're possibly a senior engineer, architect, or lead of some sort because they were probably hired with a plan to spearhead a specific project.
I'd say in the average case a new engineer will have enough time to at least break the ice with their new team before being put into a high pressure situation.
I gave a demo of my intro project after a month to my 100 person org at my last company and I was a new grad. In fact, when I was an intern, all interns did project demos at the end of their internship.
I agree with the sentiment but find most of my whiteboard time concerned with a much higher level type of problem solving than writing lines of code or absolutely correct algorithms. I also mostly do it with colleagues I know on shared problems we are working cooperatively to solve. Where I’m using one with strangers I’m much more often working as an expert in an explanatory manner. It feels quite remote to my experience of solving an unknown problem in an interview situation.
Clear and respectful communication is one of the most important skills for an engineer to have.
If you read the second half of my answer in the article I go into the communication side of things. In my experience there are better ways to evaluate a candidate's emotional intelligence and social skills than asking them to code in front of someone.
I'd say you should stick to your guns - I like your approach. Interviews are inherently adversarial: "are you good enough for us to hire you?", "are you better than the other 5-10-1000 candidates". As a result there's a built-in bias which makes whiteboarding a bad fit.
I've written code on white boards at work, and I've discussed about code on white board with colleagues or managers. It's not the same thing as during interviews. The schedule is never as tight, scrutiny is way laxer and the overall mood is completely different when I'm working with colleagues or even bosses.
I kind of understand why the BigCo's do it (they need a super rigid filter since they have so many candidates), but in their case they could select for people with the best pink leotard on interview day and they'd still get decent programmers, because their companies are in such high demand and they generally already control their markets (software, natural monopolies, etc.) and can afford to pay a ton so the competition would be cutthroat anyway.
Coding on a whiteboard while being questioned by two or more interviewers isn’t really a true representation of a work situation. It’s always more stressful and some people thrive on it, some people don’t. Im not sure what the better alternative is to demonstrate verba/technical communication skills however. The problem is, it’s hard to figure out if someone is a good fit with just a couple of hours of vigorous questioning. Maybe that’s the problem. I’d say I’ve worked with plenty of colleagues where it took me weeks or months to fully appreciate how extremely valuable they are to the team.
Then test that. Having someone code on a whiteboard, even if they are explaining what they are doing while they go along, is simply not the same thing as being good at using on the fly visual aides while communicating. In any case, that isn’t coding either.
That is the same as saying who can code the fastest rather than who does the best job. But at the PhD level there will be a presentation given regardless.
I communicate my ideas using a whiteboard too. But I never produce those ideas using a whiteboard in front of my coworkers under pressure of being fired.
Well this is hardly the fault of white board interviews. That’s just interviews. Any interview (regardless of the style) is going to be in front of “coworkers” and “under pressure of being fired.”
My experience agrees with this. I am a different, significantly less comfortable person to be around when strangers are asking me so many questions about myself. I think it’s just about the worst way to get to know me.
When I get hired this way, I always feel a little bit of guilt because I know they actually hired somebody else.
What can be done? I’m not sure, but I can say one thing: my level of discomfort is multiplied by the number of strangers in the room. Does this ever get considered with interviews?
I am far more comfortable coding in front of 1 or 2 people who each give me a little bit of background about their coding experience; just so I know. If they are experienced engineers, it’s probably not going to change what I say aloud, but it just makes me more comfortable. I guess more overlap of technologies in our background does help. It’s nice to not feel pressure of worrying if my solutions reflect general programming conventions enough to be language agnostic. I have never really used Java, for example.
The "produce the ideas during the interview" part is the fault of whiteboard interviews. There's nothing that says an interview should cover brand-new material as opposed to material the candidate is supposed to be familiar with.
If communicating ideas is part of your job, I'd bet any amount that you decide what ideas you want to communicate well before you do the actual presentation (or other physical delivery of the ideas).
I don’t think there’s much criticism of those types of whiteboard interviews. I’ve had programmer interviews that involved sketching out a system architecture or database scheme on a whiteboard, which I think is perfectly reasonable.
It's one thing to do it with colleagues for whom you have rapport and the risk of saying something wrong is low. It's a different thing when doing it around total strangers and a new job is on the line.
Ugh, communicating with a whiteboard has no overlap with developing a solution to a completely unexpected quiz (if you didn't cheat) on a whiteboard in front of someone scrutinizing your every move with your future career on the line.
Whiteboard interviews rarely test the ability to communicate on a whiteboard because the interviewer knows the answer they are looking for and the presenter does not.
If you defend whiteboard leetcode using this excuse you are deluding yourself and you are part of the problem. The only thing the modern FANG interview hires for is people that can solve algorithms problems in a silo under pressure. No checks for software engineering skills, collaboration skills, testing skills, reviewing skills, and on and on...
The whiteboard interview is why so many engineers unexpectedly suck.
> Whiteboard interviews rarely test the ability to communicate on a whiteboard because the interviewer knows the answer they are looking for and the presenter does not.
They you are asking the wrong questions. I use whiteboard for developers, and for junior developers I give them a simple task such as reverse an array, turn a string into a palindrome (and explain what it is if they don't know). If they want to be a developer they must be able to solve those simple tasks. I don't really care how, as long as they think loud. Then I use it as a starting point for enhancement discussions, perhaps stack/heap questions, recursion, assignments ect. During all this I coach them, teach unknown concepts in simple terms - this is what they can expect when they ask their mentor a question so in that sense they get a feel for us too.
Some argue that it's still not good, and that we're filtering out people that work best alone. That is true, but that is by design. We work as a team, having 10 individual developers not working together unless forced is an architectural nightmare.
If you think someone looking over your shoulder while you solve a problem with your future career on the line is anything like "working as a team", I weep for whatever work environment your developers work in.
So a simple problem solved while communicating, asking for clarification, and telling your thoughts is "someone looking over your shoulder while you solve a problem"?
I'm not asking them to impelment van Emde Boas tree because I spend countless of hours implementing and analyzing it getting my degree. I ask them to turn a string into a palindrome, or return the sum of all even index in an array of ints, ect. They're free to ask all the questions they like. It's a litmus test to see if they know simple programming, akin to seeing if a contractor knows how to drive in a nail using a hammer.
Is it high pressure while your future career is on the line? Absolutely, but they are not special snowflakes, and it's the case for any interview for any kind of position. I'm not going to throw thousands of dollars after someone just to figure out that they have in fact never written a functioning for loop.
I always thought it would be a good idea for the interviewee to also come up with a question and then have the interviewer try and solve it with their guidance. I imagine it may be more helpful than the other way around.
If the interviewer leaves frustrated because they couldn't solve anything and it was a mess, or the question was formulated terribly, or the interviewee was under prepared, etc etc, then it provides a lot of insight into the capabilities of the interviewee. It almost provides more insight into the qualities required of a developer than just putting them on the spot.
>> What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
In that case I would recommend reading the candidate's resume beforehand, look for prior experience publishing, presenting, or teaching and ask them about it.
I think now a days, selection through the whiteboard process simply comes first to how much you grind leet code and the like.
Ironically this leads to min- maxxers who ignore a good chunk of their courses simply to master leetcode. My smartest peers are ones who work on interesting projects and research, thus never finding time to grind interview questions.
Then there are also companies that expect your white board code to compile and have perfect syntax, which imo is very unreasonable.
I agree, and as an employer I find a happy medium is to test on a whiteboard, but be completely okay writing filler, a comment saying "code for this goes here", etc. So long as they're building the correct general approach, that's all I need to see.
Whiteboard interviews where the interviewer says "it's actually `.toLowerCase()`, not `.lowerCase()`, you fool!", or "oops! You forgot to `position: relative;` the containing element that you `position: absolute;`d below!". you're not testing anything useful. Maybe if we were still coding everything on punchcards and using massive user manuals, but docking applicants for things their editor would highlight or one ctrl-r on the page would find is ridiculous.
VPE at Eaze here (the original author of the quote above) -
You're 100% correct. We give our candidates the choice between working through a problem at the office (on the spot as you say) or doing a take home project asynchronously. Not everyone has the time to do homework, but not everyone wants to do an in person coding interview either. We try to stay flexible.
How do you compare the resulting apples & oranges? I can do a lot more at home (especially if I cheat on time, which everyone who really wants your job and can afford to do will do)
We're less concerned with the resulting code than we are about your ability to explain the code that you wrote and your ability to prove to us that you have good judgement when making technical decisions.
Also, we're growing fast enough that we're rarely comparing candidates. If we can hire both the apple and the orange (assuming both candidates are evaluated to be good for the role), we will.
I'm actively looking/interviewing and really like the "cut of your jib". (both your answers here, plenty of aspects of Eaze as a company, the prospect of moving back to SF...)
For both the sr backend and devops positions, how important to the roles is familiarity/experience with your technical stack? I might have some of the skill set/experience you're looking for -- sr backend engineer, sr systems engineer, devops are all positions I've held -- but honestly I don't know JS, .NET, or Chef. Bluntly, I don't want to waste your (or my) time. ;)
<tiny>Also, I wish HN supported private replies</>
We care less about your particular technical skills than we do about your generalized engineering skills. Specific stacks are easy to teach; fundamentals, emotional intelligence, and social skills are harder to teach.
In fact, if someone is too into their stack, we see that as a bad sign.
So I'd be super excited if you decided to apply :)
Do you want to work for PlayStation Network team? We have plenty of open positions from DevOps to server or client side developers. And variety of work also there - commerce, social, video streaming, distributes databases, big data, ml.
Thank you for the response. I appreciated your take on the situation in the original article and I am glad to hear that you accommodate developers like me that don’t perform well for strangers on the spot. This type of thing really pushes a company up the ranks if I’m interviewing with them.
I think this is a great approach. It keeps the company flexible to hire a broad array of talent under differing circumstances while also letting the interviewee play to their strengths.
Really? I have multiple in person discussions per week about how we are going to tackle some problem where everyone chips in and those are just the weekly problems we are working towards. There are also the much larger design docs and while some of the feedback on those happens asynchronously, a lot also happens in real time during meeting reviews. Oh, and both of those problems are decently well covered by your standard interviews (the former generally being data structure/algorithm design, the latter generally being system design).
A team I was on at Google (XLA, the compiler for TPUs) used whiteboards almost daily to express algorithms to each other. Definitely helpful for us, but I can see that just being a single team's dynamic.
No comment on it being an effective interview process.
I've aced every whiteboarding interview when the problem was the same or similar to some problem I solved before, and I failed pretty much every interview when the problem was new to me (except when the problem was trivial, ala fizzbuzz). I think that proves how whiteboarding just tests whiteboarding skills and nothing else.
It's funny, when I saw the title I immediately thought "Hell yeah, whiteboard interviews are absolutely necessary". As I read the article, I realized it meant actually making people code on whiteboards, which is absurd. I've always used whiteboards for high level design and architecture questions, and there mostly to aid the candidate in organizing their thoughts. If I want to see someone's code, I'll ask them to provide a code sample.
Nay. I do not understand the obsession with whiteboards. I've had coworkers that went gaga when they discovered that whiteboard paint was a thing, and they could cover their office in whiteboard surface. Doodles and doodles and doodles all over the place, but not very much working code ever makes its way into production from those offices. Lots and lots of movement and noise and grandiose planning, signifying nothing, too often.
Lock me in a dark closet under the stairs with just the glow from a couple LCDs and throw pizza and requirements documents through the slot, like you're feeding the Rancor. /s
Yeah, but that's also stupid. All to often it comes down to "Make standup a completly working client server system for this arbitrary problem from scratch in 30 minutes, starting... NOW!"
You end up spending all your time dicking around with missed semicolons and looking up library routines and shit like that. Sure, it's all stuff you have to do as part of a job, but it's also all trivial stuff.
The last time I had to do this, I spent time googling how to open up a port in python, and then read from stdin. It's just a waste of time.
Find a reasonably self-contained problem that doesn't involve too many interactions with the "environment". For example, ask the candidate to refactor an existing piece of code.
My favorite interviews have been those that had a small project as a screener and then proceeded to have you modify and/or explain decisions regarding the project at the in-person stage.
I haven't had the opportunity yet but I really want to take a piece of code from our system, de-optimize it a bit and add some bugs, then give users a unit test around it.
Most of what we do as engineers is trying to read code to figure out what it's doing, how to fix it and how to enhance it. Going through real code with a candidate and assessing their ability to understand brand new code (and what questions they ask) seems like it would give good insight to how they would perform on the job.
In one of the best interviews I had, the interviewer had printed off an actual (undocumented) class in one of their projects (I got the job, so I confirmed it later), then asked me to tell him what it did, and if there was anything I would do to improve it. I think he might have added a couple of logic or syntax issues as well, but I can't quite remember anymore.
I've been on dozens of interviews, and that's the only time I've ever encountered that, but I think it does a really good job representing the job, as that's invariably what I'm going to have to do when you hire me, is sit down and familiarize myself with the code base.
He also asked me a handful of questions, said "Okay I know you know enough to do the job, let's see how much you really know," asked me a bunch of really low level stuff, and proceeded to teach me concepts when I told him I didn't know certain things. I walked out of that interview knowing more than when I walked in, something else that has never happened in all the other interviews I've had (except maybe I learned a new term that I never heard before that I was apparently supposed to regurgitate back because that's what was written down as the answer on HR's answer sheet).
Coding challenges either take home or live session.
Most whiteboard questions are always some esoteric algorithm any engineer worth their salt would in real life would look up before ever consider using to verify their own understanding before using it.
Putting someone on the spot saying whiteboard x is not ok.
However if the test is to see if they can take part in a meeting, then by all means do a white board, but ask them draw a workflow diagram or some other 40,000 foot type of diagram. Do not ask them to put code on a whiteboard.
It's not just that real working engineers would go look it up. It's that the culture around these "prove you can code" problems got into an arms race, where companies kept trying to come up with ever more difficult problems and stringent requirements on solutions. It finally reached the point where, at some companies which use them, the bar for "can you code at all" is actually "given a problem you've never seen before, can you match or beat the absolute best solution the world's best CS researchers have ever come up with, in twenty minutes, on this whiteboard". If you aren't able to, they label you a bozo who can't even write a for loop, and pronounce you unqualified to do any type of programming, and pat themselves on the back for having kept "fake coders" out of their company.
(and if you wonder how anyone ever gets hired there, the answer is: by cheating. If you're going to interview at one of those places, odds are their interview problem and an accepted solution will be online somewhere, so you just go look up and memorize)
Yes, my favorite was a while back, someone's list of algorithmic interview questions got posted on here, and a couple of his answers were wrong! And he STILL insisted they were good interview questions!
Our field is full of some of the smartest morons on the planet.
The best I have come up with if you are requiring them to write actual code is to give them a problem to solve for with code prior to interviewing. Have them bring that code along and then have them explain the code and why they did things the way they did. It is not 100% verifiable that they wrote the code, but being able to talk to why things were done the way they were tends to be good enough. I have also found that understanding how people approach life and problems in general are more telling of how good they will be than actually evaluating their coding skills. Monitoring coding exercises will weed out many good candidates as well as the poor ones in my experience.
During my recent job search I was asked many questions I didn't know the secret algorithm to, but if those companies had given me the problem ahead of time and allowed me to research it -- just like what happens in real life -- I could have coded solutions to all of them.
And if you're worried about people just plagiarizing existing solutions, have them talk through it, as you say. Should be fairly clear if they understand each line or not.
I really wish we had multiple formats that the candidate could choose from:
1) Whiteboard
2) Review and technical discussion of existing code or projects
3) Paid take-home project
4) Pair programming on live code
5) Technical discussion only for rare cases for those Top Secret/NDA candidates that really can't discuss their past projects but are also time constrained
Some people do actually like to whiteboard. Keeping it around is kind of like legacy support for people who practiced hard to excel in those interviews.
This is pretty much the union of technical interviews that are well-known. If you switch out a technical interview format and can't get meaningful insights into a candidate, you're testing for the wrong things.
We could be like real Engineers (civil, certain classes of mechanical and electrical) and require a license. One 8 hour exam covering fundamentals and application of them along with simple project management and soft skills.
Depends on "why" you are recruiting and what the end result of employing someone is.
So much of the technical hiring process tends to be about trying to find people that match what you expect people to be. Which might be fine, you need to fill a particular position which requires a specific set of skills. You need to validate those skills. You do that by doing something semi realistic while trying to minimize the pressure of an interview situation.
But, then there is the other kind of approach where you are looking to see what the candidate can offer, trying to find people who know different things to you and do things differently than you, in this scenario you are more inquisitive and let the candidate direct the interview process and get them to show you things that sound interesting (including coding).
The first approach is all about finding specific skills.
The second approach is all about being clear about the values you are looking for.
The second approach requires more skill and more time while wearing the hat of a technical recruiter.
You may need a little bit of both approaches.
So the big thing is to be clear about who you want to work with at the end of the process.
Just talk. Have a conversation. There is no point in trying to put someone outside of his comfort zone. You should take him to the very center ot it, and then see how well he can managd.
Any question that the interviewer already knows the answer to is not a good question.
Honestly I’m not afraid of whiteboards as earlier in the career. The interviever is whether some fresh graduate smartass trained in a narrow domain and it’s easy to route the discussion out of their comfort zone, or someone with particular problem - well, I’m not able to pull out immediately answers to the world problems and if this is the case I’ll just admit it. In the worse case I’ll not get hired - I hadn’t got hired too many times in my life to care.
It seems like there are two things at play - whiteboarding interviews vs. using whiteboards in your day-to-day job.
If the test is to see how well a person works in a whiteboarding interview to solve a problem, then it should be a simulation of collaboration, while trying to avoid the Clever Hans effect.
Otherwise your DP whiteboarding problems are worthless.
Nay, unless the candidate wants the whiteboard. Even Google doesn't mandate whiteboard anymore: you can write your code on the provided Chromebook. Some people choose whiteboard, though, for reasons unknown. I think whiteboard (or a sheet of paper) is reasonable for what it gets used for IRL: for sketching out what you're going to do, by drawing or writing very high level description, but it sucks for everything else.
I did a interview with Google last summer. They didn't use a whiteboard, but instead had me code in a shared GoogleDoc. Pretty similar issue, limited time, somebody staring over your shoulder, not your usual development environment. Even more frustrating when you _know_ they have a web-based IDE for internal software development but make you write code in a GoogleDoc.
I did one about ten years ago, two of the guys asked me to do the program on legal pads, two on a whiteboard, it was kind of funny because the Google HR person gave out a pdf explaining they were going to use a whiteboard and to practice on a whiteboard before coming on site.
With doc you don’t have to stare over the candidate’s shoulder. The interviewer can just share the doc with him/herself and watch as you type. That’s what I used to do. The main goal is actually to make sure the candidate stays on the right track. Some folks are prone to veering way out into the woods otherwise. Unsupervised, they would just run out of time.
Was that an on-site interview or a technical interview over the phone? I did an interview at Google about 7 years ago and that sounds like the Technical Interview.
When they brought me on-site after that, I interviewed face to face with 5 or 6 people in a row (except for a lunch break), and every one of them asked me a two or three questions and then switched to tricky whiteboard problems.
I remember one was 'write a function that draws a 2D skyline from a bunch of 3D buildings', which I got totally stuck on because there's so many possible configurations of heights and widths and how they overlap that complicates it, so I didn't really know how to properly solve it.
I didn't get the job. I thought the experience was interesting at the time, and basically treated it as a free tour of Google and a chance to eat In and Out burger again, but I don't want to go through that grueling exercise again (or spend a ridiculous amount of time prepping for it... as it was I spent a solid two weeks studying for it the first time around), so I've said no to Google recruiters every time they've gotten in touch with me since.
I might have given it one more shot, though, if I were willing to move to Silicon Valley at this point in my life, but I just bought a pretty decent house and I'm not really wanting to give that up to take out a mortgage five times what I have here just to live in a smaller house in the Valley and be able to say I worked at Google.
I may be an outliner but I prefer them. Mixing presenting with randomly writing things is better than talking for an hour. Makes me less nervous doing two things at once.
Whiteboard interviews would be fine if they weren't taken so damn literally. Writing code on the board is dumb.
Spitballing solutions to a problem collaboratively and with access to reference resources (a nice little programmer's bookshelf in the room) followed by actually coding a little bit up seems that it would be way less stressful and way more likely to give you a sense as to what someone is capable of and what they're like to work with.
In general, an interview is to understand what a candidate brings to the table. Therefore, one of the best ways to assess "glass half full" is to talk about their past work. The interviewer is a subject matter expert who can definitely ask pertinent questions about the candidate's work. In many cases, more so for scientific positions - the candidate's work is publicly available.
Currently, many interviewers go for - as I like to call it -"trivia questions". Personally, it just doesn't seem appropriate because "Everything may be obvious, once you know the answer". In some such interviews, I've asked my own set of trivia questions to the interviewers. The interviewers weren't pleased as they couldn't answer some questions for which I knew the answers beforehand.
Unfortunately, some companies' HR departments actively encourage "trivia questions", because they ask interviewers to ask every candidate the same questions so that a "fair" comparison between candidates.
I think you've touched on a crucial point here about making the comparison "fair". In my view, I find the "trivia questions" approach to be an abdication of responsibility for the difficult, nebulous task of assessing a candidate's abilities. It gives you an objective measure by which to compare candidates... even though the measure itself is usually arbitrary and irrelevant.
The only "trivia" questions I like are ones that don't necessarily have a single "correct" answer (i.e. not "guess what I'm thinking") and can actually lead to further discussion.
For example, one I like to ask (in person, with no computer access), ostensibly to gauge familiarity/intimacy with the Unix CLI, is:
"For each letter of the alphabet, see if you can name a Unix command (anything one can type at a shell prompt that won't return a "command not found error") that starts with that letter."
I preface the question with something suggesting that it's supposed to be fun, and I encourage skipping letters and coming back to them to avoid pressure (nor do I write down the answers). Most people are surprised at how many they remember without much thought.
Although not particularly deep, further discussions can arise from differences between shells with respect to built-in commands, as well as differences between OS flavors. Also, I've discussed how a somewhat new language like Java has made one of the otherwise difficult letters much easier.
Regardless of the trivial nature of the question/exercise, it provides a spectrum of jumping-off points for deeper knowledge discussions, all of which are signaled/chosen by the candidate.
> "Whiteboarding" has become too large of an umbrella term, one that groups together everything that's wrong with the interview process.
That general premise is right, we've lumped too much into the term "whiteboarding", and it's worth unpacking into the different interviewing practices we may like or dislike. Some things people don't like have nothing to do with an actual whiteboard (e.g. testing esoteric CS knowledge or deliberately creating a stressful environments), and some things that involve a whiteboard IMO are ok (e.g. sketching out a system diagram).
Yes. The whiteboarding that gets hate is effectively the one where you would be asked to write a depth first traversal or similar, in some verbose language, correctly. On a whiteboard.
That is STILL a thing (judging by HN) and it's useless. I don't think anyone would ever think it's a bad idea to have a whiteboard on hand if a candidate is asked to describe how they would design some nontrivial system. Being asked to describe something without a whiteboard in that case, is even harder.
The objection you often get for the latter kind is that it tends to benefit the extroverted kind that likes to babble and sketch, whereas a candidate who would prefer ten minutes alone with a pen and paper might not do so well. I think are some merits to that complaint, but I also know from experience that being able to babble and sketch is really important.
They're fine but they need to be understood properly. Whiteboard interviews don't really capture how well someone codes. They are better at capturing thought process, high level understanding, system architecting ability, and a candidate's communication skills.
If you want to see how well someone codes, look at their code. Or ask them to write something for you. Don't do a whiteboard interview and think it is the end all be all. Take it as a data point.
To me, it's not about the code on a whiteboard vs a computer and correctness.
It's about the thought process when breaking down a problem and the ability to organize and communicate their thoughts. It's also about exploring a problem space. How carefully does an engineer consider edge cases? What kinds of things are important to them?
A whiteboard to me is a much simpler and accessible medium through which to explore a problem vs setting up an environment and having to type code and dealing with all the minutiae that comes with actual development.
Of course, it isn't the ultimate method of evaluating a candidate but it's certainly valuable.
I think it's important to measure what's actually relevant to the job. If the job entails large periods of coding on the whiteboard, then yes, it's a perfect metric.
If, however, the job requires careful consideration of dependencies, time domain, and, yes, edge cases - then you are NOT testing for that.
Lock the candidate in a room and give a few hours to let her to produce the result would be a better measurement.
When I do whiteboard interviews I want a little bit of code because I'm stunned by the number of people who can't write a loop. But mostly I want to see the design process. I ask for a simplified version of a real feature that has an algorithmic core. This lets us code something but lets us discuss all sorts of edge cases and real world complexities that would come up in a true deployment.
Lots of interviewers suck. That isn't a property of whiteboard interviews.
>I'm stunned by the number of people who can't write a loop
I won't say this is you, but it's funny the number of people who will interview and say something like this and then still give a non-trivial problem/question to whiteboard.
Reminds me of a guy who interviewed me several months ago at a place in Mountain View (not Google, but another).
He started by asking me to create an object with a couple attributes. Then he slowly began to add to the problem by asking for X, then Y, then Z.
Little by little, the code became more complex because he wanted iteration, etc.
He would say "I just want to know that you're able to code."
Later on, I found out he didn't like what I wrote, meanwhile all the basic elements of coding he wanted I was able to add without any problems and I met his specs.
If there's a "communication issue" in whiteboarding, perhaps the interviewees shouldn't shoulder all of the blame. Perhaps the interviewers need to temper their expectations and learn to communicate them better, because obviously there is some disconnect in a situation like this, and it's not the first time I've seen this before.
If the goal is to test the thought process why not give interviee a self-directed project then ask them to present a 10 minute talk about it? You get some code written to verify the engineer can build something, plus you get confirmation that they can organize their thoughts and effectively communicate them to another engineer.
Isn't that a more realistic scenario? How often do you give an (employed) engineer a task then ask them to immediately explain what they'll do to implement it?
It is more realistic, but you can't really expect someone to be novel, and if they're not being novel, you should expect someone else to solve the problem. So this method has problems, and it kind of demands payment for results.
I'm confused by what you mean by self-directed. Would you give the person a specific problem to solve? Or would they be coming up with both the problem and the solution?
Simple, I want them to work on the same topic as other engineers I've interviewed, so I can benchmark them against each other, and ensure the topic is close to what we work on, and not just what the candidate happens to know a lot about.
I've seen a respected engineer that I worked with for years, be told by an outsider that he was incompetent because he couldn't fizzbuzz an interview. Yet, they had no problem acquiring the project he designed and wrote.
"Why couldn't he do fizzbuzz with someone else watching his every move while he wrote on a blackboard with no interpreter to test his solution or catch his mistakes?" seems like the better question.
Many people believe that it is completely unreasonable to ask an interviewee to write code on a whiteboard at all. I disagree with such an absolute position. I do think it can be a reasonable thing to do, but only if you are asking the interviewee to code something really simple.
When I say simple, I mean really simple, like FizzBuzz simple.
There are interesting questions you can ask that are really simple which are not FizzBuzz. I've used the Fisher-Yates shuffle algorithm and more recently Selection Sort. I should stress in both of these cases that I carefully explained the algorithm before asking the interviewee to code it up. Even though these algorithms are quite simple, I do not expect the interviewee to have memorized them.
I do expect someone who is a professional software engineer to understand the algorithms after a decent explanation and then write code which will implement them.
I also don't necessarily expect the interviewee to get the code exactly right. I just expect them to demonstrate that they could write correct code given enough time and access to an actual computer to test the code on.
If you're looking for them to do something _really simple_, like FizzBuzz, then you're doing it wrong already. This is something that could have taken place over a technical phone screening.
Once they hit the on site interview it should be more about how they can work together with a team, how they would design systems, what they would do if they needed help, etc.
Coding up FizzBuzz on-site is a waste of everyone's time.
We're a small startup, and we don't have a really structured process. Also I have been the only person asking a detailed coding question (in consultation with my boss). Everybody else on the team has been responsible for those other questions.
I should also add that I personally prefer to do coding questions in person on a whiteboard, rather than doing them during a phone screen with some sort of collaboration software.
My technique of explaining the algorithm before asking the interviewee to implement it works way better with both people in front of a whiteboard.
I'm not saying everybody should do it this way. I'm just saying that this is my current approach and it seems like it's working OK.
I mean if it works for you then great but I've seen that back fire far too many times where I had to interview an incoming candidate who didn't do any coding over collaboration software and then couldn't do simple FizzBuzz. Basically wastes their time traveling and interviewing and my time when a quick check just to make sure they can do a few, quick basic things over the phone can save so much time.
While I dislike the way most large tech companies handle their interview process this one I don't mind as much.
How exactly could you evaluate FizzBuzz over the phone? Its designed as a weed out question early on, if they can't do that (and yes some have failed) then it makes a decision pretty easy
Most places I've interviewed at (or was the interviewer) talked to the candidate over the phone but had something like Google Docs open so you could watch some live coding. Usually just go through some very basic exercises to make sure the person at least knows how to code and ask a few, general programming questions. Stuff like that.
My most recent interview involved a coding assignment, which I topically don't do, but this one was a very interesting challenge that would make a good blog post afterwards so I did it. What was surprising is the interview not only included going over my code from the assignment, but also whiteboard problems about recursion. I've still never used recursion in my day job. Can interviewers really not gauge technical ability from my GitHub as well as chatting about problems and solutions?
Interviewer: I see you have built an app just like ours five years ago. That's great. OK now let's get to the important stuff. How many cakes can you steal from the queen if your duflle bag hold 5 pounds and there are 10 different kinds of cakes?
I can only think of a couple occasions in my professional career where recursion made more sense than iteration. Both involved traversing trees with unknown depths (parsing XML and scanning a file system).
That hasn't stopped it from showing up in every programming interview I've ever done.
I haven't really used it for anything other than super simple things with a very tightly bound max input. Stack overflow and honestly many recursive algorithms are quite hard to parse in your head especially once you add some edge cases and some other entropy.
All of these interviewing "tools" (or tricks) attempt to be time/cost efficient proxies for doing the damn job, and they all suck.
* Whiteboard coding is awesome for software development shops that don't actually own computers or use punch cards to load programs.
* Shared coding environments with a time limit works well when you want to double screen for someone who can also diffuse suspicious packages that arrive at the office
* Riddles and puzzles are good when your workplace has a chicken coop, an office fox and you can't figure out how everyone can go for lunch without leaving the fox alone.
* Behavioral interviews are appreciated by candidates who took Psychology 100 back in school for an easy A.
* Take-home tests go over well with the huge population of talented software developers who can't find a job and have loads of time they want to spend decoding the operational cost of a bubble sort
The only thing I've seen that's at all realistic and effective is a very short, __paid__ project. I did a two-day one for Indeed (ironically one of the worst promoters of all this bullshit) and a 4-hour one for my current employer. The payment doesn't even have to be market, it's more important as a signal to candidates that the company values your time.
I ask for 3 things from perspective employers:
1. value my time like you value your own (both the number and composition of your interview steps)
2. keep me updated as to where we're at in the process and when the next stage/decision will be made
3. Move forward in the process in a timely manner. It should not take more than 2 weeks from when you initially contact me to the process concludes.
I've never gotten more than two of the above from a single organization, but I will someday, and I'm betting that a company that treats potential employees that well will treat actual employees better as well.
What? Just let the programmer to it in his homeland. Easy. Just as outsourcing.
I wish this project approuch was more common. It would probably be cheaper than wasting time on multiple days interviewing reality shows where people are voted out one by one like some companies do it.
It doesn't work when they are already working in the foreign country. They aren't going to travel 12 hours to do a homework assignment for you legally.
I agree with much of what you say, heck, nearly all of it -- but it's not reasonable to expect, or even ask for a _very_ short paid project. I see no reason that a possible employer should pay actual money for some useless task.
If it's very short, then almost by definition it's useless.
That being said I've had great results from doing, say, a tiny consulting project, and ending up with full time employment. The best jobs of my life have followed the pattern of "am either offered, or I manage to ask/negotiate for a small consulting project" -> "before the project actually ends, am brought on full time".
So is their time. Takes candidate time to do, takes employer time to test/evaluate. Both parties are risking wasted time. You're wasting time driving to interviews and answering calls etc etc. Its the cost of doing business.
Asking someone to spend an hour on something is not a big deal, but I've seen/heard of stuff where they're talking about half a day or even an entire day. For someone with other things going on like a family (I know it may seem outlandish to younger readers here, but it does happen) and a current job, that's asking an awful lot. It does not take the company an entire day to review that exercise, making the time commitment very asymmetric.
Riddles are the worst, I was recently asked the "You have a 5 litre and a 3 litre container, how do you get 4 litres" question (never heard it before).
It's so simple to figure out if you're on your own and just play around with the idea. When asked on the spot I just kept thinking "hey these people want an answer NOW, don't make them wait" while feeling their stare, and couldn't figure it out without help.
I guess if you are old enough you'll know that one from the movie. Doesn't mean you remember how they solved it (because what did I care at the time?). Nothing to do with programming unless someone asks you to write a short program to figure out the most efficient (least steps) on how to do it. Behind a computer.
I have found the best course is to treat the interview as a situation where a colleague has come to me with a problem that we are hashing out together. I feel a lot less pressure with that mindset, though I do get thrown off by interviewers who take an adversarial approach. But I find this an acceptable loss.
How did you follow "take-home tests suck" with "two day paid projects are great"? Sure you're at least compensated for your time but most employment agreements almost certainly block this (unless you're a contractor), and that's assuming I'm willing to use 2 vacation days to maybe get an offer.
I've been on both sides of this.
Whiteboard interviews are good for a couple of things:
Can the candidate explain a problem on a whiteboard?
Is his hand-writing legible?
It's not sufficient to decide if someone can code.
You want some actual code written by that person.
And you don't need everyone on your team being able to explain things well on a whiteboard.
Just try to figure out how they are thinking and how they are approaching a problem.
I feel like this topic comes up on HN every six months and we all lament the broken state of technical interviewing, and the ardent supporters of all the different styles come in and push their position based on success perceived from anecdotal evidence. I have a hypothesis that hiring a good candidate is mostly luck.
When we view the hiring process in the context of what it actually is, a sales relationship, we realize that the customer (i.e. the employer), is ideally looking to buy services and time from the business (i.e. the employee/contractor). Viewed in this context, we quickly realize that many different things become part of the equation. Think about hiring a contractor for your home. If you happen to understand the job they do, you might be able to assess a the potential contractors previous work, quote and reputation effectively. Most people can't because they generally don't understand what it might take to do a job and have no experience in executing it even if they do. So you either heavily rely on recommendations from others, or go for a lower or higher priced contractor based on your perceived value (cheaper job, w/e the results vs more expensive job and risk mitigation) and hope for the best.
Considering most software development doesn't actually happen at "tech" companies, the reality is most companies are like most people and likely can't properly assess talent no matter what. They essentially get lucky if they get a good candidate or unlucky if they don't. That would lead us to think that maybe the "tech" companies are able to do a better job of it. In essence they do, but not because of their methods. I'd say it's a form of selection bias.
Google became successful and happened to be run by generally smart people technically capable people. As such they were able to attract generally smart technically capable people to work for Google. I'd venture a guess that 90% of people who make it to the live interview stage at Google would be qualified to work there, but since Google has it's pick of the litter they need to create a filtering system somehow. They happened to make it the "whiteboard" interview and similar high achieving Peer companies all tend to do the same thing. It became a thing, and as such all non-high achieving companies followed suit so they can behave like the high-achieving companies. The filter a company like Google applies is only necessary because they have too many candidates. Most companies aren't in a position to apply that filter if they really want talent. So really, they need to get lucky enough to get those who don't interview well with the "tech" company method of interviewing and hope they get some of those prime candidates.
I'm pretty sure the tech talent pool, as with all things in life, follows a Gaussian distribution. If you interview enough people you're bound to find some decent ones, even if they don't actually interview all that well.
I think Google is seeing a positive signal on their hiring process because a) they get a high percentage of the best programmers applying there at some point in their lives, and b) they run de-facto IQ tests on the onsites to pick the cream of the crop from that pool. It might not be the "best" raw programmers (that's impossible to define anyway) but being able to perform well on the algorithm tests means the people in general are highly intelligent.
Exactly. In general, if you do well it means you're probably pretty smart (whether you've memorized or figured it out on the spot). If you don't do well, it just means it's not your type of test most likely. I'd say this creates an opportunity for non-Google like companies to moneyball their teams and get some high quality talent at a discount because they suck at passing the Google filter. If these companies were smart, and considering how much luck is involved in the process anyway, they should look for a different set of criteria to evaluate a candidate on.
The term "whiteboard interview" seems to mostly refer to "solving algorithmic/coding/puzzle problems on a whiteboard", which I agree is almost always a waste of time for a software engineering position.
As the article mentions though, the term is a bit vague and is overly broad; for example if you're doing a system design question, I'd argue that the whiteboard is the best choice for an interview, since that's how you're most likely to collaboratively sketch the design of a system in the real world.
I think the problem can be restated at a more general level; does the interview test the sort of skills that the job requires? Most people here are correctly pointing out that the skill/knowledge required to write out pseudocode for quicksort is almost entirely uncorrelated to the skills involved in most software jobs.
If yes, the problem should be simple and relevant, and the session highly interactive. The assessment should be about (a) How interested the candidate is in the problem (b) How receptive they are to feedback.
White board coding interview are testing only one thing: how well candidate is prepared for it. And here what is wrong with it:
The assumption that candidate should spend his/her valuable time preparing for someone's assessment is arrogant. I do understand why Google and Facebook do it (the do other arrogant things), they assume you want to work for them so much you will study to make the cut. So, they track bunch of metrics, which makes THEIR life easier. They have baselines, calibrations, and other things you never have in the real life. Hilarious part is that they are both risk averse (better to not hire a good candidate than hire a bad one) and using brood-force approach (they will interview you endless number of time).
And if you are not prepared - you will most probably fail, unless you are really good at it or lucky. So, why should I prepare for Google's or someone's else interview? I have a interesting and very busy job, I just don't have time for this. I have commitments to my team to work on real products, not on fake problems from Cracking Google Coding Interview.
Just think how absurd it is?
So, starting from 6 month ago I:
1) Tell recruiters upfront I will not be spending a minute preparing for their interview process
2) I decline coding on the whiteboard. High level designs are fine, writing code - no, no exceptions
3) And the most important, I stopped asking candidates to code on the white board.
The most important skill of the software engineer is (IMO): ability to keep complex problems in context for long time (not 5 boxes and months), ability to manipulate them and ability to convince others that you idea is worth pursuing.
Do I? There are a lot of great companies, why should I waste time on Google? Working there is not as fancy as you think.
>> Everyone is busy, that's a terrible excuse not to brush up on your interview skills.
I believe that their approach is arrogant, they just don't value candidates time.
I have a confession to make, I spent some time at Google. And after I left I was interviewing people using their approach and I regret it now.
P/S/
When you interview for a company, you actually taking more risk that the company (talking about folks who are headhunted into interviews). If it doesn't work out, you will be out searching for a job, you career growth may be slowed down. And what is the company risk?
> Do I? There are a lot of great companies, why should I waste time on Google? Working there is not as fancy as you think.
I'm definitely confused. You don't want to waste time preparing for an interview, but you will waste time interviewing at a company for a job you don't want?
Yes, there are lots of companies hiring. But in my experience, in preparing for one software interview, you're preparing for others as well.
I interviewed at Dropbox, Amazon, Valve, and Indeed, within a span of a few weeks, and the prep work (Cracking the Coding Interview) was applicable to all.
What LaserToy is saying is that it's not just a little preparation; a few hours here and there. There are people who turn it into a literal hobby and do nothing else. If grinding through competitive programming questions is your idea of fun, then do it. Not everyone has interest in spending 2+ hours a day for a month to prepare.
That point I get, people may not enjoy prepping for an interview.
That's different than asking "Why should I prepare for a job?"
If you want the job, you'll put in the effort to prepare, regardless of your feelings on the process. That sacrifice (2 hours a day for a month) is minor in the grand scheme of things. Granted to the OP's point, Google probably isn't worth it. But that's a value decision each candidate has to make for themselves.
I'm preparing for my brown belt in Krav Maga. The test is 6 1/2 hour grueling physical exam that includes a comprehensive test of all the material from the previous 4 belts. Not only that, we're the first group eligible to test, so the instructor wants to set the standard with us.
Not everyone wants to put themselves through that hell. But those that do will put in the effort to be ready come test time.
I would talk about candidates experience, try to find some project he/she are proud of and dive deeper.
I might ask to write some code, but it is not going to be CS exam and it will be in real IDE.
I might ask a design question.
At the end of the day I need to answer 1 main question: "are you passionate about what you do or it is just 9-5 work"? Cause if you are you will be able to learn new things, and if you don't - knowledge of merge sort not going to save you.
>>The assumption that candidate should spend his/her valuable time preparing for someone's assessment is arrogant.
Plus, if some one has to 'prepare' for an interview that is supposed to test people's capabilities to do 'everyday' jobs, then you are hiring the wrongest possible people out there.
Whiteboard coding does a good job at assessing that, regardless of the candidates ability to actually solve the problem (for all the reasons why whiteboard coding sucks).
Do you analyze the problem?
Write test cases first?
Write a mini spec?
Gather Requirements?
Ask Questions?
Jump into code?
Google?
Verbalize the problems you encounter?
In that respect, engineering coding interviews work well. But too often, they are essentially a binary Yes/No: Candidate solved the problem the way I want them to solve it.
369 comments
[ 3.3 ms ] story [ 329 ms ] thread> I am a quote
This makes the quotation doubly-distinct, and still readable on mobile devices.
===
To the actual point I wanted to make: plenty of engineers at companies code on a whiteboard. They schedule a meeting, grab a room, and talk things out, while making notes, diagrams and sometimes even actually writing out some code on a whiteboard.
I hope nobody asks people to write compilable code on a whiteboard, but asking a person to talk through a problem and jot down some pseudocode doesn't seem like an awful thing to me. Communication is a pretty critical part of our job.
Can't do that in a whiteboard session.
Vast majority of the technical problems have been solved and its delivery a solution to a business problem that is the most important aspect.
Yes, I've been turned down for jobs over the white board but I've had more positive results than negative.
My sister, a pediatric ER nurse would disagree with you. The interviews are largely behavioral and are a breeze. No dummy is wheeled in with head trauma, random "new" diseases aren't invented and asked to be treated, etc. The simple fact of the matter is if hospitals hired in the same way, they would have no staff.
I know it's totally anecdotal, but we've all heard horror stories about candidates who couldn't write a for loop; some of us have witnessed these things first hand. And yes, phone screens should be filtering out those sorts of candidates long before they start sweating with a dry-erase marker in their hand, but, well.
How likely is an applicant for a nursing job to flat out not know how to stitch up a wound, or to take a patient's pulse and temperature?
"I know it's totally anecdotal, but we've all heard horror stories about candidates who couldn't write a for loop; some of us have witnessed these things first hand. And yes, phone screens should be filtering out those sorts of candidates long before they start sweating with a dry-erase marker in their hand, but, well."
Yes, it is a failure of how you have set up your hiring pipeline which you are now band-aiding. The majority of those folks can be screened by one look at a resume or in the first 30 seconds of the phone interview. Other hiring managers in our company repeatedly had this problem until we got them to focus of the right candidate qualities and ask the appropriate questions.
Take for example your local symphony orchestra; they have the same problem where people with visions of "making it" show up not being able to play at all. Want to audition? Send us an audition tape and a check. When you show up for the audition, you'll play a selection from these pieces and be asked to sight read this music. It is ironic that for an industry that is in a sense so subjective that the gates in the hiring process are more concrete.
To make an analogy, the software world is akin to:
Interviewer: "I see you are interviewing for the 1rst chair violin. The 3rd chair tuba player is really into experimental music, he would like to transpose Vivaldi's Four Seasons into a new scale with 12.5 notes per octave with a slight progressive jazz leaning. Oh, and since we all know there is pressure in performing in front of an audience, you have 30 seconds to think before the 2nd chair begins to throw rotten food at you. Reaction to stress and all you know...here is your tuba."
"But I don't play tuba...Are you asking me to play tuba? Am I going to be playing this nutcase's new music as part of our program?"
"Sigh...you don't know music do you?"
Not a significant enough majority, imo.
"Here's a couple of pages of unfamiliar sheet music that a second-year student should be able to play. You have an hour to figure out how to muddle through it on an instrument of your choice."
People hiring musicians don't do that, because they instead prefer to give candidates 16 bars of complex sheet music, and expect them to play it perfectly during the audition.
The programming equivalent would be to give someone a hard take-home problem, let them stew on it, bring them into the interview, and ask them to type in their solution, from memory, into a text file, on a keyboard with a broken Backspace key. That they will then compile, run, and compare the result of to that of the other 60 candidates auditioning for the role.
Are you sure you want to do auditions, instead of interviews?
> It is ironic that for an industry that is in a sense so subjective that the gates in the hiring process are more concrete.
That's because there's fifty thousand correct ways to solve a trivial programming problem, but only 'one' way to correctly play second violin in Vivaldi Four Seasons.
Music is a subjective art. Playing music is a mechanical process. My iPod can play music. My iPod can't implement a sorting algorithm.
I have no idea what the candidate did in their CS undergrad. Maybe they cribbed all their work from their roommate. Maybe they went to a party school. Maybe they spent the last 4 years as a 'Senior Developer' at FooCorp copying files from hard drives to floppy disks, and posting a few paragraphs a day on the company's WordPress install. Maybe they are an Architecture Astronaut who can talk for six hours about how great Haskell is at doing multi-manifold monadic trivariable entaglement, but has no idea how to do any real work.
Or maybe they spent the last decade building Bigtable and MapReduce, and Spanner, and TensorFlow at Google. I'm not an expert on Bigtable, or MapReduce, or Spanner, or TensorFlow, though - and I can't definitively, in 60 minutes, tell if the person I'm talking to is bullshitting me. I can't tell if they actually did any of that work, or they coasted. I can't tell if the complicated problem they are describing to me is actually hard, or if they are embellishing it. Even if I felt confident that I could make that conclusion, my opinion would be incredibly colored by personal biases.
Oh, I should check their GitHub, you say? Well, guess what - Jeff Dean - the guy who did spend the last decade building Bigtable and Mapreduce, and Spanner, and TensorFlow - doesn't have a GitHub account. Presumably because he has better things to do with his free time, then work on OSS.
Oh, I should hire fast and fire fast? Don't get me started on why that doesn't work...
Once you're an RN or an MD with all the appropriate qualifications, assuming you're not outright faking them, they can be reasonably confident that you are, in fact, a competent nurse or physician and move on from there.
We don't have that, and so we have to spend a lot of time making sure that a prospective candidate even has the basic skills and qualifications of a software engineer.
I had to do that at my last interview. I got through it ok-ish. The more interesting question was a more general system design question - how would I architect a system to do X, under constraints Y and Z. What would I do if a new constraint came up? Ok, now how would I make it more resilient? There, the whiteboards is just a tool I can use, not the primary focus.
Honestly, we all might as well read chicken entrails because that would be about as predictive as our current interview practices.
But it's the interview equivalent of your driving instructor making sure you adjust your side and rear view mirrors before you start the car.
I'm not opposed to white-boarding, but to reinforce this point I'll note that I would have done much better on some of these problems when I was just out of college than I would today, because they were all fresh in my mind.
I assure you I wasn't a better programmer or employee then.
Not always a whiteboard, either; one of the best engineers I ever knew flunked a Google interview because of a situation where he had to write a bash script and read it over the phone to the interviewer, who apparently didn't transcribe it correctly on the other end. But "they only hire the best", you know, and their practices are rock-solid and proven.
https://www.youtube.com/watch?v=V7MXfg2wt28&list=PLqJcvcLs4C...
I, too, have a side project I'd love to quit my job to work on, but I think it's less monetizable than an actual game.
Actual Code? Not a chance. Portability, Durability, Efficiency - It's the worst of all worlds.
I've _never_ seen someone, or have asked to participate in, some sort of whiteboard coding thing. Even with pseudo code. That just doesn't seem helpful to me, at all.
an advice to candidate: when you've been asked to do whiteboard coding - run away. this is a negative signal.
And interviews are not pressuring, they are simple. No hard desisions, no anxiety whether it is going to work. You in and 5 hours later you out. Lame.
What if you consider part of someone's job to be communicating concepts to people, possibly with the help of visual aids and diagrams?
I hate this concept that if it's not typing code into an editor then it's not "real work".
I absolutely communicate with my coworkers using a whiteboard and pseudocode. I reject the idea that being put on the spot is necessarily "artificial". To the contrary, I think that the number of engineering jobs where you can assume you'll never be put on the spot or have to communicate complex ideas verbally or visually is relatively low.
Now if this particular skill isn't interesting to an employer and they'd rather spend the time talking about some other candidate capability, that's a whole different story.
thinking and presenting are two separate skills, and, in my opinion, can rarely be executed at the same time.
I think the complaint that programmers pretty much never have to think about code under pressure is a fair criticism. The whiteboard really puts you on the spot. But if it's your company and your hiring managers are playing a game of gotcha on the whiteboard, you need better interviewers.
which also means the team might consist of people selected by using this metric. this is a good signal ;-)
Whiteboard coding interviews are one thing that I have to go out of my way to practice for and get better at over time when interviewing (ie over the course of a few failed interviews). I don’t feel more skilled or smarter by the end of the set of interviews, but I inevitably do better at these kinds of problems. Should the interview be testing my competency at day-to-day work skills or how well I’ve practiced my interview skills, because whiteboard coding problems only achieve the latter.
I do coding interviews in the following manner: I select an interesting problem from a book. I personally complete the problem, measuring what was challenging and the time it took for me to complete the problem. I check the textbook solution, making notes of how my solution differs from the textbook's. If I like the problem, and if it fits with the development role, I invite the candidate to bring a development laptop, I give them a hard time limit of twice the time it took me to complete it, and have them share their screen while they attempt to complete it.
Afterwards we test their solution, I ask them some questions about it. I grade based on code taste, the number of hints they need, and their completion time. All candidates for a position are given the same question. Performance in coding interview is about two third's of the overall candidate evaluation. The ability to sit down and write good code in a time constraint manner is a very important part of being a developer.
First, your definition of "interesting problem" probably includes a lot of personal biases. What you find interesting probably touches stuff you've had experience with, but just because you've done work on say, customized implementations of binary search, doesn't mean it's a good baseline for all developers everywhere. Maybe they've worked on different domains or solved different problems and can't relate to questions you personally find "interesting".
It'd be much fairer to select a boring question that involves standard stuff that everyone has to face at some point in their careers, like string manipulation. And even then, it just involve normal transformations ('look for and remove special characters') and not wacky, HackerRank-style rules ('no two bs after cs but before as').
On the other hand, I think there needs to be some proxy for "how well does the interviewee navigate complexity?" Because a programmer who does it well is far more valuable than a programmer who doesn't, and there's high variance on this between programmers and between fields of expertise.
The systems design interview sort of gets at it, and whiteboard interviewing often tries to get at it. I feel like I've yet to settle on an approach I'm happy with.
You have a good point, I noticed this about myself as well. However, I don't think whiteboard interviews are special in that regard. I noticed that I got noticeably better at the "sit down and code me a simple web API endpoint" interview, and I got noticeably better at the systems design interview, and so on.
I do think that whiteboard interviews tend to lean very algorithm-heavy, which doesn't reflect the role's actual day to day. And practicing algorithm interviews really does feel like an inefficient use of time unless your role truly demands algorithmic proficiency (like if you're a graphics dev).
It's not; I couldn't agree more.
In real life when using engineering "skill", not only am I not anxious/stressed by trying to write on a whiteboard, but I'm using my laptop, with my editor, as well as access to man pages, docs, and google to look things up.
In addition to the overemphasis of performance responses, I think "how much do you know from memory with 10 seconds of thinking" vs "how much do you know with 10 seconds of google" is useless. The only fairly complex datastructures I can perfectly describe from memory are the ones which I used most recently...
If you want to test communication skills amongst programmers, I dunno...ask them to write something in coherent English. Or here's a crazy thought: ask them to document some code. I guarantee that 80% of working "rockstar coders" will fail (but not before whining, crankily, about how unfair it is that you would actually make them do such a useless thing, since, y'know...never actually documents anything IRL, dude).
Programmers like whiteboarding because it lets them believe that interviews can be reduced to purely objective functions, and because GooAmaSoftBook does it. They're too scared to deviate from the pack, lest people think they aren't as "elite" as everyone else.
Think how much you might actually learn about a candidate if your interview process replaced a day of solid whiteboarding with a code review, some pair programming, a session of documenting someone else's code, behavioral interviews, etc. It's almost as if you'd be treating them like a...person!
I feel like we're probably at an impasse if I can't convince you that a whiteboard is a decent medium to communicate ideas around datastructures and algorithms, but I appreciate your point of view.
I can only say my experience, which I hope you will take into account as one anecdote. I don't read books about "cracking the coding interview" or do leetcode or hackerrank. I left school 14 years ago or so my stash of cs trivia/secrets/gotcha isn't particularly full. I've done whiteboard interviews where I come up with at best a naive solution.
And yet I've received offers for fairly senior engineering positions at Amazon and Twitter and (hopefully tomorrow) from Google. Most of the whiteboard interview isn't even around the code, although that's a small part. Most of it is analyzing the problem, discussing constraints, discussing tradeoffs, walking through data structure manipulations, drawing arrows and boxes, that kind of stuff. Some code, maybe 30% of the interview. I just keep having this experience where nobody wants to play the gotcha game, they want to know how well I can communicate while solving a problem and they think they get that information out of the interview (I agree with them).
That experience makes it hard for me to understand a viewpoint that believes that whiteboard interviews are about memorizing secrets in advance.
Every set of interview prep material I've seen from Facebook, Amazon and Google recruiters asking me to interview in the last 3 months has included links to things suggesting "cracking the coding interview" is a good start and pointing to similar docs otherwise.
Are you sure you're not just replacing one stereotype with another here? Do you think it's fair to say that 80% of people who do well at whiteboard coding value documenting their software so little that they'd be fail at it while disparaging the concept?
There seems to be a bunch of ingenuous assertions here that is characteristic of someone who doesn't think one should have to prove oneself. I'm at a FAANG and we don't even ask anything crazy hard for algorithms/data structures like Google or Facebook does, but our interviews still manage to be intense for technical and soft skills - they will challenge even strong candidates and one cannot pass the interview without displaying strong technical acumen or communication skills.
There is so much more to technical assessment than just coding. A whiteboard is just a standard medium for communicating ideas in a collaborative manner that still works the best for any sort of deep collaborative design when compared to any other medium so far.
Perhaps it would be more effective to have the candidate whiteboard a concept that they are already familiar with, be it a high-level engineering principle or a system/solution they have built in the past. Attempting to solve a problem you have just been presented with AND communicating the solution effectively is a big ask.
I know my opinion is unpopular, but I sometimes do think I've figured out something other people miss. I think when it comes to whiteboard inteviews, candidates are often playing the wrong game. They think it's about gotcha questions and they think they fail because they didn't leetcode hard enough. I don't think that's true, they are just trying to game the thing that's easy to measure.
In my experience with the "terrible" FANG companies, it's not about gotcha questions. I get the offers even though I don't often find a non-naive solution. The people I'm in the room with really do want to see my thought process and they really do want me to communicate the tradeoffs with them. People don't fail the gotcha questions because they don't know trivia or because they forgot a detail from their CS classes. They fail the whiteboard interview because they see an unfamiliar question and say: "I don't know that trivia" instead of drawing out possible solutions and having a conversation with their interviewers.
I'm inclined to describe it as a sort of interrogation technique, you're offering a stimulus (the problem) and aggregating a number of reactions to form an opinion, and open up new lines of questioning. Very delicate work.
I think problems arise when unskilled interviewers present an excessively obtuse problem to the candidate, then read far too much into their responses (this approach is also easily "hacked" by those who've memorised common problems of this ilk). To stick with my interrogation analogy, it's the equivalent of screaming in the suspect's face, noticing that they gulp before shifting in their seat and scratching their face, then deciding that "they must be lying".
But ... if you read some of the feedback from interviewers at "those companies" that rely on these interviews, they say the reason they failed someone is exactly because they "didn't leetcode hard enough". It is manifested as:
"Well other candidates got the same solution as you, they just did it 10 minutes faster"
or
"You missed an edge case, even though your core algorithm was correct".
This is a huge issue with these interviews. It's all too easy for interviewers to evaluate candidates based on how fast, correct, neat or "complete" you answer was. It's easy and takes no time.
That may have been a reason why I have failed some interviews but I feel like most interviewers are actually good about this and will say something along the lines of "what about this input?" where my code does not work and then I have that "oh shit, that won't work for that" and then I fix it.
If you can't give me good examples from your past, I'm going to throw my own questions at you. I tend not to do coding on the whiteboard, though. Lots more boxes and lines and schema and system interaction-y stuff.
This is incredibly common, and you tell the candidate in advance that they will be expected to give an n-minute presentation on a subject of their choosing (need not even be tech) so they can prepare.
Tell me this: when did a mob last ambush you at your desk and demand that you invert a red-black tree on a whiteboard that they happen to have with them right now? I'm guessing that never happens where you work. If it does then fair enough :-)
I think it's safe to say that being put on the spot in front of people you don't know who are judging you is quite different from being put on the spot in a team meeting with someone from product who will watch your demo of a potential new project/idea/concept.
We've never rejected somebody for having a "bad" answer, because they were able to at least talk about what they're doing. We have had people flat out refuse to even try, and that seems like a pretty big red flag.
In my most recent on-site interviews, my interview would be in a room with full floor to ceiling glass windows, the next interviewer would come in, ask one or two personal questions, and then either cut me off after a couple minutes or just simply say "we're going to move on to the whiteboard now, do this". Meanwhile people are walking by staring into the room, looking at the whiteboard, etc.
Which is just too rigid, IMO.
When I interview a candidate, I want to know that they'll be able to work together with myself and my current teammates as a team. So I would rather do something similar to what you described.
So it seems to me entirely reasonable to still be nervous.
Mind you, I don't have a better answer, but I don't think just explaining that suboptimal is ok and it's a discussion really helps.
Also, generally, if I'm whiteboarding in front of my coworkers/team/etc. I'm discussing something I've thought about for more than the few moments I have to digest the question while standing at the board during an interview.
> Can we assume I know how to do argument checking?
I would probably question you at some point about a particular value of an argument and almost all candidates will get points for thinking out load that of course a NULL would get it crashing and that a simple check could stop it. Then it leads to discussion about how it should be handled. Sometimes a NULL means the program should cast an exception, other times it should just return without any side effect.
IMO the hate for whiteboard questions is really a symptom of poor interview technique.
Granted, I will give you that optimized code has definitely been important. One of those interviews even had me use a bit array (at least I think it was that) to efficiently a store a list of booleans but at least they gave me the relevant function calls to use an already implemented one.
Picking on whiteboard code for not compiling is not good interview technique. But it's also a common enough failing that it's hard to say a bad interviewer in that regard means a bad workspace, unfortunately.
The only place I’ve worked that was “good” instead of “treats people poorly but where else are they going to go” was a small, boutique financial company.
Small companies that are randomly led by non-dummies are the best, but are exceedingly rare. Out of the vast majority comprising the other stuff, it’s mostly all so miserable that you’re doing yourself a favor to skip it unless the pay is some dramatic increase for you or you otherwise have some emergency requiring you to take some job.
(And yeah, the pay makes a difference. Not a lot of places where you can make a few hundred thousand dollars a year.)
(I’m making a distinction between the idea that a job “is fine” where one “is happy” merely because the culture is at least not worse than elsewhere while the pay is better vs actually feeling positively about one’s company’s culture and corporate behavior.
For example, my friends who work at Facebook are “happy” with their jobs, mostly because of pay and because they know if they switch to other companies, politics, corporate misbehavior, etc., will just continue. But these same people tell me frequently how sad, upset, soulless, disappointed they regularly feel because of their employer.
That feeling I think is extremely widespread in tech, almost all employers. That’s the only part in my viewpoint that matters for whether someone “is happy” at their job, and it’s that type of bad culture I think can be easily flagged by little stuff, like bad whiteboard interviews.)
Nobody has any idea what’s going on; nobody has clarity. And when someone does have clarity, middle management is only interested if they can set it up like a battery to harvest credit from. If they can’t, their incentives are usually more aligned with intentionally abusing process to “manage out” innovative people.
the one thing i'd add is that i do take particularly thoughtful / empathetic interview processes, interviewers who actually know and understand the question (rarer than you'd think), etc. as a fairly strong signal, as those sorts of things don't just happen by chance-- they are the byproduct of a culture of thoughtfulness (and thoughtful people tend to be thoughtful about most things).
Ya I did not get that job lol.
Right. And a competency based interview goes better if you have a perfect example for every question. But it isn't an auto-fail if some of your answers are mediocre
Personally though - I'd rather hire someone who is able to properly communicate their thought process over an imperfect solution, than someone who was unable to discuss/explain the perfect solution they scrawled down.
When I give interviews candidates often say they don't remember some API. I tell them I don't care and they should make something up and I'll figure out what they are doing from context. The only time this burns candidates is when they clearly don't understand an ADT well enough to give it a sane interface.
I've had my share of the second kind, which were without exception some of the most unpleasant interviews I've ever gone on.
If only all interviewers were like you.
I’ve interviewed at Facebook as well and although the interview was not successful I have never been nitpicked about syntax and compilabiltiy/runnability of my whiteboard solutions. I came off with a very healthy appreciation of the way they try to engage the candidate and reduce stress.
Fizz-buzz tests have their place, but I think an onsite interview is past that point. In general I have found that white board exercises don't allow me to either understand or convey how I can apply my expertise to provide value for the organization.
Except at the end of the day this is an interview, you are judging their answers and evaluating their performance. I have been part of white boarding sessions like this and it does not reduce anxiety and still feels completely removed from what I actually do on a daily basis.
We at least on a weekly basis somebody has some small problem they need help working through, it generally ends up being diagraming or pseudo-coding to figure out the best approach. If somebody can only participate by hiding in a corner typing out code, then the process worked for both sides.
I highly doubt a normal engineer is going to be giving a demo to their team with high stakes in the first three or so months. I understand if they're possibly a senior engineer, architect, or lead of some sort because they were probably hired with a plan to spearhead a specific project.
I'd say in the average case a new engineer will have enough time to at least break the ice with their new team before being put into a high pressure situation.
Clear and respectful communication is one of the most important skills for an engineer to have.
If you read the second half of my answer in the article I go into the communication side of things. In my experience there are better ways to evaluate a candidate's emotional intelligence and social skills than asking them to code in front of someone.
I've written code on white boards at work, and I've discussed about code on white board with colleagues or managers. It's not the same thing as during interviews. The schedule is never as tight, scrutiny is way laxer and the overall mood is completely different when I'm working with colleagues or even bosses.
I kind of understand why the BigCo's do it (they need a super rigid filter since they have so many candidates), but in their case they could select for people with the best pink leotard on interview day and they'd still get decent programmers, because their companies are in such high demand and they generally already control their markets (software, natural monopolies, etc.) and can afford to pay a ton so the competition would be cutthroat anyway.
What you're testing is the ability to discuss a problem with your peers. That is the critical thing.
Do the whiteboard but don't make the coding the standard. Don't get too hung up on the details of the problem.
Make the problem easy but fuzzily specified.
Do write lots of sequence diagrams, block diagrams, ui sketches, arch diagrams and very occasionally someone might draw a data structure.
When I get hired this way, I always feel a little bit of guilt because I know they actually hired somebody else.
What can be done? I’m not sure, but I can say one thing: my level of discomfort is multiplied by the number of strangers in the room. Does this ever get considered with interviews?
I am far more comfortable coding in front of 1 or 2 people who each give me a little bit of background about their coding experience; just so I know. If they are experienced engineers, it’s probably not going to change what I say aloud, but it just makes me more comfortable. I guess more overlap of technologies in our background does help. It’s nice to not feel pressure of worrying if my solutions reflect general programming conventions enough to be language agnostic. I have never really used Java, for example.
If communicating ideas is part of your job, I'd bet any amount that you decide what ideas you want to communicate well before you do the actual presentation (or other physical delivery of the ideas).
Nope. There was a great submission a few years ago about a guy who applied to two categories of jobs: management and software development.
In the SD roles, everything was oriented toward rejecting him. Any possible "red flag".
In the management roles, the interview was focused on finding areas where he could contribute to the company.
Whiteboard interviews are a problem in and of themselves, and the propagate terrible attitudes into the rest of the interviewing process.
Whiteboard interviews rarely test the ability to communicate on a whiteboard because the interviewer knows the answer they are looking for and the presenter does not.
If you defend whiteboard leetcode using this excuse you are deluding yourself and you are part of the problem. The only thing the modern FANG interview hires for is people that can solve algorithms problems in a silo under pressure. No checks for software engineering skills, collaboration skills, testing skills, reviewing skills, and on and on...
The whiteboard interview is why so many engineers unexpectedly suck.
The reason many engineers unexpectedly suck is because it’s a high paying job where performance is difficult to measure let alone predict.
White boarding interviews are a symptom of the problem, not the cause of it.
They you are asking the wrong questions. I use whiteboard for developers, and for junior developers I give them a simple task such as reverse an array, turn a string into a palindrome (and explain what it is if they don't know). If they want to be a developer they must be able to solve those simple tasks. I don't really care how, as long as they think loud. Then I use it as a starting point for enhancement discussions, perhaps stack/heap questions, recursion, assignments ect. During all this I coach them, teach unknown concepts in simple terms - this is what they can expect when they ask their mentor a question so in that sense they get a feel for us too.
Some argue that it's still not good, and that we're filtering out people that work best alone. That is true, but that is by design. We work as a team, having 10 individual developers not working together unless forced is an architectural nightmare.
I'm not asking them to impelment van Emde Boas tree because I spend countless of hours implementing and analyzing it getting my degree. I ask them to turn a string into a palindrome, or return the sum of all even index in an array of ints, ect. They're free to ask all the questions they like. It's a litmus test to see if they know simple programming, akin to seeing if a contractor knows how to drive in a nail using a hammer.
Is it high pressure while your future career is on the line? Absolutely, but they are not special snowflakes, and it's the case for any interview for any kind of position. I'm not going to throw thousands of dollars after someone just to figure out that they have in fact never written a functioning for loop.
If the interviewer leaves frustrated because they couldn't solve anything and it was a mess, or the question was formulated terribly, or the interviewee was under prepared, etc etc, then it provides a lot of insight into the capabilities of the interviewee. It almost provides more insight into the qualities required of a developer than just putting them on the spot.
In that case I would recommend reading the candidate's resume beforehand, look for prior experience publishing, presenting, or teaching and ask them about it.
Ironically this leads to min- maxxers who ignore a good chunk of their courses simply to master leetcode. My smartest peers are ones who work on interesting projects and research, thus never finding time to grind interview questions.
Then there are also companies that expect your white board code to compile and have perfect syntax, which imo is very unreasonable.
Whiteboard interviews where the interviewer says "it's actually `.toLowerCase()`, not `.lowerCase()`, you fool!", or "oops! You forgot to `position: relative;` the containing element that you `position: absolute;`d below!". you're not testing anything useful. Maybe if we were still coding everything on punchcards and using massive user manuals, but docking applicants for things their editor would highlight or one ctrl-r on the page would find is ridiculous.
You're 100% correct. We give our candidates the choice between working through a problem at the office (on the spot as you say) or doing a take home project asynchronously. Not everyone has the time to do homework, but not everyone wants to do an in person coding interview either. We try to stay flexible.
Also, we're growing fast enough that we're rarely comparing candidates. If we can hire both the apple and the orange (assuming both candidates are evaluated to be good for the role), we will.
For both the sr backend and devops positions, how important to the roles is familiarity/experience with your technical stack? I might have some of the skill set/experience you're looking for -- sr backend engineer, sr systems engineer, devops are all positions I've held -- but honestly I don't know JS, .NET, or Chef. Bluntly, I don't want to waste your (or my) time. ;)
<tiny>Also, I wish HN supported private replies</>
In fact, if someone is too into their stack, we see that as a bad sign.
So I'd be super excited if you decided to apply :)
You name it, we are doing it.
No comment on it being an effective interview process.
I still only do OK at whiteboarding in interviews.
Lock me in a dark closet under the stairs with just the glow from a couple LCDs and throw pizza and requirements documents through the slot, like you're feeding the Rancor. /s
You end up spending all your time dicking around with missed semicolons and looking up library routines and shit like that. Sure, it's all stuff you have to do as part of a job, but it's also all trivial stuff.
The last time I had to do this, I spent time googling how to open up a port in python, and then read from stdin. It's just a waste of time.
All this does is test the wrong things.
Most of what we do as engineers is trying to read code to figure out what it's doing, how to fix it and how to enhance it. Going through real code with a candidate and assessing their ability to understand brand new code (and what questions they ask) seems like it would give good insight to how they would perform on the job.
I've been on dozens of interviews, and that's the only time I've ever encountered that, but I think it does a really good job representing the job, as that's invariably what I'm going to have to do when you hire me, is sit down and familiarize myself with the code base.
He also asked me a handful of questions, said "Okay I know you know enough to do the job, let's see how much you really know," asked me a bunch of really low level stuff, and proceeded to teach me concepts when I told him I didn't know certain things. I walked out of that interview knowing more than when I walked in, something else that has never happened in all the other interviews I've had (except maybe I learned a new term that I never heard before that I was apparently supposed to regurgitate back because that's what was written down as the answer on HR's answer sheet).
Most whiteboard questions are always some esoteric algorithm any engineer worth their salt would in real life would look up before ever consider using to verify their own understanding before using it.
Putting someone on the spot saying whiteboard x is not ok.
However if the test is to see if they can take part in a meeting, then by all means do a white board, but ask them draw a workflow diagram or some other 40,000 foot type of diagram. Do not ask them to put code on a whiteboard.
(and if you wonder how anyone ever gets hired there, the answer is: by cheating. If you're going to interview at one of those places, odds are their interview problem and an accepted solution will be online somewhere, so you just go look up and memorize)
Our field is full of some of the smartest morons on the planet.
During my recent job search I was asked many questions I didn't know the secret algorithm to, but if those companies had given me the problem ahead of time and allowed me to research it -- just like what happens in real life -- I could have coded solutions to all of them.
And if you're worried about people just plagiarizing existing solutions, have them talk through it, as you say. Should be fairly clear if they understand each line or not.
1) Whiteboard
2) Review and technical discussion of existing code or projects
3) Paid take-home project
4) Pair programming on live code
5) Technical discussion only for rare cases for those Top Secret/NDA candidates that really can't discuss their past projects but are also time constrained
Some people do actually like to whiteboard. Keeping it around is kind of like legacy support for people who practiced hard to excel in those interviews.
This is pretty much the union of technical interviews that are well-known. If you switch out a technical interview format and can't get meaningful insights into a candidate, you're testing for the wrong things.
So much of the technical hiring process tends to be about trying to find people that match what you expect people to be. Which might be fine, you need to fill a particular position which requires a specific set of skills. You need to validate those skills. You do that by doing something semi realistic while trying to minimize the pressure of an interview situation.
But, then there is the other kind of approach where you are looking to see what the candidate can offer, trying to find people who know different things to you and do things differently than you, in this scenario you are more inquisitive and let the candidate direct the interview process and get them to show you things that sound interesting (including coding).
The first approach is all about finding specific skills. The second approach is all about being clear about the values you are looking for.
The second approach requires more skill and more time while wearing the hat of a technical recruiter.
You may need a little bit of both approaches.
So the big thing is to be clear about who you want to work with at the end of the process.
If the test is to see how well a person works in a whiteboarding interview to solve a problem, then it should be a simulation of collaboration, while trying to avoid the Clever Hans effect.
Otherwise your DP whiteboarding problems are worthless.
When they brought me on-site after that, I interviewed face to face with 5 or 6 people in a row (except for a lunch break), and every one of them asked me a two or three questions and then switched to tricky whiteboard problems.
I remember one was 'write a function that draws a 2D skyline from a bunch of 3D buildings', which I got totally stuck on because there's so many possible configurations of heights and widths and how they overlap that complicates it, so I didn't really know how to properly solve it.
I didn't get the job. I thought the experience was interesting at the time, and basically treated it as a free tour of Google and a chance to eat In and Out burger again, but I don't want to go through that grueling exercise again (or spend a ridiculous amount of time prepping for it... as it was I spent a solid two weeks studying for it the first time around), so I've said no to Google recruiters every time they've gotten in touch with me since.
I might have given it one more shot, though, if I were willing to move to Silicon Valley at this point in my life, but I just bought a pretty decent house and I'm not really wanting to give that up to take out a mortgage five times what I have here just to live in a smaller house in the Valley and be able to say I worked at Google.
Spitballing solutions to a problem collaboratively and with access to reference resources (a nice little programmer's bookshelf in the room) followed by actually coding a little bit up seems that it would be way less stressful and way more likely to give you a sense as to what someone is capable of and what they're like to work with.
Currently, many interviewers go for - as I like to call it -"trivia questions". Personally, it just doesn't seem appropriate because "Everything may be obvious, once you know the answer". In some such interviews, I've asked my own set of trivia questions to the interviewers. The interviewers weren't pleased as they couldn't answer some questions for which I knew the answers beforehand.
For example, one I like to ask (in person, with no computer access), ostensibly to gauge familiarity/intimacy with the Unix CLI, is:
"For each letter of the alphabet, see if you can name a Unix command (anything one can type at a shell prompt that won't return a "command not found error") that starts with that letter."
I preface the question with something suggesting that it's supposed to be fun, and I encourage skipping letters and coming back to them to avoid pressure (nor do I write down the answers). Most people are surprised at how many they remember without much thought.
Although not particularly deep, further discussions can arise from differences between shells with respect to built-in commands, as well as differences between OS flavors. Also, I've discussed how a somewhat new language like Java has made one of the otherwise difficult letters much easier.
Regardless of the trivial nature of the question/exercise, it provides a spectrum of jumping-off points for deeper knowledge discussions, all of which are signaled/chosen by the candidate.
That general premise is right, we've lumped too much into the term "whiteboarding", and it's worth unpacking into the different interviewing practices we may like or dislike. Some things people don't like have nothing to do with an actual whiteboard (e.g. testing esoteric CS knowledge or deliberately creating a stressful environments), and some things that involve a whiteboard IMO are ok (e.g. sketching out a system diagram).
That is STILL a thing (judging by HN) and it's useless. I don't think anyone would ever think it's a bad idea to have a whiteboard on hand if a candidate is asked to describe how they would design some nontrivial system. Being asked to describe something without a whiteboard in that case, is even harder. The objection you often get for the latter kind is that it tends to benefit the extroverted kind that likes to babble and sketch, whereas a candidate who would prefer ten minutes alone with a pen and paper might not do so well. I think are some merits to that complaint, but I also know from experience that being able to babble and sketch is really important.
If you want to see how well someone codes, look at their code. Or ask them to write something for you. Don't do a whiteboard interview and think it is the end all be all. Take it as a data point.
If that were the case, ask me about a past problem and my solution and to defend it, not the proper way to invert a b-tree.
It's about the thought process when breaking down a problem and the ability to organize and communicate their thoughts. It's also about exploring a problem space. How carefully does an engineer consider edge cases? What kinds of things are important to them?
A whiteboard to me is a much simpler and accessible medium through which to explore a problem vs setting up an environment and having to type code and dealing with all the minutiae that comes with actual development.
Of course, it isn't the ultimate method of evaluating a candidate but it's certainly valuable.
If, however, the job requires careful consideration of dependencies, time domain, and, yes, edge cases - then you are NOT testing for that.
Lock the candidate in a room and give a few hours to let her to produce the result would be a better measurement.
Lots of interviewers suck. That isn't a property of whiteboard interviews.
I won't say this is you, but it's funny the number of people who will interview and say something like this and then still give a non-trivial problem/question to whiteboard.
Reminds me of a guy who interviewed me several months ago at a place in Mountain View (not Google, but another).
He started by asking me to create an object with a couple attributes. Then he slowly began to add to the problem by asking for X, then Y, then Z.
Little by little, the code became more complex because he wanted iteration, etc.
He would say "I just want to know that you're able to code."
Later on, I found out he didn't like what I wrote, meanwhile all the basic elements of coding he wanted I was able to add without any problems and I met his specs.
If there's a "communication issue" in whiteboarding, perhaps the interviewees shouldn't shoulder all of the blame. Perhaps the interviewers need to temper their expectations and learn to communicate them better, because obviously there is some disconnect in a situation like this, and it's not the first time I've seen this before.
Isn't that a more realistic scenario? How often do you give an (employed) engineer a task then ask them to immediately explain what they'll do to implement it?
So yeah.
When I say simple, I mean really simple, like FizzBuzz simple.
There are interesting questions you can ask that are really simple which are not FizzBuzz. I've used the Fisher-Yates shuffle algorithm and more recently Selection Sort. I should stress in both of these cases that I carefully explained the algorithm before asking the interviewee to code it up. Even though these algorithms are quite simple, I do not expect the interviewee to have memorized them.
I do expect someone who is a professional software engineer to understand the algorithms after a decent explanation and then write code which will implement them.
I also don't necessarily expect the interviewee to get the code exactly right. I just expect them to demonstrate that they could write correct code given enough time and access to an actual computer to test the code on.
Once they hit the on site interview it should be more about how they can work together with a team, how they would design systems, what they would do if they needed help, etc.
Coding up FizzBuzz on-site is a waste of everyone's time.
I should also add that I personally prefer to do coding questions in person on a whiteboard, rather than doing them during a phone screen with some sort of collaboration software.
My technique of explaining the algorithm before asking the interviewee to implement it works way better with both people in front of a whiteboard.
I'm not saying everybody should do it this way. I'm just saying that this is my current approach and it seems like it's working OK.
While I dislike the way most large tech companies handle their interview process this one I don't mind as much.
https://medium.freecodecamp.org/how-to-organize-your-thought...
Just embrace it. It will be okay.
True story.
A question I've had in an interview was what is the difference between recursion and iteration. Fairly easy to explain, including examples of usage.
That hasn't stopped it from showing up in every programming interview I've ever done.
Well, yeah, other than tail recursion with tail call optimization, stack consumption, if not stack overflow, is always an issue.
> and honestly many recursive algorithms are quite hard to parse in your head especially once you add some edge cases and some other entropy.
I find lot of things are easier to conceptualize recursively than iteratively, though parsing really depends a lot of the language.
* Whiteboard coding is awesome for software development shops that don't actually own computers or use punch cards to load programs.
* Shared coding environments with a time limit works well when you want to double screen for someone who can also diffuse suspicious packages that arrive at the office
* Riddles and puzzles are good when your workplace has a chicken coop, an office fox and you can't figure out how everyone can go for lunch without leaving the fox alone.
* Behavioral interviews are appreciated by candidates who took Psychology 100 back in school for an easy A.
* Take-home tests go over well with the huge population of talented software developers who can't find a job and have loads of time they want to spend decoding the operational cost of a bubble sort
The only thing I've seen that's at all realistic and effective is a very short, __paid__ project. I did a two-day one for Indeed (ironically one of the worst promoters of all this bullshit) and a 4-hour one for my current employer. The payment doesn't even have to be market, it's more important as a signal to candidates that the company values your time.
I ask for 3 things from perspective employers:
1. value my time like you value your own (both the number and composition of your interview steps)
2. keep me updated as to where we're at in the process and when the next stage/decision will be made
3. Move forward in the process in a timely manner. It should not take more than 2 weeks from when you initially contact me to the process concludes.
I've never gotten more than two of the above from a single organization, but I will someday, and I'm betting that a company that treats potential employees that well will treat actual employees better as well.
They are all imperfect proxies due to various constraints.
I wish this project approuch was more common. It would probably be cheaper than wasting time on multiple days interviewing reality shows where people are voted out one by one like some companies do it.
If it's very short, then almost by definition it's useless.
That being said I've had great results from doing, say, a tiny consulting project, and ending up with full time employment. The best jobs of my life have followed the pattern of "am either offered, or I manage to ask/negotiate for a small consulting project" -> "before the project actually ends, am brought on full time".
My time is valuable. I see no reason to work on a useless task for free.
It's so simple to figure out if you're on your own and just play around with the idea. When asked on the spot I just kept thinking "hey these people want an answer NOW, don't make them wait" while feeling their stare, and couldn't figure it out without help.
t0:{[3,5],[0,0]},
t1:{[3,5],[0,5]},
t2:{[3,5],[3,2]},
t3:{[3,5],[0,2]},
t4:{[3,5],[2,0]},
t5:{[3,5],[2,5]},
t6:{[3,5],[3,4]},
}
Thinking about making this an algorithm, are we forced to use recursion?
* Paid projects are appreciated by candidates who are currently unemployed.
It's not sufficient to decide if someone can code. You want some actual code written by that person.
And you don't need everyone on your team being able to explain things well on a whiteboard. Just try to figure out how they are thinking and how they are approaching a problem.
When we view the hiring process in the context of what it actually is, a sales relationship, we realize that the customer (i.e. the employer), is ideally looking to buy services and time from the business (i.e. the employee/contractor). Viewed in this context, we quickly realize that many different things become part of the equation. Think about hiring a contractor for your home. If you happen to understand the job they do, you might be able to assess a the potential contractors previous work, quote and reputation effectively. Most people can't because they generally don't understand what it might take to do a job and have no experience in executing it even if they do. So you either heavily rely on recommendations from others, or go for a lower or higher priced contractor based on your perceived value (cheaper job, w/e the results vs more expensive job and risk mitigation) and hope for the best.
Considering most software development doesn't actually happen at "tech" companies, the reality is most companies are like most people and likely can't properly assess talent no matter what. They essentially get lucky if they get a good candidate or unlucky if they don't. That would lead us to think that maybe the "tech" companies are able to do a better job of it. In essence they do, but not because of their methods. I'd say it's a form of selection bias.
Google became successful and happened to be run by generally smart people technically capable people. As such they were able to attract generally smart technically capable people to work for Google. I'd venture a guess that 90% of people who make it to the live interview stage at Google would be qualified to work there, but since Google has it's pick of the litter they need to create a filtering system somehow. They happened to make it the "whiteboard" interview and similar high achieving Peer companies all tend to do the same thing. It became a thing, and as such all non-high achieving companies followed suit so they can behave like the high-achieving companies. The filter a company like Google applies is only necessary because they have too many candidates. Most companies aren't in a position to apply that filter if they really want talent. So really, they need to get lucky enough to get those who don't interview well with the "tech" company method of interviewing and hope they get some of those prime candidates.
I'm pretty sure the tech talent pool, as with all things in life, follows a Gaussian distribution. If you interview enough people you're bound to find some decent ones, even if they don't actually interview all that well.
As the article mentions though, the term is a bit vague and is overly broad; for example if you're doing a system design question, I'd argue that the whiteboard is the best choice for an interview, since that's how you're most likely to collaboratively sketch the design of a system in the real world.
I think the problem can be restated at a more general level; does the interview test the sort of skills that the job requires? Most people here are correctly pointing out that the skill/knowledge required to write out pseudocode for quicksort is almost entirely uncorrelated to the skills involved in most software jobs.
If yes, the problem should be simple and relevant, and the session highly interactive. The assessment should be about (a) How interested the candidate is in the problem (b) How receptive they are to feedback.
White board coding interview are testing only one thing: how well candidate is prepared for it. And here what is wrong with it: The assumption that candidate should spend his/her valuable time preparing for someone's assessment is arrogant. I do understand why Google and Facebook do it (the do other arrogant things), they assume you want to work for them so much you will study to make the cut. So, they track bunch of metrics, which makes THEIR life easier. They have baselines, calibrations, and other things you never have in the real life. Hilarious part is that they are both risk averse (better to not hire a good candidate than hire a bad one) and using brood-force approach (they will interview you endless number of time).
And if you are not prepared - you will most probably fail, unless you are really good at it or lucky. So, why should I prepare for Google's or someone's else interview? I have a interesting and very busy job, I just don't have time for this. I have commitments to my team to work on real products, not on fake problems from Cracking Google Coding Interview.
Just think how absurd it is?
So, starting from 6 month ago I: 1) Tell recruiters upfront I will not be spending a minute preparing for their interview process 2) I decline coding on the whiteboard. High level designs are fine, writing code - no, no exceptions 3) And the most important, I stopped asking candidates to code on the white board.
The most important skill of the software engineer is (IMO): ability to keep complex problems in context for long time (not 5 boxes and months), ability to manipulate them and ability to convince others that you idea is worth pursuing.
Because you want the job? Everyone is busy, that's a terrible excuse not to brush up on your interview skills.
By the tone of your post, my sense is that prior to your recent epiphany you weren't very kind to candidates who you interviewed.
>> Everyone is busy, that's a terrible excuse not to brush up on your interview skills. I believe that their approach is arrogant, they just don't value candidates time.
I have a confession to make, I spent some time at Google. And after I left I was interviewing people using their approach and I regret it now.
P/S/ When you interview for a company, you actually taking more risk that the company (talking about folks who are headhunted into interviews). If it doesn't work out, you will be out searching for a job, you career growth may be slowed down. And what is the company risk?
I'm definitely confused. You don't want to waste time preparing for an interview, but you will waste time interviewing at a company for a job you don't want?
Yes, there are lots of companies hiring. But in my experience, in preparing for one software interview, you're preparing for others as well.
I interviewed at Dropbox, Amazon, Valve, and Indeed, within a span of a few weeks, and the prep work (Cracking the Coding Interview) was applicable to all.
That's different than asking "Why should I prepare for a job?"
If you want the job, you'll put in the effort to prepare, regardless of your feelings on the process. That sacrifice (2 hours a day for a month) is minor in the grand scheme of things. Granted to the OP's point, Google probably isn't worth it. But that's a value decision each candidate has to make for themselves.
I'm preparing for my brown belt in Krav Maga. The test is 6 1/2 hour grueling physical exam that includes a comprehensive test of all the material from the previous 4 belts. Not only that, we're the first group eligible to test, so the instructor wants to set the standard with us.
Not everyone wants to put themselves through that hell. But those that do will put in the effort to be ready come test time.
Plus, if some one has to 'prepare' for an interview that is supposed to test people's capabilities to do 'everyday' jobs, then you are hiring the wrongest possible people out there.
Whiteboard coding does a good job at assessing that, regardless of the candidates ability to actually solve the problem (for all the reasons why whiteboard coding sucks).
Do you analyze the problem?
Write test cases first?
Write a mini spec?
Gather Requirements?
Ask Questions?
Jump into code?
Google?
Verbalize the problems you encounter?
In that respect, engineering coding interviews work well. But too often, they are essentially a binary Yes/No: Candidate solved the problem the way I want them to solve it.