The tech interview process is different for every company but it's definitely not broken.
As a CTO, one of the things I look for is experience. You can look up all the "basic shit" you want but if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem. If I'm interviewing a senior engineer, I want you to be able to show your seniority one way or another, if we talk about architecting something, I want you to be able to talk about why we want to solve it this way, pros-cons etc. Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
> Why does this matter? Why do you want this? How do you correlate what you're stating you want here with actual job performance?
It matters to me because we want to be able to show that we can think through solutions. If we're building a feature that has something to do with streaming video, it would be a plus if you've shown that you have experience building streaming services or at least some knowledge along the topic. This should correlate with job performance because of the experience and working on this topic is already known. Now, there would be a case where we have 0 experience with this problem space but again it comes down to thinking through solutions and thinking about race conditions, side-effects etc and being able to identify these.
In the macro, there is a ton of demand for engineers yet short supply (particularly for universities). Thus the very high false negative rate observed at nearly every company indicates the interview system is broken.
What would help is more disclosure from hiring managers and interviewers about what they want and the success rates of candidates. That information is still largely today only available to hiring managers and executives. We need less information arbitrage for a more efficient hiring system.
More information just makes it more difficult to filter out the scripters trying to automate their job applications. That's the biggest issue I have seen, personally. There's too much noise on the channel, and giving any advantage to the CV spammers is not going to make it less noisy.
> Thus the very high false negative rate observed at nearly every company indicates the interview system is broken.
What high false negative rate? Can you point to some data?
By "false negative", are you referring to rejecting a good candidate? Are false negatives (rejecting a good candidate) more important than false positives (hiring a bad candidate)?
> We need less information arbitrage for a more efficient hiring system.
How do you define efficient? Whose efficiency are you considering, the company or the candidate? Why should efficiency be a primary goal of the hiring system, as opposed to simply finding the best candidates in a reasonable time frame?
Engineers are of course not a uniform thing and every role is looking for different experiences and traits, so you’ll get a lot of good engineers knocked back from interviews who go on to get a job somewhere else. This might be trite but thought it is worth mentioning as what seems “unfair” can often just be this.
> Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
If you feel that way, here's a viewpoint from Derek Sivers which talks about "Idea being a multiple of execution" [0]. If you genuinely believe that "solving a problem within a few minutes" can be reached, I'm going to ask you - What problems are you asking people to solve in a limited time frame?
In my experience, speed is the opposite of key for longer term success. Speed is how my companies have ended up with designs and implementations that weren't maintainable and/or had big gaps in them.
I'm sure the usefulness of speed varies by context
In my experience, getting a bad hire is worse than not hiring at all. It takes a huge amount of resources to properly onboard a new team member, and then if they are not performing well the work they were doing ends up becoming a liability instead of an asset to the team.
I think part of the problem is that places with coding interviews usually pay better than those without, and the ones with the really painful, idiotic, grueling interviews that everyone hates offer the top pay, so the system ends up sorting all the worst candidates to the companies that don't do "normal" coding interviews.
In fact, I think that effect is exactly why the top-paying places do it, because it makes their hiring easier. At this point, if any of them unilaterally drops the hellish, silly, hazing-interview process, they'll be inundated with good candidates they've been excluding plus a shitload of bad ones who'd already been filtered out by the fact that their process is hellish and silly. So none of them will.
Someone wrote a really great article for how you should play this as a company if you can't pay the best.
Unfortunately, don't have the link.
The premise is: you should be aware that there's a large subset of good engineers who aren't particularly great at coding interviews - and you should set up your interview process to select for those candidates.
I find that pair programming with the candidate on their laptop, using their preferred tools works well.
It takes time and in many ways it requires more energy from me than if I were to just sit and observe. However, it gives a better feel for how the person goes about solving problems from the abstract, architectural stuff to the nitty gritty. I love it when pairing with a fellow vim-user as I will inevitably learn some new trick.
Totally! On their laptop being the key. I interviewed at Pivotal around ~6 years ago and the person had some customized VIM setup which was a PITA to work with. He had a very good workflow with it and I honestly looked dumb trying to achieve the same things as him in his computer...
I think that they want to hear is thinking. So every time I am during technical interview, I just talk out loud what I see in front of my eyes. That is great because they get to know my thinking process and it works as a rubber ducky for myself.
This is something a great interviewee does. I don't try and trick interviewees or make things unnecessarily hard. I really just want to see "can you think like a programmer". When an interviewee gets stuck and isn't saying what they are thinking about, it makes it really hard for me as an interviewer to give good prompts.
Obviously, I can't give away a solution, but at the same time some people will get stuck on things like "Can I use X" or "can I use Y". And for me, that answer is almost always "Yes, absolutely use X!". Because that's what I WANT from someone I'm working with. I don't want you to write a sort from scratch, I want you to call `Collections.sort` or even have the wherewithal to ask someone/google "Hey, I know Java has some inbuilt sorts, how do I invoke those?".
The only time I'll maybe mark you lower is if you start the interview by saying "Yes, java is my favorite language to program in and I use it all the time!" and then struggle to do `new ArrayList<>()` (which has happened on multiple interviews). I always try to get someone to use whatever language they are most comfortable with. Because, to me, languages are stupidly easy to pick up and which one you are most familiar matters far less than how you use it.
There is a limit to the proposed approach. Most interviews I conduct (algorithms) include puzzle-like problems. So if you just Google them - you would find a complete answer. In that case, what's left for the interviewee to solve?
(Whether algorithm questions should be a part of the interview cycle is a different matter.)
I interviewed at SkyScanner around 5 years ago and they gave me some problems that I already had seen since I was interviewing like crazy. I solved them very easily and the interviewee had nothing else to say. Then her feedback to the interview was negative because it looked like I had already seen the problem.. like... :shrugs:
I've seen some rather complex problems being asked in interviews. I've always wondered why my credentials don't seem to matter when it comes to programming skills. I have a masters degree where i did some very tricky algorithms and I even open-sourced it for all to see. But every time I've been looking for a job, i have to rehearse some leetcode problems so that i can put on a good show for 45 minutes.
I get that deep-dives matter a lot and i can respect some of those exercises. These are crucial for senior positions since ideas are more important than code. Then I really get the life story kind of interviews because relating to coworkers and culture fit are hugely important.
It's just the dang leetcode that ticks us all off isn't it?
I was also a fan of this tweet until I found out what inverting a binary tree actually is. Being asked to code relatively complicated algorithms like balancing a binary tree is one thing, but this guy essentially failed FizzBuzz. At that point it’s not about testing the limits of your candidate but filtering out the low hanging fruit.
Obviously I don’t have the whole story of the interview, but it makes me a little worried for homebrew.
Inverting a binary tree is essentially trivia in the context of most programming, honestly, if I had to do it without a library I'd be extremely worried I'd fuck it up in some subtle way. Max Howell obviously has a strong understanding of file systems, web protocols, package management, and a whole other suite of skills I'm too stupid to understand.
No, but I assume that the inteviewer will elaborate if you ask them what they mean. I don't consider people who don't know what FizzBuzz is terrible developers, not knowing how to implement it after an explanation is a red flag though.
No, it's not FizzBuzz. It's a recursive algorithm, which FizzBuzz isn't. (Or doesn't have to be. Let's not cite crazy repos with dozens of solutions deliberately obfuscated.) That automatically takes it out of FizzBuzz range.
If you're a functional programmer, it's probably something you can just bang out. I've done functional programming. But it's been a while. In an interview situation, I'd definitely need some time to struggle through it.
What gives? I claim to be a senior developer of ~25 years experience? Why can't I just bang out the roughly ten-line solution it would be in Haskell, including the requisite data declarations? Well, in my 25 years of experience, yes, I've covered everything that is necessary to do that problem, but I've got to drag it out of storage and rehydrate it manually, because I don't do this sort of thing every day lately. Or ever, really, because what senior dev is running around implementing Computer Science 301 data structures every day anymore? Pretty much just one currently implementing a brand new language that needs those basic structures in its standard library.
I'd get there. It wouldn't even necessarily take long. But I can understand why an interviewer asking this question ten times a week who knows the answer forward and backward might find taking even 5 minutes to be an unimpressive performance for something that such an expert should clearly just be able to scribble down in one shot, right?
And I'm gonna be honest. If you ask me to balance a possibly-unbalanced sorted binary tree, I'm basically going to be re-deriving the answer from scratch. Yeah, I covered it. Probably even did a homework assignment on it in C. But my job for the past 25 years has not been implementing balanced binary trees.
People cheat their way through credentials all the time, having a masters degree doesn't tell you anything about a person except they maybe sat in a room on their phone while someone discussed the nuances of Church's Thesis. And most interviewers aren't valued enough by a company to really give a shit who you are, there is just too much happening and too many interviews, so they just can't be asked to look at your Github.
The game and well are poisoned at many levels. There's just too much money on the table at most modern tech companies to trust anything.
I do quite a few coding interviews for my company and, frankly, the pass rate on the coding interview is dismal. It's one of our most effective filters for bad candidates.
What sort of torturous questions do I ask? Merge 2 sorted arrays is one of my favorite.
How many people can pass that? less than 50%. A good number of people don't even get to the "concat the 2 arrays into 1 array and sort" sort of solution. It's really that bad. These are people with "15 years of software development experience" failing really basic coding problems.
I really wish I didn't have to ask that sort of question, but at the same time, I don't want to work with anyone that can't solve it (even "under pressure").
The fact is, a lot of programmers will go off the deep end with simple problems. This joke[1] still relatively accurately describes the mentality of a bunch of programmers in the industry.
I got to wonder, are all those people really incompetent? See, another take is that they all clutch when being interviewed. And maybe not every interview; maybe it matters how the interviewer approaches it.
It's not 'under pressure'. Real pressure is schedules and deployments and compute constraints. Not somebody looking over their shoulder.
I'd love a different solution, but this is about the best I've got. I can certainly see that there are some false negatives, but to me, I'd rather take that risk than take the risk of hiring someone that's a true negative. Having worked with some in the past, I really don't want to touch code written by a dev that can't code.
I'm not sure how people couldn't merge two sorted arrays when a competent fifth grader could figure that question out without ever touching a computer.
Some people, even those with 100s of hours of schooling, just never learn how to think computationally. They just "fake it till they make it", which really just means they have no ability to provide value to any company.
Did you know that there are Go problems that professional players can't solve, but that newbies solve trivially? Becoming a professional go player creates blind spots, basically any board that could never happen in a real game just don't make any sense to someone at that level.
And while, I don't equate merging 2 sorted arrays with being an unsolvable problem for an experienced dev, I have personally experienced similar increasing difficulty to toy problems as I gain increasing real world experience. Common things like timestamps and full names are not as simple as they seem, and so now, I'm less likely to just jump to an answer that would have seemed obvious to me 20 years ago. And it's not like I come up with the solution and discard it, it's more like the previously obvious solution is no longer the first things I think about.
I can definitely write code. I've been doing it quite a while. I've been the "smart guy" to go to with hard problems, at more than one place. Feedback has been very good at most places I've worked. I'm the guy who figures out weird platforms and things like that. I also clean up alright and can talk to people. I'm not bad at the job, and would be, so far as the technical side of things, a good hire for probably 95+% of all software development jobs there are that even somewhat fit in the set of things I've already worked on (which is broad). I'm good under on-the-job pressure (great, even—I kind of like emergencies, in a perverse way. Clears the mind, demands focus on a clear goal.)
I am also 100% certain that I've given interviewers the impression I don't know how to code whatsoever on at least three occasions, including as recently as a couple years ago. Hell, I'll probably do it at least a couple more times before my career's up. I guarantee that the interviewers were chatting after and were like, "see this is why we have to do this, because you get guys like that lying piece-of-shit yahoo, who've somehow had a full career and have great references and can talk about software and processes and architecture and all that, but inexplicably can't code at all".
I'm terribly self-conscious when people watch me code in the context of an interview where they're explicitly judging me so I get very slow even at using tools I'm familiar with ("I think that shortcut does what I want but... what if it doesn't? I'll just do it the slow way, oh no, now I look clumsy and stupid, fuck fuck fuck!"), and I tend to do stuff like forget the syntax to invoke a method in [language X that I've written 500 lines of in the last two weeks] when I start writing with a blank file (no existing syntax to crib off of, and c'mon, can you ask to google shit that basic in an interview? LOL no). Basically I have intense social anxiety but only for specific parts of interviews.
What I can do is talk through architecture, and any amount of conceptually-how-things-fit-together I have in my head, including low-level stuff. I can do 100% of the talking parts of an interview just fine. I'm told I come off exceptionally good in that part, actually (I can't tell, I just go by feedback). I'm sure this just reinforces the impression that I'm some kind of god-tier social manipulator who's, for some unknown reason, decided to scam his way into mid-tier tech jobs rather than applying such outstanding bullshitting skills to something that would pay for them more directly, and probably even be compensated higher if I were that good.
Ok, so here's my advice for you in that sort of situation.
Just talk. A lot. What are you thinking, why are you thinking it? What are the alternatives you are thinking about. All of these will give your interviewer a good opportunity to give you feedback and can make the problem a lot easier than you might be expecting.
Yeah, I know, I do try to do that. I just don't really interview often enough to get much better at getting over the anxiety, because despite that I've never really had trouble finding good jobs. I only bring it up to confirm that yes, some of those "fakers" (maybe lots of them) aren't faking, they're just exceptionally shit at a particular kind of interview. Truly, I expect that the set of people who can do a really good job of talking the talk but aren't actually competent at writing code is pretty small, because faking being able to program and talk off-the-cuff about technical topics reflects a fair amount of talent at something else that ought to pay pretty well on its own, in other roles.
I think the way those algo-quiz or live-code-a-thing filters are used tends to catch people who can't talk well about it or do the job (this is the majority of would-be "bad hires", and you'd catch them anyway), some set who are afflicted with extremely bad general social anxiety so who are just fucked at interviews generally (I feel bad for those folks, especially since some of them are probably fine in normal work-social situations); the ones like me who can do the job, and can talk about it fluently, but suck at "coding interviews"; plus some very small set of actual social geniuses who've decided lying their way into so-so tech jobs is what they want to do with their lives, and have the talent (at faking it) to back that up, which are the ones people seem to think are super-common in the wild, for some reason, which seems very implausible to me for the simple reason that, again, that's a skill that's per se valuable elsewhere, if you're actually that good.
(frankly, from what I've seen on the other side of the table, I suspect you could weed out a very high percentage of actual-fakers by just blanket rejecting anyone with any kind of graduate degree in CS, who's applying for a mediocre position rather than something flashier and better-paying, though obviously that would come with its own set of pretty bad false negatives and I wouldn't actually do it, and anyway even those people usually don't "talk the talk" well enough to be convincing, if you get them off-script)
[EDIT] incidentally, FAANG and finance and a few of their peers are the ones where these kinds of aggressive weed-outs do sorta make sense, because their comp is so high that those "social geniuses" I mentioned might actually make more money faking their way into those tech jobs than applying their outstanding non-tech skills more directly to another role, such that it's worth it despite the stress and risk of being "found out".
I could definitely see ditching the coding questions on a lower paying job or a small company. You'd learn a lot more by bringing someone in for a month long probation (so long as they can talk the talk through the interview).
I could also see doing different interviews if what I did didn't have a lot of algorithm writing requirements. It wouldn't make sense for someone to consider algorithmic efficiency if what they are doing all day long is writing basic CRUD apps.
For us, we've got a lot of specific finance logic and calculations that we need to implement. It's not (always) straight forward "write some crud".
In my 20+ year career as a web developer and software engineer, I have never ONCE had to merge two sorted arrays as part of my job.
If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
> In my 20+ year career as a web developer and software engineer, I have never ONCE had to merge two sorted arrays as part of my job.
Cool. It doesn't come up in my job either. That's not the point of the question. The point of the question is "given an algorithm specification, can you implement it" and that problem, particularly at my current work place, comes up very frequently.
> If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
All my coding interviews are done via hackerrank which has an OK IDE. I generally also allow googling or questions from the interviewer.
> Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
> At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
Congratulations!
Honestly, I think you are letting this sort of question intimidate you too much. It is an easy question to answer and if you sat down for a few minutes I'm sure you could solve it. It's not a trick question. It's not one that's designed to filter someone of your qualifications. It's simply a "can you write an algorithm" question. Something that's really important in finance because a lot of what we deal with (particularly where I'm at, financial reporting) is writing code that matches financial models and algorithms. There's not a library you can reach for that computes everything we do.
Give it a shot in your favorite language and I think you'll be pleasantly surprised at how easily you can write a solution.
Oh, don't get me wrong. I have done my fair share of "leet" and toy code interview problems over the years, and for a while I even used them when I was interviewing and hiring other devs myself.
But then I had some personal experiences that changed my perspective...
There were a couple of jobs that I applied for where I was perfect for the role. Every single piece of technology they used and listed in the job description was something that I had mastered and used with success at other companies.
I got turned down for those roles, however, because I didn't ace their stupid leet-code bubble sort style questions during their interviews. These were trick problems that had NOTHING to do with the actual day-to-day work I would have been doing.
There were a hundred other things they could have looked at to determine that I was a good fit. They could have asked me to show some of my other work. They could have looked at the 100+ projects in my Github. They could have looked at the 20+ written recommendations on my LinkedIn profile all vouching for me (including glowing recommendations from other developers I worked with over the years, many of whom are now working at companies such as Facebook, Apple, and Google.
But nope. Instead of doing any of that, they would rather rely on some leet-code trick questions to gauge my abilities. As a result, they missed the forest for the trees and they lost out on what could have potentially been a perfect fit and a valuable hire.
So yeah. That's why I don't do that type of interview anymore.
If the job is actually for a role that literally solves algorithm problems on a day-to-day basis, then a code test for algorithm problems makes sense. If not, then please don't use them!
Notably, he was never asked to invert a binary tree and I don't believe he ever got confirmation about why he was rejected. Given how he responded, I wouldn't be stunned if the answer was not his technical performance.
I have definitely been part of interview processes where the team unanimously rejected experienced candidates who express visceral negative and incredulous reactions to being asked to demonstrate basic programming proficiency. The story here has hints that it may be in this category. Some candidates with decades of experience expect to immediately have philosophical discussion about where software is headed, and offered their choice of management/architect positions without talking about code, and get upset when the interview starts out with some softballs. The irony is they get ejected before the interview gets to the good stuff, we do ask about philosophy and management, but a bad attitude can take that off the table very quickly.
I'm not sure I understand your zinger. Are you saying that it's unreasonable to ask experienced programmers to demonstrate a small amount of their programming ability, or to ask basic or easy questions?
We don't know the full story behind that tweet. Inverting a binary tree is not exactly a difficult problem. They could have even spoon fed the author with "Ok, here's the code to traverse a binary tree. This is what it means to invert a subtree. How would you invert the rest of the tree?"
I've had plenty of candidates with great credentials who utterly bombed relatively simple coding problems. Not fizz-buzz but also not that much more difficult.
Our tech interviews focus on watching you work the way you actually work: Google allowed, your IDE of choice, your boilerplate project templates, etc. Yes, you still have to show progress against a problem in 30-60m -- but sometimes rapid prototyping is important to the job!
Hold on, how are they “fundamentally broken” if the author is happy with the format with just a slight tweak (allowing use of Google, which many places do already)?
I see that phrase a lot and it’s often deployed without a great deal of thought, which is a good indicator that what I’m about to read is a pretty low-effort article, blog post or tweet.
I think the author is trying to say that interviews are testing for the wrong thing (ability to solve tricky problems quickly, on the fly, and with no help, which is unrealistic in the workplace) when what they should be testing for is good analytical and research skills, with the assistance of the internet available, just like in real life.
Most likely because an interview is not a probationary period where the candidate does actual work. Neither can it simulate the important parts of doing the work. So you have to look for proxies you believe correlate well with being able to do work and test for those instead. Things that can be reasonable measured in 45 minute chunks, thus the proxies you pick will not at all look like someone doing actual work, they will be far removed from that. Hence they will not be "realistic". But that doesn't mean they aren't informative.
So while it's true that having a candidate be able to write code that implements a stack and or whiteboard a simple app is not a realistic scenario of what they will be doing day to day, if they are able to do that well than they are much more likely to be able to do the work than if they are unable to do it well. So you use that as a proxy. Then 5 people interview for an hour, each using a different proxy, and you look at all the information the proxies are telling you to decide whether the candidate should be hired.
>Neither can it simulate the important parts of doing the work.
For sure it can. You can combine problem solving, ability to read code, write tests, avoid common pitfalls, clarify requirements all in the space of 45 mins.
I typically do this by taking a chunk of real code out of the code base, uncoupling it so it can run on its own and writing some stories.
Without the background information you are not really simulating what happens. But I also took some examples from our code base, abstracted them a bit to remove dependencies on our libraries, simplified the logic and used them in interviews as a "find the bug" type question. I liked that it because you have to read the code, be able to describe what it is trying to do, and then figure out where it goes wrong. Those types of questions have served me well and I always try to ask one when I am asked to interview.
But that also was not a realistic description of what happens during a regular workday, but it tries to cover one of the (many) skills you do need, so I think it was a decent proxy.
I am not advocating for lots of weird trick questions or brainteasers, but simulating the entire process of design, implementation, test, debug, document, sprint planning and the like. You can't realistically do that in an interview, you pick a few proxies and hope for the best.
It sounds like you achieved a close enough approximation.
I find interviews like this give very strong (& often surprising) signals.
I don't really object as much to people who try to approximate and don't quite get there than people who decide to just test something that nobody would ever actually do in their job.
It's like the difference between having buggy code and a product that just does completely the wrong thing.
Neither can it simulate the important parts of doing the work.
I disagree with this. With some effort, you can create a small sandbox app that recreates an exact bug or existing implementation that needs a a feature added to it straight from your day to day work.
You can even translate your obscure problem domain into the appropriate analogy.
A small pet peeve of mine is when someone criticizes the interview process without having been on both sides, and assumes that they know what the acceptance criteria is. This article is a good example of that:
> They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web
This is not quite right. It's right in the sense that if your resume says 3 years of JS experience and you're spending 20% of the time fumbling with unmatched brackets, it says something about you. It's right in the sense that if you say you've done React at two previous jobs but proceed to call setState multiple times in series and then scratch your head when stuff doesn't work, that says something about you. Etc. However, it doesn't mean what a lot of newbies seem to think (that when receiving some leetcode-like task, they're supposed to have the exact ideal answer memorized)
In my interviews, one of the first things I tell candidates is in fact "feel free to google stuff, I'm not here to test whether you've memorized stuff, I'm more interested in how you approach problems".
At least for FAANG/bay area tech, good interviewers looks for signals, and not just purely technical rote memorization: on a basic level, one looks at whether the candidate has working familiarity w/ syntax and APIs that they ought to know, but on deeper levels how they think about code structure, refactoring, testability concerns, design trade-offs, how effectively they communicate (both speaking and listening to feedback), etc. There may also be smaller considerations for legitimate "fit" factor, for example whether a candidate is more startup oriented or more enterprise oriented (this mostly matters when you think about their career aspirations vs what the job is able to provide in terms of challenges).
You can get a pretty decent feel for how "senior" a candidate is, and then acceptance becomes more a question of placing the candidate on a sliding scale based on how demanding of a role one is trying to fill.
The one criticism I have about the interview process is that many interviewers don't tweak their expectations based on level quite as much as they should, especially if they only hire sparsely for low seniority levels. For a junior hire, there should be an expectation that some ramp up time will be required. Expecting upfront solid experience with a laundry list of technologies from them does a service to no one.
This is a bad take. Coding interviews help you see how someone solves a problem. And good coding interviews let you search for whatever you want except for "how to do this problem in x language."
I've done more interviews over the last two years than I'd like, and honestly - I have no problem with folks googling "how do I solve X in language Y".
Frankly, we tell folks up front - "This is not a memory test, please feel free to ask questions, use google/stack overflow/man-pages/etc."
I'm in the interview to give you a problem that looks as close as possible to the work we're going to ask you to do, and then discuss with you how you solved that problem.
If you copy from stack overflow and it works, great! Now we can talk about possible edge cases, performance issues, testing, add a new requirement, etc.
I don't really care how you get the code into the file, or where it comes from - I care that you understand it, and can think through how that code will perform in different situations. Most importantly of all - I care that you can tell me that information in a coherent fashion.
What is your process for evaluating "how someone solves a problem" and how does that inform your hiring decision? Does it simply come down to "does this person solve the problem the way I would"? How do you address confounding issues like performance anxiety induced by constraints not present in a normal work scenario?
My observation is that many companies don't address those problems well.
I like the idea of letting the candidate do whatever they would if they were actually coding. I even think it could be interesting (in the initial part of the process) to ask the candidate what question they would like to be asked, and let them implement that.
Few things to point out though:
1. you can ask the interviewer questions
2. the problems are usually simple enough that you don't need to google them, and should have prepared for them in the first place
3. you should be somewhat efficient (being able to do basic stuff without googling) in at least one language, since you can choose the language for the interview do that
I'm lead dev on a rather large frontend project, and I still feel like I'd struggle in a frontend-focused interview today. Underappreciated by many I think is the ability to solve problems. I solve them (almost) every day, but sometimes it's easy, and sometimes it's really tough. But eventually, if you know how to ask the right questions, you'll get to a workable answer. That kind of real-world experience doesn't seem to surface well in a typical interviewer/interviewee type setting.
And once you are a lead you are doing less of the stuff day to day that would make you shine in a coding interview. Especially the ones where “we need a c# engineer so let’s dust off those weird exception handling edge cases again and test on it” when in real life you look it up or run a unit test over it (better!)
If the title was a comment, it would be breaking the site guidelines.
More importantly, it's throwing out absolute opinions on why interviews are broken without demonstrating a real problem. There have been hundreds of articles saying hiring/interviews are broken because <logic/opinion>, and almost none that point to data showing that companies are not able to hire developers, nor that developers are statistically unable to get jobs.
Interviews can be difficult and frustrating for both sides, but that doesn't mean they're not working. Despite the fact that the article is correct that looking things up is routine in a dev job, I still may prefer to hire the candidate who doesn't need to during my interview question that was specifically designed to not really need to.
I don't know how often coding challenges disallow using the internet, but the last time I interviewed it was allowed. I've seen lots of companies allow use of any language, and spend your time searching the internet if you want.
We literally have thousands of research studies demonstrating that place of birth, income, gender, and even names bias hiring.
If it was this prilliant data driven meritocratic nirvana of scientific precision, we would never have issues with discrimination again minorities, the poor and people who looks funny and have tatoos.
The burden of proof is on folks that keep pushing these silly tasks, where is the data "we've added some stupid questions to our interviews, and now we are doing twice better"?
> The burden of proof is on folks that keep pushing these silly tasks
This strikes me as a somewhat bizarre and myopic way of thinking about jobs. There is no burden of proof at all, because there is no claim being made. The deal here is someone will pay you money, if you agree to do the work they ask. This opportunity is available to people who make it through the interviews in order to demonstrate they meet the qualifications for the job. Companies that are hiring are the sole decider of what these qualifications are, and can choose to have them demonstrated in any way they choose (up to what we agree are legal limits.) If you don't want money, then don't submit to "silly tasks". (I've never witnessed silly tasks in my interviewing, ever. I've had questions for which I don't have experience, and questions that are too hard, but the questions have always been pretty focused on figuring out what I know and how good of a programmer I am, and by extension how much they are going to pay me...)
Your reply is written as if a system that is trustworthy is being accused of being untrustworthy.
Instead, a system that we know runs poorly in 10 ways is claimed to be run poorly in 11 ways.
That is a dramatic difference and this would be a valid argument in a court of law - if the system was never run properly, why do you expect it runs properly when it comes to this particular matter? Why would anyone put their faith in it?
> That is a dramatical difference and this would be a valid argument in a court of law
What are you talking about?
> Why would anyone put their faith in it?
They need money, maybe?
Because software developers are, by and large, employed (not to mention paid well)?
Because the economy is functioning?
I don't understand what you're suggesting, what is the alternative to interviewing? What do you want fixed? Why are you coming down hard on me for suggesting that not being allowed to Google the answer to your interview question isn't a serious problem that needs solving? How would you hire people if you had a company?
What does trust have to do with interviewing? Does one need to have trust in order to get a job? Do you have a job? Aside from cultural bias, what are examples of the system breaking trust in large numbers that demonstrate that hiring is "broken"? Where is the data that shows this?
Since you want to talk about trust, why should a company trust your or your claim of being qualified for a job without asking you to demonstrate you're qualified, in a time-bound and resource-bound setting? The interview questions you called "silly tasks" are nothing more than a request to demonstrate a very small amount of your claimed education and ability. Is that unreasonable? Why or why not?
The following statement is not very convincing: "maybe hiring is broken up when it comes to culture, gender, race, personal connections and class, but when it comes to programming tests, we certainly have it right"
Mistakes in other areas indicate mistakes are very likely in this area. Parallel with courts was meant to illustrate that
"They need money, maybe?"
If thsts the only reason, they won't put their faith in it, they will game the system to the maximum possible degree.
I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
Maybe we need qualifications that can actually be trusted, thats one possible solution.
> Then they won't put their faith in it, they will game the system to the maximum possoble degree.
Hence the request to demonstrate basic proficiency?
You seem to be inventing reasons to criticize and argue with me about subjects that neither I nor the article I responded to brought up. I didn't claim that programming tests are "right", so this is like your 4th straw man in a row. You don't seem to be answering any questions about how to improve the situation. Aside from bias, you haven't yet demonstrated a problem with hiring from the company side or the candidate side that has data to back it up.
What would you do to improve your perceived imperfections in the interview and hiring process?
> I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
This comment makes it seem like you are ignorant of and not even remotely aware of what the qualifications and interviews are like in other fields, and how much worse and more difficult they are than software. Surgeons go through years of low paid residency, and accreditation, and they suffer a whole battery of knowledge based tests where they're not allowed to Google the answer. Same for structural engineering. The fields of surgery and structural engineering both have cultural bias problems, just like software.
I do have data: I and a friend of mine have refused to do coding tests. It was really funny because frequently hiring folks start acting as if I insulted him. And we could still find a job, and it's not overrun by idiots or incompetents. The standards are not 'lower' and the company does not perform worse or pay worse.
So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
> So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
This is false. Your assertion that doctors don't perform in order to demonstrate their qualifications is wrong. They do have to perform multiple dissections and surgeries before they can get hired anywhere.
> I do have data: I and a friend...
That's not data, that's personal experience.
Your "data" is proving my point that coding tests aren't causing a hiring problem.
Last chance, third time I'm asking: what would you do to fix it?
But Third time I am answering: have trustworthy qualifications.
But you ignore most of what i write, and provide no evidence to support your position that this charade of hiring actually achieves what you think it does, so at this point I no longer believe you are debating in good faith
You didn't start this argument in good faith, you started with a straw man and then added several more. You have also misquoted me and ignored most of what I wrote.
> have trustworthy qualifications.
This doesn't answer the question. People already have trustworthy qualifications, so this solves nothing. Your own point proves your answer wrong: people will game the system. You're contradicting yourself.
Doctors and structural engineers achieve trustworthy qualifications by doing a 4 year residency / apprenticeship program after their degree. Is that what you want in software? You would prefer years of supervised and low paid work rather than spend a total of a few days of your life doing easy coding tests?
So trustworthy qualifications solve nothing and we already have them, except we don't and they do solve the issue for structural engineers and doctors as you pointed out exactly they are more trustworthy than anything we have in software.
For the fourth time, yes, thats exactly what I want. Having two years of supervised work would dramatically improve this industry
> For the fourth time, yes, thats exactly what I want. Having two years of supervised work would dramatically improve this industry
You do see that this is the first time you've mentioned in this thread spending time (years!) on a certification process apart from school, right? I've re-read all the text above and can't find the other three times that you answered my question with anything concrete.
What problem, exactly, does this solve?
> So trustworthy qualifications solve nothing and we already have them, except we don't and they do solve the issue for structural engineers and doctors as you pointed out exactly they are more trustworthy than anything we have in software.
I feel like you're not being either clear or fair with your jumping between talking about interview process and certification process for other fields. Maybe I'm not being clear either. Let's be more clear.
When I said we already have trustworthy qualifications, I was talking about school degrees, because that is the primary thing we have in software. In your reply here, you are conflating apprenticeship programs (doctor's qualifications) with school degrees (software developer's qualifications).
School degrees, unfortunately, while they are trustworthy in what they provide, are not providing a guarantee that an applicant is fit for industry work. That's what an apprenticeship program does. Again, like above, the issue is not one of trust. The issue is what the qualification is qualifying, and what the hiring company wants to see.
Ultimately, I'd rather do a few easy coding tests when I change jobs than have to spend 2-4 years in some kind of certification program. People who have to do certification programs complain all the time that the work is bureaucratic and jumping through hoops and irrelevant to their eventual jobs.
And, certification programs have not solved the only concrete reason you've cited as a hiring problem: cultural bias.
Where are you interviewing that they don't let you google stuff? I've gotten to the point that I don't even ask anymore, I just open a new tab and explain what I'm looking for. If the interviewer takes issue with it that's a big red flag for me.
I see posts like this all the time on HN and I really wonder if there's an alternate world of really stupid interview practices out there that I've just been lucky enough to avoid.
In this case: I've never heard of, participated in or administered a programming interview where the candidate couldn't Google things.
I remember when I interviewed at Braintree, I spent a good chunk of the pairing interview searching for Linux command syntax because I was new to using it. I got the job and spent 7 years there.
I'm not sure there's much of a clear line between the two. Google asked me to solve what was really a brain teaser, but solve it using code. Don't think I was allowed to Google anything.
That's fair, in that situation there's probably a valid case for disallowing use of a search engine. Whether or not that's an effective interview is another story :)
Were you the only candidate? I mean maybe they interviewed a few people, who already knew apt. So it’s a choice between hiring someone who already knows the tech stack they use, and someone that doesn’t.
To take this even further, it is possible that they had this same situation before, went with the one that didn't know the stack, and regretted it terribly.
Yeah I guess I was just thinking that this kind of thing seemed totally irrelevant to the job description which was for a CUDA application engineer, but maybe it was a lot more important for their people to be facile with ubuntu cli in front of customers.
When I interviewed for Facebook, the interviewer berated me for the coding pad losing focus with an alt-tab for a second ("What are you doing?").
I wasn't aware they were tracking this, of course, nor that it wouldn't be kosher, and I specifically asked about whether I could look up Java API at the start and did not get a "no" ("We don't necessarily require working code here" is pretty ambiguous).
I passed the technical round, and then they ghosted me after asking what my technical level at Google was (which I didn't have to tell, as it's hidden even within Google).
Probably very controversial opinion, but so far, my favorite type of interviews have actually been take-home problems.
Of course, it sucks if you've invested time into those, and get ghosted afterwards - but luckily I have yet to experience that.
Some people excel on taking test, some people excel on projects, others in both - I've personally never been the type to perform well on banging out code on whiteboards, but I will perform well on projects.
edit: Let me expand a bit on this
All my take-home problems have been very reasonable. Usually they've been more big-picture / architecture oriented, and not some specific implementation - some coding, some presentation, usually takes 2-6 hours to complete, depending on how much you put into it.
It is of course unfortunate if someone decides to code up something for a whole weekend, like 20-30 hours, only to get ghosted or similar. I guess it comes down to identifying free work from actual interview problems.
Personally I don't find making my tax situation more complicated doesn't help with the stress of job applications. I did a work trial last year and it added a whole new layer of anxiety about figuring out how i'm supposed to declare ad-hoc income (which in this case was foreign, making it even more complicated).
If you're used to something like the UK's PAYE system, being paid for interviewing is far from a no-brainer.
In this case, given the choice, i'd just have opted to do a few more interviews rather than investing a week in a work trial. I was burning through my savings due to being unemployed for ages, so exchanging a week of my time for a charitable donation wasn't really on the cards -- i've no doubt they would have done that if i'd asked.
Not to mention that I now consider mandatory work trials to be an unethical practice. The stress of an interview, but stretched to last an entire week. A week where you feel you can't make any mistakes. I'll never do another one, it's not worth it.
The fundamental problem with take homes is that they produce a huge asymmetry in the cost of interviewing. The company can waste hours of random people's time without having to spend a single minute on them. So while I understand that paying someone for a take home complicates things for some individuals, some kind of contractual agreement that the company will reciprocate some amount of good faith effort is necessary to avoid abuse. Payment is basically a proof of work, and money deposited in my account is a sure way I know of that a company expended some resource on my part. I'd say that even just offering to pay standard contract rates would make me more likely to actually do a take home.
* Here's a problem that takes 6 hours now do it in two.
* Ghosted
* Bizarre feedback that suggests they mixed me up with a different candidate.
* Emailed while I'm half way through the task to let me know that the position has been filled.
* We aren't prepared to spend 5 minutes looking at your portfolio to see if you can code to a base level but please do our take home problem that takes an hour and a half to prove that you can code to a base level.
The problem with take home interviews as I see it is that it costs the candidate X time and the company roughly 0 time. This incentives companies to give them to anyone and everyone. So you may be considered a marginal candidate who has little chance of passing unless you prove to be an utter genius in the take home. With in person interviews you'd just get rejected but with a take home they have nothing to lose by giving it to you. So it's a system that can work when few companies do it but utterly implodes at scale.
I've had some take home interview assignments that were paid at my freelancing rate. In my opinion that's win-win; the company has the right incentives and also saves engineering time since interviews are shorter while in the worst case the candidate made a little money on the side.
At my company, we usually do the more in-person interview, but if we suspect you have potential and your performance is negatively impacted by the stress of the interview process, we propose a project interview to wrap up the decision.
Since it's more time consuming we don't offer it up front, most people don't like them, but it's a useful tool to have to size up a candidate.
The post-project debrief provides a lot of information on how you think and how you work, the end result is not expected to be perfect, or even complete, as long as the approach is reasonable and we can picture if the person-team fit is there.
A good question for ppl in the coding challenge camp.
What new information do you neeed from a coding challenge that you don't get talking shop with the individual?
I don't think low level technical stuff is something you can BS your way through an interview. If it works on you, then you as an interviewer has major gaps
"Just 4 hours" for a chance at the job, and how high a chance depends on the employer (i.e. how early in the process they drop the take-home exercise on you, whether you've already come to terms on pay at that point, et c.)
In general I just wish places that weren't FAANG would be more open about their process. Do I need to brush up on leetcode shit before applying, or no? Are you gonna quiz me on obscure corners of a language that I reflexively avoid because they're terrible, so haven't thought about in a while? Gonna talk architecture? Will I be writing code with someone looking over my shoulder? Pairing? On what kind of thing, generally? Is your filter for the 200 candidates who apply to hit them with a take-home assignment or longish automated test first thing, so I know to just skip applying at all (screw burning that much time to apply for the privilege of getting an interview at just one place)? With FAANG you know what you're getting in to, so at least you know you're about to step in a big pile of shit and can put on your waders first. The blind roulette wheel of hiring outside FAANG is its own kind of trash-fire.
I remember this with a newborn I spent 20 minutes on such a test. Recruiter tells me they said no because code quality so i gave it another shot spending 2 hours. Remember with multiple applications these times added up.
Got to the interview and they were so obsessed about this being written in some “perfect way” (see the enterprise fizz buzz someone linked to) that they knocked me back when I was explaining it at the interview.
It’s the coding equivalent of “ooh so you are not using a suse vide” at a mates bbq.
When I interview using coding problems, you just need to know very basic Java or python to solve the problem.
We also nudge interviewees in the right direction to make sure they don't get stuck, which would be in no ones interest.
There is also no hard requirement that you need to solve the problem completely. If you talk to someone for 45 minutes about a coding problem you will get a good sense of their performance, even if they don't solve the problem.
Coding interviews are great (except making the intervieww very nervous), but you have to be mindful as an interviewer.
I disagree that coding challenges are stupid, I actually find them having some of the best signal-to-noise ratio of any step in the interview. Maybe it depends a lot on what type of coding problem it is?
One important aspect of doing the coding interview well is to make it generic enough and not look for a specific exact solution, so things like "write a function for a movie theater ticket dispenser" vs "write the most efficient string reversal" function.
I am also totally fine if a candidate needs to look something up, but I don't think it is unreasonable to expect candidates to be able to work though a general coding exercise without needing to stop and google things frequently. Especially if they can pick their own preferred language!
It's largely in the execution. Coding exercises are probably the best thing you can do, but many interviewers use them to ask bizarre trivia and look down on people who don't have it memorized.
After convincing the boss to fire a guy who laughed at me using vim saying it was really old and then proceeded to push broken code to production at 4am, trying to blame it on me when the logs were really not in his favour, I ended up in the position of doing the hiring for a guy to replace this yahoo. I live in a country where people's resumes fit in a damn manila envelope because there's just SO MUCH emphasis on degrees and stuff like that, and while the education here is pretty good (A lot of focus on hardware in the compsci curriculum here actually), really all I wanted after rejecting a bunch of prospects for technobabbling bullshit at me with their huge, multi page resumes i barely read was someone to tell me humbly enough that he'd just have to google the solution to the really damn easy fizzbuzz i was presenting them. You know how hard it was to find that in a braindrain situation? My boss thought I was being too mean, but really I just can't work with dishonest people. It's the most important thing to me that I actually like my coworkers for being good people. A coding interview of the MOST BASIC kind can act as a sort of filter for this.
All I want is to know if you know how to work with a modulo operators or not in an honest way, if you know basic third grade level integer division with remainders. If you give me bullshit instead of that, chances are you're not going to get hired. A probationary period for a couple of weeks after on the job after that to see how good you actually are, paid, with the knowledge that if you're just a terrible programmer you don't make it past that.
As a job searcher, I want to be able to spend as little time as possible on the interview process for one company. I don't mind researching the company or talking about their specific process, but I want to minimize the amount of time I have to take to prove my competence to them, because that's something that should transfer well between the different roles I'm pursuing.
I understand that not everyone has an OSS portfolio, but it's better than having 5 different companies give me a unique take-home project, and then having another 3 companies give me an algorithms assessment. If everything was "let's take a look at one repo you've spent <10 hours on" or "let's do leetcode problems" that would be preferable to the current state of affairs, because as it stands I have the worst of both worlds.
The author hasn't had the more dynamic interactive form of tech interview that I (as a platform dev) am used to.
> Coding is solving problems. A big part of that is looking stuff up. Like, seriously, the most important skill a developer can have it knowing how to research solutions to problems. Specifically…
Breaking problems into parts
Identifying what to search for
Knowing how to sort good info from junk
Rather than have them Google things, there's an open dialog with the interviewer where the candidate thinks out loud and the interviewer gives the 'search results' for them to use now, later, or not.
The interview is not about 'if' the candidate can solve the problem. Even it they don't the result can be positive. It's about how they think, break things down, compose the parts into a solution. No one should care if you memorized the syntax of anything.
Agree with this. In my experience interviewers don't mind if you would normally look something up, but you need to state "hmm I always forget which direction Array.sort sorts based on your return value" and they'll usually say something like "yeah me too, let's try it this way and flip it if needed." If you say nothing and freeze or implement it incorrectly without making it clear that you half expected it, the same situation feels much worse.
In general, I've been seeing some conflation between the "tech interviews are broken" sentiment and the "tech interviews are too hard" sentiment. There's probably some truth to both, and some specific things that both interviewers and candidates can do differently to make more interviews more productive. In all cases, being calm and congenial and trying to put the other person at ease (even if you're the candidate) can work wonders.
That's fine too. Often if I'm thinking out loud it's for the benefit of the interviewer so they don't get bored or confused later when I write a bunch of stuff after a long period of silence.
I also tend to do a lot of scribbling of intermediate picture/thoughts that I wipe and redo so it's not a complete void.
Where it can penalize you is if you don't finish and not having talked/shown your work to that point doesn't count for as much.
The best tech interview is if you describe the strategy, algorithm outline and datastructures that you would be using and the interviewer says, "that's good enough, next question".
Why don't you ask the candidate about problems that they've solved recently though? Why give them a specific problem out of the blue that they might not have any experience with? Wouldn't it be better to ask them about the hardest stuff they've recently done and then ask them about what constraints and tradeoffs they hit and things like that?
I do both. The problem with asking only about experience is - as surprising as it may seem - people are often not prepared for this question. This often means they just talk about a highly specific mundane technical task they're currently working on (for example, I've had answers like "implement redux reducers for my project") and when pressed about challenges, they can't come up with anything and just wing it with some generic canned response (e.g. "the challenge was using APIs from backend team to make the product work" or some other mundane aspect that one typically would expect to be present in every such job). Another variation is that they can't organize their thoughts quickly enough to summarize effectively, so they ramble semi-incoherently for 10 minutes about the specifics of their tech debt.
Mind you, while answers can be highly uninspiring, they often help contextualize where a candidate is in the spectrum. It's easy to spot ticket-closers with this question for example, and conversely you can quickly spot highly opinionated individuals. But like in scientific studies, there's different value in self-reported data vs data you collect from your own observation. There's plenty of "talk-the-talk" people that can articulate prose extremely well, but struggle with the most basic of technical tasks when it comes to actually writing some code. This is surprisingly common with people pivoting from startup jack-of-all-trade roles to highly specialized roles in bigcos.
Definitely both. This is the part of the interview that I get the strongest signal from. If they can still remember a problem, failed and working solution with great detail some time after the situation then they must have understood it at a level more than lookup trial and error fixes. It's great when you can riff off each other thinking about various avenues of discovery, potential causes and fixes.
I don't understand why we collectively treat interview questions as this big gotcha trade secret we have to spring on interviewees in real time. It feels like a masochistic hazing ritual.
I've taken to pre-emailing the general structure of questions I expect to ask ~48 hours before the interview and I've found that a) the interviewees are enormously thankful (which helps with closing candidates) and b) the quality of signal I get is much higher.
I think there are two answers: the bad answer is that people think of interviews as school finals, with all the same connotations about potential for cheating, cramming, memorization-without-retention, etc.
The good answer (IMHO) is that the format allows the interviewer to evaluate how a candidate deals with prioritizing conflicts of interest (i.e. the desire to write clean code vs desire to just get something out due time constraints pressure). It also opens up opportunities to explore refactoring, whereas a primed candidate might just bang out "perfect" code the first time around, and not leave much room for evaluating their iterative process.
160 comments
[ 0.14 ms ] story [ 14.6 ms ] threadAs a CTO, one of the things I look for is experience. You can look up all the "basic shit" you want but if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem. If I'm interviewing a senior engineer, I want you to be able to show your seniority one way or another, if we talk about architecting something, I want you to be able to talk about why we want to solve it this way, pros-cons etc. Experience also shows me that it wouldn't take you a long time to solve the problem, and in our industry, at least my experience in startups, speed is key.
For example:
> if we're talking about a specific problem, I want you to be able to give/walk me through a solution even if we don't finish the problem"
Why does this matter? Why do you want this? How do you correlate what you're stating you want here with actual job performance?
It matters to me because we want to be able to show that we can think through solutions. If we're building a feature that has something to do with streaming video, it would be a plus if you've shown that you have experience building streaming services or at least some knowledge along the topic. This should correlate with job performance because of the experience and working on this topic is already known. Now, there would be a case where we have 0 experience with this problem space but again it comes down to thinking through solutions and thinking about race conditions, side-effects etc and being able to identify these.
He is talking about coding challenges specifically.
What would help is more disclosure from hiring managers and interviewers about what they want and the success rates of candidates. That information is still largely today only available to hiring managers and executives. We need less information arbitrage for a more efficient hiring system.
What high false negative rate? Can you point to some data?
By "false negative", are you referring to rejecting a good candidate? Are false negatives (rejecting a good candidate) more important than false positives (hiring a bad candidate)?
> We need less information arbitrage for a more efficient hiring system.
How do you define efficient? Whose efficiency are you considering, the company or the candidate? Why should efficiency be a primary goal of the hiring system, as opposed to simply finding the best candidates in a reasonable time frame?
If you feel that way, here's a viewpoint from Derek Sivers which talks about "Idea being a multiple of execution" [0]. If you genuinely believe that "solving a problem within a few minutes" can be reached, I'm going to ask you - What problems are you asking people to solve in a limited time frame?
[0] https://sive.rs/multiply
I'm sure the usefulness of speed varies by context
I am glad I left.
In fact, I think that effect is exactly why the top-paying places do it, because it makes their hiring easier. At this point, if any of them unilaterally drops the hellish, silly, hazing-interview process, they'll be inundated with good candidates they've been excluding plus a shitload of bad ones who'd already been filtered out by the fact that their process is hellish and silly. So none of them will.
Unfortunately, don't have the link.
The premise is: you should be aware that there's a large subset of good engineers who aren't particularly great at coding interviews - and you should set up your interview process to select for those candidates.
It takes time and in many ways it requires more energy from me than if I were to just sit and observe. However, it gives a better feel for how the person goes about solving problems from the abstract, architectural stuff to the nitty gritty. I love it when pairing with a fellow vim-user as I will inevitably learn some new trick.
Bad assumption. If you really did understand why , you wouldn’t say it’s stupid (unless for clickbait).
Obviously, I can't give away a solution, but at the same time some people will get stuck on things like "Can I use X" or "can I use Y". And for me, that answer is almost always "Yes, absolutely use X!". Because that's what I WANT from someone I'm working with. I don't want you to write a sort from scratch, I want you to call `Collections.sort` or even have the wherewithal to ask someone/google "Hey, I know Java has some inbuilt sorts, how do I invoke those?".
The only time I'll maybe mark you lower is if you start the interview by saying "Yes, java is my favorite language to program in and I use it all the time!" and then struggle to do `new ArrayList<>()` (which has happened on multiple interviews). I always try to get someone to use whatever language they are most comfortable with. Because, to me, languages are stupidly easy to pick up and which one you are most familiar matters far less than how you use it.
The reason for your predicament is in your comment.
I've seen some rather complex problems being asked in interviews. I've always wondered why my credentials don't seem to matter when it comes to programming skills. I have a masters degree where i did some very tricky algorithms and I even open-sourced it for all to see. But every time I've been looking for a job, i have to rehearse some leetcode problems so that i can put on a good show for 45 minutes.
I get that deep-dives matter a lot and i can respect some of those exercises. These are crucial for senior positions since ideas are more important than code. Then I really get the life story kind of interviews because relating to coworkers and culture fit are hugely important.
It's just the dang leetcode that ticks us all off isn't it?
Obviously I don’t have the whole story of the interview, but it makes me a little worried for homebrew.
Asking what FizzBuzz is is similarly useless and would also make an abysmal question.
So do you consider youself a terrible, unhire-able developer? Presumable your still has quadrupled since you found out this information?
If you're a functional programmer, it's probably something you can just bang out. I've done functional programming. But it's been a while. In an interview situation, I'd definitely need some time to struggle through it.
What gives? I claim to be a senior developer of ~25 years experience? Why can't I just bang out the roughly ten-line solution it would be in Haskell, including the requisite data declarations? Well, in my 25 years of experience, yes, I've covered everything that is necessary to do that problem, but I've got to drag it out of storage and rehydrate it manually, because I don't do this sort of thing every day lately. Or ever, really, because what senior dev is running around implementing Computer Science 301 data structures every day anymore? Pretty much just one currently implementing a brand new language that needs those basic structures in its standard library.
I'd get there. It wouldn't even necessarily take long. But I can understand why an interviewer asking this question ten times a week who knows the answer forward and backward might find taking even 5 minutes to be an unimpressive performance for something that such an expert should clearly just be able to scribble down in one shot, right?
And I'm gonna be honest. If you ask me to balance a possibly-unbalanced sorted binary tree, I'm basically going to be re-deriving the answer from scratch. Yeah, I covered it. Probably even did a homework assignment on it in C. But my job for the past 25 years has not been implementing balanced binary trees.
The game and well are poisoned at many levels. There's just too much money on the table at most modern tech companies to trust anything.
I do quite a few coding interviews for my company and, frankly, the pass rate on the coding interview is dismal. It's one of our most effective filters for bad candidates.
What sort of torturous questions do I ask? Merge 2 sorted arrays is one of my favorite.
How many people can pass that? less than 50%. A good number of people don't even get to the "concat the 2 arrays into 1 array and sort" sort of solution. It's really that bad. These are people with "15 years of software development experience" failing really basic coding problems.
I really wish I didn't have to ask that sort of question, but at the same time, I don't want to work with anyone that can't solve it (even "under pressure").
The fact is, a lot of programmers will go off the deep end with simple problems. This joke[1] still relatively accurately describes the mentality of a bunch of programmers in the industry.
[1] http://4.bp.blogspot.com/-Hk1mt-RKYLc/UOkxShm6NrI/AAAAAAAACq...
It's not 'under pressure'. Real pressure is schedules and deployments and compute constraints. Not somebody looking over their shoulder.
Some people, even those with 100s of hours of schooling, just never learn how to think computationally. They just "fake it till they make it", which really just means they have no ability to provide value to any company.
And while, I don't equate merging 2 sorted arrays with being an unsolvable problem for an experienced dev, I have personally experienced similar increasing difficulty to toy problems as I gain increasing real world experience. Common things like timestamps and full names are not as simple as they seem, and so now, I'm less likely to just jump to an answer that would have seemed obvious to me 20 years ago. And it's not like I come up with the solution and discard it, it's more like the previously obvious solution is no longer the first things I think about.
I am also 100% certain that I've given interviewers the impression I don't know how to code whatsoever on at least three occasions, including as recently as a couple years ago. Hell, I'll probably do it at least a couple more times before my career's up. I guarantee that the interviewers were chatting after and were like, "see this is why we have to do this, because you get guys like that lying piece-of-shit yahoo, who've somehow had a full career and have great references and can talk about software and processes and architecture and all that, but inexplicably can't code at all".
I'm terribly self-conscious when people watch me code in the context of an interview where they're explicitly judging me so I get very slow even at using tools I'm familiar with ("I think that shortcut does what I want but... what if it doesn't? I'll just do it the slow way, oh no, now I look clumsy and stupid, fuck fuck fuck!"), and I tend to do stuff like forget the syntax to invoke a method in [language X that I've written 500 lines of in the last two weeks] when I start writing with a blank file (no existing syntax to crib off of, and c'mon, can you ask to google shit that basic in an interview? LOL no). Basically I have intense social anxiety but only for specific parts of interviews.
What I can do is talk through architecture, and any amount of conceptually-how-things-fit-together I have in my head, including low-level stuff. I can do 100% of the talking parts of an interview just fine. I'm told I come off exceptionally good in that part, actually (I can't tell, I just go by feedback). I'm sure this just reinforces the impression that I'm some kind of god-tier social manipulator who's, for some unknown reason, decided to scam his way into mid-tier tech jobs rather than applying such outstanding bullshitting skills to something that would pay for them more directly, and probably even be compensated higher if I were that good.
Just talk. A lot. What are you thinking, why are you thinking it? What are the alternatives you are thinking about. All of these will give your interviewer a good opportunity to give you feedback and can make the problem a lot easier than you might be expecting.
The more you talk, the better.
I think the way those algo-quiz or live-code-a-thing filters are used tends to catch people who can't talk well about it or do the job (this is the majority of would-be "bad hires", and you'd catch them anyway), some set who are afflicted with extremely bad general social anxiety so who are just fucked at interviews generally (I feel bad for those folks, especially since some of them are probably fine in normal work-social situations); the ones like me who can do the job, and can talk about it fluently, but suck at "coding interviews"; plus some very small set of actual social geniuses who've decided lying their way into so-so tech jobs is what they want to do with their lives, and have the talent (at faking it) to back that up, which are the ones people seem to think are super-common in the wild, for some reason, which seems very implausible to me for the simple reason that, again, that's a skill that's per se valuable elsewhere, if you're actually that good.
(frankly, from what I've seen on the other side of the table, I suspect you could weed out a very high percentage of actual-fakers by just blanket rejecting anyone with any kind of graduate degree in CS, who's applying for a mediocre position rather than something flashier and better-paying, though obviously that would come with its own set of pretty bad false negatives and I wouldn't actually do it, and anyway even those people usually don't "talk the talk" well enough to be convincing, if you get them off-script)
[EDIT] incidentally, FAANG and finance and a few of their peers are the ones where these kinds of aggressive weed-outs do sorta make sense, because their comp is so high that those "social geniuses" I mentioned might actually make more money faking their way into those tech jobs than applying their outstanding non-tech skills more directly to another role, such that it's worth it despite the stress and risk of being "found out".
I could definitely see ditching the coding questions on a lower paying job or a small company. You'd learn a lot more by bringing someone in for a month long probation (so long as they can talk the talk through the interview).
I could also see doing different interviews if what I did didn't have a lot of algorithm writing requirements. It wouldn't make sense for someone to consider algorithmic efficiency if what they are doing all day long is writing basic CRUD apps.
For us, we've got a lot of specific finance logic and calculations that we need to implement. It's not (always) straight forward "write some crud".
If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
Cool. It doesn't come up in my job either. That's not the point of the question. The point of the question is "given an algorithm specification, can you implement it" and that problem, particularly at my current work place, comes up very frequently.
> If you asked me that question in a whiteboard interview without access to my favorite IDE, a REPL, or google, there is a good possibility that I would fail to impress you.
All my coding interviews are done via hackerrank which has an OK IDE. I generally also allow googling or questions from the interviewer.
> Never mind the fact that I have been involved with the development of multiple highly successful, multi-million dollar projects where I wrote massive portions of the codebase.
> At my current company, they didn't give me a code test or anything like that. They just gave me a small project as contract work (migrating some of their infrastructure to the cloud). I knocked it out, they paid me for the work, and then they hired me full time. I've been here for two and a half years and the CEO once publicly stated that hiring me was one of the best things the previous CTO ever did.
Congratulations!
Honestly, I think you are letting this sort of question intimidate you too much. It is an easy question to answer and if you sat down for a few minutes I'm sure you could solve it. It's not a trick question. It's not one that's designed to filter someone of your qualifications. It's simply a "can you write an algorithm" question. Something that's really important in finance because a lot of what we deal with (particularly where I'm at, financial reporting) is writing code that matches financial models and algorithms. There's not a library you can reach for that computes everything we do.
Give it a shot in your favorite language and I think you'll be pleasantly surprised at how easily you can write a solution.
But then I had some personal experiences that changed my perspective...
There were a couple of jobs that I applied for where I was perfect for the role. Every single piece of technology they used and listed in the job description was something that I had mastered and used with success at other companies.
I got turned down for those roles, however, because I didn't ace their stupid leet-code bubble sort style questions during their interviews. These were trick problems that had NOTHING to do with the actual day-to-day work I would have been doing.
There were a hundred other things they could have looked at to determine that I was a good fit. They could have asked me to show some of my other work. They could have looked at the 100+ projects in my Github. They could have looked at the 20+ written recommendations on my LinkedIn profile all vouching for me (including glowing recommendations from other developers I worked with over the years, many of whom are now working at companies such as Facebook, Apple, and Google.
But nope. Instead of doing any of that, they would rather rely on some leet-code trick questions to gauge my abilities. As a result, they missed the forest for the trees and they lost out on what could have potentially been a perfect fit and a valuable hire.
So yeah. That's why I don't do that type of interview anymore.
If the job is actually for a role that literally solves algorithm problems on a day-to-day basis, then a code test for algorithm problems makes sense. If not, then please don't use them!
I see that phrase a lot and it’s often deployed without a great deal of thought, which is a good indicator that what I’m about to read is a pretty low-effort article, blog post or tweet.
So while it's true that having a candidate be able to write code that implements a stack and or whiteboard a simple app is not a realistic scenario of what they will be doing day to day, if they are able to do that well than they are much more likely to be able to do the work than if they are unable to do it well. So you use that as a proxy. Then 5 people interview for an hour, each using a different proxy, and you look at all the information the proxies are telling you to decide whether the candidate should be hired.
For sure it can. You can combine problem solving, ability to read code, write tests, avoid common pitfalls, clarify requirements all in the space of 45 mins.
I typically do this by taking a chunk of real code out of the code base, uncoupling it so it can run on its own and writing some stories.
But that also was not a realistic description of what happens during a regular workday, but it tries to cover one of the (many) skills you do need, so I think it was a decent proxy.
I am not advocating for lots of weird trick questions or brainteasers, but simulating the entire process of design, implementation, test, debug, document, sprint planning and the like. You can't realistically do that in an interview, you pick a few proxies and hope for the best.
I find interviews like this give very strong (& often surprising) signals.
I don't really object as much to people who try to approximate and don't quite get there than people who decide to just test something that nobody would ever actually do in their job.
It's like the difference between having buggy code and a product that just does completely the wrong thing.
I disagree with this. With some effort, you can create a small sandbox app that recreates an exact bug or existing implementation that needs a a feature added to it straight from your day to day work.
You can even translate your obscure problem domain into the appropriate analogy.
> They want to get a sense for what candidates actually know versus what they’ve copy/pasted from the web
This is not quite right. It's right in the sense that if your resume says 3 years of JS experience and you're spending 20% of the time fumbling with unmatched brackets, it says something about you. It's right in the sense that if you say you've done React at two previous jobs but proceed to call setState multiple times in series and then scratch your head when stuff doesn't work, that says something about you. Etc. However, it doesn't mean what a lot of newbies seem to think (that when receiving some leetcode-like task, they're supposed to have the exact ideal answer memorized)
In my interviews, one of the first things I tell candidates is in fact "feel free to google stuff, I'm not here to test whether you've memorized stuff, I'm more interested in how you approach problems".
At least for FAANG/bay area tech, good interviewers looks for signals, and not just purely technical rote memorization: on a basic level, one looks at whether the candidate has working familiarity w/ syntax and APIs that they ought to know, but on deeper levels how they think about code structure, refactoring, testability concerns, design trade-offs, how effectively they communicate (both speaking and listening to feedback), etc. There may also be smaller considerations for legitimate "fit" factor, for example whether a candidate is more startup oriented or more enterprise oriented (this mostly matters when you think about their career aspirations vs what the job is able to provide in terms of challenges).
You can get a pretty decent feel for how "senior" a candidate is, and then acceptance becomes more a question of placing the candidate on a sliding scale based on how demanding of a role one is trying to fill.
The one criticism I have about the interview process is that many interviewers don't tweak their expectations based on level quite as much as they should, especially if they only hire sparsely for low seniority levels. For a junior hire, there should be an expectation that some ramp up time will be required. Expecting upfront solid experience with a laundry list of technologies from them does a service to no one.
Frankly, we tell folks up front - "This is not a memory test, please feel free to ask questions, use google/stack overflow/man-pages/etc."
I'm in the interview to give you a problem that looks as close as possible to the work we're going to ask you to do, and then discuss with you how you solved that problem.
If you copy from stack overflow and it works, great! Now we can talk about possible edge cases, performance issues, testing, add a new requirement, etc.
I don't really care how you get the code into the file, or where it comes from - I care that you understand it, and can think through how that code will perform in different situations. Most importantly of all - I care that you can tell me that information in a coherent fashion.
My observation is that many companies don't address those problems well.
Few things to point out though:
1. you can ask the interviewer questions
2. the problems are usually simple enough that you don't need to google them, and should have prepared for them in the first place
3. you should be somewhat efficient (being able to do basic stuff without googling) in at least one language, since you can choose the language for the interview do that
More importantly, it's throwing out absolute opinions on why interviews are broken without demonstrating a real problem. There have been hundreds of articles saying hiring/interviews are broken because <logic/opinion>, and almost none that point to data showing that companies are not able to hire developers, nor that developers are statistically unable to get jobs.
Interviews can be difficult and frustrating for both sides, but that doesn't mean they're not working. Despite the fact that the article is correct that looking things up is routine in a dev job, I still may prefer to hire the candidate who doesn't need to during my interview question that was specifically designed to not really need to.
I don't know how often coding challenges disallow using the internet, but the last time I interviewed it was allowed. I've seen lots of companies allow use of any language, and spend your time searching the internet if you want.
If it was this prilliant data driven meritocratic nirvana of scientific precision, we would never have issues with discrimination again minorities, the poor and people who looks funny and have tatoos.
The burden of proof is on folks that keep pushing these silly tasks, where is the data "we've added some stupid questions to our interviews, and now we are doing twice better"?
Meritocracy, incidentally, is a term that was invented to demonstrate bias, not a signifier of the ideal system. https://en.wikipedia.org/wiki/Meritocracy#Early_definitions
> The burden of proof is on folks that keep pushing these silly tasks
This strikes me as a somewhat bizarre and myopic way of thinking about jobs. There is no burden of proof at all, because there is no claim being made. The deal here is someone will pay you money, if you agree to do the work they ask. This opportunity is available to people who make it through the interviews in order to demonstrate they meet the qualifications for the job. Companies that are hiring are the sole decider of what these qualifications are, and can choose to have them demonstrated in any way they choose (up to what we agree are legal limits.) If you don't want money, then don't submit to "silly tasks". (I've never witnessed silly tasks in my interviewing, ever. I've had questions for which I don't have experience, and questions that are too hard, but the questions have always been pretty focused on figuring out what I know and how good of a programmer I am, and by extension how much they are going to pay me...)
Instead, a system that we know runs poorly in 10 ways is claimed to be run poorly in 11 ways.
That is a dramatic difference and this would be a valid argument in a court of law - if the system was never run properly, why do you expect it runs properly when it comes to this particular matter? Why would anyone put their faith in it?
What are you talking about?
> Why would anyone put their faith in it?
They need money, maybe?
Because software developers are, by and large, employed (not to mention paid well)?
Because the economy is functioning?
I don't understand what you're suggesting, what is the alternative to interviewing? What do you want fixed? Why are you coming down hard on me for suggesting that not being allowed to Google the answer to your interview question isn't a serious problem that needs solving? How would you hire people if you had a company?
What does trust have to do with interviewing? Does one need to have trust in order to get a job? Do you have a job? Aside from cultural bias, what are examples of the system breaking trust in large numbers that demonstrate that hiring is "broken"? Where is the data that shows this?
Since you want to talk about trust, why should a company trust your or your claim of being qualified for a job without asking you to demonstrate you're qualified, in a time-bound and resource-bound setting? The interview questions you called "silly tasks" are nothing more than a request to demonstrate a very small amount of your claimed education and ability. Is that unreasonable? Why or why not?
The following statement is not very convincing: "maybe hiring is broken up when it comes to culture, gender, race, personal connections and class, but when it comes to programming tests, we certainly have it right"
Mistakes in other areas indicate mistakes are very likely in this area. Parallel with courts was meant to illustrate that
"They need money, maybe?"
If thsts the only reason, they won't put their faith in it, they will game the system to the maximum possible degree.
I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
Maybe we need qualifications that can actually be trusted, thats one possible solution.
Hence the request to demonstrate basic proficiency?
You seem to be inventing reasons to criticize and argue with me about subjects that neither I nor the article I responded to brought up. I didn't claim that programming tests are "right", so this is like your 4th straw man in a row. You don't seem to be answering any questions about how to improve the situation. Aside from bias, you haven't yet demonstrated a problem with hiring from the company side or the candidate side that has data to back it up.
What would you do to improve your perceived imperfections in the interview and hiring process?
> I think what we are doing in this industry is idiotic, we do not ask surgeons to do a 30-minute surgery on a dog to test their skills in surgery on people, we do not ask structural engineers to make a bridge out of matchsticks to test their skill.
This comment makes it seem like you are ignorant of and not even remotely aware of what the qualifications and interviews are like in other fields, and how much worse and more difficult they are than software. Surgeons go through years of low paid residency, and accreditation, and they suffer a whole battery of knowledge based tests where they're not allowed to Google the answer. Same for structural engineering. The fields of surgery and structural engineering both have cultural bias problems, just like software.
So why can we hire a doctor without asking him to transplant a dog's kidgey during an interview, but not a developer?
This is false. Your assertion that doctors don't perform in order to demonstrate their qualifications is wrong. They do have to perform multiple dissections and surgeries before they can get hired anywhere.
> I do have data: I and a friend...
That's not data, that's personal experience.
Your "data" is proving my point that coding tests aren't causing a hiring problem.
Last chance, third time I'm asking: what would you do to fix it?
But Third time I am answering: have trustworthy qualifications.
But you ignore most of what i write, and provide no evidence to support your position that this charade of hiring actually achieves what you think it does, so at this point I no longer believe you are debating in good faith
> have trustworthy qualifications.
This doesn't answer the question. People already have trustworthy qualifications, so this solves nothing. Your own point proves your answer wrong: people will game the system. You're contradicting yourself.
Doctors and structural engineers achieve trustworthy qualifications by doing a 4 year residency / apprenticeship program after their degree. Is that what you want in software? You would prefer years of supervised and low paid work rather than spend a total of a few days of your life doing easy coding tests?
For the fourth time, yes, thats exactly what I want. Having two years of supervised work would dramatically improve this industry
But i am the one arguing in bad faith.
You do see that this is the first time you've mentioned in this thread spending time (years!) on a certification process apart from school, right? I've re-read all the text above and can't find the other three times that you answered my question with anything concrete.
What problem, exactly, does this solve?
> So trustworthy qualifications solve nothing and we already have them, except we don't and they do solve the issue for structural engineers and doctors as you pointed out exactly they are more trustworthy than anything we have in software.
I feel like you're not being either clear or fair with your jumping between talking about interview process and certification process for other fields. Maybe I'm not being clear either. Let's be more clear.
When I said we already have trustworthy qualifications, I was talking about school degrees, because that is the primary thing we have in software. In your reply here, you are conflating apprenticeship programs (doctor's qualifications) with school degrees (software developer's qualifications).
School degrees, unfortunately, while they are trustworthy in what they provide, are not providing a guarantee that an applicant is fit for industry work. That's what an apprenticeship program does. Again, like above, the issue is not one of trust. The issue is what the qualification is qualifying, and what the hiring company wants to see.
Ultimately, I'd rather do a few easy coding tests when I change jobs than have to spend 2-4 years in some kind of certification program. People who have to do certification programs complain all the time that the work is bureaucratic and jumping through hoops and irrelevant to their eventual jobs.
And, certification programs have not solved the only concrete reason you've cited as a hiring problem: cultural bias.
Be careful what you wish for.
In this case: I've never heard of, participated in or administered a programming interview where the candidate couldn't Google things.
I remember when I interviewed at Braintree, I spent a good chunk of the pairing interview searching for Linux command syntax because I was new to using it. I got the job and spent 7 years there.
How would that even work? You’d just Google for the answer to the brain teaser and find it because there’s only so many brain teasers out there.
Once bitten twice shy and all that.
I wasn't aware they were tracking this, of course, nor that it wouldn't be kosher, and I specifically asked about whether I could look up Java API at the start and did not get a "no" ("We don't necessarily require working code here" is pretty ambiguous).
I passed the technical round, and then they ghosted me after asking what my technical level at Google was (which I didn't have to tell, as it's hidden even within Google).
Of course, it sucks if you've invested time into those, and get ghosted afterwards - but luckily I have yet to experience that.
Some people excel on taking test, some people excel on projects, others in both - I've personally never been the type to perform well on banging out code on whiteboards, but I will perform well on projects.
edit: Let me expand a bit on this
All my take-home problems have been very reasonable. Usually they've been more big-picture / architecture oriented, and not some specific implementation - some coding, some presentation, usually takes 2-6 hours to complete, depending on how much you put into it.
It is of course unfortunate if someone decides to code up something for a whole weekend, like 20-30 hours, only to get ghosted or similar. I guess it comes down to identifying free work from actual interview problems.
If you're used to something like the UK's PAYE system, being paid for interviewing is far from a no-brainer.
Not to mention that I now consider mandatory work trials to be an unethical practice. The stress of an interview, but stretched to last an entire week. A week where you feel you can't make any mistakes. I'll never do another one, it's not worth it.
I’d view a week trial as a knee jerk “skip this opportunity” signal.
Either they grade you at 0 and still hire you or I can make some extra cash turning in a bunch of empty applications.
* Here's a problem that takes 6 hours now do it in two.
* Ghosted
* Bizarre feedback that suggests they mixed me up with a different candidate.
* Emailed while I'm half way through the task to let me know that the position has been filled.
* We aren't prepared to spend 5 minutes looking at your portfolio to see if you can code to a base level but please do our take home problem that takes an hour and a half to prove that you can code to a base level.
screw that
Since it's more time consuming we don't offer it up front, most people don't like them, but it's a useful tool to have to size up a candidate.
The post-project debrief provides a lot of information on how you think and how you work, the end result is not expected to be perfect, or even complete, as long as the approach is reasonable and we can picture if the person-team fit is there.
People who can hold targeted low-level technical conversations are generally well equipped to do the work you are expecting him/her to do.
What's wrong with this approach?
What new information do you neeed from a coding challenge that you don't get talking shop with the individual?
I don't think low level technical stuff is something you can BS your way through an interview. If it works on you, then you as an interviewer has major gaps
Even the take home things are not fantastic, though - if you already have a job, a family and other stuff going on, "just 4 hours" is kind of a PITA.
In general I just wish places that weren't FAANG would be more open about their process. Do I need to brush up on leetcode shit before applying, or no? Are you gonna quiz me on obscure corners of a language that I reflexively avoid because they're terrible, so haven't thought about in a while? Gonna talk architecture? Will I be writing code with someone looking over my shoulder? Pairing? On what kind of thing, generally? Is your filter for the 200 candidates who apply to hit them with a take-home assignment or longish automated test first thing, so I know to just skip applying at all (screw burning that much time to apply for the privilege of getting an interview at just one place)? With FAANG you know what you're getting in to, so at least you know you're about to step in a big pile of shit and can put on your waders first. The blind roulette wheel of hiring outside FAANG is its own kind of trash-fire.
Got to the interview and they were so obsessed about this being written in some “perfect way” (see the enterprise fizz buzz someone linked to) that they knocked me back when I was explaining it at the interview.
It’s the coding equivalent of “ooh so you are not using a suse vide” at a mates bbq.
We also nudge interviewees in the right direction to make sure they don't get stuck, which would be in no ones interest.
There is also no hard requirement that you need to solve the problem completely. If you talk to someone for 45 minutes about a coding problem you will get a good sense of their performance, even if they don't solve the problem.
Coding interviews are great (except making the intervieww very nervous), but you have to be mindful as an interviewer.
One important aspect of doing the coding interview well is to make it generic enough and not look for a specific exact solution, so things like "write a function for a movie theater ticket dispenser" vs "write the most efficient string reversal" function.
I am also totally fine if a candidate needs to look something up, but I don't think it is unreasonable to expect candidates to be able to work though a general coding exercise without needing to stop and google things frequently. Especially if they can pick their own preferred language!
Since I'm not allowed to google stuff now, would that mean I won't be allowed to do that while working for your company as well?
Like sheesh, I'm not looking for this:
https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
All I want is to know if you know how to work with a modulo operators or not in an honest way, if you know basic third grade level integer division with remainders. If you give me bullshit instead of that, chances are you're not going to get hired. A probationary period for a couple of weeks after on the job after that to see how good you actually are, paid, with the knowledge that if you're just a terrible programmer you don't make it past that.
I understand that not everyone has an OSS portfolio, but it's better than having 5 different companies give me a unique take-home project, and then having another 3 companies give me an algorithms assessment. If everything was "let's take a look at one repo you've spent <10 hours on" or "let's do leetcode problems" that would be preferable to the current state of affairs, because as it stands I have the worst of both worlds.
> Coding is solving problems. A big part of that is looking stuff up. Like, seriously, the most important skill a developer can have it knowing how to research solutions to problems. Specifically…
Rather than have them Google things, there's an open dialog with the interviewer where the candidate thinks out loud and the interviewer gives the 'search results' for them to use now, later, or not.The interview is not about 'if' the candidate can solve the problem. Even it they don't the result can be positive. It's about how they think, break things down, compose the parts into a solution. No one should care if you memorized the syntax of anything.
In general, I've been seeing some conflation between the "tech interviews are broken" sentiment and the "tech interviews are too hard" sentiment. There's probably some truth to both, and some specific things that both interviewers and candidates can do differently to make more interviews more productive. In all cases, being calm and congenial and trying to put the other person at ease (even if you're the candidate) can work wonders.
I also tend to do a lot of scribbling of intermediate picture/thoughts that I wipe and redo so it's not a complete void.
Where it can penalize you is if you don't finish and not having talked/shown your work to that point doesn't count for as much.
The best tech interview is if you describe the strategy, algorithm outline and datastructures that you would be using and the interviewer says, "that's good enough, next question".
Mind you, while answers can be highly uninspiring, they often help contextualize where a candidate is in the spectrum. It's easy to spot ticket-closers with this question for example, and conversely you can quickly spot highly opinionated individuals. But like in scientific studies, there's different value in self-reported data vs data you collect from your own observation. There's plenty of "talk-the-talk" people that can articulate prose extremely well, but struggle with the most basic of technical tasks when it comes to actually writing some code. This is surprisingly common with people pivoting from startup jack-of-all-trade roles to highly specialized roles in bigcos.
I've taken to pre-emailing the general structure of questions I expect to ask ~48 hours before the interview and I've found that a) the interviewees are enormously thankful (which helps with closing candidates) and b) the quality of signal I get is much higher.
The good answer (IMHO) is that the format allows the interviewer to evaluate how a candidate deals with prioritizing conflicts of interest (i.e. the desire to write clean code vs desire to just get something out due time constraints pressure). It also opens up opportunities to explore refactoring, whereas a primed candidate might just bang out "perfect" code the first time around, and not leave much room for evaluating their iterative process.