Tell HN: Stop Accepting Shitty Interviews
https://news.ycombinator.com/item?id=29892437
One of the recurring themes is the "leet code" interviews. I will go further and say that any interview that doesn't mimic the job is a guaranteed way for me to turn the company down.
Stop accepting shitty interviews
When I was talking to trading firms about coming back to the finance industry, about half of them admitted they already use my open-source software. And yet, only one firm bothered to ask me about how some of my stuff was implemented. That's the firm I'm working at now.
Instead, I got plenty of firms that still insisted on asking me questions that are not remotely related to the job. Tons of Hacker Rank and Codility, even though I've never had a time limit when building something in real life. Tons of intern-level questions as if I was a college freshman. And tons of detailed questions about how a specific programming language works, as if we're going to build a compiler for that language.
And to reiterate, those were the firms that already admitted to using my software!
Stop accepting shitty interviews
The stupid assessment practices won't change unless employers suffer the consequences for this nonsense. I have always made it my mission to reject any firm that asks me syntax questions or brain teasers. (Seriously, the New York Times doesn't assess journalists by having them conjugate verbs; it's just weird to ask this.)
Yes, people get mad when I terminate the interview on the spot. I'm sure I've been blacklisted from a few places because of my stubbornness. But damn if I'm going to allow stupid shit like this.
Stop accepting shitty interviews
126 comments
[ 4.3 ms ] story [ 211 ms ] threadIn your case, they should've certainly dug into your open source, at least if it was directly in their line of business.
I certainly love it when I ask a candidate "tell me about a project you found interesting" and they can go into details, talk about tradeoffs, etc. Sadly, most candidates can't. They can't come up with anything better than "well I rewrote the thing, and it was challenging since I hadn't used Kotlin before, learning Kotlin was interesting." So at that point I'm unimpressed, but I'll give them a chance to code still because (a) not everybody has had opportunities to work on interesting stuff and (b) I wouldn't have enough people to hire otherwise.
But if "has open source software we use" is the bar, you're gonna fail a lot more good engineers than you would using algo questions.
* implementation choices with pros and cons
* details about how the underlying system works
* domain knowledge, where applicable
* sample code where the candidate writes to mimic something from the system
My interview style has always been: "Tell me what you know, and we'll see if that's true or not." Surprisingly, most candidates fail even that since they don't seem to understand their own projects. But I find this approach the fairest of all since it allows the candidate to lead the interview and it doesn't assume anything about the candidate's knowledge or prior experience.
The interview that's always worked for me in hiring is where we talk about the issue and we see if the person understands the problem. There's an infinite number of things we can get into, and if the person is experienced they'll never run out of things to say. I've never hired someone who turned out not to know the basic things like some particular algorithm.
I suppose if we're talking the leetcode style interview loops, they normally have a system design part. That might be the closest to what I do.
This approach simply doesn't always work. Some people are amazing-sounding but terrible in practice. The reverse is also true - some people are terrible at talking about their work, but produce amazing code. (Their lack of communication skills might or might not hamper their actual job performance, depending on multiple factors.)
On the other hand, leet code questions definitely don't show that the candidate can actually write maintainable, well-documented code with test suites and easy-to-use APIs.
Though I will say that having some kind of fizz-buzz style phone screen is still, in my mind, a good idea. It saves a lot of time if it turns out that your candidate can't even write a simple for loop (and so many can't, to my great dismay).
Start there. Let the candidate walk through the problem. Ask questions, and introduce hypothetical limitations. "We need to display the data in a table on a webpage." (frontend) ... "We need an API that can query the data." (backend) ... "We need to sort the data." (discuss algos) It's not that hard to find people's strengths, and make sure you're hiring qualified folks.
Making commits to FOSS a general requirement in interviews would exclude a whole lot of highly skilled and experienced engineers, limit the chances of beginners, and lead to loads and loads of BS open source projects that exist solely to fatten resumes.
It seems you are asking for a miracle: an interview that properly assesses the candidate's job readiness without time involvement. How much time do people spend preparing for interviews and then have to do it again the next time they want to interview? At least with this system, you have a history that is reusable.
The closest might be white-boarding potential solutions in meetings, but there, most of the time, I have a lot of context wrt. the company and situation.
In the past I haven't minded presenting solutions to contrived problems in interviews, but these days I wouldn't. I also wouldn't take a job without significant equity attached, but I'm probably not representative of most devs.
This is just nonsense, since the only reason the developer can answer the question is because they crammed it in preparation for the interview. Leetcode questions are not tricky, they're just tricks.
From the interviewee side: It gives you a known set of material to prepare for (ofc there may be newer questions). But this alone helps a lot. With the explosion of frameworks, libraries, languages it is very hard to prepare for everything.
From the interviewer side: Again it will give you a standardized set of questions which can be asked and you can compare across candidate due to this.
>> about half of them admitted they already use my open-source software
Similar to how people don't have time to prepare for leetcode, there may also be people who don't have github repos with open source code etc. Take my case, I work for a company where I have to get permissions if i am gonna start a open source project, even though it may not be related at all to my current work.
Is this common ? Or legal ?
https://leginfo.legislature.ca.gov/faces/codes_displaySectio...
https://apps.leg.wa.gov/rcw/default.aspx?cite=49.44.140
For morale and tech brand reasons, some companies have a process to officially bless releases of work they might have had a good case for owning.
What you're describing are not advatanges as much as ways to make the job easier, without accounting for if it produces better results.
I've got a little anecdote on an interview that I should have ended, but I was a junior so I didn't really know better at the time.
At this interview, one of the things they wanted me to do was to do a series of one minute coding challenges. They weren't leet code or anything, but just difficult enough that they required a lot of knowledge, and I wasn't allowed to look up documentation. This one guy stood behind me and looked over my shoulder with a stop watch. After a minute was up, I had to put down the code I was writing and move on to the next one, and the process would repeat. I think I did about 12 of those. It was some of the most ridiculous bullshit I ever did in an interview. If you didn't remember the exact array method off the top of your head then you were just screwed. Mind you, they were hiring for a junior position. These days there's a possibility I'd survive it, but I wouldn't even bother at this point.
In my experience these are terrible. It becomes an arms race, what the company intended to become a 5 hour take home assignment becomes a 20 hour assignment because everyone else that's doing the take home assignment is also spending 20 hours. Take homes are a trap.
Location, pay, workload, social aspects, stocks. Not to mention work environment, chance at promotion, career progression. Deciding on interviewing types can be part of the equation.
Changing employer's tactics is not going to happen at the interview feedback level. This movement needs a gurus/books and a cargo cult mantra which filters into magazine articles and popular culture.
You are better off going in a different direction and using this information to hire the best employees for your own company or offering a service that places your candidates in roles.
If your startup operates with a 50% or 75% target rejection rate, you absolutely can and should operate differently, and this could be an important competitive advantage for you.
I swear sometimes it might just be easier or even fairer to throw all resumes/applications in the air and see which ones land inside of a predetermined circle.
When I was a responsible for hiring interns, we used to joke about this all the time. Just throw away 75% of applicants, everyone that got thrown out is unlucky, and we don't want unlucky people bringing the company down.
Steve Yegge made that observation many years ago regarding Google. For every person working at Google, there is a panel of interviewers that would have rejected that person.
Few companies are actually in a position, given their knowledge, stability of their industry, and marginal quality of hiring success, to succeed at deliberately loosening or tightening their interview process to arrive at just the right number of final candidates.
With automated resume pruning, it shouldn't take more than two hours for initial pruning. Actually, the higher number of applications means they can put in stricter pruning conditions, leading to lesser manual pruning later on. Assuming 2 hours/person interviewing, it should take less than a 40 hours-person to close a position.
I'd guess there's also some benefit w.r.t company loyalty when hiring a good-enough candidate instead of the best-available. They would be less likely to switch jobs compared to someone who can solve a leetcode hard problem in 15 minutes.
https://pandas.pydata.org/docs/reference/api/pandas.merge_as...
The biggest open-source thing I've ever written is Empirical, a language for time-series analysis.
https://www.empirical-soft.com
I'm also top 1% on Stack Overflow. A lot of companies have found me through there.
https://stackoverflow.com/users/478288/chrisaycock
What makes these "shitty" interviews? They're very simple and they test for your ability to write code.
They set the bar clearly and you have the ability to study for them with ease. Solve 200 of these problems once in your life and you'll never have an issue again. You'll breeze through all of these interviews.
Compared to the alternative, where people ask insane unrelated questions and that are incredibly subjective, these aren't all that bad.
Do they? Or do they test for your ability to memorize a set of common problems?
I don't have another solution, but just because they are the best we have come up with doesn't mean they aren't shitty.
...
1. The question you are posing isn't hard. It's easy to borderline trivial, I would think so even for junior engineers who are recent grads.
2. Being skilled in a specific FE framework or any library/API isn't what you should be testing for unless you're hiring for simple task work. You really want to uncover ability (IQ) not familiarity. You want people that will grow with the company, not drag you down.
Edit: If you're looking for puzzle solving skills then give someone a puzzle
Knowing how to implement some arcane algorithm is also not what you should be testing for. It tells me ZERO about this persons ability to code in the real world.
It may not even tell me they understand the algorithm...they may have just memorized the pseudocode implementation well enough.
> You really want to uncover ability (IQ) not familiarity.
a) Cognitive ability isn't accurately measureable. Not with IQ tests, and certainly not with standardized puzzle questions the internet is making lists of which one to learn to crack interview @ company X
b) Familiarity is EXTREMELY important when working with complicated systems.
b) of course it is! no dispute whatsoever. that's why it's 6 months to become productive in any new job in the first place. But you want someone that can BECOME familiar and can UNDERSTAND the framework du jour.
When I hire an appliance repairman I want a guy that understands how the machine WORKS and can troubleshoot it from that knowledge base. Not someone that can swap in a fuse as long as the machine tells him which fuse is bad. I want someone who is knowledgeable with DISHWASHERS not someone that is an expert in MIELE seeing as I have a BOSCH or I've modified my miele beyond recognition anymore.
If you need seat warmers the story is different. Then sure, you just want a miele certified repairman.
The point where I think we disagree, and please correct me if I'm wrong, is: are LC style questions an effective testing method for these skills.
I say no, they are not, for 2 reasons:
a) implementing something like a non-trivial graph algorithm doesn't come up often in day-2-day work, and if it comes up, even some experienced developers look it up, just to make sure they got it right from memory. To borrow your dishwasher-analogy: I want the repairmen to know how a generalized dishwasher works, so he can repair Miele, Bosch, whathaveyou, but I don't need him to be able to program the DWs microcontroller on a whiteboard, or forge one of the pipes with nothing but a firepit, a hammer, anvil, crucible and some copper-scrap.
b) since all these questions are known problems, both in their solution AND their usage as test questions, we are facing a "learning for the test" dilemma: people can, and many do, just memorize these answers to pass interviews, same as many students in standardized test environments just learn how to pass the tests.
This is fundamentally different from understanding the subject. Again, I agree with you: I want someone who can answer these questions because he can apply generalized knowledge about algos and data-structures.
But I am not confident tests which rely on pre-fabricated and well known puzzles will give me that.
Either one is a red flag, and a perfectly great reason to bail from an interview. If hiring/HR is rotten, the rest of the company is likely similar.
This statement is an oxymoron.
Do you have objective evidence that this is an effective test of "the ability to write code", or is that just your belief?
It's a small portion of the interview, the majority of which is asking them to talk about problems, technologies, strategies, etc., but the easy question (not fizzbuzz, but also not a DP question) makes sure they actually know how to code. My general issue with leet code interviews is that once I get nervous, my brain stops being able to come up with clever solutions to problems, and if a problem seems hard, this only compounds.
When you give someone an "easy" question, they loosen up and can actually start thinking creatively. You can see how they think it through, verify their solutions, and talk to you about requirements and approaches. As opposed to giving them a hard problem and watching the amygdala do its thing for an hour.
Do crossword puzzles test my ability to write novels? No they don't, because remembering some obscure name for a medieval dignitarys hat of office doesn't mean I can write compelling immersive stories.
By the same logic, memorizing how some obscure algorithm is implemented doesn't test my ability to think about real world applications or code in the context of a large project.
However, for hiring people who don't have a public body of work to point to - it remains pretty effective. (I am not affiliated with hackerrank/hackerearth/codility - just been on the hiring side for the last many years)
Also, in my current org - we've been able to trust people lacking the "expected" graduate degrees - solely because they were able to prove their acumen in the interview by solving algo-data-structure problems. (Of course, we have to be creative here - and avoid like the plague the same bunch of questions which are present all over the web)
I suspect the answer to this question is the same as to why people do the same for interview questions.
Why should I stop? My personal experience is that many of the best companies to work for have shitty interviews. All I need to do is to study a bit and go through a 5 hour interview and I have a cushy high paying job? Sign me up. Complaining about interviews is missing the forest from the trees, I don't spend the majority of my working life interviewing, I spend it working. So if preparing a lot for interviews and getting my ego bruised a little means I get a good job so be it.
I’d choose a much higher pay over a reasonable interview.
Otherwise you might not just bother, because it's usually a reflection of what the company values and how it thinks
I’ve got a sickness, and the only prescription is more bloom filters, baby
Embedded space sadly suffers from a dearth of good implementations for different parameters than CPU cycles.
Most of the population couldn't do 200 leetcode problems if they were given two months full time one on one tuition and told they'd get $100,000 in cash at the end. That would probably hold true even if you restricted it to college graduates and definitely would for high school graduates. Most people are really, really bad at mathematical thinking. It's more than just a bit of study.
Most of the population are not programmers.
For someone who already knows a PL or 2, the picture is very different: There are maybe 32-50 questions that get repeated in almost every "code puzzle"-style interview. Which ones these are is actively monitored and curated in several communities.
A person intelligent enough to know a PL, can simply memorize these questions. "Learning for the test" is a standard, and successful, strategy to game predictable test environments. We see it in educational systems using standardized multiple choice tests, why would it be different in bad code interviews?
Also more meta-level commentary, I find the word "should" interesting, as in general I have found that comments on HN which contain this word reflect a level of frustration that X is X instead of Y, because "X should be Y". Someone else may come along and say "X should be Z", and all the meanwhile, X is X.
There's a whole leetcode culture on sites like teamblind.com. A lot of people are happy to play the game.
Not my cup of tea either, but to each their own.
If you can tell if someone's good within fifteen minutes of talking to them you should be able to get rich by setting up a software consulting company or startup with other people's false negative rejects. That's equivalent to being able to take mine tailings and spit out gold.
> In my experience, I can gather more information about a candidate's ability in just 15 minutes talking 1 on 1 with them rather than hours of Leetcode busywork
This is your perspective as someone doing hiring. My perspective is from someone being hired. I really don't think leetcode is that difficult all things considered so a system based on leetcode is just not that bad in my opinion. It's really not that difficult, there is a set of 15 or so algorithms that are all 40 or so lines long and you just need to memorize them to the point of instant recall. Now you can get hired almost anywhere.
How many people did you interview and hire?
Just asking since I interview and hire a lot of people, and find it very hard to asses someones capabilities.
In those few LC questions, how do I know the person can write good, maintainable, reliable software with a team of engineers and didn't just learn for the test?
Yes, interviews are hard. Assessing a persons capabilities is a difficult job.
You think you can gather more information. I doubt that your decision is more accurate than leetcode interviews on average.
I've spent 30min talking to candidates about their projects, experience, tech they used, choices made, etc. and been super excited and positive about them.
Then I ask them to find the intersection of two lists in better than O(n^2) and they completely fail to write functional code in their preferred language in the next 30 minutes. This isn't even "leetcode" level of challenge.
If I'm hiring you to be a professional software engineer for 6 figures income to spend 8-10 hours a day designing, writing, and debugging code, you need to be able to write code.
The disparity between the conversation and the coding performance can be significant, and both need to be evaluated.
If there's a better way to evaluate if someone can write code, I'm open to try it. But take home tests, portfolios, pair-programming, "a day in the life", etc. all have their problems and disadvantage different groups.
That commitment can be exploited.
It's like Nigerian Prince scams, those willing to jump through the hoops are more easily to convince and get jumping through future hoops.
It's also likely they will rationalize that their job is great because they had to do all this shit to get it. The employer could think "if we can get their foot in the door and convince them to fully enter the room even though we make it difficult to fully open the door, they're less likely to leave and more likely to fight to stay in the room." It keeps uncommitted folks out, but doesn't necessarily filter for competent people, just committed people.
Cognitive dissonance is a powerful thing.
As to shitty interviews. I was approached by Google for a non-programmer role a few years back. Failed the coding part as I’ve not been a professional programmer for a 20 years, despite being able to write all the code that I need for my research. Was told they’d be in touch for a development program as I’d apparently aced the rest of the interview. Never heard anything. Was approached again after a while and I agreed under the condition that there was no coding interview - I knew I had failed that and was going to fail again. Was assured there’d be no coding interview. Only to have one scheduled immediately.
So an organization that manages what’s probably the largest information system on the planet cannot for the love of their life implement a simple sales lead process? Why? Because they fundamentally do not care about you.
Do you expect medical doctors to do an unpaid "trial" surgery as part of their interview? Do you expect lawyers to litigate a case for free before they'll be considered for a job?
As far as I know software engineering is the only field where in addition to experience & education you have to waste time gaming the interview process. Every other field seems to get away just fine hiring based on experience and/or education.
How much of the content on the MCAT is "useful" for a doctor's daily work? I would argue that the majority of it is moot. In the same vein, the LSAT has a lot of "useless" logic puzzles that aren't applicable to their daily work as a lawyer. Fact of the matter is, we're lucky in tech to have this system that ANYONE can study for "useless" shit, doesn't matter your background, what college you went to, etc. If you get the interview and nail it, you're in.
https://joelgrus.com/2016/05/23/fizz-buzz-in-tensorflow/
My experience in the last year or so categorically states that leetcoding is the only game in town in CA. I hate it, I am bad at it, I find it humiliating and irrelevant. I was told by Asian teenagers that I am not good at coding after 20 years building software. I have failed multiple interviews because of dynamic programming&Co even at companies where I had a decade of highly relevant experience. Nobody ever wanted to hire me for my production experience with all of Scala/Kotlin/Java even though I routinely meet companies where they don't use Scala/Kotlin because of hiring difficulties.
It's just the way it is nowadays. The world is getting progressively crazy. If you're in a position to not accept such interviews I envy you big time. I don't see it happening even in smaller (i.e. not-FAANG compensation) companies though. Some European/remote companies occasionally give you a home coding challenge. But they are an exception.
Secondly it rewards people who at least put the effort in to prepare for an interview. These questions are a very reliable way to eliminate candidates.
And to take the journalism example, I actually think that'd make a pretty solid test because if a journalist doesn't have a basic grasp on formal language I would have to reconsider how strongly they're committed to their craft. It's honestly only the case in software, an engineering discipline to boot, that people question the necessity of being able to solve basic theoretical problems. Would anyone trust a civil engineer who can't answer first semester math questions?
> it rewards people who at least put the effort in to prepare for an interview
Is this a good thing? Would you rather hire smart people, or people who studied a few leetcode questions the night before the interview?
> Would anyone trust a civil engineer who can't answer first semester math questions?
Civil engineers use that knowledge in everything they do. A really small subset of developers need to implement tree traversal algorithms from scratch on a daily basis.
I also think its a little backward to expect the candidate to be doing very much prep work in advance of the interview. They are in all likelyhood currently employed fulltime and busy with their life outside of that.
Instead I would focus on simply seeing if the candidate would be interested in the sort of work they would be doing, and giving as much detail as possible on what the job would be like for them. Job listings and corporate websites are often very opaque. It's one thing to get someone competent, but if they hate the job they will be gone in a year and you will be in the exact same place as you are today. If you get someone who might not know all the required tooling, but is very interested in the work being done, they will step up and be one of your most prolific employees before long and contribute to lasting institutional knowledge within your organization, rather than running off with their skills as soon as someone dangles a better paycheck in front of them.
You should not be trying to approximate an IQ test (and leetcode does exactly this).
And can you guarantee that this is what LC questions test for?
We are talking about questions here were the answers are known, their usage for interviews is known, and curated lists of questions commonly used for interviews exist .
So yeah, they may test for a good grasp of algorithms, data structures and math.
But they can also just be memorized and replayed at will given enough time.
Even if it is annoying to spend hours and not get an offer, I can see why they ask these types of questions to see your problem solving ability first hand.
I would expect any decent engineer to be able to solve leet code style question, or at the very least be able to model the problem reasonably well and give a recurrence. Even if the code contains errors or there's missing edge cases then it allows you to see how someone approaches a problem.
Leet code style questions are basically a bunch of combinatorial algorithms. Even if they seem like a different trick each time there's general techniques that can be used to approach each problem.
There’s a class of delusional people who think a GitHub profile is sufficient. Outside of a minority of technical influencers that is absolutely wrong and unreasonable.
If you're not kidding, you are so delusional that I don't even know where to start.
If you are kidding than to did a damn good job!