I've decided I'm going to start interviewing with a set of PR's per each role we hire for. each PR will have obvious mistakes, complicated logic problems, as well as code that could be "refactored" once the whole PR has been read.
this provides a few things; it gives us an ability at how the interviewee problem solves. next we can see how they respond too obvious fixes (would they be someone you'd want to send a PR too?). finally, it tests their knowledge of the language and APIs, hopefully much better than Leet Code can. I would also like to see if the user can spot obvious bugs in the setup code (say, package.json, pyproject.toml, etc)
I am going to make an example PR for:
- frontend (React/NextJS, TypeScript, CSS)
- backend (Django, Python)
- DevOps (potentially some Pulumi code for deploying to a Kubernetes cluster?)
I think there are flaws with this approach too, but I wish this approach was more common so that there was industry wide rapid iteration to a more optimized version of this.
What I’ve seen are time trials, which don’t show you how well a candidate will adapt to a codebase, wont show you how they will do tickets for your sprint. Only reinforces a flawed idea of the employer about how they wont “hit the ground running” despite having that job opening for 8 months.
What you described might not be a time trial, its what ive seen though.
Although this is better than leet code, PRs have been shown to be a poor way to prevent bugs. I find they are mostly useful for disseminating knowledge.
I once a test with badly written code and gave my comments. A bot immediately asked me to implement those changes, but I assumed a human would respond. I made a YAGNI change, trying not to disturb existing code too much. They responded saying the comments raised during PR were good, but writing a big switch statement for all subclasses violates Liskov Substitution Principle.
> Alternatively, look at under-performing people and find what they are lacking
This is great suggestion. While the “look at their github” one is a bad suggestion. Github polishing is theatre more suited for theatre majors instead of people actually working with integrity before coming to your company. Its very similar to the issue with the leetcode interviews as its geared towards people with time to optimize that instead of a day to day job.
Oh you focused on the “polishing” and not the “there’s nothing to show” part
People good at their jobs because they do their job: nothing on github
People in the business of performance theatre because they dont have a job to be good at: plenty on github. It can be legit code they wrote, that was not the point at all.
I don't like the idea that every developer has to have code on github. If they want to code for work and nothing more, then that's fine and shouldn't disqualify them from any jobs. We shouldn't expect people to spend years of employment building up a portfolio for the next time they're looking for a job.
However, I don't see how looking at the code people write isn't informative. You can see many things, from small-scale code style decisions to how they structure an application just from looking through their github profile.
Writing code isn't a performance, it's literally what you want to pay them for.
Of the people with code on github, a lot of that is forked projects with small contributions, and the rest is small convenience or learning projects that werent meant for scrutiny. If its curated its because they are making a performance that also has nothing to do with being with an employee before meeting you.
Github presence is just a non signal. Thats the only point.
We agree that if you have a different way to see how theyll actually structure code for you, then its useful
Okay, so you've addressed profiles with low-quality code and curated portfolios; but what about normal people who just put their projects on the platform? Their profiles will be a good representation of how they generally do things.
Depends on what they have on there though, my GitHub mostly has projects from ~2010 when I was a very different engineer to the one I am now (this was before my first dev job)
But commits are timestamped though? It's easy to see if a github profile is active or not, and which projects likely represent someones current level of skill.
The stuff I have on Github is a mix of university projects (about 30 years ago), forked stuff I keep in sync with main repo (just in case they go away), a couple of half baked projects that I occasionally touch on rainy days and lack the rigeur of what gets deployed into production at work.
But hey, maybe HR will be happy I have a GitHub account.
> People good at their jobs because they do their job: nothing on github
Even better if they have both, worked on open-source projects or have created useful open-source software used by other companies and are already working in their other job(s) or have personal real-world projects they can point to; which those are clear advantages and a simple quick filter to use.
No need to ask about frivolous leetcode questions around re-implementing sorting algorithms or wasting more time asking the candidate to write proofs for those algorithms where realistically you're going to just import it from a library or look up the solution on StackOverflow.
Unless you're Google, a FAAMNG company, university or general research related position or if the position isn't for a typical CRUD application development, then there is little to no justification for wasting everyone's time on pointless leet-code puzzles and this applies to the majority of companies.
I have 30+ years software dev and almost nothing on github. There are valid reasons why a person who writes lots of code, including on weekends/spare time, would not be on github.
For my personal projects I prefer bitbucket. For professional work, it is proprietary and thus cannot be shared (esp not in an interview!!).
Sure, but if you do have something meaningful on GitHub or where ever, and you share this in your CV, I would expect that the interviewers look at that. Or at least, I would not like to do some stupid coding exercises that are less complex, less authentic, and basically one-dimensional compared to my work on these open source projects.
These articles always seem to have an underlying assumption that really great people are being denied access to jobs because they can't get through these interviews but it is implying that the people who do get through these are not also great but are less hassle, easier to measure their ability and so what if they have swatted up on LeetCode to help their application? That means they are driven, that they have learned stuff on the way and are more likely to get something right the first time.
I have been on the receiving end of applying to an agency and being told I didn't make the grade technically. I was disappointed because I know I am a good engineer but I didn't expect them to magically know this. I can also see how my approach to the technical tests might have made me look less than what they were looking for, which is fine.
I am also not sure of any good alternatives because someone will always object to any alternative which they cannot achieve for some reason. A "take home" project is good for real life work but some people cannot (or will not) invest the time even if they are paid for it; discussions can be great for helping nervous people but that is not how work usually is, there are challenges, pressures etc. and the able people object that it is not fair that people are getting in too easily.
A realistic problem similar to what they’ll be expected to d in the job followed by a discussion to ensure they actually understand it makes much more sense than leetcode. Ideally with time for them to complete some of it independently with full access to the internet, an editor, etc.
> I am also not sure of any good alternatives because someone will always object to any alternative which they cannot achieve for some reason.
When I was on the market, a couple of companies actually gave me a variety of options, which I appreciated. I don't want to spend hours on a take-home and I also don't want to do leetcode, but an open question/answer plus some code review and live debugging was an acceptable combination for me.
Providing the options definitely has the potential to take up developer time, but I think the tradeoffs are worthwhile for both the interviewee and the hiring team.
Whenever they ask me to do a take-home I make a mention that I have about 150k lines of code on GitHub; this is usually ignored. Now, I somewhat understand why larger companies just "follow procedure, even if it doesn't strictly makes much sense" for various reasons, but what baffles me is that even a lot of smaller companies do this; which seems odd since a lot of places I interviewed at are kinda desperate for developers, and often can't match the salaries of larger companies either.
The exception is my current position (only started last week): "oh yeah, we looked at your GitHub already and it's actually similar to the take-home anyway, so little point in that". Instead, they prepared an "alternative" interview where they posed some scenarios with "what would you do? How would you handle this?", which was intended to test both some technical skills, but also social/attitude things. I wrote down some answers, which took me about 30 minutes, and then we discussed them, taking a further 30-45 minutes.
The questions were a bit clunky because they were looking for someone ASAP and there was only 2 days between the first and second interview, but I felt that was a much better approach for the company as well, because they got a lot more information this way: they could already verify basic coding skills themselves, and this way they got a new chunk of information they wouldn't have had otherwise.
>but what baffles me is that even a lot of smaller companies do this
You named the exact problem with the industry.
No one is going to complain if FAANG does this with top tier, life changing salaries. No one will object to learning and memorizing DS&A at a job where DS&A is used heavily.
What people are upset about is your average no-name company hiring individuals based on things they won't use during the job, of which the knowledge is still incredibly varied[0], and they still complain about not being able to find anyone and play the "woe is me" card.
[0]: Dare I say it, DS&A is such a big topic not everyone learns the same things. Thinking in terms of a tree is different from thinking in terms of a linked list, graph, tree, heap, stack, queue, you name it. Most LC medium/hard require time you won't get, information you might not have. This knowledge isn't as universal among skilled graduates as people like to believe. And we all know the moment candidates are able to just memorize questions about linked lists, hiring will jump to the next topic.
> what baffles me is that even a lot of smaller companies do this
Thoughtfully creating an interview process and the procedures/questions is a skill that has to be learned... and when people haven't or don't have the opportunity to learn that skill, it makes sense to turn to resources like books or articles, the most popular of which are often based on FAANG practices.
I also know from my own experience structuring and performing interviews that there can also be a lot of pressure from the top to eliminate candidates following the belief that "the last one standing" is the best, rather than actually trying to evaluate the strengths and weaknesses of every candidate.
That way tends to lead toward interviews of 5-7 rounds that act as sieves.
This is all just my limited personal experience though and I'd love to hear from other people who've done interviewing at/for smaller companies!
> When I was on the market, a couple of companies actually gave me a variety of options, which I appreciated. I don't want to spend hours on a take-home and I also don't want to do leetcode, but an open question/answer plus some code review and live debugging was an acceptable combination for me.
That sounds excellent!
This would allow the company to determine what the applicant believes is their strongest suit.
Then, when it comes time to sit down as a team, and review all the "top shelf" applicants, people can decide, based on a number of criteria.
If an applicant avoided technical challenges, but spoke well, then some managers might like that, but others, might not be comfortable. Maybe they might devise a technical challenge, customized to the applicant, and using real-world problems.
The main deal, is that the vetting has been done. The chaff has been filtered out, so more attention can be paid to the wheat.
At my old corporation, headcount was something I had to fight like crazy to get (I was a manager). We seldom hired, so it was well worth it to spend a great deal of time on each candidate, as they would be responsible for important work, and would have a great effect on the corporate bottom line.
That sounds like a lot of small startups, to me. BigCorp (MAANG, et al) hires thousands of engineers per year. They need to have a cookie cutter system. Smaller companies do it, simply because they want to be like the Big Boys.
Also, and this is neither here, nor there, but binary tree tests are a great "young-pass filter."
The alternative is to push companies to pay for long interviews, be realistic with their requirements (stop asking LC for simple CRUD work), have some faith in schooling, start carrying some risks again, and use the probationary period for what it's designed.
What's happening is companies are putting the burden of the risk on candidates more and more. Because they can. If candidates would put their foot down and stop accepting this, most of these shenanigans would stop. The junior market shows what happens when people are desperate for jobs and willing to bend over for any whim corporate has that might help their hiring process (even when most of it is completely unproven).
> use the probationary period for what it's designed
This is probably fine in an environment where the candidate has lots of other options, but think about this scenario: a person applying to multiple companies, possibly rejecting some offers, possibly relocating or otherwise changing their life, accept that one offer, and then be let go in their first weeks. Changing jobs can be emotionally difficult and being let go even more. If you have to restart your job search, because you were let go during the probation period, all the other roles you have applied to might have been filled. On top of this, contrary to when you were looking for a job last time, now you're actually unemployed and potentially under pressure to find something new.
So, in essence I don't think this is good for candidates.
You obviously can't take the above in a vacuum and think it's fine. You could say the same for the advice of "give people work which resembles what the company does" leading to a situation akin to having interns doing minimum wage work for free, putting pressure on others as a result.
But as things stand, nothing is preventing companies from doing the above anyway. If they think you're a bad fit, they will use the probationary period to cut ties with you. This is perfectly viable today. Your example assumes current filtering methods do in fact increase the ratio of true : false positives, and taking some of them out would decrease that ratio. This is not something that has been proven, and I'd even argue it's something that can't be reasonably proven within the next few years. This is even worse when considering a few interview rounds can only filter for the most obvious dummies, but can't decisively tell you the performance of that individual a few weeks down the line.
What your example does show is how much power employers have over employees. It just isn't healthy for individuals to have to carry this amount of risk while corporates continue to reap the benefits.
I keep seeing this kind of article that leetcode is not the answer. Yes it is not the answer but the best strategy to filter out not dumb, low IQ people who are going to panic when you give a vague, open ended problem. There are many people who call themselves engineers while trying to talk everything out instead of diving deep into problem.
You know what else will filter out dumb people? A vague open ended problem that is relevant to the position you're hiring for. Have them write an abstraction layer for two different implementations of [insert relevant problem here] or refactor messy duplicated code into reusable [methods/components/modules]... Better yet, find an actual problem that your engineers actually had to solve in the past and see how they do it.
> filter out not dumb, low IQ people who are going to panic when you give a vague, open ended problem
The article addresses this as well. Stage fright does not make someone “dumb”. Slow thinkers aren’t “low IQ”. And reversing a tree likely doesn’t make someone apt for a job.
I don't really understand the point almost all of the thing they list as downsides of using Leet Code are actually benefits. If someone can't manage to code some simple questions during an interview I can't imagine they'd ever make real contributions.
What if they suffer from social anxiety and they just struggle under the pressure of an interview?
I’ve seen so many people over the years do terrible on these types of test that went on to be amazing contributors. We only found this out because we essentially ignored the outcome of these tests, which makes you wonder why you’re doing these tests at all!
if the "simple questions" are realistic problems they might face day to day, fair enough. But the classic example in these articles is an exercise like invert a binary tree, which most people never have nor will need to do, making it a poor test.
I think the worst part is that half the candidates will never have seen the problem before, and half of them will. Which makes it really hard to make fair comparisons.
That doesn't mean that someone would be unable to program it. Inverting a binary tree tests if you know how to traverse a tree. I may not traverse trees all day at work, but I can easily do a tree traversal if I needed to and I expect that to be true of most people who can actually program and not just talk the talk.
I understand what you mean, I could also probably pass the test. But what is the utility in an arcane test like this? If I'm hiring web developers, I'd want them to see how they apply their knowledge to a realistic scenario. There is a real chance that a potentially great hire with decent experience in a desirable technology will fail the inversion task.
sure, I often have to do tree and graph traversals in my day to day but, I will 100% fail to do that under pressure with people looking at me and expecting me to "talk throught my thought process".
If you really really care about me traversing a graph, leave me alone with the task for a while, let me take my time, give me access to the internet even. Why do you care that I can write it on the spot? If anything that just proves that I memorised it just before the interview, not that I actually had to think much about it
- interviews are inherently more stressful than even a very busy day at work,
- they cover topics unrelated to what you'd do as part of your job so if you're good at the job you might still fail the interview
- people with time (==money) to prepare for interviews will come out ahead of those who don't have that luxury, despite the fact they may be far better prepared for the actual job.
So you get a bunch of false positives, a bunch of false negatives, and on top of that you're discriminating against people who are in a worse financial situation or have interview anxiety. I can hardly think of a worse outcome for a seemingly sensible hiring method.
I haven't done many Leet Code-style interviews, but the ones I have are usually simple-ish problems I'd have no difficulty with under normal circumstances, and yet I fail them at least 50% of the time.
There are a couple of things at play here:
1. I'm regularly complimented on how sharp my mind is, but I can't reliably recruit that sharpness on-demand. If I get any kind of brain fog during a timed problem (with no opportunity for a break), it's game over.
2. Coding in an unfamiliar environment, like Coderpad.
The problems are hard enough to make them easy to fail if you can't spot a good approach immediately, but easy enough that a "good" answer gives very little signal relative to the time being dedicated to the problem. I've done interview processes where 50% of the total process is given over to these kinds of problems, and the process concluded with me feeling I didn't get any real opportunity to demonstrate my strengths.
And last but not least they are geared towards puzzles of the type encountered during CS education. Thus they are age discriminating and they are non CS discriminating.
Just talk to the person, ask them about projects they’ve worked on, problems they’ve solved etc. you’ll learn far more about them that way than getting them to put on a dog and pony show at a whiteboard!
I've found that it's relatively easy to figure out if someone is bullshitting. It just takes some work on your end by researching the candidate's application a bit deeper, thinking about relevant questions beforehand, and then drilling down during the conversation. I've hired 'non-silver-tongued' candidates this way, as well as filtered out people who were bullshitting. The key is to actually have a conversation, instead of just letting them produce canned stories and responses.
But, yes, this is more work on the recruiting side.
As a side note, I totally get it why large organizations would use Leetcode though. It does help filter out false positives if you're willing and able to bear the cost of passing on many good candidates.
Yep. I've found that you can give someone enough rope to hang theyself. If you let them explain their processes and ways of working, most of the bullshitters will out themselves :D
If you hear bullshit, then that's easy: you filter out such people.
Not really silver-tongued candidates? Well, what's the problem? Unless you are looking explicitly for silver-tongued candidates, you should ignore this "trait".
I think an extra hour to talk will give me more information than watching them balance a binary tree.
For many people the stress of the leetcode session is just awful, I don’t think it’s a nice thing to inflict on someone if you’re ultimately going to disregard the outcome anyway!
I have to say this is how I've always interviewed people - I tend to be optimistic and assume people don't lie about their skills and experiences. So when I've been asked to interview people I mostly want to talk about projects, hobbies, and general things about developing, debugging, and the process by which people solve problems.
I don't want to know if you can do specific things, if you can't you'll learn, but I do want to know if I'm gonna find it easy to get along with you, if you seem like an interesting person, and if I can stand being in a small room with you for 7 hours a day.
I suspect this is just another kind of bias, but I've had good results.
Great to hear this. We use a similar approach, we sometimes get flattery in the form of "this was the nicest interview I ever had". I believe in human centric approach (the golden- and silver rules), and if they have the fundamentals right, technical skills etc. they can learn the rest on the job! Quickly too. And stay working for many years, low turnover, more cost effective.
I thought that is one reason there are the standard 90 day trial period. Instead a lot of time is spent in interviews finding the perfect candidate (which doesn't exist).
It's much better than the alternative and ideally only one part of the interview process.
Ultimately the proof is in the pudding, I've had loads of candidates that could barely code and Leetcode-style problems are a great filter against that.
Otherwise you risk just getting PM-style bullshitters as engineers, who talk persuasively about projects that other people actually implemented.
People who can do leetcode challenges are only roughly correlated with great engineers.
The more I learn about software development the more I try to _not_ have leet code style parts in my code - in the very rare circumstances would I see something like that and say - yes there is _no_ other library out there that has a battle tested algo to solve this particular problem, _and I have to code my own_.
Worse people who excel at Leet Code could start to bring that sort of thing into your codebases, optimising small parts of your code to perfection, and then leaving the whole thing a mess or just plain reject to work on solving business issues and concentrate on polishing their small bit of algorithm.
Of corse there are positions where that sort of mindset is welcome and sought after. Especially in very big companies, but for the most developers out there leet code is _just_ for the interview part and they would (should) never use that kind of problem solving in their day-to-day work.
When I have to interview somebody, and there's leet code involved:
a) they are _really_ comfortable with those types of challenges - that tells me they have spent a lot of time preparing - either for this interview or just generally, either way that wouldn't _really_ tell me much about their problem solving skills
b) they feel uncomfortable and fail to solve anything due to stress or anxiety - again no really knowledge gained, as real-world work environments tend to try to minimise those, at least in the places where I work in.
c) they feel uncomfortable but get the hang of it - now I _might_ have a glimpse of their problem solving skills, but I've put a human being in a very uncomfortable position just to _test_ them. There _has_ to be a better way.
The way I like to conduct interviews where possible is to ask people to walk you through some of the code they've written and ask various questions about decisions - much more relaxed and I still get the sense of how they organise their stuff and how they work.
If that's not possible - a take home task or something. Or even "hire fast fire fast" approach works too.
I think it's fair to do two Leetcode Easys - ideally with problems that allow you to ask further questions about memory management (when are allocations triggered, etc.), recursion, stack frames, adjacent memory location for efficient reads and so on.
The problem is that a take-home task can be impossible for some people already in a full-time job with kids, etc. and so has its own bias. And the "hire fast fire fast" approach doesn't really exist here in Scandinavia, even though there is a probation period it's still frowned upon to use it for anything but extreme circumstances, and if done en masse would likely raise issues with the trade unions.
>memory management (when are allocations triggered, etc.), recursion, stack frames, adjacent memory location for efficient reads and so on.
This sounds incredibly specific. Most places requiring at most LC easy don't really need any of this, and I'd be worried about the interviewer looking for key words and specific answers over actually testing whether the candidate grasps things or can learn what is necessary.
Then part of the process should be to allocate part of the interview schedule to the take home. Say, 1 hour interview block, at minute 1 candidate is sent an exercise and at minute 45 a zoom call starts to discuss things.
That way, some of the pressure is off of coding with someone looking over your shoulder, and you still have time allocated off already.
> c) but I've put a human being in a very uncomfortable position just to _test_ them. There _has_ to be a better way.
> ...
> Or even "hire fast fire fast" approach works too.
Comment "c" shows a high level of care/concern for the candidate. "hire fast fire fast" does not.
What if the candidate left an unsatisfying but steady job to join your company? And they get fired in the first few weeks. Now they are unemployed. This is much more uncomfortable than an awkward interview question.
Leetcode should only be used for easy problems - FizzBuzz types - where you are checking that the candidate can code at all, probably as the very first round of an interview. This way, I think it is totally fair, including a relatively tight time limit. The harder the problem is, the more you focus on the special skill of leetcoding, and not much else.
It’s absolutely hilarious you think leetcode would give you the skills to design anything larger than a simple website. I don’t recall leet code ever talking about how to set up an infrastructure that has reliability on an enterprise-level where seconds of downtime means millions of dollars lost. But you’ll be really good at hash maps and quicksort which make for good worker drones, not necessarily good engineers. that requires outside the box thinking, The opposite of the type of memorization leetcode encourages
“Absolutely hilarious” that you’re forgetting there is usually a system design round that tests things like “setting up infrastructure on an enterprise-level”.
The coding round is supposed to test that.. you can code.
For the record I’m also not a fan of the Leet Code style rounds, even though I actually find completing them (outside of an interview) quite fun.
> It’s absolutely hilarious you think leetcode would give you the skills to design anything larger than a simple website.
The point of leetcode is not to show what you can do, it is to show what you can't do.
To keep up with your analogy, being able to design a simple website doesn't make you able to set-up an enterprise-level infrastructure, but if someone isn't able to design a simple website, I don't want to hire him for my enterprise-level infrastructure.
As for "outside the box" thinking, it only works if you know where the box is, otherwise you are just being clueless.
And by the way, because it is an argument I see way too often, calling the "sort" function of your favorite library instead of doing the exercise "as intended" is not "outside the box" thinking, it is literally the most obvious thing to do. What could count as "outside the box" would realizing that you can solve the problem more efficiently by not sorting anything at all, which requires you to know your sorting algorithms to show that it is indeed more efficient.
There's quite the distance between FizzBuzz and LeetCode medium/hard questions. If you want to filter out candidates that barely can't code, just have them walk you through a small code-base containing bugs.
What does the code do? Does it work? How would you get it working? How do you test it? What is the complexity? etc.
I really disagree here; I've been a successful engineer at a Fortune 500 company for a decade. I'm a front end engineer. I still have absolutely no idea why companies choose to test FE engineers with leetcode style questions (which I am terrible at). My job has almost NOTHING to do with leetcode, and yet it's a huge hurdle to go over for some unknown reason.
I'm a great engineer, I just suck at leetcode questions because...shocker! They have absolutely nothing to do with my day-to-day. It's similar to puzzles; I hate puzzles and doing puzzles/escape rooms/etc, but I'm a pretty smart guy.
Leetcode is merely a gatekeeping methodology that proves almost nothing and weeds out a lot of amazing engineers who aren't skilled at grinding leetcode or hate brain teaser shit.
Yup! And they work well as a filter for candidates too. If a company leans heavily on these then you know it’s probably a dehumanising corporate hellscape.
> And they work well as a filter for candidates too
I agree. A couple of years ago I was asked by two companies to solve leet code problems even before there was a screener meet-and-greet interview. They were quickly crossed off my list.
Was contacted by Facebook recently for a data engineering position and was told I had to grind some leetcode to prepare for the interview process. I just could not be bothered and I can not be the only one who made that decision. My guess is that companies relying on leetcode style interview are also missing on some very capable engineers who just don't want to grind leetcode outside working hours.
But that's not a problem. These companies pay so much that they have too many candidates applying all the time, so missing good engineers is not a problem, but filtering out false positives is.
Though, I don't agree leetcode is a good interview practice.
Has this ever been tested properly? Has anyone ever given a failed candidate the job and monitored how they did compared to candidates who passed the LC?
There are many issues with this approach, starting with using the same tools to evaluate vastly different candidates, e.g. a college kid would likely outperform a senior guy in Leetcode questions, so that offers no proof of competency.
In a previous job of mine, we would show candidates a printout of some buggy code, and ask them to find the bugs. We would leave the room and let the candidate work through it on their own. The code in question was basic algorithms and data structure stuff in C++, such as inserting into a doubly-linked list. I always thought it was a good exercise. Suits slow-thinkers and nervous people, and it's a good test of coding ability since if you can spot a bug then you can clearly read and understand the code.
Man, I would rock that so hard and I wish more companies would do this. My current job did a live coding exercise in the interview, which went okay. The interviewer gave me a chance to iterate on the code and improve it after our call and send it by email an hour later. My solution by then was far better than what I'd had at the end of the interview, once I had a chance to gather my thoughts in peace and think everything through thoroughly.
I've had a couple of sysadmin-interviews which were pretty interactive.
You're presented with a laptop connected to a remote VM, and told "Fix MySQL", or "Rewrite the git history in this repository".
Usually these are simple problems, which have obvious solutions. Every now and again you might get a surprise like an immutable-bit set on a file, or SELinux blocking access to specific files/paths, but the good thing about these kind of "challenges" is that you'll usually also have full google access.
I guess l33tcoding isn't really a thing for sysadmins, but I do appreciate a (fair) simple test like that, especially being given the opportunity to talk through the process.
So you basically implemented an analog version of Stack Overflow? :) That sounds really nice, once you get over the idea that people will show deliberately broken code (most folks on SO just don't know better, or at least that's my hope). Thanks for sharing!
Was the bug in the algorithm resulting in wrong results? Was it a kind of implementation bug that happens due to peculiarities in the language/compiler?
It was a bug in the algorithm, resulting in wrong results. For example, I recall that to fix the linked-list insertion code, you needed to add a line of code to increment a pointer.
Yes, I do exactly the same with a contrived example in a take-home coding assessment (with some failing test cases to not waste their time). Always seems to be a good indicator, especially of their ability to communicate technical concepts.
Only the first one you mention (a task to be solved in real time with both the interviewer and the candidate on the same video call) is the only one that respects the candidate's time. The other ones do not. A take home assignment could mean hours or days of effort for the candidate, while it may take at most 1h or so for the interviewer to review it. Totally asymmetric.
I think it depends on the perspective (and the state of the market): two scenarios a) "candidates are willing to work for companies, therefore companies have a bit more of control over the whole interview process", and b) "companies are looking for (scarce) good talent out there, therefore candidates have a bit more of control over the whole interview process".
When I was a junior developer I used to deal with a). Now that I have more years of experience I usually deal with b).
Instead, I offer to share a private GitHub repository, (if not some open source) with the prospective employer. It offers them the ability to see how my code changed over time, how it ended up, and the quality and calibre I may or may not devote to projects.
It invites a conversation, in depth, about software construction, quality, and decision making from the point of view of real work. It also tells me if a company wants to simply filter. If the organization is unwilling to invest in a candidate interview, as I do to be interviewed - I in turn learn a lot, and decline to pursue accordingly.
My experience is that employers refuse to do this (I have dozens of repos, with tens of thousands of lines of code, entire shipping applications, a decade of checkin history, dozens of Web articles, and hundreds of pages of documentation and project planning artifacts). I even had one indicate that “I probably faked it.” It was jaw-dropping.
In fact, I have so much stuff out there, in the public realm (I’ve been doing open-source software for decades), that employers could easily evaluate me technically, without ever contacting me, and all they’d need to find out, is whether or not I’d be a decent team fit.
I suspect that the main reason they ignore my portfolio, is that they have already decided that they aren’t going to hire me, and don’t want to waste their time, reviewing my work.
It also helps me filter out companies that are either too lazy, or too strict in their procedures, or both, i.e. there is no value in asking the same questions, regardless of candidate’s resume.
As an example, a month or so ago I went through a couple rounds of interviews with this company. I told them forthright that I wasn’t doing any code assessments, and if they wanted to, they could take a look at my GitHub account. They seem willing to consider. Third round comes in and they ask me to write some sorting algorithm, to which I refuse.
Ironically enough, one of my repos does have an implementation of an advanced data structure, so they could as well just take a look at it.
Benefit #175 of having some kind of open source project in your company: you can open all types of different issues and user stories, and use them to discuss with candidates. You can even attach bounties to them so that candidates don't feel like they are doing free work, and it would still be cheaper than paying a recruiter that will likely just source candidates by spamming different sites.
I love this. Picking an issue from some well-organized dependency has worked for me (I wonder how I’d present this to AI/ML positions). I hadn’t thought of the positive signal of issuing a bounty.
Leet Code tests can be a great way to find good engineers - after you've told them what complex puzzle you want them to code, you can reject any that don't ask you why you need it! :)
I recently failed interviews at Google and Amazon, exactly at this sort of algorithmic problems. And that's quite far from my client application daily job. I knew it's not my strong side, I would have applied before they approached me otherwise. Now I can tell everyone - I knew quicker than Google, that I'm not the right fit ;)
Don't get me wrong I hate LeetCode-style interviews as much as the next guy. In fact I really, really, really suck at them! Not sure if that says more about my ability then anything but c'est la vie
In the defence of LeetCode-style questions, I do think they work, and very well may I add - with the caveat you have the throughput of candidate to make it work well? Their ability to filter out 'those who can't code' in an efficient manor while sacrificing a small amount where it filters out 'those who can code' greatly out weighs the alternatives. The alternatives needing to fit into a 1 hour timebox, be objective while also favouring the positive cases (I think I got that the right way round).
My two cents would be more around the way in which they are conducted; in my experience I've found conflict with the interviewer more then the process itself - with interviewers in my past lacking.... empathy (may not be the right word) for the person on the other end of the screen/table feeling flustered, nervous or down right stupid that they're struggling to solve a simple fizz-buzz/reverse string problem, leads to a snowball effect and pilling onto that can effect the candidate in quite a spectacular way. Best interviewer I've had asked if I was alright and got me a glass of water, props to that guy!
I dunno - I've just come to terms with having to learn how to play the game, even if I find that part of the game really hard and to some parts unfair. Such is life
There's an assumption you're making that these challenges only filter out a small amount of qualified candidates. I suspect the percentage is actually quite significant. In my experience it's usually bigger "looks good on a resume" companies that do these challenges, which suggests they're getting enough candidates in the funnel to be able to afford turning away a lot of more-than-qualified applicants.
I've failed more than my fair share of these challenges, but never (being subjective here) because I wasn't actually capable of (1) solving the problem or (2) doing the job. My take here is that I _may_ have been unqualified for these roles, but that the interview failed to actually uncover it, due to spending all the available time on low signal exercises.
Big companies also have the headache of standardizing across thousands of interviews. That said you can do that more with practical exercises (build this thing…) than leetcode
> There's an assumption you're making that these challenges only filter out a small amount of qualified candidates. I suspect the percentage is actually quite significant.
Statistically that’s irrelevant, as they optimize for “do not let through rotten apple”, rather than “find good apple”.
Which is horseshit. I'm currently working at my first bigco company, and I would consider the MAJORITY of people they have hired in the last year to be horrifically bad hires, because they have optimized for new hires that can leetcode, but can't actually code.
We just fired a person on my team who didn't understand pass by value vs pass by reference, or how to debug in an ide, but she could manipulate strings in leetcode!
That assumption isn't being made at all. That's why it says with sufficient throughput of candidates. If you need to hire 100 people, have 10,000 candidates, 1,000 of whom are qualified, and a 90% false negative rate, you'll get the 100 true positives you need, while leaving 900 well-qualified people pissed off. The process is bad for most of the candidates, but works fine for the company doing the hiring.
The problem comes when smaller companies that don't have the same high rate of new applicants use the same process and then complain they can't find anyone.
> "Their ability to filter out 'those who can't code' in an efficient manor while sacrificing a small amount"
This is the assumption I was referring to, that the "sacrifice" is small. It's suggesting that the false-negative rate for LeetCode challenges is small, and I'd argue it's actually quite high -- as you also suggest (your rate is 90%).
Any hiring process will necessarily filter out a large percentage of qualified candidates so it's not a major concern. If I'm hiring for X positions, and there are X + Y qualified candidates, then any hiring process whatsoever will necessarily have to filter out at a minimum Y candidates, and usually Y will be much much greater than X.
The problem is that the total number of candidates who apply for the position, Z, is significantly higher than X + Y by a very very large margin, and I mean orders and orders of magnitude. For every position I post I get on the order of 600-1000 applicants in a matter of a week, even though I'm only looking to hire maybe 2-3 people. Of those 1000 applicants, 80% of them are simply unqualified, and that's being really really generous just for the sake of argument (I'd wager the figure is closer to 90-95%). So once again just for the sake of argument that means 200 of them are qualified, and I'm hiring three, which means any process I choose whatsoever will filter out a minimum of 197 out of the 200 qualified people, no matter what I do.
Given that calculus, it's better for me to focus on making sure that I filter out the 800 people who are simply unqualified for the position even if that means I end up filtering some of the 200 good developers, because I have no choice but to filter out at least 197 of the good developers anyways no matter what, whereas I do have a choice about filtering out the 800 bad ones.
I have had a great experience in a recent interview that used Byteboard. The format was two parts: First there was a design document in a Google Doc for a hypothetical system with three implementation options. All options were defendable, you just had to defend one in an essay-style answer. There were also various comments to respond to throughout the document.
The code part was a small existing codebase simulating the system in said design document. You you are given three tasks, and explicitly told you are not expected to complete all of them. I ended up using virtually all of the time (70 minutes) completing the first two tasks, and using my remaining minutes writing comments about how I'd complete the third task. When that was complete, I was given 15 minutes or so to describe what I would do if I was given another 15 minutes of time to work on the project.
My only real complains were that the time limit added some pressure (that I was able to manage reasonably) and that the grading process is opaque. I know a human grades it according to a rubric, but I don't see any of my results. The company I was interviewing for just said "Everything looks great, we're moving you forward to the next stage of the interview process".
The code didn't involve writing any fancy algorithms, but instead getting to know a (very small) existing codebase and understanding how to use it to add functionality. This is much more realistic a gauge of how good of an employee you are than how well you can implement a search algorithm from memory.
I interviewed hundreds of C++ developers as a freelance assessment interviewer. The most candidates I interviewed wanted to work in the automotive industry in Europe.
Many candidates (like: maybe the half) are fancy talkers without any skill in writing code. I really don't know why they are applying for dev jobs. It is easy to filter out these persons with a very simple coding test.
I agree with the article that 'leetcode' tests (find that complicated algorithm in 30 minutes while I am staring at you) are bad. But I think coding tests are good! Give the candidate just a really simple coding task with stuff they normally do every day. Create and delete object, fill arrays, iterate over arrays, and so on. 50% of the candidates will fail! The rest are OK engineers.
Exactly. Give a simple coding task that the candidate does on their own time (but not one that's too complicated, it's rude to waste the time of people who aren't even working for you), and then, crucially, have them briefly explain their solution during the interview. The task can be so easy that reasonable solutions are just 10-50 lines of code – my experience has been that people's explanation of their own work very clearly tells you whether or not they're a decent coder.
What if that test was done once by an external company, and then the stamp of approval is good for a year of job interviews. That would save the candidate and interviewer a lot of hassle!
I agree. You would expect that from a bachelor or master degree. But nowadays you have a lot of people coming out of coding bootcamps, and their level can vary a lot.
> I agree with the article that 'leetcode' tests (find that complicated algorithm in 30 minutes while I am staring at you) are bad. But I think coding tests are good! Give the candidate just a really simple coding task with stuff they normally do every day. Create and delete object, fill arrays, iterate over arrays, and so on. 50% of the candidates will fail! The rest are OK engineers.
Do the candidates know that ahead of time?
Your approach seems similar to what I've done as a coding interviewer and to what my interviewers did when I last interviewed (at Google back in 2007). Assess people's coding with something relatively simple. Make sure they can gather requirements, describe why they chose this approach instead of a couple alternatives, and (if they make a mistake) that they can diagnose it if you describe the symptoms. If you have extra time, have them review some bad code. See if they spot the problems and how they gently help the author understand/fix them. (They also should be tested on system design, but that's a whole other interview slot.)
I'm studying to be a coding interviewee for the first time in 15 years. This process is stressful in part because they just tell you coding on hackerrank/leetcode/coderpad, which is so broad. If the problem they pick is as you describe, I should be fine. If it's for example some advanced dynamic programming problem...well, those haven't come up for me in the last 17 years, so I'm probably in trouble. It's a perfectly valid area of computer science, but it's not one that matches my experience or what I'd likely be doing if accepted. I'm not excited about taking the time to prepare for that, but I also don't want to make a fool of myself if they do. They probably won't pick this area...and if they do, it's probably a bad sign about the company or my understanding of the role...but the possibility is stressful nonetheless.
Someone who builds a truly novel technology solution involving hundreds of hours of effort gets filtered out of an interview involving contrived scenarios. You may have built the next generation X, but given an array of strings and a fixed width, can you format the text such that each line has exactly maxWidth characters and is fully justified -- in the next 30 minutes? Maybe you should have cultivated that skillset instead, because around here we value parlor tricks more than real world accomplishments.
> What do you suggest for the 99%+ other candidates?
What about (instead of forcing a months long decision process upon the candidates and the company) bringing them into the company after a short interview (maybe 2hrs), and making sure they can afford housing, food and everything else they need.
If you like their work, they stay employed. If, say after one month, you do not like what you see, you can easily let them go. Of course you tell them upfront what the deal is.
We could call it, I don't know, maybe trial or probationary period.
That's a big overhead for both the candidate and the company. Only an unemployed candidate could do that, and even then they'd have to stop interviewing at other places to dedicate the month. No thanks.
I think we'd need to share more detail (and fwiw, I'm half from europe and half from Canada :).
Certainly companies have probation periods. And on paper, that reality and what's proposed in previous post are similar.
But I think there's a massive real world difference between "Default stay hired" and "Default not stay hired".
Probation, as it has currently been implemented in most companies I've worked in, exists, is formal, can and has been used, but is an exception. It's used when there's a massive, unanticipated, egregious problem in performance.
What is sometimes proposed in these threads is effectively replacing long/multiple interviews, with a probation period. While such probation period may look similar or same on paper, I think it's a completely different approach: "We're sure of you (though possibly wrong) so we're hiring you" vs "We're not sure of you so let's hire you and see!". I for one would have only touched the latter with a 100ft pole maybe once in my life. Certainly, I imagine anybody with current job and monthly obligations, would be quite wary in taking a "we don't know so let's try it!" approach to hiring. No, let's figure it out first please :)
I've only done the latter in the form of being brought on as a contractor at (high) contractor rates but with the understanding they'd prefer to have me join full time, at a time when I was already doing contracting and had other clients in parallel covering parts of my costs. In that situation I was not taking on any more risk than I had already chosen (and planned for) by contracting, so it was fine.
It's the only kind of context in which I'd ever consider the "we don't know so let's try it" approach.
My expectation with such an arrangement is that if you do decide to hire me, since I am now a known quantity you won't have an excuse to pay anything other than top of the market rates. "You said you hire only the best right, and you've seen me work, you want to hire me, looks like the best make $X."
It would be fair if the company has to pay you 5-11 months of salary if they decide no after the evaluation period. That would leave ample time to find another job. Also, in many jurisdictions, this kind of arrange isn't legal, for good reason, as the company has way more power over the individual worker.
We could make sure that a person only has to do that trial once in their career and then every other company should accept that they've done it because they've proven they've done it. We could call it an Apprenticeship or an Engineer-In-Training stage. (Where have I heard those before?~)
> But then why do people with that kind of reputation still (at certain companies) have to jump through these hoops?
Your article points out that in this example: "I'm not allowed to check in code, no... I just haven't done it. I've so far found no need to.".
> If, say after one month, you do not like what you see, you can easily let them go. Of course you tell them upfront what the deal is.
> We could call it, I don't know, maybe trial or probationary period.
You make it sound like it's a better solution for candidates, but it's way worse for many of them and it has been explained by other commenters already.
How many companies actually do this? At which scale?
Some companies increased their difficulty to hire by having aggressive PIP objectives. Likewise, having a "real" probation period where you fire, say, 10%+ of employees is not gonna make you competitive when candidates compare their offers.
I have over a decade of experience, including driving big technical change at one organisation, and was filtered out by a timed leetcode test. I put together a repository of leetcode practice, as I had a feeling that I would have bad luck on the day. They didn't look at this.
The internal recruiter said it kept happening for seniors and people with a lot of experience, but his hands were tied, as the leetcode process was deemed important by the CTO.
If you think it’s about flexibility or brilliance or intuition or experience, you couldn’t be more wrong. Leetcode and others and purely about practice and that too recent practice, all questions follow similar patterns and if you have it fresh in memory you can write it in 5 mins. But writing it from brainstorming can take more time tha allotted.
So are top leet coders better programmers? Not really, a lot of them are colleges students who have time to practice and they are in similar competitive circle. I’ve interviewed many and couldn’t hire even one
People straight out of college will spend a month doing 100+ Leetcode problems targeted at the companies they're aiming for. Leetcode tracks problems used at interviews, so there's a very good chance they'll see a problem identical or similar to one they've solved already, and it's just pattern recognition.
After the 3rd or 4th time doing this, practicing the same problems just to pass interviews isn't very appealing, especially if you've saved money and can do something more interesting.
You could try to "wing it" and derive it on the spot, but you'll be outcompeted by someone doing a lookup of a solution + alternate solutions from cache.
Nah, it’s just that LC problems take practice to get good at. Similar to weight lifting you wouldn’t expect to bench 250 lbs your first week, but after training and making progress you’ll get there (or close).
If you’ve never done LC (or any competitive programming) you’ll struggle. Practice more and you’ll recognize the dozen or so patterns.
Also LC is not “novel,” maybe at the time when the algorithm was first devised but not when you have 20 minutes to solve one.
Bingo. I was asked recently to find the maximum subarray of an array, in a live coding exercise. In TypeScript. So, JS, but with type annotations.
The interviewer themself said "this is probably more aimed at someone who just graduated."
It was for a senior data engineering role, where the odds of me implementing classic dynamic programming problems on the regular are slim to none.
They made me an offer anyway, but I just wonder what value they found in that. Oh, he knows about Big O? He's heard of memoisation?
They were big on FP, apparently. But not Scala, there was too much FP in that for them, hence the TypeScript.
Mind you my first ever rejection was for a Python role back in 2011 when Python was still very niche in my country, and while I had a portfolio of, imo, pretty decent Python code, they weren't interested because I didn't have a degree, and people without degrees write unstructured code.
Which is a very long way of saying, every interview process ultimately devolves into people hiring people like them.
I was once asked "how do you sort a list of integers?" - the question was so ill defined that I thought it was a joke (in a database?, how long is the list?, in a flat file?, memory only?, on an embedded system?, a dataset in spark? ...).
Thinking, ok, this must be an ice breaking joke I responded, "I don't know, how _do you_ sort a list of integers?". In a condescending tone they responded "are you even a programmer?". At the time, I had been in the game for more than 10 years.
I am really not sure how one could successfully build several companies and have shipped several products without knowing "how to sort a list of integers".
Trying to find a good place to grow your career is difficult on many levels, but if you find leetcode questions silly, you might be too advanced for entry level jobs. ...and you probably don't want to work there.
I've forgotten most of the algorithm stuff I learned at university that I haven't used in my job. I know how dynamic programming works and I can recognise such a problem, but don't ask me to implement it in an hour. I know how graphs work and what the various algorithms are, but why would I implement any of them when I can just import networkx?
The tests aren't testing for competency at the job, and after a decade of experience writing software you have long ago realized that party tricks and cute algorithms are a fairly rare part of the job (generalizing here of course), so you stop thinking about them as much and get out of practice. When they do show up, you certainly don't have to do them in 10 minutes, and I think everyone would rather you didn't anyway, so that you write a robust solution rather than a clever one.
Students are often better at leetcode because school has been drilling this shit into them for the past three years, but it will probably be the last time they see such a compelling algorithmic challenge until their next leetcode exam.
but why must it be mutually exclusive? Are you implying all "robust" solutions, whatever that means, are dumb? Surely you put some thought in it to make it "robust"?
In general in this type of use, "clever" and "dumb" would better be called "tricky" and "obvious" respectively. Usually people describe very tricky solutions as "clever", and much more obvious solutions as "dumb" jokingly.
For example, storing some flag in the high bits of a pointer field of a struct is a "clever" solution, whereas having a separate bool field is a "dumb" solution. In most cases, the "dumb" solution is much more robust over time (less likely to cause bugs as the code changes and is modified by various people). Of course, the "clever" solution is necessary in some situations (very constrained environment, critical infrastructure such as an object header used for every type etc), but should often be avoided if possible.
What's important is that the way this is often presented is that more experienced people will prefer "dumber" solutions, as experience often shows that long-time maintainability trumps many small losses of efficiency. So using "clever" and "dumb" in this way is not at all intended to put down the engineer writing the more robust version.
When it comes to everyday software solutions you are usually aiming for obvious and clear, and often 'clever' is obtuse and opaque but definitely not always.
I think there might just be some vernacular nuance here though, maybe we can call it smart and robust, versus clever and opaque. Some problems are just difficult though, and if you get to work on that kind of problem regularly then that is pretty lucky.
More likely a lot of us don't feel we have to cram for problems suitable to screen entry-level candidates at most.
Personally I'm perfectly happy to be filtered out by such tests and refuse to practice for them, as companies that use them for senior level positions are companies I really don't want to work at.
Novel solutions aren't as helpful as you think. Pragmatic, simple, vanilla solutions are reliable.
You won't create your own linked list library, you'll use one from the standard library.
General runtime analysis can be helpful - but production, real-world benchmarks trump all theoretical performance values.
Code changes - how do I make a change to a production system in a million line code base that has good test coverage and when deployed, won't bring the entire system down. That's an exercise in the coding interviews that is completely ignored but most useful in the day-to-day professional setting.
> General runtime analysis can be helpful - but production, real-world benchmarks trump all theoretical performance values
This is what I always found funny. Most of the software development work these days is related to web apps. Optimizing that nested loop won't do anything if you have to wait 300ms on some shitty API to answer anyway. It literally doesn't matter, noone cares.
Related meme I saw on reddit some time ago where senior developer says 'haha nested for loop go brrrrr':
> What do you suggest for the 99%+ other candidates?
To apply for the 90% tech companies out there. 10% of all tech companies out there are FAANG or FAANG-like. 90% of tech companies are normal tech companies (they'll care about your education and cv and the interviews are usually just a chat. No IQ tests)
I think it would be exceptionally hard to build an individual reputation in the tech community in general, props to those who have, but building a reputation in a specific industry and local community is much more achievable for everyone.
In smaller industry niches this can be true for companies more than people. At this point in my career, the fact that I worked at Company X is evidence enough that I can do the job Company Y wants me for, since it's a tight industry they essentially know of the work I was doing, even though it wasn't a groundbreaking novel technology of my own.
Wasn't there famously the story of the guy who wrote a package manager (brew?) that become massively popular and was widely used at google, and yet google rejected him for a job because he couldn't invert a binary tree or something?
Don't know all the details so I could be missing something crucial, but if not it'd seem that reputation isn't enough
I don’t necessarily think leetcode should be the only litmus test for a candidate but Max is an outlier. Not every candidate getting rejected by a leetcode question is also capable of building out homebrew on their free time.
They 100% can get a job and a decent one. I know because I fall into this category BUT the FANG salary are 2 to 2.5 times what I make at this point which pushes me down the study leetcode part.
I had a great track record that was out in the open, actually displaying vast knowledge of algorithms and data structures and exactly the stuff that's being asked in those interviews. FAANG interviewers did not care one bit about it. They actually consider it bias to look at a person's prior work.
>This person should already have enough of a reputation to get a job at many companies, if their work is public enough.
You can't just hire someone based on their reputation at a company of any maturity. That's a legal and HR nightmare. There has to be a process with a semblance of objectivity, and that process has to demonstrably apply to everyone equally, always.
Hiring based on reputation is the same as hiring based on resume. And it's extremely common in almost any company. Why would it be a legal or HR nightmare?
Which in practice means you put out a fake job posting where the qualifications uncannily mirror this persons resume to a tee, and you hand them the job formally after a week.
I've been in situations where I was already somewhat working and onboarding while the faux job ad was up for the two week or however long mandatory posting period. tallied the hours separately and got paid back after i was hired.
I can't at all imagine why this would be the case. Why would a company have any kind of liability for hiring biases such as reputation (except for systematically refusing candidates from protected groups, of course)?
FOMO is an atrocious hiring practice. Chances of your scenario, where you somehow find a person that can create something novel, that they will stick around to actually produce novel thing and then this novel thing “making it” are nonexistent.
yeah, but in real world when this kind of problem will need to be solved people will most probably sort O(NlogN), or use priority queue O(NlogK), or even will go with something like O(N*K), almost no one will go with O(N) algo and because usually N and K are rather small and this code will not be called too often time complexity may be ignored. Still any solution shorter than O(N) will be called inefficient. And in real world they will know N and K from this what kind of problem they are solving, and this will not be hidden in mist of abstraction with assumption that "candidate should ask".
Moreover, with the micro-optimized SIMD quicksort algos that are perennially cropping up on this website... I would be willing to bet that "sort and take first N" is objectively faster than my crappy Python implementation -- even if it is linear time.
I mean, in the real world you probably use a library method. If I were an interviewer I would not be expecting the candidate know about median-of-medians (O(n) worst case). I wouldn't even expect they know a-priori about quickselect (O(n) avg). But I don't think it's unreasonable that given a few hints, a candidate could understand and implement quickselect in 30 mins. Most people know about quicksort already, and quickselect is not very different. You can even give them the partition and select_pivot function at the start and then if there's time have them fill those in. In the rare situation they haven't even heard of quicksort, you can even write the shell of the algorithm for them, and have them adapt it to quickselect.
Even then, all thats probably a bonus - a priority queue implementation, or many other possible solutions are probably good enough for me.
Do you work for Google? I'm just doubting you a little. Like the guy from project zero is finding bugs in Windows via ssh to a Linux machine? Definitely going with Doubt here.
The first program you build in Noogler training takes more compute and io to build than the Linux kernel. The distributed build systems laughs at such a trivial program and barely breaks a sweat at programs 10x that size.
Google has a giant monorepo. It is too big for git. (Virtually) everything is built from source. Building a binary that just runs InitGoogle() is going to crush a laptop.
I believe that there are also a bunch of IP reasons for this policy, but from a practical perspective doing everything with citc and blaze is really the only option.
There are a few outliers, but yeah. Per policy, no code is allowed on laptops. And because Google's build tooling is very centralized basically everybody works on the same kind of machine.
The folks developing Chrome for Windows or iOS apps might have different workflows, but even then they aren't going to be using brew because of Google's third party code policies.
Why would I install homebrew on a work laptop if I cannot build or run code on my work laptop? Why would I install a dependency management system if company policy is that all third party code is checked into the repo as source and built using blaze?
When I worked for google, several of my projects were developed locally on laptops. My intern (who was developing tensorflow robotics computer vision stuff) used homebrew to install tools. Not everybody at Google used blaze.
Because you can also install applications via homebrew. I use it for a few things, including installing bat, delta and other Rust coreutil replacements that I prefer. I absolutely do not use homebrew for project dependencies.
Google lives and does by its distributed build system. Most devs don’t even have exactly have code on their local workstation either—it comes via remotely mounting a file system.
But 99.9% of the builds happen remotely. So local vs remote code just isn’t that relevant.
It is also true that Google spends millions and millions on its dev environment every year, so this isn’t your average “no code on laptops” situation.
I’m sure there are a few open source developers who use brew or Xcode directly, but most Mac builds happen in a distributed build system wired up to use remote macs. Yes, via Xcode. Not on local machines.
The mac build system is wild. It is still all done remotely with a farm of macs running xcode. You still don't build code for running on macs on your local machine.
You actually do, what Tulsi generates is an XCode project that has shell script build steps that call out to bazel. Bazel underneath the covers will end up calling the clang that comes with XCode.
You are factually wrong. Not only that, you're conflating several different processes and team rules.
Many googlers use brew to install applications on their laptops. This not against policy. Other googlers work with code stored directly on their laptop. There may even be developers who are obtaining deps (for their own builds) from brew.
The problem with the brew author is that he had every opportunity to make himself look hirable at Google but instead chose to write an incorrect screed and publish it on the internet.
You need a full Santa exemption with business reason to use brew. The average person working on some server that deploys to borg does indeed use their Macbook as a thin client. Who's obtaining deps for their builds from brew? If you're building stuff on Mac, it's via bazel and all your deps are in source control.
I never said anybody is obtaining deps for builds- that's all UncleMeat.
At the time I used brew (5 years ago) it didn't require a santa exception with business justfication (and my justification would have been "I need this for my work"). Fortunately this wasn't really a problem for me any way as I don't even look at Mac machines as anything other than a thin client.
It is true that this was the problem with the brew author. Homebrew being part of the typical Google workflow is entirely independent of the situation.
But, as usual for internet discussions, it is fun to rathole on side conversations.
The best part is, I was defending the use of homebrew (which I absolutely hate) and local development (which is far inferior, IMHO, to blaze/forge/citc/piper). I had really hoped releasing abseil/bazel would help but sadly, it was done too little, too late.
You shouldn't use Brew to obtain dependencies. This is how you end up with people complaining about a brew upgrade replacing the version of Postgres their project depends on.
You probably shouldn't be using dpkg or rpm for that, either, unless your CI and deployment targets are running the exact same version of Linux that you are, and even then—there are usually cleaner and more cross-platform/distro ways to do it, especially if you need to easily be able to build or run older versions of your own software (say, for debugging, for git-bisecting, whatever). I continue to wonder how TF people have been using typical Linux package managers, that they end up footgunning themselves with brew. "Incorrectly", I suspect is the answer, more often than not.
Where it excels is installing the tools that you use, that aren't dependencies of projects, but things you use to do your work.
Get your hammer from Brew. Get your lumber from... uh, the proverbial lumber yard, I suppose. Docker, environment-isolated language-specific package managers, vendored-in libs, that kind of thing.
I don't install project deps with Brew (it's a bad idea, but, again, so is doing that with dpkg or rpm or whatever directly on your local OS, a lot of the time) but I do install: wget, emacs, vscode, any non-Safari browsers I want, various xvm-type programs (nvm, pyenv, that stuff), spectacle, macdown, Slack, irssi, and so on.
That's fine, but almost nobody is running tools on their MBP. So for this sort of thing you'd be using the package manager distributed with glinux. And Google is also a really weird island where tons of tools are custom. You cant use some open source tool for git bisecting because Google doesn't use git. You cant use some open source tool for debugging because borg is a weird custom mess and attaching debuggers requires specialized support.
Google uses git. I used to sit next to Junio Hamano, the primary developer of git, and lots of teams that used my team's services were using git. Lots and lots of teams. There was even an extension to use git with google3, which was really nice, but was replaced with a system that used hg instead.
I was very imprecise. Git is used both for OSS stuff as well as some other stuff. But the norm is development in google3 and even if you've got a layer of git commands on top of that, the actual source and change management is being done by citc/piper.
True, although I predated citc and piper and we definitely built apps locally on machines with source code (from perforce). I was strongly advocating that more people switch to abseil and build their code like open source in the cloud (the vast majority of compiled code doesn't have interesting secrets that could be used to game ranking, or make money from ads).
what in the world does that have to do with whether he was qualified to work for google? the average developer at google is definitely a much much less proficient programmer than the creator of homebrew.
I don't do leetcodes because I am not really into programming puzzles (or parlor tricks :D ), but I roughly tried to do what you described. ~45 minutes it took, deciding in the middle to not worry about the algorithm. Guess I won't get the job ... shrugs.
You won't get the job not because you "didn't worry about the algorithm" but because you didn't ask any questions about the problem; just went straightforward to the implementation. In FAANG interviews that would be a red flag.
Yes, because getting to the right answer is not the point of the interview. Apart from anything else, getting to the right answer may mean you memorised it and are incapable of doing anything else. Always show your working and thought process. Asking questions and showing that you understand tradeoffs and that users have different requirements is a good way to do that.
Well, good to know if that were ever an opportunity for me. Probably would have to invent questions because it's hard to ask yourself out of being dumb-founded on the spot for the moment.
The interview for my current job had something simpler: something like finding random permutations, then what's the algorithmic complexity of this random algorithm. (It was years ago, I forget the details of it.) I just talked through the solution. That was nicer that having to come up with questions. :)
In the FAANG interviews I've done you're never allowed to ask questions...? Maybe for clarification of the problem space, but not about the algorithm or the promise of a particular solution.
Facebook interviews were very much interactive with my interviewer probing me on O(n) type questions and me refining it down to be more efficient. I was certainly allowed to answer questions, typically about scale. My code which was on a whiteboard certainly wouldn't compile and a good amount of the discussion was making sure that the interviewer could follow it and that he was satisfied with each of the steps.
My first round I passed with a less than optimally efficient solution, but he was satisfied every step of the way during my work.
While I was lukewarm to the prospect of working for Facebook, the interview process was very positive and reflected very well.
On a personal level, self interest would have me like the leetcode style problems because I can get most of them right on the first try during a timed interview, without studying. If I were pursuing a job at a FAANG, I might actually study them and I'm sure it would go well for the testing portion of the interview.
However, when I interview this is not what I'm looking for. I'm typically looking for someone who knows the particular language that I'm hiring for. My questions run from the very simple to as deep as they can go on either language or implementation details. From the most junior to the most senior, they get the same starting questions and I expect the senior people to go deeper and explain why they choose something over something else. I'm also testing their ability to explain it to me (not just get it right) as that is part of their job working with juniors.
I really don't even care if they have the names of things right and don't really count things wrong against them if they get the names of two things backwards for instance. For example in Go, a huge percent of the time you might use slice over arrays. Some people get the names backwards, but can identify which one they actually use and they know that one can change size. They are correct in usage and misnaming them. I inform them of the name, encourage them a bit and move on.
I've never liked the "look at this code, what's wrong with it" approach. There are too many contexts that I have to jump into at the same time. There is often an expectation that I find a specific problem with it. I'm lacking the usual tools like an IDE or compiler. What level am I looking at in the code? Does it compile? Are there off by one errors? Cache invalidation? Spelling errors? Logic errors? Business errors?
This guy has missing tests on code that needs to be refactored in order to make those tests. Maybe he has it figured out just right, but the "jump into my code" interviews I've been in on all seemed like they had secret gotchas that the interviewer expected specific answers about.
In short, I haven't seen a proper, repeatable process for interviewing for software development.
If I'm the interviewer and you don't ask questions I'm going to rate you very low on your communications skills. You may have the best algorithm in your head and write the most elegant code, but working at a company requires you to communicate your ideas and plans and code and everything else to your team. And no, communicating only at the end (code review time) is not enough. This is not a school assignment that you silently write and then turn in for a grade.
The best interviews I've experienced, both as an interviewer and an interviewee, are the ones that feel like two team members collaborating to narrow down requirements and solve a problem.
> about the algorithm or the promise of a particular solution
It's not about "the" algorithm or "a" solution. It's about you the candidate being able to propose multiple solutions, perhaps with space-time tradeoffs, to provide a recommendation based on your judgement, and to ask the interviewer what they think of your proposal.
I mean, that's great, and I feel the same way. Yet every time—since the introduction of leetcode questions, anyway—as the interviewee I've been asked not to ask questions about the algorithm or the solution, just clarification of the problem space. FWIW I have been employed at several of the FAANGs or whatever they're called now and have also been on the hiring side, where I certainly did not discourage asking questions of any sort.
This is a myth. It doesn't matter if you ask "questions" but implement an n^2 solution. Unless you implement the optimal solution, usually using a top-down DP or some array trick, you aren't moving forward in the interview process.
I've done that in interviews: "oh hey, the most efficient known algorithm is this classic named X". That tends not to win interviews either, but in the real world knowing the name of the best algorithm (or even just that a best algorithm exists and a general idea of what you'd google to find it) is more useful than knowing the details of it. If I need to reimplement a well known algorithm I can often reimplement it from the Wikipedia description, that's trivial and boring make-work (and will always be trivial and boring make-work). But I need to know which well known algorithm sometimes and that's a far more useful practical skill.
> given an array of strings and a fixed width, can you format the text such that each line has exactly maxWidth characters and is fully justified
So many questions. What character encoding is the string? What is human language? Should I honor non-breaking spaces and other similar codepoints? Is the string full of 'simple' characters or 'complex' characters? Graphme's? Emoji's? What's the min and max limits on width? How long (or short) can each string me, and how large could the array be? On what system am I running, and does the array fit in memory or is it paged off a disk? Does the font we're using support all of the graphme's present in the string?
Amusingly, I had a variation of this problem as part of an Amazon L8 IC role interview, framed as a Prefix Tree. Solved the problem, didn't get the role. :(
Sadly enough, I'm literally scoping out a new feature at the startup I work at that involves rationally splitting text into lines with a max length and buffers with a max line count so we can interface with a legacy system from the late-80s/early-90s
The best interview we've used was sharing a simple but very not idiomatic Python file with working (but slow) code and tests at the bottom. The task was to refactor and speed it up. This allows seeing the actual thought process and some basic skills, while at the same time being something any good dev could do in 20min without much pressure.
I tend toward this, especially with some canned performance report. Sometimes I take something relevant from their GitHub and introduce a glaring bug, like an impossible dependency or infinite loop.
We keep seeing posts like these on HN/Reddit, yet companies keep using them. And companies like Faang (or is it Manga now??), whose engineers I would think would hang out on HN
So it either means:
1. HN's influence is even less than we thought-- even MANGA engineers /10x silicon valley types dont hang out here
2. Everyone agrees its a good idea, but no one cares. Like everyone agrees we should care about the environment etc
MANGA engineers cant change shit. They are drones (not a criticism, I am saying like worker bees in a big colony). Startup Engineers can (but just for their startup)
Also if you have been hazed you wont vote to stop the hazing
> We keep seeing posts like these on HN/Reddit, yet companies keep using them.
Just because you see them, doesn’t mean they’re correct. They’re just a breeding ground for holy wars and discussions. On every post like that you can find tons of “I don’t like them, we need something better and no I don’t know what”. But if you dig into comments you usually find why those type of interviews are done.
You are learning that perceived consensus on an upvote/downvote site doesn’t mean much because if 60% of people agree with something because of upvotes it seems like 99% do. Reddit rediscovers this every election.
People often downvote thoughtful discussions they disagree with. It’s tiresome to see your text fade with downvotes so often people don’t bother.
If you hang out on Hacker News you might also think every engineer thinks crypto is a scam and no engineers want to return to the office. Neither situation is the case but if you disagree with the majority on those topics you will get downvoted to oblivion, your comments won’t be visible after a point, so why bother?
I personally don’t care about the endless leetcode debate but it’s a bummer there’s so much exciting technological advances in crypto that can’t be discussed on this site without an army of “crypto=scam” bros emerging from the woodworks.
I can relate so hard with the part which says it does not favour slow thinkers. I've faced this countless times - person at work asks me something, I reply "I'll think about it and get back to you", which inevitably leads to their disappointment.
I am okay with disappointing people, but it can be unnerving when that disappointment means I miss out on a good opportunity.
Having taken some interviews myself I can see some value in having the interviewee do some small coding exercise when they do not have any personal projects to show. I think it can filter out some false positives of candidates that can talk well but can’t actually write code (you’d be surprised how many candidates can’t)
What surprises me most is the lack of flexibility in the process. If a candidate shows up with a broad portfolio I’d rather talk about that then doing some random coding problem. Yet our HR manager insists on the fixed program. This is worse when the candidate is interviewing for a senior role where I don’t really care. Then I am mostly interested in their past experiences and knowledge on how to build things that don’t fall apart after six months.
Again it is definitely process over people here… not sure if it is better in other places.
I would also say that these exercises are most effective when they are quite simple. They let you test ‘can this person write a function’. The complicated ones often filter more for people who have studied those type of problems. Harder problems != better coder. At least not for the projects I work on which are more integrating existing services than investing new novel highly efficient code.
Yeah; that's what I've preferred to do. Interestingly, I've had just as good results with the hires I've made regardless of the process used, from whiteboard coding that is little more than fizzbuzz, to actual pairing with an engineer on a real problem, to leetcode, across multiple different companies. Maybe the softer parts of the the interview, which was largely the same (and prioritized people who could talk intelligently about stuff on their resume, those who when led to places they had no experience were comfortable saying "I don't know" or similar, and those who seemed interested in learning new things), was better for distinguishing mediocre from excellent (with the coding being helpful for distinguishing imposters from everyone else).
People who call for Leet Code either never tried to hire actually high skill developers or fools themselves into thinking that they're the cool kid in town.
You might think it's better than any other alternative. I agree - it's been an amazing filter to filter out companies who are on average, quite crap to work for.
From my personal experience, they are overburdened with process to a degree that even if they hired the best devs out there, they wouldn't be able to deliver anything because of all the red tape.
The best jobs I had to date, I met the person leading the company/project/team, we had a chat, talked what tech we like, dislike, how we'd structure a product, what are the preferences to the process around everything. And that's the key thing - it was always a discussion, no Q&A. The key is that the candidate is not the only one who needs to know his stuff - so does the lead.
As a side effect, all of those jobs were way above the market. Again, personal experience, but higher up you go - less BS like "we need leetcode to hire" you get. Unless you're Facebook and you have a genuine problem of too many qualified engineers constantly applying, you should aim to only disqualify truly hopeless cases.
The company can't hide behind process and expect great hires. Early in my career, in a small city I was working in (in return, in the dev community you know about what other devs are doing), our company denied so many devs that within a year or two were among the top performers, just because of the leaderships insistence of a take home tests, Q&A interviews and gotcha style questions...
So please - do continue using leetcode, it makes filtering your company out so much easier and I don't need to go through bullshit stages to know that the leadership has no balls to make the hard calls when it comes to hiring & firing.
674 comments
[ 3.3 ms ] story [ 379 ms ] threadthis provides a few things; it gives us an ability at how the interviewee problem solves. next we can see how they respond too obvious fixes (would they be someone you'd want to send a PR too?). finally, it tests their knowledge of the language and APIs, hopefully much better than Leet Code can. I would also like to see if the user can spot obvious bugs in the setup code (say, package.json, pyproject.toml, etc)
I am going to make an example PR for: - frontend (React/NextJS, TypeScript, CSS) - backend (Django, Python) - DevOps (potentially some Pulumi code for deploying to a Kubernetes cluster?)
What I’ve seen are time trials, which don’t show you how well a candidate will adapt to a codebase, wont show you how they will do tickets for your sprint. Only reinforces a flawed idea of the employer about how they wont “hit the ground running” despite having that job opening for 8 months.
What you described might not be a time trial, its what ive seen though.
This is great suggestion. While the “look at their github” one is a bad suggestion. Github polishing is theatre more suited for theatre majors instead of people actually working with integrity before coming to your company. Its very similar to the issue with the leetcode interviews as its geared towards people with time to optimize that instead of a day to day job.
People good at their jobs because they do their job: nothing on github
People in the business of performance theatre because they dont have a job to be good at: plenty on github. It can be legit code they wrote, that was not the point at all.
I don't like the idea that every developer has to have code on github. If they want to code for work and nothing more, then that's fine and shouldn't disqualify them from any jobs. We shouldn't expect people to spend years of employment building up a portfolio for the next time they're looking for a job.
However, I don't see how looking at the code people write isn't informative. You can see many things, from small-scale code style decisions to how they structure an application just from looking through their github profile.
Writing code isn't a performance, it's literally what you want to pay them for.
Github presence is just a non signal. Thats the only point.
We agree that if you have a different way to see how theyll actually structure code for you, then its useful
But hey, maybe HR will be happy I have a GitHub account.
Even better if they have both, worked on open-source projects or have created useful open-source software used by other companies and are already working in their other job(s) or have personal real-world projects they can point to; which those are clear advantages and a simple quick filter to use.
No need to ask about frivolous leetcode questions around re-implementing sorting algorithms or wasting more time asking the candidate to write proofs for those algorithms where realistically you're going to just import it from a library or look up the solution on StackOverflow.
Unless you're Google, a FAAMNG company, university or general research related position or if the position isn't for a typical CRUD application development, then there is little to no justification for wasting everyone's time on pointless leet-code puzzles and this applies to the majority of companies.
I have 30+ years software dev and almost nothing on github. There are valid reasons why a person who writes lots of code, including on weekends/spare time, would not be on github.
For my personal projects I prefer bitbucket. For professional work, it is proprietary and thus cannot be shared (esp not in an interview!!).
Leet Code is only cargo culted because many candidates let it happen.
they rather keep a set of leetcode, iq and personality test to remove you from the list of candidates
root cause is bootcamps over saturation of devs in job market, especially in web
I have been on the receiving end of applying to an agency and being told I didn't make the grade technically. I was disappointed because I know I am a good engineer but I didn't expect them to magically know this. I can also see how my approach to the technical tests might have made me look less than what they were looking for, which is fine.
I am also not sure of any good alternatives because someone will always object to any alternative which they cannot achieve for some reason. A "take home" project is good for real life work but some people cannot (or will not) invest the time even if they are paid for it; discussions can be great for helping nervous people but that is not how work usually is, there are challenges, pressures etc. and the able people object that it is not fair that people are getting in too easily.
When I was on the market, a couple of companies actually gave me a variety of options, which I appreciated. I don't want to spend hours on a take-home and I also don't want to do leetcode, but an open question/answer plus some code review and live debugging was an acceptable combination for me.
Providing the options definitely has the potential to take up developer time, but I think the tradeoffs are worthwhile for both the interviewee and the hiring team.
The exception is my current position (only started last week): "oh yeah, we looked at your GitHub already and it's actually similar to the take-home anyway, so little point in that". Instead, they prepared an "alternative" interview where they posed some scenarios with "what would you do? How would you handle this?", which was intended to test both some technical skills, but also social/attitude things. I wrote down some answers, which took me about 30 minutes, and then we discussed them, taking a further 30-45 minutes.
The questions were a bit clunky because they were looking for someone ASAP and there was only 2 days between the first and second interview, but I felt that was a much better approach for the company as well, because they got a lot more information this way: they could already verify basic coding skills themselves, and this way they got a new chunk of information they wouldn't have had otherwise.
You named the exact problem with the industry.
No one is going to complain if FAANG does this with top tier, life changing salaries. No one will object to learning and memorizing DS&A at a job where DS&A is used heavily.
What people are upset about is your average no-name company hiring individuals based on things they won't use during the job, of which the knowledge is still incredibly varied[0], and they still complain about not being able to find anyone and play the "woe is me" card.
[0]: Dare I say it, DS&A is such a big topic not everyone learns the same things. Thinking in terms of a tree is different from thinking in terms of a linked list, graph, tree, heap, stack, queue, you name it. Most LC medium/hard require time you won't get, information you might not have. This knowledge isn't as universal among skilled graduates as people like to believe. And we all know the moment candidates are able to just memorize questions about linked lists, hiring will jump to the next topic.
Thoughtfully creating an interview process and the procedures/questions is a skill that has to be learned... and when people haven't or don't have the opportunity to learn that skill, it makes sense to turn to resources like books or articles, the most popular of which are often based on FAANG practices.
I also know from my own experience structuring and performing interviews that there can also be a lot of pressure from the top to eliminate candidates following the belief that "the last one standing" is the best, rather than actually trying to evaluate the strengths and weaknesses of every candidate.
That way tends to lead toward interviews of 5-7 rounds that act as sieves.
This is all just my limited personal experience though and I'd love to hear from other people who've done interviewing at/for smaller companies!
That sounds excellent!
This would allow the company to determine what the applicant believes is their strongest suit.
Then, when it comes time to sit down as a team, and review all the "top shelf" applicants, people can decide, based on a number of criteria.
If an applicant avoided technical challenges, but spoke well, then some managers might like that, but others, might not be comfortable. Maybe they might devise a technical challenge, customized to the applicant, and using real-world problems.
The main deal, is that the vetting has been done. The chaff has been filtered out, so more attention can be paid to the wheat.
At my old corporation, headcount was something I had to fight like crazy to get (I was a manager). We seldom hired, so it was well worth it to spend a great deal of time on each candidate, as they would be responsible for important work, and would have a great effect on the corporate bottom line.
That sounds like a lot of small startups, to me. BigCorp (MAANG, et al) hires thousands of engineers per year. They need to have a cookie cutter system. Smaller companies do it, simply because they want to be like the Big Boys.
Also, and this is neither here, nor there, but binary tree tests are a great "young-pass filter."
What's happening is companies are putting the burden of the risk on candidates more and more. Because they can. If candidates would put their foot down and stop accepting this, most of these shenanigans would stop. The junior market shows what happens when people are desperate for jobs and willing to bend over for any whim corporate has that might help their hiring process (even when most of it is completely unproven).
This is probably fine in an environment where the candidate has lots of other options, but think about this scenario: a person applying to multiple companies, possibly rejecting some offers, possibly relocating or otherwise changing their life, accept that one offer, and then be let go in their first weeks. Changing jobs can be emotionally difficult and being let go even more. If you have to restart your job search, because you were let go during the probation period, all the other roles you have applied to might have been filled. On top of this, contrary to when you were looking for a job last time, now you're actually unemployed and potentially under pressure to find something new.
So, in essence I don't think this is good for candidates.
But as things stand, nothing is preventing companies from doing the above anyway. If they think you're a bad fit, they will use the probationary period to cut ties with you. This is perfectly viable today. Your example assumes current filtering methods do in fact increase the ratio of true : false positives, and taking some of them out would decrease that ratio. This is not something that has been proven, and I'd even argue it's something that can't be reasonably proven within the next few years. This is even worse when considering a few interview rounds can only filter for the most obvious dummies, but can't decisively tell you the performance of that individual a few weeks down the line.
What your example does show is how much power employers have over employees. It just isn't healthy for individuals to have to carry this amount of risk while corporates continue to reap the benefits.
The article addresses this as well. Stage fright does not make someone “dumb”. Slow thinkers aren’t “low IQ”. And reversing a tree likely doesn’t make someone apt for a job.
I’ve seen so many people over the years do terrible on these types of test that went on to be amazing contributors. We only found this out because we essentially ignored the outcome of these tests, which makes you wonder why you’re doing these tests at all!
That doesn't mean that someone would be unable to program it. Inverting a binary tree tests if you know how to traverse a tree. I may not traverse trees all day at work, but I can easily do a tree traversal if I needed to and I expect that to be true of most people who can actually program and not just talk the talk.
If you really really care about me traversing a graph, leave me alone with the task for a while, let me take my time, give me access to the internet even. Why do you care that I can write it on the spot? If anything that just proves that I memorised it just before the interview, not that I actually had to think much about it
- interviews are inherently more stressful than even a very busy day at work, - they cover topics unrelated to what you'd do as part of your job so if you're good at the job you might still fail the interview - people with time (==money) to prepare for interviews will come out ahead of those who don't have that luxury, despite the fact they may be far better prepared for the actual job.
So you get a bunch of false positives, a bunch of false negatives, and on top of that you're discriminating against people who are in a worse financial situation or have interview anxiety. I can hardly think of a worse outcome for a seemingly sensible hiring method.
There are a couple of things at play here:
1. I'm regularly complimented on how sharp my mind is, but I can't reliably recruit that sharpness on-demand. If I get any kind of brain fog during a timed problem (with no opportunity for a break), it's game over.
2. Coding in an unfamiliar environment, like Coderpad.
The problems are hard enough to make them easy to fail if you can't spot a good approach immediately, but easy enough that a "good" answer gives very little signal relative to the time being dedicated to the problem. I've done interview processes where 50% of the total process is given over to these kinds of problems, and the process concluded with me feeling I didn't get any real opportunity to demonstrate my strengths.
P. S. I hate coding interviews.
But, yes, this is more work on the recruiting side.
As a side note, I totally get it why large organizations would use Leetcode though. It does help filter out false positives if you're willing and able to bear the cost of passing on many good candidates.
"What is a binary tree" will filter out most of the bullshitters.
None of this stuff is rocket science. We have a responsibility not to be lazy when interviewing people.
Not really silver-tongued candidates? Well, what's the problem? Unless you are looking explicitly for silver-tongued candidates, you should ignore this "trait".
I think an extra hour to talk will give me more information than watching them balance a binary tree.
For many people the stress of the leetcode session is just awful, I don’t think it’s a nice thing to inflict on someone if you’re ultimately going to disregard the outcome anyway!
I don't want to know if you can do specific things, if you can't you'll learn, but I do want to know if I'm gonna find it easy to get along with you, if you seem like an interesting person, and if I can stand being in a small room with you for 7 hours a day.
I suspect this is just another kind of bias, but I've had good results.
Ultimately the proof is in the pudding, I've had loads of candidates that could barely code and Leetcode-style problems are a great filter against that.
Otherwise you risk just getting PM-style bullshitters as engineers, who talk persuasively about projects that other people actually implemented.
The more I learn about software development the more I try to _not_ have leet code style parts in my code - in the very rare circumstances would I see something like that and say - yes there is _no_ other library out there that has a battle tested algo to solve this particular problem, _and I have to code my own_.
Worse people who excel at Leet Code could start to bring that sort of thing into your codebases, optimising small parts of your code to perfection, and then leaving the whole thing a mess or just plain reject to work on solving business issues and concentrate on polishing their small bit of algorithm.
Of corse there are positions where that sort of mindset is welcome and sought after. Especially in very big companies, but for the most developers out there leet code is _just_ for the interview part and they would (should) never use that kind of problem solving in their day-to-day work.
When I have to interview somebody, and there's leet code involved:
a) they are _really_ comfortable with those types of challenges - that tells me they have spent a lot of time preparing - either for this interview or just generally, either way that wouldn't _really_ tell me much about their problem solving skills
b) they feel uncomfortable and fail to solve anything due to stress or anxiety - again no really knowledge gained, as real-world work environments tend to try to minimise those, at least in the places where I work in.
c) they feel uncomfortable but get the hang of it - now I _might_ have a glimpse of their problem solving skills, but I've put a human being in a very uncomfortable position just to _test_ them. There _has_ to be a better way.
The way I like to conduct interviews where possible is to ask people to walk you through some of the code they've written and ask various questions about decisions - much more relaxed and I still get the sense of how they organise their stuff and how they work.
If that's not possible - a take home task or something. Or even "hire fast fire fast" approach works too.
The problem is that a take-home task can be impossible for some people already in a full-time job with kids, etc. and so has its own bias. And the "hire fast fire fast" approach doesn't really exist here in Scandinavia, even though there is a probation period it's still frowned upon to use it for anything but extreme circumstances, and if done en masse would likely raise issues with the trade unions.
This sounds incredibly specific. Most places requiring at most LC easy don't really need any of this, and I'd be worried about the interviewer looking for key words and specific answers over actually testing whether the candidate grasps things or can learn what is necessary.
That way, some of the pressure is off of coding with someone looking over your shoulder, and you still have time allocated off already.
> ...
> Or even "hire fast fire fast" approach works too.
Comment "c" shows a high level of care/concern for the candidate. "hire fast fire fast" does not.
What if the candidate left an unsatisfying but steady job to join your company? And they get fired in the first few weeks. Now they are unemployed. This is much more uncomfortable than an awkward interview question.
The coding round is supposed to test that.. you can code.
For the record I’m also not a fan of the Leet Code style rounds, even though I actually find completing them (outside of an interview) quite fun.
The point of leetcode is not to show what you can do, it is to show what you can't do.
To keep up with your analogy, being able to design a simple website doesn't make you able to set-up an enterprise-level infrastructure, but if someone isn't able to design a simple website, I don't want to hire him for my enterprise-level infrastructure.
As for "outside the box" thinking, it only works if you know where the box is, otherwise you are just being clueless.
And by the way, because it is an argument I see way too often, calling the "sort" function of your favorite library instead of doing the exercise "as intended" is not "outside the box" thinking, it is literally the most obvious thing to do. What could count as "outside the box" would realizing that you can solve the problem more efficiently by not sorting anything at all, which requires you to know your sorting algorithms to show that it is indeed more efficient.
They aren't suggesting to not do programming tests, they're saying make the tests more aligned to the role and domain being hired for.
I use a pretty similar approach and find it works very well.
What does the code do? Does it work? How would you get it working? How do you test it? What is the complexity? etc.
I'm a great engineer, I just suck at leetcode questions because...shocker! They have absolutely nothing to do with my day-to-day. It's similar to puzzles; I hate puzzles and doing puzzles/escape rooms/etc, but I'm a pretty smart guy.
Leetcode is merely a gatekeeping methodology that proves almost nothing and weeds out a lot of amazing engineers who aren't skilled at grinding leetcode or hate brain teaser shit.
I agree. A couple of years ago I was asked by two companies to solve leet code problems even before there was a screener meet-and-greet interview. They were quickly crossed off my list.
Though, I don't agree leetcode is a good interview practice.
You're presented with a laptop connected to a remote VM, and told "Fix MySQL", or "Rewrite the git history in this repository".
Usually these are simple problems, which have obvious solutions. Every now and again you might get a surprise like an immutable-bit set on a file, or SELinux blocking access to specific files/paths, but the good thing about these kind of "challenges" is that you'll usually also have full google access.
I guess l33tcoding isn't really a thing for sysadmins, but I do appreciate a (fair) simple test like that, especially being given the opportunity to talk through the process.
Followed by "this code needs refactoring, go nuts, and then let's talk about what you did and why afterwards".
But i have also heard people say "Oh my god! They asked me to do a debugging exercise - ON PAPER!!!? What were they thinking?!".
in my opinion
I agree it's maybe a little less respectful of a candidate's time, but as long as it's not too big a task I don't think it's unreasonable either.
When I was a junior developer I used to deal with a). Now that I have more years of experience I usually deal with b).
It invites a conversation, in depth, about software construction, quality, and decision making from the point of view of real work. It also tells me if a company wants to simply filter. If the organization is unwilling to invest in a candidate interview, as I do to be interviewed - I in turn learn a lot, and decline to pursue accordingly.
In fact, I have so much stuff out there, in the public realm (I’ve been doing open-source software for decades), that employers could easily evaluate me technically, without ever contacting me, and all they’d need to find out, is whether or not I’d be a decent team fit.
I suspect that the main reason they ignore my portfolio, is that they have already decided that they aren’t going to hire me, and don’t want to waste their time, reviewing my work.
It also helps me filter out companies that are either too lazy, or too strict in their procedures, or both, i.e. there is no value in asking the same questions, regardless of candidate’s resume.
As an example, a month or so ago I went through a couple rounds of interviews with this company. I told them forthright that I wasn’t doing any code assessments, and if they wanted to, they could take a look at my GitHub account. They seem willing to consider. Third round comes in and they ask me to write some sorting algorithm, to which I refuse.
Ironically enough, one of my repos does have an implementation of an advanced data structure, so they could as well just take a look at it.
In the defence of LeetCode-style questions, I do think they work, and very well may I add - with the caveat you have the throughput of candidate to make it work well? Their ability to filter out 'those who can't code' in an efficient manor while sacrificing a small amount where it filters out 'those who can code' greatly out weighs the alternatives. The alternatives needing to fit into a 1 hour timebox, be objective while also favouring the positive cases (I think I got that the right way round).
My two cents would be more around the way in which they are conducted; in my experience I've found conflict with the interviewer more then the process itself - with interviewers in my past lacking.... empathy (may not be the right word) for the person on the other end of the screen/table feeling flustered, nervous or down right stupid that they're struggling to solve a simple fizz-buzz/reverse string problem, leads to a snowball effect and pilling onto that can effect the candidate in quite a spectacular way. Best interviewer I've had asked if I was alright and got me a glass of water, props to that guy!
I dunno - I've just come to terms with having to learn how to play the game, even if I find that part of the game really hard and to some parts unfair. Such is life
I've failed more than my fair share of these challenges, but never (being subjective here) because I wasn't actually capable of (1) solving the problem or (2) doing the job. My take here is that I _may_ have been unqualified for these roles, but that the interview failed to actually uncover it, due to spending all the available time on low signal exercises.
Statistically that’s irrelevant, as they optimize for “do not let through rotten apple”, rather than “find good apple”.
We just fired a person on my team who didn't understand pass by value vs pass by reference, or how to debug in an ide, but she could manipulate strings in leetcode!
The problem comes when smaller companies that don't have the same high rate of new applicants use the same process and then complain they can't find anyone.
This is the assumption I was referring to, that the "sacrifice" is small. It's suggesting that the false-negative rate for LeetCode challenges is small, and I'd argue it's actually quite high -- as you also suggest (your rate is 90%).
The problem is that the total number of candidates who apply for the position, Z, is significantly higher than X + Y by a very very large margin, and I mean orders and orders of magnitude. For every position I post I get on the order of 600-1000 applicants in a matter of a week, even though I'm only looking to hire maybe 2-3 people. Of those 1000 applicants, 80% of them are simply unqualified, and that's being really really generous just for the sake of argument (I'd wager the figure is closer to 90-95%). So once again just for the sake of argument that means 200 of them are qualified, and I'm hiring three, which means any process I choose whatsoever will filter out a minimum of 197 out of the 200 qualified people, no matter what I do.
Given that calculus, it's better for me to focus on making sure that I filter out the 800 people who are simply unqualified for the position even if that means I end up filtering some of the 200 good developers, because I have no choice but to filter out at least 197 of the good developers anyways no matter what, whereas I do have a choice about filtering out the 800 bad ones.
But did you consider the number of qualified candidates that did not even apply because they know there will be leetcode questions?
The code part was a small existing codebase simulating the system in said design document. You you are given three tasks, and explicitly told you are not expected to complete all of them. I ended up using virtually all of the time (70 minutes) completing the first two tasks, and using my remaining minutes writing comments about how I'd complete the third task. When that was complete, I was given 15 minutes or so to describe what I would do if I was given another 15 minutes of time to work on the project.
My only real complains were that the time limit added some pressure (that I was able to manage reasonably) and that the grading process is opaque. I know a human grades it according to a rubric, but I don't see any of my results. The company I was interviewing for just said "Everything looks great, we're moving you forward to the next stage of the interview process".
The code didn't involve writing any fancy algorithms, but instead getting to know a (very small) existing codebase and understanding how to use it to add functionality. This is much more realistic a gauge of how good of an employee you are than how well you can implement a search algorithm from memory.
Many candidates (like: maybe the half) are fancy talkers without any skill in writing code. I really don't know why they are applying for dev jobs. It is easy to filter out these persons with a very simple coding test.
I agree with the article that 'leetcode' tests (find that complicated algorithm in 30 minutes while I am staring at you) are bad. But I think coding tests are good! Give the candidate just a really simple coding task with stuff they normally do every day. Create and delete object, fill arrays, iterate over arrays, and so on. 50% of the candidates will fail! The rest are OK engineers.
A hole in the market!
Do the candidates know that ahead of time?
Your approach seems similar to what I've done as a coding interviewer and to what my interviewers did when I last interviewed (at Google back in 2007). Assess people's coding with something relatively simple. Make sure they can gather requirements, describe why they chose this approach instead of a couple alternatives, and (if they make a mistake) that they can diagnose it if you describe the symptoms. If you have extra time, have them review some bad code. See if they spot the problems and how they gently help the author understand/fix them. (They also should be tested on system design, but that's a whole other interview slot.)
I'm studying to be a coding interviewee for the first time in 15 years. This process is stressful in part because they just tell you coding on hackerrank/leetcode/coderpad, which is so broad. If the problem they pick is as you describe, I should be fine. If it's for example some advanced dynamic programming problem...well, those haven't come up for me in the last 17 years, so I'm probably in trouble. It's a perfectly valid area of computer science, but it's not one that matches my experience or what I'd likely be doing if accepted. I'm not excited about taking the time to prepare for that, but I also don't want to make a fool of myself if they do. They probably won't pick this area...and if they do, it's probably a bad sign about the company or my understanding of the role...but the possibility is stressful nonetheless.
This person should already have enough of a reputation to get a job at many companies, if their work is public enough.
What do you suggest for the 99%+ other candidates?
But then why do people with that kind of reputation still (at certain companies) have to jump through these hoops?
https://www.theregister.com/2010/04/21/ken_thompson_take_our...
> What do you suggest for the 99%+ other candidates?
What about (instead of forcing a months long decision process upon the candidates and the company) bringing them into the company after a short interview (maybe 2hrs), and making sure they can afford housing, food and everything else they need. If you like their work, they stay employed. If, say after one month, you do not like what you see, you can easily let them go. Of course you tell them upfront what the deal is.
We could call it, I don't know, maybe trial or probationary period.
They don't. Your link is not about the interview.
1. Untenable for employees - of I have a mortgage or family or plans or obligations let alone a current job, taking this kind of risk is unacceptable
2. Untenable for companies - that's way too much investment.
Companies do have probation periods formally but they are exceeeeedingly rarely invoked, for above reasons.
Works here. My org recently ditched a bad hire with it.
Certainly companies have probation periods. And on paper, that reality and what's proposed in previous post are similar.
But I think there's a massive real world difference between "Default stay hired" and "Default not stay hired".
Probation, as it has currently been implemented in most companies I've worked in, exists, is formal, can and has been used, but is an exception. It's used when there's a massive, unanticipated, egregious problem in performance.
What is sometimes proposed in these threads is effectively replacing long/multiple interviews, with a probation period. While such probation period may look similar or same on paper, I think it's a completely different approach: "We're sure of you (though possibly wrong) so we're hiring you" vs "We're not sure of you so let's hire you and see!". I for one would have only touched the latter with a 100ft pole maybe once in my life. Certainly, I imagine anybody with current job and monthly obligations, would be quite wary in taking a "we don't know so let's try it!" approach to hiring. No, let's figure it out first please :)
It's the only kind of context in which I'd ever consider the "we don't know so let's try it" approach.
Your article points out that in this example: "I'm not allowed to check in code, no... I just haven't done it. I've so far found no need to.".
> If, say after one month, you do not like what you see, you can easily let them go. Of course you tell them upfront what the deal is.
> We could call it, I don't know, maybe trial or probationary period.
You make it sound like it's a better solution for candidates, but it's way worse for many of them and it has been explained by other commenters already.
How many companies actually do this? At which scale?
Some companies increased their difficulty to hire by having aggressive PIP objectives. Likewise, having a "real" probation period where you fire, say, 10%+ of employees is not gonna make you competitive when candidates compare their offers.
The internal recruiter said it kept happening for seniors and people with a lot of experience, but his hands were tied, as the leetcode process was deemed important by the CTO.
Why would this be? Old brains not being as "flexible" to think up novel solutions?
So are top leet coders better programmers? Not really, a lot of them are colleges students who have time to practice and they are in similar competitive circle. I’ve interviewed many and couldn’t hire even one
After the 3rd or 4th time doing this, practicing the same problems just to pass interviews isn't very appealing, especially if you've saved money and can do something more interesting.
You could try to "wing it" and derive it on the spot, but you'll be outcompeted by someone doing a lookup of a solution + alternate solutions from cache.
If you’ve never done LC (or any competitive programming) you’ll struggle. Practice more and you’ll recognize the dozen or so patterns.
Also LC is not “novel,” maybe at the time when the algorithm was first devised but not when you have 20 minutes to solve one.
The interviewer themself said "this is probably more aimed at someone who just graduated."
It was for a senior data engineering role, where the odds of me implementing classic dynamic programming problems on the regular are slim to none.
They made me an offer anyway, but I just wonder what value they found in that. Oh, he knows about Big O? He's heard of memoisation?
They were big on FP, apparently. But not Scala, there was too much FP in that for them, hence the TypeScript.
Mind you my first ever rejection was for a Python role back in 2011 when Python was still very niche in my country, and while I had a portfolio of, imo, pretty decent Python code, they weren't interested because I didn't have a degree, and people without degrees write unstructured code.
Which is a very long way of saying, every interview process ultimately devolves into people hiring people like them.
Thinking, ok, this must be an ice breaking joke I responded, "I don't know, how _do you_ sort a list of integers?". In a condescending tone they responded "are you even a programmer?". At the time, I had been in the game for more than 10 years.
I am really not sure how one could successfully build several companies and have shipped several products without knowing "how to sort a list of integers".
Trying to find a good place to grow your career is difficult on many levels, but if you find leetcode questions silly, you might be too advanced for entry level jobs. ...and you probably don't want to work there.
Students are often better at leetcode because school has been drilling this shit into them for the past three years, but it will probably be the last time they see such a compelling algorithmic challenge until their next leetcode exam.
but why must it be mutually exclusive? Are you implying all "robust" solutions, whatever that means, are dumb? Surely you put some thought in it to make it "robust"?
For example, storing some flag in the high bits of a pointer field of a struct is a "clever" solution, whereas having a separate bool field is a "dumb" solution. In most cases, the "dumb" solution is much more robust over time (less likely to cause bugs as the code changes and is modified by various people). Of course, the "clever" solution is necessary in some situations (very constrained environment, critical infrastructure such as an object header used for every type etc), but should often be avoided if possible.
What's important is that the way this is often presented is that more experienced people will prefer "dumber" solutions, as experience often shows that long-time maintainability trumps many small losses of efficiency. So using "clever" and "dumb" in this way is not at all intended to put down the engineer writing the more robust version.
I think there might just be some vernacular nuance here though, maybe we can call it smart and robust, versus clever and opaque. Some problems are just difficult though, and if you get to work on that kind of problem regularly then that is pretty lucky.
Personally I'm perfectly happy to be filtered out by such tests and refuse to practice for them, as companies that use them for senior level positions are companies I really don't want to work at.
You won't create your own linked list library, you'll use one from the standard library.
General runtime analysis can be helpful - but production, real-world benchmarks trump all theoretical performance values.
Code changes - how do I make a change to a production system in a million line code base that has good test coverage and when deployed, won't bring the entire system down. That's an exercise in the coding interviews that is completely ignored but most useful in the day-to-day professional setting.
This is what I always found funny. Most of the software development work these days is related to web apps. Optimizing that nested loop won't do anything if you have to wait 300ms on some shitty API to answer anyway. It literally doesn't matter, noone cares.
Related meme I saw on reddit some time ago where senior developer says 'haha nested for loop go brrrrr':
https://i.redd.it/lmrf72ko0ro41.png
To apply for the 90% tech companies out there. 10% of all tech companies out there are FAANG or FAANG-like. 90% of tech companies are normal tech companies (they'll care about your education and cv and the interviews are usually just a chat. No IQ tests)
In smaller industry niches this can be true for companies more than people. At this point in my career, the fact that I worked at Company X is evidence enough that I can do the job Company Y wants me for, since it's a tight industry they essentially know of the work I was doing, even though it wasn't a groundbreaking novel technology of my own.
Don't know all the details so I could be missing something crucial, but if not it'd seem that reputation isn't enough
Apple homebrew is used by tens of millions of people daily.
You can't just hire someone based on their reputation at a company of any maturity. That's a legal and HR nightmare. There has to be a process with a semblance of objectivity, and that process has to demonstrably apply to everyone equally, always.
“I have an array with positive numbers, find the n^th largest”
Even then, all thats probably a bonus - a priority queue implementation, or many other possible solutions are probably good enough for me.
https://en.cppreference.com/w/cpp/algorithm/nth_element
Dont mind that half of Google is using his work for free.
Also, nobody at google is using homebrew for work.
I'll admit that surprises me greatly, I can't see why it's considered more efficient, but hey, Google.
A poor little laptop would break down and cry.
I believe that there are also a bunch of IP reasons for this policy, but from a practical perspective doing everything with citc and blaze is really the only option.
The folks developing Chrome for Windows or iOS apps might have different workflows, but even then they aren't going to be using brew because of Google's third party code policies.
But 99.9% of the builds happen remotely. So local vs remote code just isn’t that relevant.
It is also true that Google spends millions and millions on its dev environment every year, so this isn’t your average “no code on laptops” situation.
No one’s running Xcode on the web or on a Linux machine.
Many googlers use brew to install applications on their laptops. This not against policy. Other googlers work with code stored directly on their laptop. There may even be developers who are obtaining deps (for their own builds) from brew.
The problem with the brew author is that he had every opportunity to make himself look hirable at Google but instead chose to write an incorrect screed and publish it on the internet.
At the time I used brew (5 years ago) it didn't require a santa exception with business justfication (and my justification would have been "I need this for my work"). Fortunately this wasn't really a problem for me any way as I don't even look at Mac machines as anything other than a thin client.
But, as usual for internet discussions, it is fun to rathole on side conversations.
You probably shouldn't be using dpkg or rpm for that, either, unless your CI and deployment targets are running the exact same version of Linux that you are, and even then—there are usually cleaner and more cross-platform/distro ways to do it, especially if you need to easily be able to build or run older versions of your own software (say, for debugging, for git-bisecting, whatever). I continue to wonder how TF people have been using typical Linux package managers, that they end up footgunning themselves with brew. "Incorrectly", I suspect is the answer, more often than not.
Where it excels is installing the tools that you use, that aren't dependencies of projects, but things you use to do your work.
Get your hammer from Brew. Get your lumber from... uh, the proverbial lumber yard, I suppose. Docker, environment-isolated language-specific package managers, vendored-in libs, that kind of thing.
I don't install project deps with Brew (it's a bad idea, but, again, so is doing that with dpkg or rpm or whatever directly on your local OS, a lot of the time) but I do install: wget, emacs, vscode, any non-Safari browsers I want, various xvm-type programs (nvm, pyenv, that stuff), spectacle, macdown, Slack, irssi, and so on.
The interview for my current job had something simpler: something like finding random permutations, then what's the algorithmic complexity of this random algorithm. (It was years ago, I forget the details of it.) I just talked through the solution. That was nicer that having to come up with questions. :)
My first round I passed with a less than optimally efficient solution, but he was satisfied every step of the way during my work.
While I was lukewarm to the prospect of working for Facebook, the interview process was very positive and reflected very well.
On a personal level, self interest would have me like the leetcode style problems because I can get most of them right on the first try during a timed interview, without studying. If I were pursuing a job at a FAANG, I might actually study them and I'm sure it would go well for the testing portion of the interview.
However, when I interview this is not what I'm looking for. I'm typically looking for someone who knows the particular language that I'm hiring for. My questions run from the very simple to as deep as they can go on either language or implementation details. From the most junior to the most senior, they get the same starting questions and I expect the senior people to go deeper and explain why they choose something over something else. I'm also testing their ability to explain it to me (not just get it right) as that is part of their job working with juniors.
I really don't even care if they have the names of things right and don't really count things wrong against them if they get the names of two things backwards for instance. For example in Go, a huge percent of the time you might use slice over arrays. Some people get the names backwards, but can identify which one they actually use and they know that one can change size. They are correct in usage and misnaming them. I inform them of the name, encourage them a bit and move on.
I've never liked the "look at this code, what's wrong with it" approach. There are too many contexts that I have to jump into at the same time. There is often an expectation that I find a specific problem with it. I'm lacking the usual tools like an IDE or compiler. What level am I looking at in the code? Does it compile? Are there off by one errors? Cache invalidation? Spelling errors? Logic errors? Business errors?
This guy has missing tests on code that needs to be refactored in order to make those tests. Maybe he has it figured out just right, but the "jump into my code" interviews I've been in on all seemed like they had secret gotchas that the interviewer expected specific answers about.
In short, I haven't seen a proper, repeatable process for interviewing for software development.
The best interviews I've experienced, both as an interviewer and an interviewee, are the ones that feel like two team members collaborating to narrow down requirements and solve a problem.
> about the algorithm or the promise of a particular solution
It's not about "the" algorithm or "a" solution. It's about you the candidate being able to propose multiple solutions, perhaps with space-time tradeoffs, to provide a recommendation based on your judgement, and to ask the interviewer what they think of your proposal.
So many questions. What character encoding is the string? What is human language? Should I honor non-breaking spaces and other similar codepoints? Is the string full of 'simple' characters or 'complex' characters? Graphme's? Emoji's? What's the min and max limits on width? How long (or short) can each string me, and how large could the array be? On what system am I running, and does the array fit in memory or is it paged off a disk? Does the font we're using support all of the graphme's present in the string?
Amusingly, I had a variation of this problem as part of an Amazon L8 IC role interview, framed as a Prefix Tree. Solved the problem, didn't get the role. :(
So it either means:
1. HN's influence is even less than we thought-- even MANGA engineers /10x silicon valley types dont hang out here
2. Everyone agrees its a good idea, but no one cares. Like everyone agrees we should care about the environment etc
Also if you have been hazed you wont vote to stop the hazing
Just because you see them, doesn’t mean they’re correct. They’re just a breeding ground for holy wars and discussions. On every post like that you can find tons of “I don’t like them, we need something better and no I don’t know what”. But if you dig into comments you usually find why those type of interviews are done.
People often downvote thoughtful discussions they disagree with. It’s tiresome to see your text fade with downvotes so often people don’t bother.
If you hang out on Hacker News you might also think every engineer thinks crypto is a scam and no engineers want to return to the office. Neither situation is the case but if you disagree with the majority on those topics you will get downvoted to oblivion, your comments won’t be visible after a point, so why bother?
I personally don’t care about the endless leetcode debate but it’s a bummer there’s so much exciting technological advances in crypto that can’t be discussed on this site without an army of “crypto=scam” bros emerging from the woodworks.
I am okay with disappointing people, but it can be unnerving when that disappointment means I miss out on a good opportunity.
What surprises me most is the lack of flexibility in the process. If a candidate shows up with a broad portfolio I’d rather talk about that then doing some random coding problem. Yet our HR manager insists on the fixed program. This is worse when the candidate is interviewing for a senior role where I don’t really care. Then I am mostly interested in their past experiences and knowledge on how to build things that don’t fall apart after six months.
Again it is definitely process over people here… not sure if it is better in other places.
I would also say that these exercises are most effective when they are quite simple. They let you test ‘can this person write a function’. The complicated ones often filter more for people who have studied those type of problems. Harder problems != better coder. At least not for the projects I work on which are more integrating existing services than investing new novel highly efficient code.
The best jobs I had to date, I met the person leading the company/project/team, we had a chat, talked what tech we like, dislike, how we'd structure a product, what are the preferences to the process around everything. And that's the key thing - it was always a discussion, no Q&A. The key is that the candidate is not the only one who needs to know his stuff - so does the lead.
As a side effect, all of those jobs were way above the market. Again, personal experience, but higher up you go - less BS like "we need leetcode to hire" you get. Unless you're Facebook and you have a genuine problem of too many qualified engineers constantly applying, you should aim to only disqualify truly hopeless cases.
The company can't hide behind process and expect great hires. Early in my career, in a small city I was working in (in return, in the dev community you know about what other devs are doing), our company denied so many devs that within a year or two were among the top performers, just because of the leaderships insistence of a take home tests, Q&A interviews and gotcha style questions...
So please - do continue using leetcode, it makes filtering your company out so much easier and I don't need to go through bullshit stages to know that the leadership has no balls to make the hard calls when it comes to hiring & firing.