This post ought to be the final nail on the coffin of the cult of leetcode/whiteboard style inverviewing.
Just looking at this guy's repo and his writing style -- you'd be foolish to waste his time or yours on leetcode-style hazing. Your conversation should be strictly high-level. If you must, pick a file at random from one his larger projects, and ask a few straightforward quetions ("I'm new to Go - can you explain why I would use this datastructure here?") just to make sure the repo isn't a copy of someone else's, and he isn't outright trolling you. Apologizing as you're doing so, of course ("Sorry, but do you mind if I ask a silly-sounding question or two about something in your repo?").
That's all you need to do. Most candidates won't have work samples that are quite as slam-dunk, of course. But still, your own "smell test" can save you tons and tons of time (and the candidate, agony). It's really, really hard to fake one's way through questions like these -- and really easy to tell if someone is trying to do so.
Put another way, your basic mindset should be: "Trust, but verify". Not: "Joel Spolsky wrote that post 20 years ago claiming that 99.5 percent of all candidates are utterly incompetent. Therefore I need to torture this guy until he confesses."
At least as far as answering the "can he/she code?" question is concerned.
> This post ought to be the final nail on the coffin of the cult of leetcode/whiteboard style inverviewing.
Looks like he has ADHD. Maybe somebody with more time can go through his blog post line-by-line and figure out what's going on.
Most of my Coderpad stuff compiles and runs the first time, so I don't use his incremental test method, or see a need for it. Maybe he learned programming on heavy-weight IDEs and got trapped mentally in those?
I also use vi (no syntax/color highlighting, no plugins, original vi commands only.)
I think you should talk to some other developers and see what's going on with your ability to focus. (I'm one of the top application developers in a particular language and can actually think in that language, so that level of familiarity helps me.)
I generally write some naïve code and if it doesn't work, get out the debugger and fix it from there. I can work a lot faster when everything is laid out and I don't have to guess about what the data structures are going to look like. If it's performance sensitive like some tests I've seen, I'd use a profiler. I often use the jetbrains find and replace tool, as well as the search tool, I also like that I can click into the source code for libraries that I use, so I don't have to guess about the interface or implementation. Stuff like coderpad takes away a good portion of my tools that I would use day to day, I taught myself to code in notepad, but I purchased the full suite of jetbrains tools years ago so I wouldn't have to fumble around with inferior tools.
A profiler will tell you whether a O(n log n) algorithm with a very high constant factor is better or worse for your situation than an O(n²) algorithm with a very low constant.
It's the other way around, from my experience. People who can write a program without any setup can solve it even on paper, while people that are hardwired to specific setup or IDE quickly froze when you ask them for something.
> People who can write a program without any setup can solve it even on paper,
As someone who had courses in C on paper, i can only say that it's a colossal waste of time. Tooling exists to make lives easier, gain time and productivity, why avoid it?
It'd be like saying only the farmer who can do everything by hand without any tools increasing their productivity is a real one. Or an accountant refusing to use Excel/calcuators/ERPs and writing everything on paper.
And before anyone jumps to conclusions, i often write small scripts with vim with a minimal setup, but when I'm doing anything complex i prefer to use an IDE
> It'd be like saying only the farmer who can do everything by hand without any tools increasing their productivity is a real one. Or an accountant refusing to use Excel/calcuators/ERPs and writing everything on paper.
You're trying to map manual work onto engineering.
And absolutely I'd rather work with accountnant who can write me on paper without Excel/calculator/ERP his logic.
It's like you're being obtuse just for the sake of the argument. Someone who can clearly express themselves without specific medium, be it paper, IDE or whatever is strictly superior to someone who is tied to one specific medium.
When one of the mediums actively helps and avoids small human errors, I'd prefer the person knowing using it instead of sticking an old and imperfect method out of pride.
> How does it actively helps and avoids small human errors in reasoning.
Because reasoning is only a part of everything to be done. An IDE can help with typos, best practices ( global variables, lack of error checking, etc) which are tedious things that take time. Isn't it better to the things that matter instead?
Im not a huge fan of leetcode or work samples but on the receiving end it weeded out a huge number of bad candidates. We'd get tons of copy/pasted together infra as code from top Google results that wasn't even close to runnable (usually being close to runnable with a few small bugs was a good submission and runnable with no bugs was a great submission)
People say this but then they reject answers that e.g correctly code the brute force solution or they don’t provide an algorithm to code for, so it’s not just “can you code,” it’s can you find the optimal algorithm and also code it, and usually the algorithm isn’t trivial or one you’d ever run into at work.
When they put it in the promo committee calibration doc that X level has done Y interviews per quarter, they end up on interview loops but then feel they must say no to Z% else they’re not “raising the bar”
> You let one bad, he will hire more in the future.
When your organization can recognize a poor investment but cannot act on it, you have an organizational problem. Why would someone get to hire more "bad" (for whatever that means) if you can recognize it? Even if you're doing all the hiring yourself, it's unlikely you'll have a 100% record over time.
For one thing, in many countries it's hard to get rid of a poor employee unless they're malicious or really exceptionally terrible.
For another, even if "an organisational problem" or inconvenient labour laws don't get in your way, firing employees can have a serious effect on the morale of those left behind. And for good reason! Usually companies that regularly fire people for poor performance has a different sort of culture than companies that don't. In my experience, that's the worse organisational problem.
The right solution is just to be more careful about who you hire in the first place.
Isn't this why there are trial periods ( no idea if it's a thing in the US with the "right to work" thing)?. Over here in the EU a work contract can't be terminated from the company just like that, there needs to be a specific reason ( like downsizing, or a serious error), and you can't quit just like that a mandatory notice period is required ( of course negotiable depending on the situation), usually in the 1-3 months range. Therefore there's a trial period at the beginning, 3-6 months usually, during which any party can end the employment with a very small notice ( a day or two).
So if someone lied their way through interviews, in the first couple of months you should be able to detect that and terminate their trial period.
> So if someone lied their way through interviews, in the first couple of months you should be able to detect that and terminate their trial period.
Managers hates firing people, it is really uncomfortable. So instead of firing this person to look for a new, they keep the person and say they need even more headcount and hope the next one is better.
In the US you are always on a trial period. The problem is that firing people is annoying and bad for team morale. More importantly even dealing with a bad coworker is really bad for team morale. No one wants to join a company that is likely to fire them in 3 months because the company f-ed up their own hiring process. No one wants to recommend their connections join a company where that is likely to happen. Made worse due to a lot of compensation is tied to staying a year at a company so leaving early is basically getting paid 50% for that time.
What is bad for morale is keeping someone around who is a bad fit. Netflix has solved this problem by paying everyone at the top of the scale and firing everyone who doesn’t meet their bar.
The question is whether or not your interviewing practices filter out bad candidates. If your entire company has no employees then maybe but I find that hard to believe.
A lot of candidates don't have public repos and you need an equal comparison across all candidates (you can introduce a lot of bias trying to gauge two candidates against different sets of criteria)
Long before open source was a thing, and repos were even thought of -- I always had a few 1,000+ line personal projects I could dig up and show people.
right, and plenty people don't have large personal projects, or not personal projects that align with their work skillset, and the question is extremely vague what scope you're looking for.
This is my experience hiring developers. If I hire without a code screen, it's a 50-50% chance the developer cannot close a ticket in their first month. With a code screen, I've had one developer not work out (personal problems). And I'm talking about "ensure that no trailing spaces are being inserted to the database in this csv import function" kind of tickets.
>usually being close to runnable with a few small bugs was a good submission and runnable with no bugs was a great submission
This is exactly what we do: check the solution, and if it's close we continue the hiring process. There is a huge difference between an almost working solution and what you get 70% of the time: code that will never, ever, ever work, even if it could run.
A lot of hiring managers have no concept of experiment design.
You can’t invent a measure (or copy it from a blogs about how other companies hire), look at who it rejects, and conclude they are bad. You don’t know if they were good or bad. You never worked with any of them. You only know that you applied your measure.
The only candidates you have any information about are the ones you hired, yet you are insulting the others on the internet today
You never worked with any of them. You only know that you applied your measure.
What's interesting is that the people doing all this "weeding out" seem so sure that they're bravely defending the fort against an onslaught of incompetent hordes... and yet they're missing this extremely basic point about the nature of the process they've chosen to use.
I'd say much has changed since he wrote that, and even for the time it contains some odd advice which he tries to force down the throat of the reader, but I'd be more interested to hear other's thoughts on the article.
To Joel Spolsky's defense, leet code questions where probably a great idea for him, when it was not an industry wide thing.
He also had this, what seems to me, nightmarish, dev. speed and progress estimation tracking app he made. I don't remember what blog post, but he described it and jokingly wrote that Lisa needed to get the speed up, or something. Fast forward to today and it is a predecessor to our agile nightmare. But I don't think that was his intent.
I remember when he made that post ("Evidence Based Scheduling"), and was also appalled. Yet, here we are today and programmer velocity is a now a measured metric in Agile.
The general problem with Scrum like velocity tracking is that devs will start to lie to hit the estimates to keep the commissars of their backs. The best liar gets the straightest graph. And as devs learns to lie, you get the impression of improved estimation.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better."
I mean ... why God. I don't know but maybe poor Milton has the hardest tasks, does not cut corners or what not. Brandon seems to "get it" and has silly narrow error range - I wonder how ...
Hard to explain the absence of something other than it's just not there. The list of projects/accomplishments are fine but the reality is that people lie about stuff like that. I poked around their recent GitHub commits but it was mostly bug reports. Nothing really that points to being obviously great at coding.
I see that on paper they're a great candidate, but the reason that companies do these tests is that often the candidate doesn't measure up to the resume
Not gp, but obviously yes. I'm yet to meet someone who can pass a leetcode-style interview with sober and conscious interviewers, but can't code. Have you?
Leetcode and coding ability is an A -> B but B -/-> A situation: failing leetcode doesn't mean you can't program, but the reverse is absolutely the case.
Leetcode can't filter out other potential issues that may impact success (e.g. poor anger management, or poor judgement).
I'm yet to meet someone who can pass LC, but can't code.
I should perhaps be clear that by "overwhelming proof of ability" I meant "the ability to program well". Which by definition does intersect with one's capacity for judgement, on a certain level. (Though is largely orthogonal to other behavioral issues, I find).
I see LC as providing proof of, at best, minimal ability. And I've seen plenty of people who can run circles around LC-style tasks, but when it gets to the bigger picture -- you know, writing good, maintainable software -- somehow, it just never comes together right.
We agree that obtaining proof of minimal ability is important. I just think that proof can be obtained in other, far less mutually degrading (and error-prone) ways.
Your post ought to be ample proof that subjective opinions (like the one you display here) are not solid grounds for hiring decisions.
When looking at this person's Github page, I -unlike you- see a vast number of trivial projects that can best be described as regurgitations of other people's work. Even worse, if you actually look at his first pinned project (https://github.com/kevinburke/nacl), you'll see that it is nothing but a wrapper with trivial changes/updates... an anti-project.
In the end, I see code that provides the illusion of competence but, when actually inspected, is actually telling me not to hire this person. The fact that pretty much anyone can declare oneself "a software engineer" today and easily present a veneer of competence together with powerful financial incentives, makes it absolutely necessary for any serious engineering team to have objective measures in place in order to evaluate candidates.
There are so many terrible engineers out there (and some even slip through the cracks and land at places like FAANG) that doing anything less is irresponsible.
I never said you should just hire someone on the basis of their repo.
But rather, if you're looking for a basic smoke test which can answer the question "Can this person actually code? Do I want to spend more time engaging them?" -- there are better ways to do that than expecting them to cram on a list of algorithms to recite over the phone. Like looking at an actual work sample.
The fact that you looked at this work sample -- and came to your own conclusion about whether they would be worth engaging -- validates this very simple and basic point.
Which is your call of course. He may not meet your bar, but it would be absurd to say this guy can't code, and hasn't been around the block in terms of general CS concepts.
That said -- my own subjective opinion is that you're being exquisitely savage in your assessment of his NaCl implementation. Yeah, it's wrapper -- like it says in the Readme, and like a lot of working code out there. It's a pretty established way of solving a problem, in fact -- in some cases a very efficient and elegant way. It's not the same thing as writing 20k of primitives from scratch, of course. But that's not what he's presenting it as. He's not saying it's a "tour de force". He's saying "I think this shows we can talk about other things besides FizzBuzz".
And then other things you're like saying... like "an anti-project"? Wow.
Engineering is not about whether someone "can code", but whether someone can _engineer solutions to problems_. Has this person demonstrated that through the projects he chooses to put on display (and advertise as signal for his ability) on Github?
This is also what a lot of those engineering interviews are trying to ascertain, in a quantitative manner. I couldn't care less about someone's degrees or ivy league schools or Github projects (in isolation). These are all secondary -if that- considerations, problem solving ability being primary.
Has this person demonstrated that through the projects he chooses to put on display (and advertise as signal for his ability) on Github?
Using the example you disparaged (his NaCl implementation), I would say:
"On first appearances, clearly yes. The problem was to adapt the a set of standard Go functions to a foreign interface - something one does quite a lot in an engineering environment. Also, the fact that he's aware of projects like NaCl and seeks to learn from the likes of Bernstein and Lange is a positive signal."
If I wanted to, perhaps I could drill down into the code and/or the NaCl spec itself. I don't know Go, so for all I know, maybe his code is actually horseshit. But all I'm saying is - from first appearances, definitely a positive signal. Infinitely more informative that FizzBuzzing (which would insult both his intelligence and mine).
This is also what a lot of those engineering interviews are trying to ascertain.
I've already made it clear that a glance at a repo does not obviate the need for an actual interview.
So I wrote that because a client had a bunch of existing nacl messages that were signed using Python, but the existing Go library uses a different protocol to sign messages than the standard Python library. The existing Go library also didn't support all of the API's supported by nacl, I figured if I was running into this, then other people would too. Clearly based on the number of stars it seems like a lot of people _are_ finding it useful.
I guess in your eyes I should have also done the cryptography, however, I am not a cryptographer or a mathematician and one of the first things that you learn when you start doing work with cryptography is that you shouldn't roll your own crypto.
I decided to make that library prominent because in my consulting practice I saw a lot of clients using cryptographic primitives that were clunky and/or had a lot more overhead, for things like cookies or public key authentication or just sharing messages. nacl is difficult to use incorrectly, and is fast and efficient in terms of overhead. I wanted to promote use of nacl more than the work that I was doing for it in particular.
For that client (a smart lock company), I rewrote the backend, that someone else wrote in C for a specification that did not match the needs of the business. The C software was extremely difficult to read and maintain - socket writes in a callback in one part of the code and socket reads in a completely different part of the code, shelling out to curl, simple Mongo queries that would take 50 lines to set up. As a result the engineering team could hardly make any progress deploying new features. I ended up rewriting the whole backend in Go, a few thousand lines of understanding what the C code was doing, and porting it, to the point where the team could actually release patches and new features on a regular cadence, and figure out what on earth was going on with the Mongo queries and shell-to-curl.
That's not public though, if you're looking for public work, you could look at Logrole (github.com/kevinburke/logrole), a HTML client for your Twilio call, SMS, phone number, and recording data. Of course, you could say "that's just a regurgitation of whoever did the API client", which is true-ish, but I also wrote the API client (github.com/kevinburke/twilio-go). Of course, you could say "that's just a regurgitation of the Twilio API," but I also helped build and maintain the Twilio API, as well as the documentation and information architecture (https://www.youtube.com/watch?v=sQP_hUNCrcE), and the client libraries (https://www.youtube.com/watch?v=C_UJHqR_2Mo).
Most recently I was hired as the first engineer at Meter (meter.com) where I built out the entire backend API, the deployment infrastructure, the CI/CD infrastructure, a pipeline for the in-the-field servers to update themselves, and a method for access points to retrieve configuration from the backend. All in all this was probably about 30-50,000 lines of code over two years. I think the API has had one customer facing error in two years and no downtime.
It is true that anyone can call themselves "a software engineer" but not everyone can get contracts for a sole member consultancy and, crucially, get their contract renewed multiple times at every single client. If I was selling snake oil and not delivering anything, that would not have happened.
I have 15 years of experience, and seem to woo teams every time we do screening calls.
Could be how he describes the problem. Most of the time during code screens they don't want you to perfectly solve it. Talk it out. Someone who feels that perfect or bust is the only way to go will fail most of the time. Idk feels like something is missing. He's right, feedback loops suck.
Question is... Can he practice doing a phone screening with a friend? Does he know anyone who interviews? Does his current job allow him to interview or sit in some? Just being part of it can give great interview experience.
Around 10 years of experience is when I learned a ton of interview skills. Interviewing is indeed a skill in itself.
I personally hate this approach. Solving programming problems requires my full focus and vocalizing while trying to think is a challenge (partially due to my stutter). I can either focus on communicating clearly and professionally or I can stfu and code, not both.
I’m happy to solve a problem silently and talk about it after I’m done, but I won’t do a phone screen that asks me to word vomit in the middle of coding.
How do you feel about collaborative problem solving? A big reason talking-it-out is popular is because it provides signal on your communication, problem solving and collaboration skills.
If you sit there for 30 minutes banging out a solution in silence, I get a bit of signal. If you sit there for 30 minutes in silence doing nothing, then you're stuck and you've failed the interview as I didn't get the opportunity to assist.
In every real world work problem that is solved collaboratively had some time before where people understood the context and the problem by themselves before. At least they know the codebase, the business goals, the colleague.
There is no situation where a boss call two random engineers, throw a completely new problem at their hands and ask them to solve it in 45 minutes.
> There is no situation where a boss call two random engineers, throw a completely new problem at their hands and ask them to solve it in 45 minutes.
I don't know what to say to this beyond, absolutely yes this situation exists. The easiest scenario is you are on call and get paged and must solve a critical issue in as few minutes as possible.
What kind of shop are you working in which requires you to solve algorithms puzzles to resolve outages?
Even if we think we see a way to solve an outage with something like this, we generally consider it inappropriate for an outage mitigation scenario, always preferring to rollback / flip feature flags / add capacity / other simple and minimally invasive interventions.
Implementing a better or more scalable or more resilient system is important, yes, but is done after the issue is mitigated... not at gunpoint.
> always preferring to rollback / flip feature flags / add capacity / other simple and minimally invasive interventions.
Sure, that's nice, but you're living in a fantasy if you expect that to always be possible.
I also didn't say anything about implementing scalable or more resilient, or even algorithm puzzles. That's not what we're discussing.
I'm talking about communicating when problem solving. If solving an outage was always a mundane rollback or simple intervention, then it should've been automated. For example, say a new deploy corrupted the database subtly. How do you recover it? Perhaps you rollback the deploy, create a new database instance from a backup, perform a cutover, lock writes, write a migration to copy records from the old instance to avoid losing data written since the backup, validate the restore, unlock writes, monitor. You probably also need to work with stakeholders to draft and send comms to customers, document the steps you took, answer questions from product folk.
This is what I am trying to get signal on in an interview. How do you work on your feet, when it matters most. If you expect to be able to sit in silence for 30 minutes after you were paged while you draft a solution in your head then you're going to have a bad time.
For a junior candidate, these expectations are reduced, but this is the bar for a senior candidates. If I wanted you to solve an algorithm puzzle I'd give you the puzzle and tell you to come back tomorrow. If I didn't think you could solve an algorithm puzzle, we wouldn't be having a conversation.
I think talking through an algorithm puzzle solution during an interview gives you no signal at all of how well an employee would communicate in the scenario you described.
I agree it isn't a real job requirement to be able to switch back and forth quickly between thinking and communicating. I really have a hard time decoding what another person is thinking when I'm solving a problem. It's like I use the same part of my brain for both things, and it has to reboot to switch between them.
To mitigate this I try to constantly verbalize things that I'm thinking anyway, like, "This isn't the OO way to do it, but I think the OO way would be a poor match for the problem," and, "In real code I would model each of these item types with a class, but to save time and hopefully get further with the problem I'm going to model them as lists." The more I talk, the less they talk, and the less often I have to reboot into human mode.
I'm much more pro-leetcode-style-interviews than the median HNer, but I agree with this criticism. It's a very unnatural way of working, and really dilutes the skill signal. In principle, you are learning about how they communicate, but I believe in practice what you suggest, where the candidate has a chance to produce a solution and then explain it afterwards, would be more effective at this as well.
Can't speak of others, but when I'm doing interview I never "take points off" for not talking while coding - as long as they're building the answer in the correct direction, that's all I want.
I think "talking while coding" is, at least in theory, for the benefit of the candidate - if they're "thinking aloud" and about to make a mistake which may take 20 minutes to recover, the interviewer can gently nudge them in the correct direction.
It's not about talk while coding. It's about letting your interviewer know what strategy you employ.
I fully expect Sr engineers to talk about why they choose the approach they do, and then implement it. I can adjust their approach while I hear about it, and then let them code it up.
This whole being silent and focus coding is bs I think. When you're doing the typing part, don't talk, just code. When you're doing the strategy part, talk. I want to see how you'll be able to express your strategy to your coworkers when a real actually challenging problem occurs.
Also. Algo problems are worthless. They only tell me you know how to cram. Don't do those, they don't actually answer anything. If I see that in an interview that's a flag for me that their eng culture might have issues. Only exception would be if you're applying to a job where you'll be doing that all day.
I can solve programming problems OR carry on a conversation. I cannot do both at the same time.
It isn't just coding. The same quirk shows up in other ways. For example, I also cannot carry on a conversation and navigate. I can drive safely while conversing, but I cannot navigate. When my daughter was a teenager she exploited this quirk for laughs: get in the car with Dad; start for destination; bring up interesting topic; wait to see what random place we end up going; profit!
I can explain my thinking after the fact. I just can't do the thinking while conversing.
I've been working as a consultant for a while now. Communication is a big part of the job. My current client is one that has come back repeatedly, and that pretty much lets me set the terms I want, and gives me broad discretion to decide how things are to be done. I presume that means they're happy with the results so far and want more of the same.
I do have to communicate effectively in that job. I collaborate with a VP to set directions, vet R&D proposals, choose implementation strategies, pick tools, guide other contributors, and so on. So I have to be able to communicate effectively.
I also write the majority of working code in the current set of projects we're working on.
What I don't have to do is write code while carrying on conversations. It's a good thing I don't have to do that, because I can't.
I can pair program, as long as my collaborator understands that when I need to really think about a problem, I have to leave the session to do it. I can rejoin once I've figured out what I'm thinking about, and I can explain what I thought of. The collaborators I'm working with now are used to it by now, and it's a normal part of our collaborative cadence.
I can hunt bugs in a pair session, similar to the way that I can drive just fine while conversing; I just can't navigate, or do involved creative problem solving. I'm not exactly sure what the difference is; just that there's a certain kind of thinking I need to do that gets blocked by conversation.
"I'm trying to think, but nothing happens."
I'm useless in tech-interview coding, so I just don't do it anymore. I've told several companies that wanted me to go through their coding interviews that they should just give me the lowest possible score on that part of the interview, and if they still wanted to talk to me about the job, I'd be happy to talk to them. Nobody has taken me up on that specific proposal, but I have gotten nice offers a couple of times after I thoroughly bombed coding interviews. Those experiences made me question how seriously those companies took their own processes.
I talked it out recently and it quickly became apparent that the interviewer considered my list of "this is how I wouldn't do it" as a list of "these are approaches I would actually consider".
I conceptually like the idea of dropping coding screens, etc - but how do you objectively identify what you're looking for in a candidate without adding bias to the process?
I.e., at least with a coding question, you can set up objective criteria. If you ask someone about their work, it might be a good experience but you're also setting yourself up to bias towards people who talk/think like you.
If you really want to test someone's coding, let them do some coding in their own time with their own tools and send it to you. This isn't ideal, since it can be time consuming, but I think it's more humane than live coding in some weird environment.
I would argue that using less of someone's time is strictly "more humane" than using more of someone's time, because an individual's reaction to a situation is... individual
There's no reason an asynchronous coding assignment needs to be more time-intensive than a synchronous coding interview - that's the convention but it's not a necessary one. Lots of companies do 4 hours of technical interviews, it's not hard to come up with an effective programming assignment that takes less time than that.
I think that staying employed as a software developer for 10+ years is a good indication of skill, far better than any random coding assignment.
Put another way, if someone has 10+ years of experience, in an ideal world there isn't much point in quizzing them on algorithms or giving them homework. They've done all of that before, passed, were promoted, etc. You can see that on their resume, and you can verify that with a few phonecalls.
Of course, in reality, you have to quiz everyone equally, and hire based on this objective metric, because even a whiff of non-objective hiring is dangerous.
> I think that staying employed as a software developer for 10+ years is a good indication of skill, far better than any random coding assignment.
As someone who has been doing interviews for as long as 2 years, wrong. I've seen at least 15 candidates that had 5-15 years of experience and we're complete shit. As in no redeemable qualities in terms of tech assignment.
One thing I realized recently while interviewing candidates at my company is that it’s one of the few opportunities that individual contributors like myself have to exert significant power over someone else, and not everyone is prepared to handle it well.
Recently, we had a candidate come through that wasn’t a good fit for us (he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time). One of my coworkers took some pot shots at the candidate, being deeply critical and rude, cutting the candidate off mid-sentence, etc. I was honestly embarrassed to be on the call and I thought it reflected poorly on my company. I’d never seen my coworker act like this before and it changed the way I felt about him.
I wonder how much of the leetcode interviewing culture is rooted in this desire to dominate candidates, since ICs rarely have opportunities to wield such power over others.
They could just need training and feedback. A lot of places just throw any rando engineer into an interview with no coaching on how to behave, how to set guide rails for the candidate, how to empathize and pace the interview, what the evaluation criteria and rubric is. And then after the interview, the person has no idea if he conducted the interview properly. Totally open loop. Just yeet him at candidates untrained and then wonder why he can’t interview effectively.
I still recall an interview I did at a recently public company in New York, for a DevOps position. 4/5 of the interviews went sailingly, with heavy emphasis on networking, scripting, and AWS knowledge. Then the lead engineer sailed in, five minutes late. He gave me an algorithm question straight from the Google playbook (I don't remember the question, sadly), then after I asked for a few minutes to ponder, proceeded to smirk and interrupt me every 30 seconds to ask if I needed help. In between he was chatting on his phone with the audible keybord ticks enabled.
Some companies will ask for feedback after the interview and there's sites like Glassdoor.
The last place I did interviews at, we'd bring all interviewers into a quick debrief meeting and talk about whether to give the candidate an offer (going off the notes we took during the sessions). It was pretty obvious if someone had a particularly bad session and could be weighted against other peoples' input
> I wish there was a way to report bad interviewers.
You can always tell the recruiter/coordinator. It's just tricky because you don't want to burn bridges, doing it before you hear back could be read as you asking for a do-over, doing it after could come across as bitter.
We are hiring at my company. We ALWAYS value feedback. Last week we heard someone had a bad experience and the feedback was taken seriously, and without resentment.
If you have a good or bad experience, PLEASE consider telling the recruiter know.
This happened to me at a recent Google interview. It was the last interview of 5 and the guy gave me a gnarly graph problem. I was trying to wrap my head around it, but he kept interrupting me and giving me unwanted help. I ended up getting writer's block and just mentally shutdown because he wouldn't let me have time to think. He ended the interview early because of poor performance.
It made me feel like I didn't know anything and was very embarassing. It also changed my outlook on Google, and I'll never interview with them again.
Please go off in a corner and ponder why such casual ageism is as unwelcome as casual racism or sexism and maybe just stay there until you come up with an answer.
Substitute woman for boomer in that sentence and listen to what it sounds like.
I could substitute it for child, another group of people who are broadly less likely to know where the setting is, and it wouldn't be offensive that way either.
As someone who was born before the moon landing and saw one of the last IMPs get installed on the ARPANET, yeah that was offensive. You should apologize.
Knowing to silence your phone in meetings is a matter of having manners, not age.
Just as no-one would begrudge a dominated race from poking fun at the dominant race, I don't see why you are so on your high horse about the dominant age group getting the same. 'Punching up' is fair game where I come from.
I'm afraid I just don't get your outrage at an innocent comment poking fun at the most dominant section of society. But, as you're the second person to have taken offence, I'm going to delete it anyway.
First of all, I'm not a particularly big fan of generational stereotypes in general, especially on a board like this that aims to do better. If I made a comment to the effect of "Zommers are the most entitled group I have ever seen," I would be (rightly) most likely downvoted to oblivion.
Secondly, I have never encountered it as a problem--at least in a way that affected me (never could really get into a dot-com company but we know how that mostly ended up). But that's mostly because I've had various roles where experience really trumped other things. But ageism is clearly an issue in tech broadly so I take issue with with the justification that it's OK to make casual cracks with the justification that it's punching up.
To say nothing that it added absolutely nothing to your comment. Rude interviewers are the issue. Not some imagined inability for older individuals to silence their phones.
Finally, I'm not sure doubling down on the oppressed race thing helps your argument--especially given what the age breakdown in tech looks like.
Ah yes, racial humor. That will never fail you in business provided you deliver it the appropriate way. Good luck with your interviews, you’re going to do great!
> 'As someone who was born before the moon landing'
That would include any and all of us in this thread. No man has ever come anywhere near the moon.
Unless, of course, you happen to know of some kind of magical (presumably still classified) rocket nozzle tech that would somehow avoid leaving a crater in the soft 'lunar dirt' as the 1960s tin can touched down on the 'lunar surface.' Not to mention an actual solution to various other hard technological problems of space travel, like radiation shielding, heat rejection (no, space isn't 'cold'), battery power, lack of any visible stars, workings of a camera in a high radiation environment, etc, etc. And how about that lunar module 'computer'? Anyone ever replicated that 1960s technological wonder? Oh, the schematics and tech were all 'lost'. Someone accidentally recorded over the original telemetry tapes with Gilligan's Island reruns. I see. What a terrible accident.
Not to detract from your main point, of course; tye interviewer was terribly rude. Selfish. Just like the liars who faked the 'moon landing' in the Nevada desert, near Indian Springs.
See the capacitor in this closeup that kens took of a power supply module in the Apollo Guidance Computer? My dad's company made it. You are telling the wrong person that the landing was faked.
Yes, everyone has got an uncle or a cousin or a buddy they knew once who was involved in some ancillary way on the outskirts of the Apollo Simulation, which they take as being absolute proof that it was 100% real. They even angrily call me names, and of course, absolutely refuse to review any of the concrete evidence that the whole thing was, in fact, a fraud.
Just like you did, just now. I still haven't heard your explanation of how a large diameter rocket nozzle, which we are told was firing for the entire descent, with 10 psi of hot gases escaping from its large volume, failed to disturb so much as a speck of dust underneath the 'lander.' That's only the most obvious of many glaring plot holes in the script.
If you were to, say, enter 'evidence that the moon landing was fakee' into any search engine, naively thinking such evidence would be presented to you, instead you would get only page after page of laughter, ridicule, and condescension--and none of the actual evidence. Certainly you would never, ever be presented with the name of the very first book on the subject, which launched moon landing skepticism into the mainstream in the early 70s. Nowhere in the first ten pages of results is this man or his book mentioned.
The book is called We Never Went To The Moon by Bill Kaysing, who was a technical writer for the Saturn V rocket program at Rocketdyne. Wouldn't you think a man with such a background would be worth listening to? Google doesn't think so.
Another book of interest is NASA Mooned America, by Ralph Rene. This one is more technical in nature, and drives the final nails in the coffin. Google also pretends this one doesn't exist.
If you type the author and name of the book into Google followed by the keyword 'pdf' however, it will take you right to it on the first page. At least it did as of a year ago. Funny how that works.
Read those books, then come back and tell me again how sure you are.
When you find out how fake and full of lies the world really is, you're going to be pissed.
I have been doing React development for 4 years now and I have never set it up from scratch and only have a rough idea how it’s done. I still apply for react jobs and do fine. Installing something from scratch is one of the least important tests because it’s something that companies typically do once every few years, they can afford to flounder for a few days on that.
I reinstall Linux from scratch about once every 5 years, long enough that I totally forget how to do it. So I keep a file with all the commands to do it.
I recall voicing an opinion that we should record interviews we give candidates and re-review it during the hiring committee, much like we do code reviews. Have the interviewer justify certain beliefs about the candidate with hard timestamps rather than platitudes ("not confident", "wanted to see how they react under pressure", etc. etc.).
My comment always gets pushback. Privacy, respect for interviewer ("If I work here, then I myself can judge fine!"), not enough time, etc.
Hiring processes are about as contentious as CI methodologies, it seems
I'm not comfortable with the idea of being recorded, especially when I'm already under the pressure of an interview. I don't know how they'll use my video, if it will be used in training, leaked, etc. I might never see that company again, but they could use my video forever without me knowing.
If I thought my interview was being recorded, I'd assume it was being used for this internally. If given the option to have it recorded, I'd assume that I'd get "minus points" for declining.
This is very unfortunate, because I love your idea. I'd just need stronger legal protections to ensure that I wasn't perpetuating part of a dystopia.
There is mostly no such things as "optional" in those sorts of situations. It's like "Here's a take home project. It's purely optional but it would give us a better sense of what you can do."
In other words, maybe if you utterly wow us in-person you'll get in anyway. But probably not.
I don't agree with recorded video, but I am now convinced that there should always be an random "observer" (interviewer does not get to pick) who does nothing but watch and take notes along with an interviewer.
You can give whatever excuse you want "Adding a new person to the loops," "normalizing the loop level," whatever. But I am now convinced that interviewers cannot be trusted alone with candidates anymore.
Maybe I'm naiive that they ever could be, but if one of my guys did half of the stuff people keep reporting to a candidate and I found out about it, I'd have his head and maybe his badge.
Fsck you very much whichever FAANG HR department decided that "interview loop" was something to be added to corporate evaluations for points and then promulgated it to other companies. Thus making both sides of interviewing even shittier than it was before.
Isn't interviewing in pairs fairly common? When being interviewed, there's always been two people across from me for the technical rounds; anything 1-on-1 is limited to initial phone screens and final rounds with the bigwigs. When conducting interviews, I've always been one of two, with the exception of one time when no one else was available to join me, which I'd prefer not to repeat.
I would go so far as to say a random passive observer is still a great idea in addition to two active interviewers.
It wasn't at any point that I knew. Interviewing is an expensive operation--it ties up quite senior folks for a fixed amount of time. If you pair that, you double the expense.
And it seems like phone screens are exactly the point where you most need a pair from these anecdotes.
Large companies have a fairly rigid process with strict rubrics for interviewers. In general the goal as I see it is to remove as much individual decision making as possible. Doesn't stop these issues as the comments here show.
You also then encode a single group's biases (the rubric) rather than a mix of everyone's biases. In my experience that doesn't actually make things better overall just slightly more consistently bad.
I have noticed that sense of "domination" come out in interviews as well. I remember long ago having one interview where the interviewer was asking me to code something which I knew would be impossible to complete in the time allotted. It just seemed so ridiculous to me that I was supposed to write this complete solution in about 30 minutes. I remember how superior he was acting at the start of the interview too, and then after I wrote up the code (unable to complete in the time given, but apparently good enough to make him happy), his tone changed completely of one where he was very friendly as though somehow I passed his "test". I was now worthy of speaking to as an equal, whereas half an hour or more earlier I was some nobody to him.
Another interview I had years later, the interviewer clearly didn't think I was worth his time because he literally got up and started playing with a yo-yo as he threw challenging question after challenging question at me and pressing me harder and harder for solutions ("Imagine you were given task X and nothing more. How would you solve this? What would you do next? If that didn't work, now what would you try?") I remember it was incredibly disrespectful and left a bad taste in my mouth about the team. Thankfully I didn't get the job.
For example: The idea that there are many programmers who cannot even write fizzbuzz sounds like a very strong claim. It seems far more likely that there are many programmers who have socially-induced anxiety.
It's entirely believable that many programmers couldn't write fizzbuzz without searching Google in the days before it was universally memorised. The modulo operator is somewhat rare in real code and is defined slightly differently in different languages.
> The modulo operator is somewhat rare in real code
Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction.
Modulo might be slightly different depending on the language, but for fizzbuzz that's not important and most likely not what you're expected to know in such an interview question. And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
> Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction.
And they'd probably fail some interviews for doing so.
> And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
This is true, but I also expect some otherwise-competent programmers would fail a test that expected them to remember the specifics of bitwise operators or bitshifting. It's been years since I've needed to use any of the three in a paid job - they're just not very common in my day jobs, whereas they crop up a lot in side projects.
I have interviewed senior candidates (more than two, fewer than five) who couldn’t write code to sum an array of ints (ignoring overflow). *
Not in the sense of missed a bit of punctuation, but rather in the sense of “couldn’t get started with anything that vaguely smelled like an answer, didn’t understand the type system of the language they claim 5 years of experience with, etc”.
I’ve probably interviewed fewer than 400 candidates in-person and maybe 200 of those would be senior ICs. To have more than a percent of them be utterly incapable here is a signal that there are candidates out there taking their best half-court buzzer-beater shot to try to land the prize. If you’ve done 100 first round in-person interviews and never run into someone that causes you to check up on how the phone screen happened, your screening processes work pretty well.
Is it possible that these candidates are capable programmers in daily work? Sure, it’s possible, but I’d bet against it.
* We used to ask a candidate to average an array of ints. It’s a question that was designed to tease out whether the candidate would discuss/inquire about overflow, what the return type should be, etc. After a couple years, we realized that candidates would stumble on the loop and sum more than anywhere else and we got more signal/minute from asking the vastly simpler summation question. Candidates who could fluently write the loop usually knew about overflow and the fact that the average of ints isn’t mathematically an int but that many times an int is what you want and can discuss the reasons.
I love this because I agree 100%. I accepted the idea that they were "out there" but only really internalized it after I interviewed someone with a Masters in physics who couldn't make any progress on my simple problem. (The problem does assume you can understand the concept of a geometric line, but I give the two line equations most people first run into in junior high, among other things, I'm not testing recall.) Especially on the part about making a question even simpler in the name of getting what you actually care about faster. I've also found it useful to combine things from separate questions into one question that can answer both in less time.
I still wonder if we couldn't remove most of these games though if we improved our hiring, onboarding, and firing processes in other ways than just trying to make a better hiring filter. Accidentally letting in someone wholly unsuitable for the job shouldn't be a big deal if 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small 3) they actually start very soon after the decision to hire is made 4) you can fire them immediately after deciding a mistake was made and they don't work another day.
Startups can handle this better already and I think is reflected in a lot of them still doing more casual "single lunch conversation" style interviews before a "welcome aboard, see you on Monday" decision. For big cos, work will have to be done on getting away from the mentality of "it's ok if you're not contributing much of anything for the first x days because onboarding" where x is high (I've seen over 30), and firming up access controls (no facebook style all-access of new hires (or anyone if possible) being able to just query the production db and access the private messages of exes or whoever), and removing the "ok you're hired, see you in x weeks" policies where new hires can only start after a significant delay or all on a certain date for whatever reasons, and only getting rid of someone through a long and drawn out PIP process... Maybe it can be done some day.
On the “we can take a chance here” front, I’ve sometimes struggled with hiring someone out of a stable situation and putting them into a role where they might earn a termination. If someone’s already unemployed and that happens, they’re no worse off (and likely better off). If it happens that they resigned a stable job and that happens, they’re made worse off.
As a hiring manager or interviewer, I’m not their fiduciary, but when facing a risky hire, I wish there was a way to let the candidate make a more fully-informed choice without the knowledge that we think it’s a high-risk offer influencing them negatively in any way.
Right now, they interview and get an offer or don’t. If they get an offer that we think is high-risk, they have no reasonable way to know that.
Could you indicate it through your choice of interview questions?
Things like, "How do you feel about stretching your skills in the work force?" "What do you do when you make a mistake at work?" "How do you handle a situation where you feel out of your depth?" Etc.?
I know I would take that as a signal not to go for it if I wanted to retain stability in my life.
That's a great thought; however, we most often discover the conflict during the hiring debrief when we have mixed feedback on a candidate leaving the candidate "on the bubble".
It does give me some ideas to consider when we extend the offer (as @Jach's sibling comment also suggests).
Yeah, it's trickier when they're coming in with an existing job, and the current professional quitting culture dictates that you couldn't start any time sooner than in two weeks at least. Add that to the list of difficulties involved in lower time-to-start dates.
I've been fortunate (or not) that the hiring decisions I've played a part in have all been non-urgent with no struggle to find more candidates, so in the face of risk the org could afford to wait. The moral hazard you describe is interesting though. I don't think I would worry about it, as you say the hiring side isn't their fiduciary, and you don't necessarily know that their current situation is all that stable. I might try and assure myself that even if things don't work out, they're unlikely to be put out on the streets, since if they got past us (even with our "risky" assessment) they can probably get past someone else quickly enough, and possibly even go back to where they left depending on how stable it really was.
I'd consider being honest with them about the risk if an offer is extended. If I were on the receiving end, I'd prefer to be told before I accept something along the lines of "Honestly we're a bit unsure how well you'll adapt to our work [because of risk factors x], but we see your potential and think you'll rise to the occasion. We hope you'll come join us!" I don't think having my risk factors pointed out would affect me negatively at all, even if I think they're BS things the company shouldn't worry about, though I can see it possibly affecting the company somewhat negatively in that they're potentially giving the candidate who all-in-all they think is worth taking a chance on an extra reason to decline (and thus they'll need to continue searching which sucks). Whether that decline happens really depends on the candidate. With that information they might conclude/agree it's too risky to leave their current role for you right now, or they might actually be motivated to do better and start proving themselves right from the start especially at a company that is so forthcoming about such things. And they might fail, but at least they leave with the knowledge of what went wrong, and even if what went wrong wasn't an identified risk factor they still have your list of risk factors (some possibly valid, some BS, some that can be worked on, some that can't) that other companies might be looking out for to have for future interviews.
> 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small...
But the cost is actually minimized already by rejecting in the interview. Just like finding a bug while writing unit tests is cheaper than QA finding it after you make a build, even if QA does so in the first hour of testing.
There's a tradeoff between the costs of trying to detect possible issues at various stages, and the costs of damage if those issues aren't caught until later. This makes the caught-earlier-is-cheaper analogy not always hold, because the costs you spend to detect an issue at some stage can easily outweigh the costs of the issue itself at any later stage. Hence the industry doesn't follow NASA practices, or even more reasonably make use of TLA+ as basic professional practice like unit tests have largely become. Many bugs have such negligible impact that they remain unfixed, regardless of when they were caught, regardless even of whether the customer found them or knows about them. We also realize that some bugs still make it past everything and get to production, so it's worthwhile to spend effort trying to reduce the cost of production bugs, which might for instance involve having a faster and smoother deploy process.
So in hiring, there are costs companies pay to avoid a bad hire, and costs that result as a consequence of bad hires -- which they mostly can only estimate because of bad hires getting through anyway, everything on the consequences side is formally probabilistic. You can think of it as hiring-filter-costs + P(bad-hire) * consequences-of-bad-hire. So yes, if P(bad-hire) is 0 because of your amazing hiring filter, then you'll never have to pay for the consequences-of-bad-hire. And supposedly, increasing hiring-filter-costs should decrease P(bad-hire), though over time we've found many practices that don't actually do that, and smarter companies have stopped doing them.
My thinking boils down to we can and should reduce hiring-filter-costs (which I want to think of as including opportunity costs and negative externalities on the whole industry, but it's not necessary), but fear prevents even experimenting at many places because there's an idea that any reduction will increase P(bad-hire) too much, and consequences-of-bad-hire are too large. So, why not focus more on identifying and lowering the consequences-of-bad-hire? Do that, and we'll also be able to lower hiring-filter-costs even if they increase P(bad-hire), which they aren't guaranteed to do anyway.
I wish America would adopt what I've seen in some European countries -- a paradigm where they introduce a fixed-time contract first, then only hire you if they like you after the contract. I have seen that in some jobs in America, but just not programming jobs.
It happens in programming jobs here but in different ways. The general style I think you're thinking of seems more likely to me for startups to try experimenting with (and more informally), but we also have a different style where you work for a general contracting agency who has you work for another company for a fixed term. Some of those after the fixed term can transform into that other company hiring you directly (and giving you higher pay not cut by the contracting company, benefits, etc. that you lacked as a contractor), or if not the contracting company helping you find a new company to contract for. The final style I can think of is already institutionalized at big companies for entry level roles in the form of hiring interns almost done with college. They work for you for a few months of the summer (often at equivalent 'hourly' salaries as a senior role) and if you like them, you offer them a full time position (at a lower entry level salary) starting next summer after they graduate. It can be a smart way to get talent that doesn't yet know it's great, or will become great, without paying a premium.
The biggest problem when hiring people with experience though, and why the probation style is pretty rare in the direct form I think you're thinking of, is that it's going to filter out the best people -- just like many other too onerous hiring practices filter them. They're just not going to put up with it when they have much better options. For Europe to the extent such a practice works at all is probably related to how universal it is for employees and employers to have a symmetric long period of notice of termination/notice of quitting that often ends up being at least a month if not 60+ days. In the US, every state is at-will, you can leave at any time and be dismissed at any time (within the bounds of certain legal protections, paranoid risk of violating which has driven larger companies to an insane PIP-style process of nudging someone out the door over a period of months). I think US companies and employees both should continue leaning in to that more as an advantage.
I have screened or interviewed hundreds of people over the past three decades. That is not a strong claim, it is simple truth. There is a large minority of so-called "professional" software developers who literally don't know what they're doing and can't anything but the dead simplest code.
They may only represent 2% of the population, but they are a much larger % of candidates searching for jobs. If you think about it a bit, you'll see why.
Asking a doctor who had completed their 12 years of medical school and residency basic biology questions would be a bit absurd.
Asking someone who claims to be a doctor of natural medicine the same might yield you more than half unable to answer.
Since there is no 12 year stringent accreditation process for being a software engineer, I'm not sure why this is a good comparison. There are many people who don't know so many basics of CS who have finished degrees, bootcamps, or have been employed in those roles for years.
Doctors simply aren't able to get accreditation or employment with the same ease as software developers.
Some people avoid the dilemma by only interviewing candidates from prestigious CS programs and with previous employment at famous companies, locking out those without the "right" credentials.
If you're going to demand that you be treated as an honest estimator of your own abilities beyond doubt don't be surprised if that ends up eliminating opportunities for people who are trying to break into the field without those credentials that put their abilities at a near certainty.
It is a strange comparison, though I don't think the "stringent process" is the issue and it's more just that they are entirely different fields with different ways of working with different kinds of feedback in the broader economy with different levels of observability. You might find that even the doctors that had years of study and paid supervised practice (residency) are unable to answer basic things. From an MD last year: "Even doctors forget the relationship between calcium and Vitamin D, and I have spoken with doctors who did not realize that 5000 units a day could cause problems. As a doctor, there is a lot to know and remember, and most doctors haven't studied calcium metabolism in many years." (https://news.ycombinator.com/item?id=24065115)
> he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time
If I was floundering because the interviewer found a hole in my knowledge (and there are many holes in Albert Hall) I'd proactively state that and say where the holes weren't and ask for a question about that.
For example, I've implemented a complete C compiler, front to back, so at one point I knew everything about C. But in implementing a new C compiler (ImportC) a few months back, I had to go over the Standard again, as I'd forgotten bits and pieces.
There's no reason at all why the interviewee cannot drive the direction of the interview, rather than being driven. For example, when I interviewed for a software job after leaving my mechanical job at Boeing, I was concerned that I wouldn't be taken seriously as a programmer. So I brought along listings of the code I'd written and pulled it out during the interview and proceeded to explain it to the interviewer, on my own initiative.
A long time after that, I was talking to the interviewer, and he said that that was what got me the offer.
> There's no reason at all why the interviewee cannot drive the direction of the interview, rather than being driven.
You know how they made a program pass the Turing test? They didn't make it smart, they hardcoded a conversation where the program drove the conversation to where it wanted it to be. That way the person interacting with the program was passive, so they never noticed that they were just interacting with a stupid program in the 5 minutes they had with it.
The same happens in interviews, if you take a person who don't understand tech, but have watched a lot of conference talks and read some papers and he can re-tell those, and he even remembers answers to common questions from the question parts there, but he doesn't understand much at all of what was said he just remembers conversations, then that guy will look like a world class expert if he gets to drive the interview. But once he starts working he didn't know anything at all, he just learned to talk about these things not do them. Some would say "just dive deeper into his knowledge!", but you can't always do that, if you aren't an expert yourself in his subject area you wont be able to drive down deep enough to find his flaws, and since he is driving the conversation most likely you wont be an expert on what he is talking about.
The person could have just learned to code in all that time spent watching conference Q&A sessions to con interviewers.
This does seem to be the threat model many interview processes are meant to protect against though.
I understand there are costs involved in hiring then firing someone, but maybe that should be reserved as a way to deal with the most outlandish fakers and let the rest of us have a quick Google or two during coding screens.
If you are smart and a hard worker then learning it properly is easier, yeah.
But if you are lazy and prefers talking to people and watching videos, so you spend your day talking about these conferences with your colleagues rather than doing work then you have the scenario I'm talking about. It doesn't require some genius conman, basically any guy who prefers to talk about things instead of doing things will more or less fit this description. The degree they lack real skills depend on the degree they prefer talking over doing, but such people will always over perform at such interviews.
Lastly since the talkers get rejected by all the leetcoding firms, you will have an abundance of talkers that can't perform in the hiring pool. So the problem with talkers is way worse today than 20 years ago.
When I have to game an obviously flawed interview because I want the job, I try to drive the interview. Make no mistake, this is a social skill. It’s not something I enjoy, it’s just a learned skill for me. But I do it because I want the job.
Often I feel like a con artist for doing this, especially if works to overcome a superficial interview. Sometimes it turns out my perception was correct and yet it turns into a better interview because I’m adding value to the interview where the interviewer had gaps…
What's really great is if you can get the interviewer talking about themselves. If you're lucky you'll spend the entire interview nodding and saying things like, "wow", "that's really cool. How did you come up with that?". At the end, if it's the kind of interviewer that will talk about themselves for an hour straight they're going to love you, just sat there and let them be the center of attention. The problem is do you want to actually work for someone like that.
This is how I got a job with a bunch of really smart folks in the early 2000s (they were all MIT grads and I was a graduate of a state liberal arts college in NY.) It was several rounds of interviews and a couple of the folks interviewing me (who clearly liked me) warned me that the interview at the end would be the hardest because the interviewer was an old non-nonsense engineer (MIT CS degree and MIT MBA.) It was intense for sure, but I had read somewhere that (as you stated) if you can get them to talk about themselves and engage them: A) they will love you and B) you'll run out the clock. I can't recall how I got him to talk about 'the good old days' of programming, but I did and I don't think I had to answer more than a handful of questions in that hour.
I suppose in my case he might have been less effective at assessing my technical abilities (there were multiple technical interviewers throughout the day.)
His feedback read like this: “Good culture fit, good soft skills. I really liked them. They seemed to understand and appreciate difficult and interesting topics.”
I assure you there are worse ways to behave in an interview. I’ve passed on people who were smart as a whip but nobody wanted to work with. And nobody wants a dead fish who just sits there waiting for the next problem to solve.
Watch out for this. It is a real phenomenon. If you "enjoy" thr interview, you are more likely to think the candidate did well. And if you spend a lot of time talking about yourself, you are more likely to think the candidate did well. Don't be fooled. Have a rubric that you can objectively measure the candidate against.
I agree that an interviewee should be an active participant (though not being is fine too given the state of interviewing), however attempting to drive it themselves can really backfire if the interviewer isn't expecting it and/or is bound by policy to actually ask certain questions or go through some process. I still remember one guy who wanted to drive things a bit too strongly, which for most of my section was fine with me because I was still able to get the data I wanted out of it and hopefully he got some of his, though at a point I really insisted he actually code a working solution to a problem in the IDE setup for him (requires like 15 lines of code, it's embedded in the context of a larger program) rather than continue to whiteboard/pen-and-paper general thoughts. He did, and I gave him a positive recommendation. But in the after-meeting my coworkers were less happy with their sections (one of them even brought up the sense he was trying to 'run the clock'), and some of them were still new to interviewing and didn't know how to handle a more 'aggressive' candidate and apart from not enjoying the experience couldn't get the info they wanted, so the eventual decision was to say no to him. I still think he would have done fine, but hey, it's just another element in how broken our interviewing practices can be.
The interviewee doesn’t know what the day to day job will look like, so they may wind up talking about entirely the wrong parts of their skills & experience. The interviewer has a much better idea what they need to learn about the candidate.
Perhaps this is less of a big deal for a generic position that’s the same at every company, but if you are matching someone to a very different role than they’ve held before, you may be interested in only certain pieces of their resume.
Walter you (with your credentials) do realize you would never go through the "standard" interview loop like the rest of us nobodies would go through right? We are lucky to be forgiven for only solving 2 LC hard problems instead of 3+ in an hour whiteboard coding interview!
I didn't have any credentials at the time, which is why I felt the need to be proactive in the job interview.
I also interviewed at Microsoft (just down the street) a few years later, and got the standard interview treatment.
I know what quicksort is, and I've implemented it more than once over the years. But I could not implement it off the top of my head at the moment without doing a quick refresher on the algorithm.
I was once assigned the task of speeding up a program written by <famous programmer you've heard of>. In perusing the code I happened to notice that there were two custom bubblesorts buried in it. I replaced them both with calls to C's qsort and got a nice boost. Was <famous programmer you've heard of> a fraud? Not at all. Anybody's code can be made better. I've been found guilty of having hidden quadratic code many times.
This is very insightful. I'd also add that having such people let loose to do interviews without training and checks and balances also is a huge red flag about the org.
I hadn't done many interviews before starting at my current job, where they seem to make a point of getting a lot more people involved in the interview process. I guess this is an idealistic view, but it should be pretty clear if you're a hiring manager whether the people you're getting involved in the interview process are likely to be jerks to a candidate.
What you describe is embarrassing, and does reflect poorly on your company. Review sites like Glassdoor do or did have interview reviews as part of their company review, and people do go look at those when deciding whether to apply at a company.
I hope you shared your feedback with your coworker, and also the hiring manager. That coworker is probably not ready to do interviews and needs some training (or maturing) first. In the future, you might interrupt the coworker and then try to help the candidate along; maybe they won't pass the interview, but you don't have to let them leave feeling like a failure, and along the way you might get some information about where they could fit in your organization. And the entire interview won't have felt like such a waste.
Not many people are gonna risk confrontation with a colleague over some candidate they don't know. Especially not during the interview, maybe afterwards when things are calm.
I'm agreeing with you here but let's stay realistic; some companies have assholes working for them, sometimes these assholes get to positions of power. Usually the company knows damn well employee X has a bit of an attitude problem but...guess he adds enough value (or acts well enough) to keep being employed.
I have 25 years of experience and I spent a year looking for a job, getting rejected after complex, time-wasting tests and endless voice interviews that went nowhere. At one point I had to take a cognitive test which I actually failed.
I eventually found something but it was really discouraging there for a long time.
Companies are really bad at interviewing. They toss the task to programmers and engineers who have never been trained in it, and expect them to not suck.
The best way, as always, is to connect with people you already know and have them short-circuit the broken process via existing trust.
23 years here and at this point I consider interviews to be 1d100 random dice rolls. No matter how much you prepare there is no way you can predict how sensible the interviewers are. I figure just deal with it and interview more. If each interview is 1% independent chance of getting an offer, and you interview 50 times, you have a 40% chance of getting an offer.
You're right, it seems totally random. For one they had the Scrum Masters interview me and they felt I didn't have enough scrum experience. For another I had to try and compile a video encoder on a three generations out of date VM and it took all day and I got $100 for my time. Madness.
This feels encouraging to read. Thanks for posting it. I only have 5 years of experience, but I've gone through something similar and it's extremely disheartening to see yourself failing these convoluted, labyrinthine tests knowing full well that you'd be able to do the job you're interviewing for pretty well given half a chance.
It's tricky to interview people lacking experience, but with >=decade of experience just do deep dives into past projects. You're basically talking to one of if not the authority on whatever it was they worked on, let them tell you all about it, down to the weeds. If they can't, then you've probably got a problem.
I've been starting to refuse 4 hour technicals myself and they all immediately drop me. I don't have your seniority but I have some. As well as a powerful online presence across Github and LinkedIn. At what level of seniority did you start doing this and what is your success rate?
they drop me too, but those companies are not worth working at to begin with
they’re not interested in hiring employees, their interest is to get candidates through the loop and to see you fail
this is fine when you’re applying to FAANG, but a terminal sickness when the company is barely known
rather than spending weeks on interviewing, i’d suggest making a open-source contribution or reading a book
the companies, especially startups have a bigger problem than you - they need to show the growth so they can attract more investors. Therefor, if they’re not getting people, they have harder time growing. If they see you as a potential candidate, but you refuse, they’ll bend their back over backwards
There's not really a seniority level where this happens.
The outrage around the technical interviews shows a lack of understanding for the situation companies face when hiring.
The reality is that something closer to 99% of the pool are applying for jobs they're not qualified for (not the same as saying 99% of developers aren't qualified - there's extreme survivorship bias in the pool).
At company X we would hire ~1/400 applicants who applied, and about ~1/20 referrals. At a smaller company I worked for we hired perhaps 1/200.
The thing is, many of these applicants have five to ten years experience (some at FAANG), but can't code at all. It's simply not possible to trust anything on a CV without verifying it. How do you verify this at scale? A pair programming exercise.
Now the training of most interviewers is often poor, but it's far better to go down this route than the HN alternative: hiring without technical screening.
I'm on your team. up front technical "gotcha" style evaluations feel like they don't respect where I'm coming from, nor my track record of work - and are a cue that the job is not the place I want to be at.
I've gotten nearly 100% of the jobs I've gotten in the past 15 years through friends. Interview process real easy after the informal "backchannel" reference check has happened. Just hi, how you doing, when can you start.
Side effect of this is probably making much less than your counterparts. If you can ace a leetcode style interview, you are much more likely to have competing offers (or retention offers later on) that allow you to make MUCH more. It’s a horrible system but just something I noticed before I got really good at them.
> I'm not fast at reasoning about code, and I often make trivial mistakes just trying to get a "first draft" of a program out. If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.
Sounds to me that the author likes to program by "sketching". I do too! Paul Graham recommends this in Hackers and Painters.
> For example, I was taught in college that one ought to figure out a program completely on paper before even going near a computer. I found that I did not program this way. I found that I liked to program sitting in front of a computer, not a piece of paper. Worse still, instead of patiently writing out a complete program and assuring myself it was correct, I tended to just spew out code that was hopelessly broken, and gradually beat it into shape. Debugging, I was taught, was a kind of final pass where you caught typos and oversights. The way I worked, it seemed like programming consisted of debugging.
>
> For a long time I felt bad about this, just as I once felt bad that I didn't hold my pencil the way they taught me to in elementary school. If I had only looked over at the other makers, the painters or the architects, I would have realized that there was a name for what I was doing: sketching. As far as I can tell, the way they taught me to program in college was all wrong. You should figure out programs as you're writing them, just as writers and painters and architects do.
On the other hand, as the post describes, interviews are geared towards a more waterfall-y approach. I find this frustrating as well, but I think that there is something really important to keep in mind that this post misses, and that most conversations about coding interviews miss. The question is whether something is _predictive_ of job performance.
For example, consider the question "What is your favorite number?". Imagine that the larger the answer to that question, the more likely you are to perform well as a developer. You might object "But I'm never doing anything relevant to my favorite number on the job!". IMHO, it doesn't matter. The goal is to predict who will perform well on the job, and if having a large favorite number does this, it should be incorporated.
Now consider the question of how well you perform on waterfall-style interview questions. Like "What's your favorite number?", it isn't something that you will find yourself doing on the job. But that isn't the right question. The right question is whether or not it is predictive of job performance.
Of course, it seems unlikely that "What is your favorite number?" would predict job performance. And it seems unlikely that if you never code in a waterfall-style on the job, that coding in that style during an interview would predict job performance. But maybe it does. I'm skeptical, but who knows. More importantly, I think that this is the question that needs to be asked. And without strong data, it's hard to be too confident in the answer.
>IMHO, it doesn't matter. The goal is to predict who will perform well on the job, and if having a large favorite number does this, it should be incorporated.
What if it was skin color, how ethnic your name is, or your gender? Did you know the majority of developers are men? Should we throw all the female applications in the trash because they're statistically not as likely to be developers?
Yeesh. Sorry if I sound hostile but... jeez. Correlation is not causation, etc etc.
Things like skin color are different from things like leetcode ability. Looking through a veil of ignorance, I definitely would not want hiring decisions to be made based on the former. But looking through the same veil of ignorance, I would be ok with hiring decisions being made based on the latter.
Well, actually, thinking about it again now, I'm not so sure. But this now becomes an ethical question. Which is certainly important. But I want to be clear that the ethical question is a different question from the question of what a purely self-interested company should do. To that self-interest question, I think the OP and lots of people argue that the answer is "leetcode-style questions having nothing to do with the job and are thus a bad idea".
My claims are that 1) this is the wrong way to think about that self-interest question. The right way is to ask whether it is predictive of job performance. And 2) although I also suspect that leetcode-style questions aren't predictive, it's hard to be confident in that without good data.
> I often make trivial mistakes just trying to get a "first draft" of a program out
In my experience, it's helpful to add simple tests as you go. I have done some live coding tests in Python recently, which I'm not super familiar with, and the first couple I did, I made a lot of trivial errors that caused my solution to fail until I went back and cleaned them up. Luckily the interviewers kept their mouths shut while I was coding, and I was able to clean up quickly afterwards, but I could tell it wasn't a great look to say, "Okay, I _think_ this solution is correct," only to have it flame out spectacularly.
After those first two, I started writing simple tests for everything I did. Not only did it avoid embarrassing fails when the solution was supposedly complete, not only was it quicker overall, but every interviewer I've done this with has commented approvingly about it.
I address this in the post and part of the problem is that this is more difficult on Coderpad, and also interviewers (n=~8 out of 10 or so) discourage writing tests.
Interviewing is really tough. The OP replaced some coding questions with other questions and if he were to interview a bunch of people with them, we'd get another set of blog posts complaining how ridiculous his interview questions were.
"Why is it bad to write a file to the disk one byte at a time?"
What are they interviewing for, low level disk experts?
"Here's a stack trace from an open source project, can you tell me what went wrong?"
Oh sure, print-based debugging is awful, you just have to read stack traces from unfamiliar projects in the blink of an eye.
"Here's a CRUD app, copy existing code patterns to implement a new CRUD endpoint"
Ah yes, the reductive "everything is a CRUD app" - this interviewer doesn't understand that most applications don't fall under the simplistic "CRUD" moniker!!!!
That’s not their point. Point is, it’s really subjective and there is no standard answer. As much hate as leetcode style interviews get, you at least have a standard interview process as that point but yeah don’t overdo it and make sure the loop is varied in the areas it covers.
I am being misunderstood, let me elaborate, point is, there is no standard answer to the question “what to ask in an interview”. They are just giving an example of how somebody will always have problem with the questions being asked and they could be right in their perspective which is just different.
Seriously. I feel like complaining about hiring for programmers is the #1 topic discussed on HN.
I actually feel like algorithm questions are somewhat of an improvement in the programmer hiring process over the alternatives. When interviewing at places that don't have them, the surface area of questions you can be asked is so much bigger.
Like one of the anecdotes in this thread about getting drilled on a testing framework (that the candidate probably added to their resume because they have to have all these keywords to make it through their automated resume scanner and maybe they only played with it for a couple hours one afternoon because they don't use it in their day job.)
With algorithmic questions:
- You're doing one type of preparation for a large amount of potential roles
- There's tons and tons and tons of great material out there to help you prepare
- Probably most controversial but I really believe it: It really does make you a stronger programmer.
For me, doing those kind of questions forced me to learn data structures more thoroughly and dramatically improved my understanding of how to write algorithms and think about performance. This stuff really does matter in terms of code quality and it really bothers me that it's always portrayed as these completely silly puzzles.
The other thing that I see pushed a lot in these discussions is this idea that every company (or most companies) that hires programmers does these exercises. That's just not true. If you're getting coderpad links for every phone screen, you're going after a certain type of company.
What are they interviewing for, low level disk experts?
While I agree that "disk" stuff is "low level" compared to a web developer, it's definitely a very relevant question. You might ask it and if the candidate is interviewing for a web developer position rephrase it after getting a blank stare: Why is it bad retrieving 50 users from a REST endpoint like `/api/users/{id}`, one at a time?
Oh sure, print-based debugging is awful, you just have to read stack traces from unfamiliar projects in the blink of an eye.
That's actually pretty relevant. If you want to be nice, make it something that uses the same stack as you're hiring for but even if not, it can tell you something about the candidate. Some candidates might react like you did in this reply. Not hired. Some candidates might go "oh well, that looks like python, which I'm not familiar with but let's see, ..." and then they try to figure something out from the function call naming and error messages shown. In any large enough company you will definitely be looking at unfamiliar stack traces, potentially from a service you are unfamiliar with and potentially a different stack than you're used to. Happens all the time at my place for example. I can't just throw up my arms and ignore it. Gotta dig in and use generic problem solving skills.
Ah yes, the reductive "everything is a CRUD app" - this interviewer doesn't understand that most applications don't fall under the simplistic "CRUD" moniker!!!!
Or maybe it's a simple test of "OK, so all of the other questions the candidate sort of failed on. Can he at least do 'stackoverflow copy and paste programming'?" Whether it's a CRUD endpoint or something else entirely doesn't matter. The point is: can this candidate work w/ something potentially unfamiliar and still get something done? Happens all the time in any large enough code base. Happened to me just this week where I had to implement something I needed in a old python service we have. I don't like python and I don't know any of the python frameworks used there. I copy and pasted, adjusted and google some basic python syntax and got it done in like half an hour. That's what they're testing for there. Sounds perfectly reasonable to me.
It’s interesting; I assume you’re using “mental gymnastics” in a derogatory sense but I love those questions and I would love answering them. I’m actually a bit peeved that no one is asking me them right now.
> Triplebyte has a quiz that actually does a pretty good job of this; you get asked thirty or so questions about databases, networking, and simple coding questions ("what is the value of 'a' when you get to line 17?").
I could absolutely see people complaining about multiple choice questions and how you're more likely to get tripped up by minor mistakes.
It feels like the other way round to me, can you clarify why you think that a multiple choice questionnaire is more subjective than an algorithmic interview?
But this is also common in algorithmic interviews, correct? Like if I have a sheet that says bubble sort is the answer, and you provide merge sort then you are marked wrong.
The interesting thing about mcq is that their administration is completely standardised so it's more objective in pretty much every sense.
Maybe you prefer algorithmic interviews, that's fine. But I really don't understand how you can believe that they are less subjective than algorithmic interviews.
They avoid the bias of using different interviewers which is massive, even if the questions are standard.
What are they interviewing for, low level disk experts?
Writing one byte at a time may be “bad” at many levels. First, on a hardware level it is inefficient because the unit is a sector/block and the access latency kicks in hard, together with a wear leveling on ssds. Embedded low level indeed. On an os/syscall level it is inefficient because of a syscall time itself. It is a system level touching C and unbuffered writes in “middle-level” languages/libs. On an application level it may be inefficient if a stream has a processing pipeline with some expensive setup/try/teardown (e.g. a jsonl parser which “tries” the buffer at every new chunk). Is this still low level?
This is a question about streaming, buffering and latency avoidance, which you may not know in hw detail, but have to have at least a vague grasp of. If a candidate seems confused, one may ask them about other situations where one-at-a-time may become “bad”: sql select one row at a time, fetch NNN items one by one, use heavy_expr().x, heavy_expr().y repeatedly or in a loop. If nothing comes to their mind, chances are you are interviewing a completely inexperienced person, because there is no way they didn’t meet that or think of that before.
I totally agree that it shouldn’t be the reason to skip a candidate, but hiring is an analyzing process, and how do you find their limit if not asking deep open questions? I think that the issue is pushing hard on these areas as if it was laughable for a professional to not know it.
I'm not sure if you are being sarcastic or not, but I think those are some pretty excellent interview questions.
The first one, it's exactly about the thing I find a lot of developers lack in: a fundamental understanding about computers and processing. I've had 'experienced' developers write algorithms where every intermediate value was written to a cache/disk only to pull it out on the next line. I feel asking some more fundamental questions can help weed ppl out.
Same with being able to look at a stack trace and see where it went wrong, how quickly are you able to hone in on a specific target. Crucial skill imo, and I've had sooooo many people at my desk -- 'it's not working' -- that did not even read the f'ing error message.
I was being sarcastic with the question responses - the point being, for any question you ask in an interview, there is a significant population of candidates that will think it is a terrible question.
At last company we used an online tool to screen junior/intermediates and the whole point of that tool was "Here's the task, here's the tests that need to pass when you're finished, feel free to write more", and someone writing more unit tests to exercise their code was a big positive.
Asking someone to implement stuff without running it sounds counterproductive to me. About as effective as asking them to write syntactically valid code on a whiteboard.
The author is focused with technical review, and as such he probably fails to see just how important personal review is. Yes, you are being reviewed technically, but you are also being reviewed personally. You may know a lot of technical things, but do I want you on my team? The phone interview is what i'm going to use to make that decision. Technical we can work over later, but only if you sound like someone I want on my team.
I'm always willing to say, "They look like a great fit for the team and our goals, and let's give them a chance to learn the tech" versus, "They know a lot of technical stuff and let's give them a chance to see if they can learn to work with other people."
I worked as a sole proprietor/consultant for several years, which involved lots of “personal review” during the sales process. I am well aware of how important that is.
> "They look like a great fit for the team and our goals, and let's give them a chance to learn the tech"
I wish more were like you.
I've definitely had my fair share of interviews that bombed so I feel like I have a good bearing of when I don't leave a good impression, but it's always the worst feeling to have an interview that you think went well based on the overall mood/feeling, only to receive the eventual rejection later in the day that they went with someone more senior or that they wanted someone with more experience in x.
My mentor who didn't got to a well-known college once told me he joined a very solid company not knowing anything about the frontend framework they used but they gave him documentation and let him learn it for weeks without touching the code base. He said it was honestly one of the strongest engineering teams he's worked on in his career. The idea of managers seeing potential in someone and thinking "let's go with that person who has never done x" seems so foreign to me.
Re: your last paragraph, I feel like that's symptomatic of a change in the relationship between capital and labor that has happened to some extent at every level of the market.
Companies are increasingly unwilling to invest in the wellbeing or value of individual workers, instead treating workers as fungible assets to minimize the cost of which serve as solutions to current (rather than long-term) problems. We see this with the near-universal embracement of short-staffing in service-level jobs, lack of career mobility from starting positions, and even in the tech industry with the expectation that you usually have to switch companies to get a raise.
But the author is specifically complaining about the technical phone screens using vastly inferior tools and processes than what you work with day-to-day. Being asked to write code on demand in a notepad with no way to test or compile is not a personal review, it is a monkey show.
Your code on github is pretty slick. I love the documentation.
Leetcode is an exam that you can study for. For me, it's pretty damned hard and I dislike it.
I can think of three reasons why someone might be good at those kinds of problems:
• You just have a knack for it.
• You are very intelligent.
• You practiced a lot.
Which one of those can you do something about?
Every time I force myself to grok a new leetcode trick, I've truly learned something. Something irrelevant in my ten years of programming, but allegedly an indicator of someone who might be good at what we do.
I have sympathy because I program in a similar way as the author. Not so much with tests but with some kind of fast feedback loop where I do lots of guess-and-check. I don't know how I'd do in this kind of interview.
However...
> If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.
My sense is that 90%+ of these interviews is maintaining your composure, asking questions and generally coming off as well-adjusted and personable even if you don't know the answer or get stuck.
> Certainly there are a lot of confident, mediocre tall white men in the tech industry
Companies are missing out on tons of great, talented individuals who are frightened to endure Leeetcode-like interviews. Are there any firms that give candidates choices on how to demonstrate they’re a good fit for open positions? Maybe you do like grinding Leetcode algorithm questions, maybe you want to create a new service that satisfies requirements and discuss the design choices, perhaps you’re really good at contributing to existing projects.
yeah and if you are an anxious person like me, (i may be on the spectrum, i have some social issues), writing code in front of people makes me very anxious, it takes me back to my childhood when my dad would scream at me for being dyslexic, i got my b’s and d’s mixed up.
i know that is not what is happening during an interview but, it’s how i feel. and this prevents me from getting ahead in an industry i am madly passionate about
I'm really sorry to hear about those traumas and your current difficulty. Shoot me an email, maybe I can help: adamzerner@protonmail.com. https://adamzerner.bearblog.dev/ if you want to internet-stalk me first.
I think it is fortunate that great engineers like OP are rejected in Leetcode style interviews. The benefit is twofold:
- It makes fresh graduate noobs (or even just noobs in general) able to compete on equal ground on high-paying job prospects. Think about it, if every high paying jobs out there value experience over leetcode style of interviews, then young noob programmers won't be able to rake great salary because all of those jobs are filled already by experienced engineers
- Great engineers should just use their talent somewhere else in startups, create more innovation, create more wealth rather than helping the big tech giants become bigger
Not so great if you are jobless and got bills to pay. “Create more innovation” is a good thing, but sometimes people need to cash those big tech giant’s checks to care for their family.
Experienced engineers and college noobs should not be competing for the same jobs.
Tech co’s seem to prefer the college grads for rank/file for many reasons. Maybe because they are easier to exploit?
Heh. I have 25 years experience and I fail phone screens too. :-D
It's fine, though. TFA goes into ways to improve the hiring process such that this sort of thing doesn't happen. But I don't mind it at all. I want to fail fast; if the company decides I'm not the right candidate, great, I'm glad it happened before we wasted time with more extensive interviews.
There are two possibilities. If they're right and I'm not a good fit, I don't want that job because I won't be successful there. If they're wrong, I don't want that job because the company isn't being successful at hiring me. The hiring process is looking for the wrong attributes, or the interviewer isn't properly trained, or... well, something, somewhere has gone wrong. There are no false negatives.
A company that's firing on all cylinders does a lot of hiring, and is good at it. They source solid candidates, and interview them thoroughly but quickly. If they're not hiring effectively, well, they're probably dysfunctional in other areas too.
Totally agree with this. One may want a Job at a prestigious FAANG or a hot startup but may not clear for culture fit whatever even tough you answered all technical questions. I say don't despair. just as you mentioned even if you got in it may not have been a good team/good manager. We should strive to find teams/managers who can appreciate what we are and do and vice versa.
Today's interview process seems eminently gameable by grinding LeetCode and watching YouTube videos on system design. In the last 4 months of job searching I've gone from barely being able to get past the question (reading under time pressure stresses me out), to convincingly passing about half of my recent phone screens. All I've done is one LeetCode per day and interviewed a couple of times per week. I'm not fundamentally any better of a software engineer than 4 months ago. The current system is great for zero-knowledge people with the time and mettle to study, and awful for accomplished but time-poor senior engineers.
And, why all companies have decided upon almost the exact same methodology, I can't understand that either. It seems to be the same double edged sword as the Common Application for college: you get more applicants because the incremental effort required to apply to college n+1 is low, but it drives down college admit rates as well as matriculation rates: the signal to noise ratio probably goes way down.
> Today's interview process seems eminently gameable by grinding LeetCode and watching YouTube videos on system design.
That is a feature. You look at what they worked on before to see if they are senior material. Then you do leetcode and system design to see if they are smart. A smart person would most likely learn whatever tech and problems they worked on, so if they do well on leetcode you just assume they did well in their past jobs. If they can't learn leetcode then you assume they did poorly in their past jobs. If they didn't practice leetcode then you got no signal on their smartness, so no-hire due to unpredictability.
So we get these possible results and what you do with it:
- Good leetcode, great and relevant experience: Hire at high level, this guy is smart so he likely learned a lot of things during his past experiences.
- Bad leetcode, great and relevant experience: no-hire, he might be smart but we don't know so we don't know if he actually performed in his past jobs.
- Good leetcode, no/minor expereince: hire at junior level, this guy will likely learn fast
- bad leetocde, no experience: don't hire, there are no positive signals.
This works really well actually. The only bad part is that some experts needs to brush up on this when they want to find a new job, but for a well paying company that isn't a problem as millions of candidates gladly brushes up for the pay bump.
As a former director of engineering who has hired many great developers and many terrible ones, I can safely say that grinding leetcode isn't a valid signal for smart, just for conformity, ego, and time. Some of the worst developers I know could crush leetcode all day.
Yes, it's an easy signal, it makes an unpredictable and messy process feel neat and binary. However, it is a false signal, and no amount of personal comfort changes that it has about as much effect on future employee productivity as hiring by height or favorite color.
Unfortunately, software productivity is never going to be measurable. Therefore we will never ever have a reliable metric to even prove to ourselves that we are more or less productive than our coworkers, much less worth hiring. This fact should sober our prideful arrogance about hiring and introduce some humility. Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
We need a heaping dose of recognition that most of our best practices are there to validate the egos of the interviewers and give a neat binary decision without any mess. We have a toxic fraternity hazing ritual to make candidates bend the knee, with exactly zero evidence it works.
We have the gall to call ourselves scientists, and we don't even measure a single thing.
Want to add agreement that grinding leetcode is a terrible metric. The ideal candidate probably already has a full time job and if you add in a family and other responsibilities, they don’t have time to focus on leetcode. Heck, they probably are tired of thinking at work all day and don’t have the mental capacity to task switch into leetcode after work (I know I don’t!)
> Unfortunately, software productivity is never going to be measurable. Therefore we will never ever have a reliable metric to even prove to ourselves that we are more or less productive than our coworkers, much less worth hiring. This fact should sober our prideful arrogance about hiring and introduce some humility. Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
I think there is something similar to art in GOOD software development, as somebody who does both. The primary reason I don't like coding for a living is the same reason I don't like writing fiction for a living: I really dislike being creative on demand and on a schedule. I find it stressful. I can't make myself get a story idea any more than I can make myself find an elegant solution to a coding issue. Sitting and staring at the screen more isn't going to solve the issue for me.
Software development is to math/CS what painting is to color theory. There is science, but there's also creation. That's why it's so hard to measure and hire for. Imagine trying to judge 1000 newly graduated oil painters who had all taken the exact same classes. How would you decide who was 'best'?
I think the opposite is true, we're doing the same thing doctors and lawyers do: artificially restrict the supply, driving up demand and pay. Not top down and centralized, but individually and decentralized. There's plenty of jobs, almost every company reports a 10% shortage, but the developers won't let in anyone.
> Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
Google has data on this, when I worked there anyone working there could view it. Performance on their technical interviews is the strongest signal they have that someone will perform well at the job. It is stronger than every other signal. They have tried a bunch of different things, Google HR hates these technical interviews since it is hard for them to game so they work hard to remove it, but they can't since they can't come up with anything with a stronger signal.
Performance on a single technical interview isn't a strong signal, but the combined performance on 5 interviews is a strong signal. If you only run a single of these instead of many with many different people then you will hire a lot of bad people anyway. Also if you pay less than Google and similar then you will get their rejects, there is probably something wrong with the people who are good at algorithms but get rejected by FAANG, not sure why you would want to hire those.
I'm very interested in what else they tried. Also very interested to see the research, is there somewhere it is published?
I do believe that performance in a technical interview corresponds closely with overall performance, but I'm very curious how they measure job performance, considering tech job performance is mostly a subjective popularity contest. I'm not convinced that is a valuable signal either. This is what I was getting at, we can't measure productivity _of employees_ with any rigor, so how can we know if our interviewing process works. Since job performance is so subjective, it's not very surprising to me that people hired with a different process aren't as well liked. The one thing I know about people is they like people like them. People who can and will grind leetcode are a very similar group indeed, so a person who didn't do that likely sticks out like a sore thumb.
Perhaps all this is accounted for, in which case I'd love to see it.
They base it on performance reviews by the managers. The mangers aren't the person rating the candidate interviews in the first place or deciding if they get hired, so it isn't just managers confirming their own hiring ratings. It isn't perfect but that is about as much as you can do, it is really hard to create a better model for evaluating hiring processes.
I do believe you can do better than Google. But I don't believe you can do much better than Google at Google's scale with that many people if the goal is to hire people who can solve hard technical problems. In a small company you can have the founder or equivalent do the hiring, like early Google did, Larry Page reviewed every single candidate until they had thousands of employees. This is what they came up with after that. Also now instead of having Larry review everyone, they have the vice presidents review every hire in their org as an extra last step to ensure the hiring process doesn't turn bad or get gamed.
Note: If you don't pay like Google, or your goal isn't to solve hard technical problems like Google has great need for, then you probably shouldn't try to hire like Google. Also if you just run a single algorithm question that is easy to look up online, then you are not copying their process, since Google forbids such questions and require multiple questions. Also if you just run an automatic algorithm challenge online then you miss all the soft skills part that comes up when you talk to a person when solving the problem, so not their process either. Also a person who isn't at least decent at algorithms cannot administer an algorithm interview properly. They need to come up with an original problem, and they need to identify when the candidate has solved the problem even if the candidate did it in a way you didn't expect. Most engineers at Google can do that since they hired for that from the start. Most engineers at most companies can't. Also if you have managers make the hiring decision instead of an independent committee then you aren't copying their process. And if your executives are not constantly checking that the process is in line with expectations and doesn't bring in a lot of bad candidates then you aren't doing their process. Etc etc.
They do so many things for this to work, if you do all of that and wants the same thing as Google then it will work for you as well. But almost no companies wants to do all that work to get this working. They just say "Google does algorithm interviews, so go and ask the candidate an algorithm question and that is our process".
Also some caveats about this process and why you hear so much bad things about it:
- Managers at Google don't like it. They would prefer to hire the candidates they want, but as is it can be hard to even fill headcount you have been allocated if you aren't an attractive team, they would love to lower the bar to fill their seats. And the few people unattractive teams manage to hire often leave the team as soon as they can for better teams. I did that, I was first on a dying project and then I did well and joined a cool project working on cutting edge machine learning infrastructure. So this process isn't pushed by managers. It is the executives who want a well run organisation who wants this process. Which is why the executive review is mandatory, this process is first and foremost for their sake. And from heir perspective, hiring really smart people, put them in a poor project to test their ability and then get the best of those into the teams that matter is how you get great engineers to those important teams. There is no reason to lower the bar for poor maintenance projects even if those managers would love to do it.
For Google the most important thing is that their search and advertisement algorithms become as great as possible, and that their infrastructure costs gets optimized down as much as possible. They need the best engineers in the world to work on those things becaus...
> even competitive programmers says that there's probably negative correlation between good engineer and competitive programmer
They don't. I've worked with some of the best competitive programmers in the world, they do think it is a positive and look for people with a strong competitive programming background.
However, if your interview process only selects for ability to solve algorithm problems, then people with a lot of practice solving algorithm problems will over perform relative their level of talent, so when you run a dumb regression analysis it might look like competitive programming experience is a negative signal. So if someone has competitive programming experience then you want to bring them in for an interview, but you should probably be harder on them in the interview than regular candidates to account for this effect.
>They don't. I've worked with some of the best competitive programmers in the world, they do think it is a positive and look for people with a strong competitive programming background.
It's written in Polish, but as I see that was about kinda different thing
>How were SPOJ and programming skills useful in the context of your studies?
>Virtually not at all. I will say more - these skills were an obstacle in obtaining the title of engineer.
><rant about higher edu institution>
>...
>How useful are SPOJ and specific, classic algorithms at the programmer's work?
>Wandering around the labor market, I got the impression that it was easier to become a programmer with a vague idea of algorithms and extensive experience in creating specific projects than being a master of all kinds of algorithms, but without experience in coding something bigger.
"Bad leetcode, great and relevant experience" - that's me after 20 years in the industry and at least 30% of that time working on very non-trivial systems. I usually don't do actual leetcode actually, I review algo books before interviewing. In the last few years it stopped working for me even at no-name companies. But I find it nearly impossible to force myself to leetcode, it feels humiliating and stupid.
Another thing I noticed is that they stopped asking questions about concurrent code or writing recursive-decent parsers. They seem to be strangely fixated on a narrow and not particularly practical subset of CS body of knowledge. We don't discuss OOP or its alternatives anymore. Companies don't seem to care about my actual hard-won real life experience even when it directly applies to the kind of domain/system they are building.
Don't get me wrong, I expect people to know how to perform depth-first traversal or the general idea behind quick sort. But we are long past that stage even in small companies with average pay at least here in the SFBA.
Quite honestly, if the job description says "we are looking for the best of the best" or "we are looking for a talented java developer" I know this job isn't for me.
Same with algorithm questions, if the interviewer asks me an algorithm question, I will switch off and politely put a cross that this role isn't for me. I don't have cs background and my CV is clear that I've not done anything like that.
Edit: I am not a great coder by any means. I just don't like to waste my time when it's clear I am not a fit for the company or the role.
I preface my interviews saying that I'm we're going to have a conversation rather than a list of trivia questions. The goal is find out what the candidate knows, rather than if they know an answer to any specific question, or specific version of a technology. If I ask a question that is too easy, it's not a trick, it's a segue to more conversation. If I ask a question that's incomprehensible, it's because I don't have a prepped list in front of me and I'm making some questions up on the fly, and the candidate should just ask for clarification. Also, that a lot of my questions are open ended, or have multiple right answers and any of them are acceptable.
It's not really a pattern of interview I expect most people can implement immediately, but it really gives a good idea of fluency. I just have to home in on what they do know. The easy questions get some momentum going and provide context for the follow ups. When they don't know an answer I'll tell them the answer. Sometimes this leads to them adding some depth beyond that answer as they know what's going on but haven't worked with it formally, or have worked with it in passing, or maybe know it by a different name.
I've only ever done this style of interview and I've never hired someone who was downright awful.
I also suspect it makes candidates more likely to accept because you can develop a bond and have a relatively enjoyable time in interviews like this.
Several times I've had the interview process be so unpleasant I lost all interest in working for the company. I try to be considerate and withdraw so I don't waste everyone's time but once I did get an offer from a company like this.
Very much this. Leetcode is terrible, but the checklist of trivia suggested in the essay is equally terrible, because it's too easy for a genuinely good programmer to simply not have experience with topic X. Or, for more experienced candidates, to not have dealt with that topic for several years and not have the knowledge fresh.
Sure, Ruby on Rails. I literally start with something like:
"You've just created a new Rails app and you change into that directory. What are some of the directories and files you find at that top level?"
"What do you find in Gemfile? Gemfile.lock? What's the difference between the two?" Maybe some discussion of resolving dependencies/versions.
There are numerous "right" answers to which files/directories are at the top level and I don't care which ones they get, but they'll get some and it gets the flow going. Most of them can be taken to greater depth. If I want to guide it away from views into models, at least they know the level of detail we're working at.
Rails is a problem interview, too, because there is so much which is take it or leave it. There are Rails apps with no front end, and someone may have not worked with views for years, or the backend may be Api based and they haven't worked with models in the same way. I've worked on a project where 95% of the work was in the workers, and the user facing code was just nudging things between work states. So I use this to guide the conversations and draw out examples of their experiences when possible.
339 comments
[ 3.2 ms ] story [ 185 ms ] threadJust looking at this guy's repo and his writing style -- you'd be foolish to waste his time or yours on leetcode-style hazing. Your conversation should be strictly high-level. If you must, pick a file at random from one his larger projects, and ask a few straightforward quetions ("I'm new to Go - can you explain why I would use this datastructure here?") just to make sure the repo isn't a copy of someone else's, and he isn't outright trolling you. Apologizing as you're doing so, of course ("Sorry, but do you mind if I ask a silly-sounding question or two about something in your repo?").
That's all you need to do. Most candidates won't have work samples that are quite as slam-dunk, of course. But still, your own "smell test" can save you tons and tons of time (and the candidate, agony). It's really, really hard to fake one's way through questions like these -- and really easy to tell if someone is trying to do so.
Put another way, your basic mindset should be: "Trust, but verify". Not: "Joel Spolsky wrote that post 20 years ago claiming that 99.5 percent of all candidates are utterly incompetent. Therefore I need to torture this guy until he confesses."
At least as far as answering the "can he/she code?" question is concerned.
Looks like he has ADHD. Maybe somebody with more time can go through his blog post line-by-line and figure out what's going on.
Most of my Coderpad stuff compiles and runs the first time, so I don't use his incremental test method, or see a need for it. Maybe he learned programming on heavy-weight IDEs and got trapped mentally in those?
I'm just like them, only my ide is notepad++.
My code never compiles on first attempt, and never runs correctly on the first attempt.
I’m very capable of focusing when there is a lot of money on the line.
I also use vi (no syntax/color highlighting, no plugins, original vi commands only.)
I think you should talk to some other developers and see what's going on with your ability to focus. (I'm one of the top application developers in a particular language and can actually think in that language, so that level of familiarity helps me.)
That sounds ad-hominem. I found the post perfectly readable.
Maybe he learned programming on heavy-weight IDEs and got trapped mentally in those?
Maybe he just thinks differently than you do? His body of work is not suggestive of someone "trapped" in an IDE or other dysfunctional midset.
I wasn't referring to his blogging style, I was referring to his description of his "struggle" development process.
> Maybe he just thinks differently than you do?
He certainly does. He might want to find out what's going on by talking to other developers.
One of his comments in this thread says he uses vim, so it's not an IDE issue.
As someone who had courses in C on paper, i can only say that it's a colossal waste of time. Tooling exists to make lives easier, gain time and productivity, why avoid it?
It'd be like saying only the farmer who can do everything by hand without any tools increasing their productivity is a real one. Or an accountant refusing to use Excel/calcuators/ERPs and writing everything on paper.
And before anyone jumps to conclusions, i often write small scripts with vim with a minimal setup, but when I'm doing anything complex i prefer to use an IDE
You're trying to map manual work onto engineering.
And absolutely I'd rather work with accountnant who can write me on paper without Excel/calculator/ERP his logic.
It's like you're being obtuse just for the sake of the argument. Someone who can clearly express themselves without specific medium, be it paper, IDE or whatever is strictly superior to someone who is tied to one specific medium.
How does it actively helps and avoids small human errors in reasoning.
> I'd prefer the person knowing using it instead of sticking an old and imperfect method out of pride.
What does even pride have to do with this?
Because reasoning is only a part of everything to be done. An IDE can help with typos, best practices ( global variables, lack of error checking, etc) which are tedious things that take time. Isn't it better to the things that matter instead?
Which applies to everyone? Take IDE out of equation and what do you have?
Along with (as was the main point of the original blog post) probably quite a few false negatives, Like the author of that post, for example.
Meanwhile, you can achieve the same filtering benefit by far less obnoxious and more accurate means (a cursory glance at their repo, fore example).
Everyone likes to think they are the good candidate and not the bad one that slipped through…
We have flat hierarchy. Just volunteer to interview because your voice is on the same level as others.
When your organization can recognize a poor investment but cannot act on it, you have an organizational problem. Why would someone get to hire more "bad" (for whatever that means) if you can recognize it? Even if you're doing all the hiring yourself, it's unlikely you'll have a 100% record over time.
Right. But we can't talk about that, of course.
Instead the corrective strategy seems to be: "Let's torture some more candidates. It's cheap and it's easy, after all."
For another, even if "an organisational problem" or inconvenient labour laws don't get in your way, firing employees can have a serious effect on the morale of those left behind. And for good reason! Usually companies that regularly fire people for poor performance has a different sort of culture than companies that don't. In my experience, that's the worse organisational problem.
The right solution is just to be more careful about who you hire in the first place.
So if someone lied their way through interviews, in the first couple of months you should be able to detect that and terminate their trial period.
Managers hates firing people, it is really uncomfortable. So instead of firing this person to look for a new, they keep the person and say they need even more headcount and hope the next one is better.
It doesn't have to be a public repo of course. They can just send zip file.
You need an equal comparison across all candidates.
"We'd like to see a code sample -- something we can build and compile, that you feel is representative of your skill level."
Sounds pretty straightforward and egalitarian to me.
Long before open source was a thing, and repos were even thought of -- I always had a few 1,000+ line personal projects I could dig up and show people.
1,000 lines isn't that large, btw - but yeah, maybe 500 would be a better number.
With Leetcode, everyone starts on equal footing (there's no bias towards people who code in their free time)
Yeah, it's extremely irritating as a candidate but it's something
How so?
> turnover rate on your team like?
2 years since I joined, hired one dev recently, no one left so far.
>usually being close to runnable with a few small bugs was a good submission and runnable with no bugs was a great submission
This is exactly what we do: check the solution, and if it's close we continue the hiring process. There is a huge difference between an almost working solution and what you get 70% of the time: code that will never, ever, ever work, even if it could run.
A lot of hiring managers have no concept of experiment design.
You can’t invent a measure (or copy it from a blogs about how other companies hire), look at who it rejects, and conclude they are bad. You don’t know if they were good or bad. You never worked with any of them. You only know that you applied your measure.
The only candidates you have any information about are the ones you hired, yet you are insulting the others on the internet today
What's interesting is that the people doing all this "weeding out" seem so sure that they're bravely defending the fort against an onslaught of incompetent hordes... and yet they're missing this extremely basic point about the nature of the process they've chosen to use.
I'd say much has changed since he wrote that, and even for the time it contains some odd advice which he tries to force down the throat of the reader, but I'd be more interested to hear other's thoughts on the article.
He also had this, what seems to me, nightmarish, dev. speed and progress estimation tracking app he made. I don't remember what blog post, but he described it and jokingly wrote that Lisa needed to get the speed up, or something. Fast forward to today and it is a predecessor to our agile nightmare. But I don't think that was his intent.
https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
The general problem with Scrum like velocity tracking is that devs will start to lie to hit the estimates to keep the commissars of their backs. The best liar gets the straightest graph. And as devs learns to lie, you get the impression of improved estimation.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better."
I mean ... why God. I don't know but maybe poor Milton has the hardest tasks, does not cut corners or what not. Brandon seems to "get it" and has silly narrow error range - I wonder how ...
(I am not bitter. Pinky swear)
Perhaps it's just me, but I took a look and I'm not convinced. Nothing jumps out to me as overwhelming proof of programming ability.
I see that on paper they're a great candidate, but the reason that companies do these tests is that often the candidate doesn't measure up to the resume
Are you suggesting that leetcode tests provide "overwhelming proof of programming ability"?
Leetcode and coding ability is an A -> B but B -/-> A situation: failing leetcode doesn't mean you can't program, but the reverse is absolutely the case.
Leetcode can't filter out other potential issues that may impact success (e.g. poor anger management, or poor judgement).
I should perhaps be clear that by "overwhelming proof of ability" I meant "the ability to program well". Which by definition does intersect with one's capacity for judgement, on a certain level. (Though is largely orthogonal to other behavioral issues, I find).
I see LC as providing proof of, at best, minimal ability. And I've seen plenty of people who can run circles around LC-style tasks, but when it gets to the bigger picture -- you know, writing good, maintainable software -- somehow, it just never comes together right.
We agree that obtaining proof of minimal ability is important. I just think that proof can be obtained in other, far less mutually degrading (and error-prone) ways.
When looking at this person's Github page, I -unlike you- see a vast number of trivial projects that can best be described as regurgitations of other people's work. Even worse, if you actually look at his first pinned project (https://github.com/kevinburke/nacl), you'll see that it is nothing but a wrapper with trivial changes/updates... an anti-project.
In the end, I see code that provides the illusion of competence but, when actually inspected, is actually telling me not to hire this person. The fact that pretty much anyone can declare oneself "a software engineer" today and easily present a veneer of competence together with powerful financial incentives, makes it absolutely necessary for any serious engineering team to have objective measures in place in order to evaluate candidates.
There are so many terrible engineers out there (and some even slip through the cracks and land at places like FAANG) that doing anything less is irresponsible.
But rather, if you're looking for a basic smoke test which can answer the question "Can this person actually code? Do I want to spend more time engaging them?" -- there are better ways to do that than expecting them to cram on a list of algorithms to recite over the phone. Like looking at an actual work sample.
The fact that you looked at this work sample -- and came to your own conclusion about whether they would be worth engaging -- validates this very simple and basic point.
Which is your call of course. He may not meet your bar, but it would be absurd to say this guy can't code, and hasn't been around the block in terms of general CS concepts.
That said -- my own subjective opinion is that you're being exquisitely savage in your assessment of his NaCl implementation. Yeah, it's wrapper -- like it says in the Readme, and like a lot of working code out there. It's a pretty established way of solving a problem, in fact -- in some cases a very efficient and elegant way. It's not the same thing as writing 20k of primitives from scratch, of course. But that's not what he's presenting it as. He's not saying it's a "tour de force". He's saying "I think this shows we can talk about other things besides FizzBuzz".
And then other things you're like saying... like "an anti-project"? Wow.
This is also what a lot of those engineering interviews are trying to ascertain, in a quantitative manner. I couldn't care less about someone's degrees or ivy league schools or Github projects (in isolation). These are all secondary -if that- considerations, problem solving ability being primary.
Using the example you disparaged (his NaCl implementation), I would say:
"On first appearances, clearly yes. The problem was to adapt the a set of standard Go functions to a foreign interface - something one does quite a lot in an engineering environment. Also, the fact that he's aware of projects like NaCl and seeks to learn from the likes of Bernstein and Lange is a positive signal."
If I wanted to, perhaps I could drill down into the code and/or the NaCl spec itself. I don't know Go, so for all I know, maybe his code is actually horseshit. But all I'm saying is - from first appearances, definitely a positive signal. Infinitely more informative that FizzBuzzing (which would insult both his intelligence and mine).
This is also what a lot of those engineering interviews are trying to ascertain.
I've already made it clear that a glance at a repo does not obviate the need for an actual interview.
I guess in your eyes I should have also done the cryptography, however, I am not a cryptographer or a mathematician and one of the first things that you learn when you start doing work with cryptography is that you shouldn't roll your own crypto.
I decided to make that library prominent because in my consulting practice I saw a lot of clients using cryptographic primitives that were clunky and/or had a lot more overhead, for things like cookies or public key authentication or just sharing messages. nacl is difficult to use incorrectly, and is fast and efficient in terms of overhead. I wanted to promote use of nacl more than the work that I was doing for it in particular.
For that client (a smart lock company), I rewrote the backend, that someone else wrote in C for a specification that did not match the needs of the business. The C software was extremely difficult to read and maintain - socket writes in a callback in one part of the code and socket reads in a completely different part of the code, shelling out to curl, simple Mongo queries that would take 50 lines to set up. As a result the engineering team could hardly make any progress deploying new features. I ended up rewriting the whole backend in Go, a few thousand lines of understanding what the C code was doing, and porting it, to the point where the team could actually release patches and new features on a regular cadence, and figure out what on earth was going on with the Mongo queries and shell-to-curl.
That's not public though, if you're looking for public work, you could look at Logrole (github.com/kevinburke/logrole), a HTML client for your Twilio call, SMS, phone number, and recording data. Of course, you could say "that's just a regurgitation of whoever did the API client", which is true-ish, but I also wrote the API client (github.com/kevinburke/twilio-go). Of course, you could say "that's just a regurgitation of the Twilio API," but I also helped build and maintain the Twilio API, as well as the documentation and information architecture (https://www.youtube.com/watch?v=sQP_hUNCrcE), and the client libraries (https://www.youtube.com/watch?v=C_UJHqR_2Mo).
Most recently I was hired as the first engineer at Meter (meter.com) where I built out the entire backend API, the deployment infrastructure, the CI/CD infrastructure, a pipeline for the in-the-field servers to update themselves, and a method for access points to retrieve configuration from the backend. All in all this was probably about 30-50,000 lines of code over two years. I think the API has had one customer facing error in two years and no downtime.
It is true that anyone can call themselves "a software engineer" but not everyone can get contracts for a sole member consultancy and, crucially, get their contract renewed multiple times at every single client. If I was selling snake oil and not delivering anything, that would not have happened.
Could be how he describes the problem. Most of the time during code screens they don't want you to perfectly solve it. Talk it out. Someone who feels that perfect or bust is the only way to go will fail most of the time. Idk feels like something is missing. He's right, feedback loops suck.
Question is... Can he practice doing a phone screening with a friend? Does he know anyone who interviews? Does his current job allow him to interview or sit in some? Just being part of it can give great interview experience.
Around 10 years of experience is when I learned a ton of interview skills. Interviewing is indeed a skill in itself.
I personally hate this approach. Solving programming problems requires my full focus and vocalizing while trying to think is a challenge (partially due to my stutter). I can either focus on communicating clearly and professionally or I can stfu and code, not both.
I’m happy to solve a problem silently and talk about it after I’m done, but I won’t do a phone screen that asks me to word vomit in the middle of coding.
If you sit there for 30 minutes banging out a solution in silence, I get a bit of signal. If you sit there for 30 minutes in silence doing nothing, then you're stuck and you've failed the interview as I didn't get the opportunity to assist.
Talking through is so important.
There is no situation where a boss call two random engineers, throw a completely new problem at their hands and ask them to solve it in 45 minutes.
I don't know what to say to this beyond, absolutely yes this situation exists. The easiest scenario is you are on call and get paged and must solve a critical issue in as few minutes as possible.
Even if we think we see a way to solve an outage with something like this, we generally consider it inappropriate for an outage mitigation scenario, always preferring to rollback / flip feature flags / add capacity / other simple and minimally invasive interventions.
Implementing a better or more scalable or more resilient system is important, yes, but is done after the issue is mitigated... not at gunpoint.
Sure, that's nice, but you're living in a fantasy if you expect that to always be possible.
I also didn't say anything about implementing scalable or more resilient, or even algorithm puzzles. That's not what we're discussing.
I'm talking about communicating when problem solving. If solving an outage was always a mundane rollback or simple intervention, then it should've been automated. For example, say a new deploy corrupted the database subtly. How do you recover it? Perhaps you rollback the deploy, create a new database instance from a backup, perform a cutover, lock writes, write a migration to copy records from the old instance to avoid losing data written since the backup, validate the restore, unlock writes, monitor. You probably also need to work with stakeholders to draft and send comms to customers, document the steps you took, answer questions from product folk.
This is what I am trying to get signal on in an interview. How do you work on your feet, when it matters most. If you expect to be able to sit in silence for 30 minutes after you were paged while you draft a solution in your head then you're going to have a bad time.
For a junior candidate, these expectations are reduced, but this is the bar for a senior candidates. If I wanted you to solve an algorithm puzzle I'd give you the puzzle and tell you to come back tomorrow. If I didn't think you could solve an algorithm puzzle, we wouldn't be having a conversation.
Communication is the whole point.
To mitigate this I try to constantly verbalize things that I'm thinking anyway, like, "This isn't the OO way to do it, but I think the OO way would be a poor match for the problem," and, "In real code I would model each of these item types with a class, but to save time and hopefully get further with the problem I'm going to model them as lists." The more I talk, the less they talk, and the less often I have to reboot into human mode.
I think "talking while coding" is, at least in theory, for the benefit of the candidate - if they're "thinking aloud" and about to make a mistake which may take 20 minutes to recover, the interviewer can gently nudge them in the correct direction.
I fully expect Sr engineers to talk about why they choose the approach they do, and then implement it. I can adjust their approach while I hear about it, and then let them code it up.
This whole being silent and focus coding is bs I think. When you're doing the typing part, don't talk, just code. When you're doing the strategy part, talk. I want to see how you'll be able to express your strategy to your coworkers when a real actually challenging problem occurs.
Also. Algo problems are worthless. They only tell me you know how to cram. Don't do those, they don't actually answer anything. If I see that in an interview that's a flag for me that their eng culture might have issues. Only exception would be if you're applying to a job where you'll be doing that all day.
I can solve programming problems OR carry on a conversation. I cannot do both at the same time.
It isn't just coding. The same quirk shows up in other ways. For example, I also cannot carry on a conversation and navigate. I can drive safely while conversing, but I cannot navigate. When my daughter was a teenager she exploited this quirk for laughs: get in the car with Dad; start for destination; bring up interesting topic; wait to see what random place we end up going; profit!
I can explain my thinking after the fact. I just can't do the thinking while conversing.
I've been working as a consultant for a while now. Communication is a big part of the job. My current client is one that has come back repeatedly, and that pretty much lets me set the terms I want, and gives me broad discretion to decide how things are to be done. I presume that means they're happy with the results so far and want more of the same.
I do have to communicate effectively in that job. I collaborate with a VP to set directions, vet R&D proposals, choose implementation strategies, pick tools, guide other contributors, and so on. So I have to be able to communicate effectively.
I also write the majority of working code in the current set of projects we're working on.
What I don't have to do is write code while carrying on conversations. It's a good thing I don't have to do that, because I can't.
I can pair program, as long as my collaborator understands that when I need to really think about a problem, I have to leave the session to do it. I can rejoin once I've figured out what I'm thinking about, and I can explain what I thought of. The collaborators I'm working with now are used to it by now, and it's a normal part of our collaborative cadence.
I can hunt bugs in a pair session, similar to the way that I can drive just fine while conversing; I just can't navigate, or do involved creative problem solving. I'm not exactly sure what the difference is; just that there's a certain kind of thinking I need to do that gets blocked by conversation.
"I'm trying to think, but nothing happens."
I'm useless in tech-interview coding, so I just don't do it anymore. I've told several companies that wanted me to go through their coding interviews that they should just give me the lowest possible score on that part of the interview, and if they still wanted to talk to me about the job, I'd be happy to talk to them. Nobody has taken me up on that specific proposal, but I have gotten nice offers a couple of times after I thoroughly bombed coding interviews. Those experiences made me question how seriously those companies took their own processes.
I.e., at least with a coding question, you can set up objective criteria. If you ask someone about their work, it might be a good experience but you're also setting yourself up to bias towards people who talk/think like you.
(I’ve also submitted code that solves the problem “perfectly” and been rejected, no idea)
Put another way, if someone has 10+ years of experience, in an ideal world there isn't much point in quizzing them on algorithms or giving them homework. They've done all of that before, passed, were promoted, etc. You can see that on their resume, and you can verify that with a few phonecalls.
Of course, in reality, you have to quiz everyone equally, and hire based on this objective metric, because even a whiff of non-objective hiring is dangerous.
As someone who has been doing interviews for as long as 2 years, wrong. I've seen at least 15 candidates that had 5-15 years of experience and we're complete shit. As in no redeemable qualities in terms of tech assignment.
I disagree.
Exp is nowhere close to being perfect proxy for skills
Recently, we had a candidate come through that wasn’t a good fit for us (he claimed to be a cypress expert so we asked him to set cypress up in a dummy react repo and he floundered the entire time). One of my coworkers took some pot shots at the candidate, being deeply critical and rude, cutting the candidate off mid-sentence, etc. I was honestly embarrassed to be on the call and I thought it reflected poorly on my company. I’d never seen my coworker act like this before and it changed the way I felt about him.
I wonder how much of the leetcode interviewing culture is rooted in this desire to dominate candidates, since ICs rarely have opportunities to wield such power over others.
I still cringe when I think back on some of the early interviews I did.
some people don’t
Maybe some tech media company will do hidden camera interviews and publish a scathing rebuke of the practice.
The last place I did interviews at, we'd bring all interviewers into a quick debrief meeting and talk about whether to give the candidate an offer (going off the notes we took during the sessions). It was pretty obvious if someone had a particularly bad session and could be weighted against other peoples' input
Unfortunately it can be really hit and miss
You can always tell the recruiter/coordinator. It's just tricky because you don't want to burn bridges, doing it before you hear back could be read as you asking for a do-over, doing it after could come across as bitter.
If you have a good or bad experience, PLEASE consider telling the recruiter know.
It made me feel like I didn't know anything and was very embarassing. It also changed my outlook on Google, and I'll never interview with them again.
That alone is a disgrace.
Substitute woman for boomer in that sentence and listen to what it sounds like.
Knowing to silence your phone in meetings is a matter of having manners, not age.
I'm afraid I just don't get your outrage at an innocent comment poking fun at the most dominant section of society. But, as you're the second person to have taken offence, I'm going to delete it anyway.
Secondly, I have never encountered it as a problem--at least in a way that affected me (never could really get into a dot-com company but we know how that mostly ended up). But that's mostly because I've had various roles where experience really trumped other things. But ageism is clearly an issue in tech broadly so I take issue with with the justification that it's OK to make casual cracks with the justification that it's punching up.
To say nothing that it added absolutely nothing to your comment. Rude interviewers are the issue. Not some imagined inability for older individuals to silence their phones.
Finally, I'm not sure doubling down on the oppressed race thing helps your argument--especially given what the age breakdown in tech looks like.
Personally, I'm not afraid of your lack of tact or humility, just disgusted by it.
I'm a Millenial, btw.
That would include any and all of us in this thread. No man has ever come anywhere near the moon.
Unless, of course, you happen to know of some kind of magical (presumably still classified) rocket nozzle tech that would somehow avoid leaving a crater in the soft 'lunar dirt' as the 1960s tin can touched down on the 'lunar surface.' Not to mention an actual solution to various other hard technological problems of space travel, like radiation shielding, heat rejection (no, space isn't 'cold'), battery power, lack of any visible stars, workings of a camera in a high radiation environment, etc, etc. And how about that lunar module 'computer'? Anyone ever replicated that 1960s technological wonder? Oh, the schematics and tech were all 'lost'. Someone accidentally recorded over the original telemetry tapes with Gilligan's Island reruns. I see. What a terrible accident.
Not to detract from your main point, of course; tye interviewer was terribly rude. Selfish. Just like the liars who faked the 'moon landing' in the Nevada desert, near Indian Springs.
https://static.righto.com/images/agc-power/transistor-closeu...
Just like you did, just now. I still haven't heard your explanation of how a large diameter rocket nozzle, which we are told was firing for the entire descent, with 10 psi of hot gases escaping from its large volume, failed to disturb so much as a speck of dust underneath the 'lander.' That's only the most obvious of many glaring plot holes in the script.
If you were to, say, enter 'evidence that the moon landing was fakee' into any search engine, naively thinking such evidence would be presented to you, instead you would get only page after page of laughter, ridicule, and condescension--and none of the actual evidence. Certainly you would never, ever be presented with the name of the very first book on the subject, which launched moon landing skepticism into the mainstream in the early 70s. Nowhere in the first ten pages of results is this man or his book mentioned.
The book is called We Never Went To The Moon by Bill Kaysing, who was a technical writer for the Saturn V rocket program at Rocketdyne. Wouldn't you think a man with such a background would be worth listening to? Google doesn't think so.
Another book of interest is NASA Mooned America, by Ralph Rene. This one is more technical in nature, and drives the final nails in the coffin. Google also pretends this one doesn't exist.
If you type the author and name of the book into Google followed by the keyword 'pdf' however, it will take you right to it on the first page. At least it did as of a year ago. Funny how that works.
Read those books, then come back and tell me again how sure you are.
When you find out how fake and full of lies the world really is, you're going to be pissed.
https://www.cypress.io/
It seems reasonable to me to not be great at setting it up.
My comment always gets pushback. Privacy, respect for interviewer ("If I work here, then I myself can judge fine!"), not enough time, etc.
Hiring processes are about as contentious as CI methodologies, it seems
I would be ok with the recording as long as they don’t make my sign the form allowing it to be used as part of some public campaign.
If I thought my interview was being recorded, I'd assume it was being used for this internally. If given the option to have it recorded, I'd assume that I'd get "minus points" for declining.
This is very unfortunate, because I love your idea. I'd just need stronger legal protections to ensure that I wasn't perpetuating part of a dystopia.
In other words, maybe if you utterly wow us in-person you'll get in anyway. But probably not.
You can give whatever excuse you want "Adding a new person to the loops," "normalizing the loop level," whatever. But I am now convinced that interviewers cannot be trusted alone with candidates anymore.
Maybe I'm naiive that they ever could be, but if one of my guys did half of the stuff people keep reporting to a candidate and I found out about it, I'd have his head and maybe his badge.
Fsck you very much whichever FAANG HR department decided that "interview loop" was something to be added to corporate evaluations for points and then promulgated it to other companies. Thus making both sides of interviewing even shittier than it was before.
I would go so far as to say a random passive observer is still a great idea in addition to two active interviewers.
It wasn't at any point that I knew. Interviewing is an expensive operation--it ties up quite senior folks for a fixed amount of time. If you pair that, you double the expense.
And it seems like phone screens are exactly the point where you most need a pair from these anecdotes.
You also then encode a single group's biases (the rubric) rather than a mix of everyone's biases. In my experience that doesn't actually make things better overall just slightly more consistently bad.
edit: If you delete your reply to this, I'll delete this. and we can unfuck this whole thing. ;)
Another interview I had years later, the interviewer clearly didn't think I was worth his time because he literally got up and started playing with a yo-yo as he threw challenging question after challenging question at me and pressing me harder and harder for solutions ("Imagine you were given task X and nothing more. How would you solve this? What would you do next? If that didn't work, now what would you try?") I remember it was incredibly disrespectful and left a bad taste in my mouth about the team. Thankfully I didn't get the job.
Imagine asking a doctor basic biology questions or what an antibiotic is used for in an interview.
Even if a candidate didn't know that modulo even existed they could still get the remainder with simple division and subtraction. Modulo might be slightly different depending on the language, but for fizzbuzz that's not important and most likely not what you're expected to know in such an interview question. And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
And they'd probably fail some interviews for doing so.
> And while it is not the most common operator it is still one of the most basic ones and present in all languages you'd use during a job interview.
This is true, but I also expect some otherwise-competent programmers would fail a test that expected them to remember the specifics of bitwise operators or bitshifting. It's been years since I've needed to use any of the three in a paid job - they're just not very common in my day jobs, whereas they crop up a lot in side projects.
Not in the sense of missed a bit of punctuation, but rather in the sense of “couldn’t get started with anything that vaguely smelled like an answer, didn’t understand the type system of the language they claim 5 years of experience with, etc”.
I’ve probably interviewed fewer than 400 candidates in-person and maybe 200 of those would be senior ICs. To have more than a percent of them be utterly incapable here is a signal that there are candidates out there taking their best half-court buzzer-beater shot to try to land the prize. If you’ve done 100 first round in-person interviews and never run into someone that causes you to check up on how the phone screen happened, your screening processes work pretty well.
Is it possible that these candidates are capable programmers in daily work? Sure, it’s possible, but I’d bet against it.
* We used to ask a candidate to average an array of ints. It’s a question that was designed to tease out whether the candidate would discuss/inquire about overflow, what the return type should be, etc. After a couple years, we realized that candidates would stumble on the loop and sum more than anywhere else and we got more signal/minute from asking the vastly simpler summation question. Candidates who could fluently write the loop usually knew about overflow and the fact that the average of ints isn’t mathematically an int but that many times an int is what you want and can discuss the reasons.
I still wonder if we couldn't remove most of these games though if we improved our hiring, onboarding, and firing processes in other ways than just trying to make a better hiring filter. Accidentally letting in someone wholly unsuitable for the job shouldn't be a big deal if 1) you can detect that like in the first day, or at least first week 2) the amount of damage they can do before detected is small 3) they actually start very soon after the decision to hire is made 4) you can fire them immediately after deciding a mistake was made and they don't work another day.
Startups can handle this better already and I think is reflected in a lot of them still doing more casual "single lunch conversation" style interviews before a "welcome aboard, see you on Monday" decision. For big cos, work will have to be done on getting away from the mentality of "it's ok if you're not contributing much of anything for the first x days because onboarding" where x is high (I've seen over 30), and firming up access controls (no facebook style all-access of new hires (or anyone if possible) being able to just query the production db and access the private messages of exes or whoever), and removing the "ok you're hired, see you in x weeks" policies where new hires can only start after a significant delay or all on a certain date for whatever reasons, and only getting rid of someone through a long and drawn out PIP process... Maybe it can be done some day.
As a hiring manager or interviewer, I’m not their fiduciary, but when facing a risky hire, I wish there was a way to let the candidate make a more fully-informed choice without the knowledge that we think it’s a high-risk offer influencing them negatively in any way.
Right now, they interview and get an offer or don’t. If they get an offer that we think is high-risk, they have no reasonable way to know that.
Things like, "How do you feel about stretching your skills in the work force?" "What do you do when you make a mistake at work?" "How do you handle a situation where you feel out of your depth?" Etc.?
I know I would take that as a signal not to go for it if I wanted to retain stability in my life.
It does give me some ideas to consider when we extend the offer (as @Jach's sibling comment also suggests).
I've been fortunate (or not) that the hiring decisions I've played a part in have all been non-urgent with no struggle to find more candidates, so in the face of risk the org could afford to wait. The moral hazard you describe is interesting though. I don't think I would worry about it, as you say the hiring side isn't their fiduciary, and you don't necessarily know that their current situation is all that stable. I might try and assure myself that even if things don't work out, they're unlikely to be put out on the streets, since if they got past us (even with our "risky" assessment) they can probably get past someone else quickly enough, and possibly even go back to where they left depending on how stable it really was.
I'd consider being honest with them about the risk if an offer is extended. If I were on the receiving end, I'd prefer to be told before I accept something along the lines of "Honestly we're a bit unsure how well you'll adapt to our work [because of risk factors x], but we see your potential and think you'll rise to the occasion. We hope you'll come join us!" I don't think having my risk factors pointed out would affect me negatively at all, even if I think they're BS things the company shouldn't worry about, though I can see it possibly affecting the company somewhat negatively in that they're potentially giving the candidate who all-in-all they think is worth taking a chance on an extra reason to decline (and thus they'll need to continue searching which sucks). Whether that decline happens really depends on the candidate. With that information they might conclude/agree it's too risky to leave their current role for you right now, or they might actually be motivated to do better and start proving themselves right from the start especially at a company that is so forthcoming about such things. And they might fail, but at least they leave with the knowledge of what went wrong, and even if what went wrong wasn't an identified risk factor they still have your list of risk factors (some possibly valid, some BS, some that can be worked on, some that can't) that other companies might be looking out for to have for future interviews.
But the cost is actually minimized already by rejecting in the interview. Just like finding a bug while writing unit tests is cheaper than QA finding it after you make a build, even if QA does so in the first hour of testing.
So in hiring, there are costs companies pay to avoid a bad hire, and costs that result as a consequence of bad hires -- which they mostly can only estimate because of bad hires getting through anyway, everything on the consequences side is formally probabilistic. You can think of it as hiring-filter-costs + P(bad-hire) * consequences-of-bad-hire. So yes, if P(bad-hire) is 0 because of your amazing hiring filter, then you'll never have to pay for the consequences-of-bad-hire. And supposedly, increasing hiring-filter-costs should decrease P(bad-hire), though over time we've found many practices that don't actually do that, and smarter companies have stopped doing them.
My thinking boils down to we can and should reduce hiring-filter-costs (which I want to think of as including opportunity costs and negative externalities on the whole industry, but it's not necessary), but fear prevents even experimenting at many places because there's an idea that any reduction will increase P(bad-hire) too much, and consequences-of-bad-hire are too large. So, why not focus more on identifying and lowering the consequences-of-bad-hire? Do that, and we'll also be able to lower hiring-filter-costs even if they increase P(bad-hire), which they aren't guaranteed to do anyway.
The biggest problem when hiring people with experience though, and why the probation style is pretty rare in the direct form I think you're thinking of, is that it's going to filter out the best people -- just like many other too onerous hiring practices filter them. They're just not going to put up with it when they have much better options. For Europe to the extent such a practice works at all is probably related to how universal it is for employees and employers to have a symmetric long period of notice of termination/notice of quitting that often ends up being at least a month if not 60+ days. In the US, every state is at-will, you can leave at any time and be dismissed at any time (within the bounds of certain legal protections, paranoid risk of violating which has driven larger companies to an insane PIP-style process of nudging someone out the door over a period of months). I think US companies and employees both should continue leaning in to that more as an advantage.
They may only represent 2% of the population, but they are a much larger % of candidates searching for jobs. If you think about it a bit, you'll see why.
Asking someone who claims to be a doctor of natural medicine the same might yield you more than half unable to answer.
Since there is no 12 year stringent accreditation process for being a software engineer, I'm not sure why this is a good comparison. There are many people who don't know so many basics of CS who have finished degrees, bootcamps, or have been employed in those roles for years.
Doctors simply aren't able to get accreditation or employment with the same ease as software developers.
If you're going to demand that you be treated as an honest estimator of your own abilities beyond doubt don't be surprised if that ends up eliminating opportunities for people who are trying to break into the field without those credentials that put their abilities at a near certainty.
If I was floundering because the interviewer found a hole in my knowledge (and there are many holes in Albert Hall) I'd proactively state that and say where the holes weren't and ask for a question about that.
For example, I've implemented a complete C compiler, front to back, so at one point I knew everything about C. But in implementing a new C compiler (ImportC) a few months back, I had to go over the Standard again, as I'd forgotten bits and pieces.
There's no reason at all why the interviewee cannot drive the direction of the interview, rather than being driven. For example, when I interviewed for a software job after leaving my mechanical job at Boeing, I was concerned that I wouldn't be taken seriously as a programmer. So I brought along listings of the code I'd written and pulled it out during the interview and proceeded to explain it to the interviewer, on my own initiative.
A long time after that, I was talking to the interviewer, and he said that that was what got me the offer.
You know how they made a program pass the Turing test? They didn't make it smart, they hardcoded a conversation where the program drove the conversation to where it wanted it to be. That way the person interacting with the program was passive, so they never noticed that they were just interacting with a stupid program in the 5 minutes they had with it.
The same happens in interviews, if you take a person who don't understand tech, but have watched a lot of conference talks and read some papers and he can re-tell those, and he even remembers answers to common questions from the question parts there, but he doesn't understand much at all of what was said he just remembers conversations, then that guy will look like a world class expert if he gets to drive the interview. But once he starts working he didn't know anything at all, he just learned to talk about these things not do them. Some would say "just dive deeper into his knowledge!", but you can't always do that, if you aren't an expert yourself in his subject area you wont be able to drive down deep enough to find his flaws, and since he is driving the conversation most likely you wont be an expert on what he is talking about.
This does seem to be the threat model many interview processes are meant to protect against though.
I understand there are costs involved in hiring then firing someone, but maybe that should be reserved as a way to deal with the most outlandish fakers and let the rest of us have a quick Google or two during coding screens.
But if you are lazy and prefers talking to people and watching videos, so you spend your day talking about these conferences with your colleagues rather than doing work then you have the scenario I'm talking about. It doesn't require some genius conman, basically any guy who prefers to talk about things instead of doing things will more or less fit this description. The degree they lack real skills depend on the degree they prefer talking over doing, but such people will always over perform at such interviews.
Lastly since the talkers get rejected by all the leetcoding firms, you will have an abundance of talkers that can't perform in the hiring pool. So the problem with talkers is way worse today than 20 years ago.
If I accidentally hired a guy like that, I'd consider him for a marketing position. :-)
Often I feel like a con artist for doing this, especially if works to overcome a superficial interview. Sometimes it turns out my perception was correct and yet it turns into a better interview because I’m adding value to the interview where the interviewer had gaps…
I assure you there are worse ways to behave in an interview. I’ve passed on people who were smart as a whip but nobody wanted to work with. And nobody wants a dead fish who just sits there waiting for the next problem to solve.
Watch out for this. It is a real phenomenon. If you "enjoy" thr interview, you are more likely to think the candidate did well. And if you spend a lot of time talking about yourself, you are more likely to think the candidate did well. Don't be fooled. Have a rubric that you can objectively measure the candidate against.
Perhaps this is less of a big deal for a generic position that’s the same at every company, but if you are matching someone to a very different role than they’ve held before, you may be interested in only certain pieces of their resume.
I also interviewed at Microsoft (just down the street) a few years later, and got the standard interview treatment.
I know what quicksort is, and I've implemented it more than once over the years. But I could not implement it off the top of my head at the moment without doing a quick refresher on the algorithm.
I was once assigned the task of speeding up a program written by <famous programmer you've heard of>. In perusing the code I happened to notice that there were two custom bubblesorts buried in it. I replaced them both with calls to C's qsort and got a nice boost. Was <famous programmer you've heard of> a fraud? Not at all. Anybody's code can be made better. I've been found guilty of having hidden quadratic code many times.
I consider myself an expert on lots of things I’d still need references and a bit of study to install properly.
What you describe is embarrassing, and does reflect poorly on your company. Review sites like Glassdoor do or did have interview reviews as part of their company review, and people do go look at those when deciding whether to apply at a company.
I hope you shared your feedback with your coworker, and also the hiring manager. That coworker is probably not ready to do interviews and needs some training (or maturing) first. In the future, you might interrupt the coworker and then try to help the candidate along; maybe they won't pass the interview, but you don't have to let them leave feeling like a failure, and along the way you might get some information about where they could fit in your organization. And the entire interview won't have felt like such a waste.
Not many people are gonna risk confrontation with a colleague over some candidate they don't know. Especially not during the interview, maybe afterwards when things are calm. I'm agreeing with you here but let's stay realistic; some companies have assholes working for them, sometimes these assholes get to positions of power. Usually the company knows damn well employee X has a bit of an attitude problem but...guess he adds enough value (or acts well enough) to keep being employed.
I eventually found something but it was really discouraging there for a long time.
The best way, as always, is to connect with people you already know and have them short-circuit the broken process via existing trust.
i have a GitHub account with 7 years worth of contributions, multimillion dollar exit, projects online and previous senior roles on my cv
if they still don’t trust me to do my job, the one i’ve provably done about a decade, their loss
nowadays it’s easier to find high-dollar consulting gigs than deal with dysfunctional hiring processes
advice: get jobs from people you know
they’re not interested in hiring employees, their interest is to get candidates through the loop and to see you fail
this is fine when you’re applying to FAANG, but a terminal sickness when the company is barely known
rather than spending weeks on interviewing, i’d suggest making a open-source contribution or reading a book
the companies, especially startups have a bigger problem than you - they need to show the growth so they can attract more investors. Therefor, if they’re not getting people, they have harder time growing. If they see you as a potential candidate, but you refuse, they’ll bend their back over backwards
The outrage around the technical interviews shows a lack of understanding for the situation companies face when hiring.
The reality is that something closer to 99% of the pool are applying for jobs they're not qualified for (not the same as saying 99% of developers aren't qualified - there's extreme survivorship bias in the pool).
At company X we would hire ~1/400 applicants who applied, and about ~1/20 referrals. At a smaller company I worked for we hired perhaps 1/200.
The thing is, many of these applicants have five to ten years experience (some at FAANG), but can't code at all. It's simply not possible to trust anything on a CV without verifying it. How do you verify this at scale? A pair programming exercise.
Now the training of most interviewers is often poor, but it's far better to go down this route than the HN alternative: hiring without technical screening.
I've gotten nearly 100% of the jobs I've gotten in the past 15 years through friends. Interview process real easy after the informal "backchannel" reference check has happened. Just hi, how you doing, when can you start.
better spend more time maintaining those connections
i myself never really practiced algorithms, so in an interview they’ll see me as a weak learner
however the reality is complete opposite - i have spent thousands of hours on piano and guitar and speak 3 languages
i have the opposite of learning disability and learn stuff quicker than many of your algorithm-knowing employees can
Sounds to me that the author likes to program by "sketching". I do too! Paul Graham recommends this in Hackers and Painters.
> For example, I was taught in college that one ought to figure out a program completely on paper before even going near a computer. I found that I did not program this way. I found that I liked to program sitting in front of a computer, not a piece of paper. Worse still, instead of patiently writing out a complete program and assuring myself it was correct, I tended to just spew out code that was hopelessly broken, and gradually beat it into shape. Debugging, I was taught, was a kind of final pass where you caught typos and oversights. The way I worked, it seemed like programming consisted of debugging. > > For a long time I felt bad about this, just as I once felt bad that I didn't hold my pencil the way they taught me to in elementary school. If I had only looked over at the other makers, the painters or the architects, I would have realized that there was a name for what I was doing: sketching. As far as I can tell, the way they taught me to program in college was all wrong. You should figure out programs as you're writing them, just as writers and painters and architects do.
On the other hand, as the post describes, interviews are geared towards a more waterfall-y approach. I find this frustrating as well, but I think that there is something really important to keep in mind that this post misses, and that most conversations about coding interviews miss. The question is whether something is _predictive_ of job performance.
For example, consider the question "What is your favorite number?". Imagine that the larger the answer to that question, the more likely you are to perform well as a developer. You might object "But I'm never doing anything relevant to my favorite number on the job!". IMHO, it doesn't matter. The goal is to predict who will perform well on the job, and if having a large favorite number does this, it should be incorporated.
Now consider the question of how well you perform on waterfall-style interview questions. Like "What's your favorite number?", it isn't something that you will find yourself doing on the job. But that isn't the right question. The right question is whether or not it is predictive of job performance.
Of course, it seems unlikely that "What is your favorite number?" would predict job performance. And it seems unlikely that if you never code in a waterfall-style on the job, that coding in that style during an interview would predict job performance. But maybe it does. I'm skeptical, but who knows. More importantly, I think that this is the question that needs to be asked. And without strong data, it's hard to be too confident in the answer.
What if it was skin color, how ethnic your name is, or your gender? Did you know the majority of developers are men? Should we throw all the female applications in the trash because they're statistically not as likely to be developers?
Yeesh. Sorry if I sound hostile but... jeez. Correlation is not causation, etc etc.
> Should we throw all the female applications in the trash because they're statistically not as likely to be developers?
What do you mean?
What if skin colour, ethnicity or gender were highly predictive of job performance? It doesn't seem very ethical to judge candidates like this.
I believe the GP is not convinced by the "all that matters is that it's predictive of job performance" idea.
Well, actually, thinking about it again now, I'm not so sure. But this now becomes an ethical question. Which is certainly important. But I want to be clear that the ethical question is a different question from the question of what a purely self-interested company should do. To that self-interest question, I think the OP and lots of people argue that the answer is "leetcode-style questions having nothing to do with the job and are thus a bad idea".
My claims are that 1) this is the wrong way to think about that self-interest question. The right way is to ask whether it is predictive of job performance. And 2) although I also suspect that leetcode-style questions aren't predictive, it's hard to be confident in that without good data.
In my experience, it's helpful to add simple tests as you go. I have done some live coding tests in Python recently, which I'm not super familiar with, and the first couple I did, I made a lot of trivial errors that caused my solution to fail until I went back and cleaned them up. Luckily the interviewers kept their mouths shut while I was coding, and I was able to clean up quickly afterwards, but I could tell it wasn't a great look to say, "Okay, I _think_ this solution is correct," only to have it flame out spectacularly.
After those first two, I started writing simple tests for everything I did. Not only did it avoid embarrassing fails when the solution was supposedly complete, not only was it quicker overall, but every interviewer I've done this with has commented approvingly about it.
Well OP, you lost me there. That's not my approach interviewing, nor is it what I've experienced.
"Why is it bad to write a file to the disk one byte at a time?"
What are they interviewing for, low level disk experts?
"Here's a stack trace from an open source project, can you tell me what went wrong?"
Oh sure, print-based debugging is awful, you just have to read stack traces from unfamiliar projects in the blink of an eye.
"Here's a CRUD app, copy existing code patterns to implement a new CRUD endpoint"
Ah yes, the reductive "everything is a CRUD app" - this interviewer doesn't understand that most applications don't fall under the simplistic "CRUD" moniker!!!!
Why would only low level disk experts be able to answer that?
I actually feel like algorithm questions are somewhat of an improvement in the programmer hiring process over the alternatives. When interviewing at places that don't have them, the surface area of questions you can be asked is so much bigger.
Like one of the anecdotes in this thread about getting drilled on a testing framework (that the candidate probably added to their resume because they have to have all these keywords to make it through their automated resume scanner and maybe they only played with it for a couple hours one afternoon because they don't use it in their day job.)
With algorithmic questions: - You're doing one type of preparation for a large amount of potential roles - There's tons and tons and tons of great material out there to help you prepare - Probably most controversial but I really believe it: It really does make you a stronger programmer.
For me, doing those kind of questions forced me to learn data structures more thoroughly and dramatically improved my understanding of how to write algorithms and think about performance. This stuff really does matter in terms of code quality and it really bothers me that it's always portrayed as these completely silly puzzles.
The other thing that I see pushed a lot in these discussions is this idea that every company (or most companies) that hires programmers does these exercises. That's just not true. If you're getting coderpad links for every phone screen, you're going after a certain type of company.
Sounds like a mental gymnastics to me.
> Triplebyte has a quiz that actually does a pretty good job of this; you get asked thirty or so questions about databases, networking, and simple coding questions ("what is the value of 'a' when you get to line 17?").
I could absolutely see people complaining about multiple choice questions and how you're more likely to get tripped up by minor mistakes.
The interesting thing about mcq is that their administration is completely standardised so it's more objective in pretty much every sense.
Maybe you prefer algorithmic interviews, that's fine. But I really don't understand how you can believe that they are less subjective than algorithmic interviews.
They avoid the bias of using different interviewers which is massive, even if the questions are standard.
Writing one byte at a time is a pretty quick way to ruin flash on a mobile device (or really, any flash).
The questions you enumerate are more subjective and depends on the interviewer's experience/maturity on how well they can gauge someone's answers.
Writing one byte at a time may be “bad” at many levels. First, on a hardware level it is inefficient because the unit is a sector/block and the access latency kicks in hard, together with a wear leveling on ssds. Embedded low level indeed. On an os/syscall level it is inefficient because of a syscall time itself. It is a system level touching C and unbuffered writes in “middle-level” languages/libs. On an application level it may be inefficient if a stream has a processing pipeline with some expensive setup/try/teardown (e.g. a jsonl parser which “tries” the buffer at every new chunk). Is this still low level?
This is a question about streaming, buffering and latency avoidance, which you may not know in hw detail, but have to have at least a vague grasp of. If a candidate seems confused, one may ask them about other situations where one-at-a-time may become “bad”: sql select one row at a time, fetch NNN items one by one, use heavy_expr().x, heavy_expr().y repeatedly or in a loop. If nothing comes to their mind, chances are you are interviewing a completely inexperienced person, because there is no way they didn’t meet that or think of that before.
I totally agree that it shouldn’t be the reason to skip a candidate, but hiring is an analyzing process, and how do you find their limit if not asking deep open questions? I think that the issue is pushing hard on these areas as if it was laughable for a professional to not know it.
The first one, it's exactly about the thing I find a lot of developers lack in: a fundamental understanding about computers and processing. I've had 'experienced' developers write algorithms where every intermediate value was written to a cache/disk only to pull it out on the next line. I feel asking some more fundamental questions can help weed ppl out.
Same with being able to look at a stack trace and see where it went wrong, how quickly are you able to hone in on a specific target. Crucial skill imo, and I've had sooooo many people at my desk -- 'it's not working' -- that did not even read the f'ing error message.
Asking someone to implement stuff without running it sounds counterproductive to me. About as effective as asking them to write syntactically valid code on a whiteboard.
The author is focused with technical review, and as such he probably fails to see just how important personal review is. Yes, you are being reviewed technically, but you are also being reviewed personally. You may know a lot of technical things, but do I want you on my team? The phone interview is what i'm going to use to make that decision. Technical we can work over later, but only if you sound like someone I want on my team.
I'm always willing to say, "They look like a great fit for the team and our goals, and let's give them a chance to learn the tech" versus, "They know a lot of technical stuff and let's give them a chance to see if they can learn to work with other people."
I wish more were like you.
I've definitely had my fair share of interviews that bombed so I feel like I have a good bearing of when I don't leave a good impression, but it's always the worst feeling to have an interview that you think went well based on the overall mood/feeling, only to receive the eventual rejection later in the day that they went with someone more senior or that they wanted someone with more experience in x.
My mentor who didn't got to a well-known college once told me he joined a very solid company not knowing anything about the frontend framework they used but they gave him documentation and let him learn it for weeks without touching the code base. He said it was honestly one of the strongest engineering teams he's worked on in his career. The idea of managers seeing potential in someone and thinking "let's go with that person who has never done x" seems so foreign to me.
Companies are increasingly unwilling to invest in the wellbeing or value of individual workers, instead treating workers as fungible assets to minimize the cost of which serve as solutions to current (rather than long-term) problems. We see this with the near-universal embracement of short-staffing in service-level jobs, lack of career mobility from starting positions, and even in the tech industry with the expectation that you usually have to switch companies to get a raise.
Leetcode is an exam that you can study for. For me, it's pretty damned hard and I dislike it.
I can think of three reasons why someone might be good at those kinds of problems:
• You just have a knack for it.
• You are very intelligent.
• You practiced a lot.
Which one of those can you do something about?
Every time I force myself to grok a new leetcode trick, I've truly learned something. Something irrelevant in my ten years of programming, but allegedly an indicator of someone who might be good at what we do.
Companies are full of people who do well at a whiteboard. You're the rare variety that companies NEED MORE of (even though they don't know it).
However...
> If I get behind or the interviewer starts interrupting to ask about the bad code I'm writing I get very stressed and have trouble both listening to the interviewer and trying to address the issues with the code.
My sense is that 90%+ of these interviews is maintaining your composure, asking questions and generally coming off as well-adjusted and personable even if you don't know the answer or get stuck.
> Certainly there are a lot of confident, mediocre tall white men in the tech industry
That's some chip on his shoulder.
i know that is not what is happening during an interview but, it’s how i feel. and this prevents me from getting ahead in an industry i am madly passionate about
- It makes fresh graduate noobs (or even just noobs in general) able to compete on equal ground on high-paying job prospects. Think about it, if every high paying jobs out there value experience over leetcode style of interviews, then young noob programmers won't be able to rake great salary because all of those jobs are filled already by experienced engineers
- Great engineers should just use their talent somewhere else in startups, create more innovation, create more wealth rather than helping the big tech giants become bigger
Experienced engineers and college noobs should not be competing for the same jobs.
Tech co’s seem to prefer the college grads for rank/file for many reasons. Maybe because they are easier to exploit?
>create more wealth rather than helping the big tech giants become bigger
should I care that some startup will become relatively significant and then sell itself once big offer appears?
It's fine, though. TFA goes into ways to improve the hiring process such that this sort of thing doesn't happen. But I don't mind it at all. I want to fail fast; if the company decides I'm not the right candidate, great, I'm glad it happened before we wasted time with more extensive interviews.
There are two possibilities. If they're right and I'm not a good fit, I don't want that job because I won't be successful there. If they're wrong, I don't want that job because the company isn't being successful at hiring me. The hiring process is looking for the wrong attributes, or the interviewer isn't properly trained, or... well, something, somewhere has gone wrong. There are no false negatives.
A company that's firing on all cylinders does a lot of hiring, and is good at it. They source solid candidates, and interview them thoroughly but quickly. If they're not hiring effectively, well, they're probably dysfunctional in other areas too.
And, why all companies have decided upon almost the exact same methodology, I can't understand that either. It seems to be the same double edged sword as the Common Application for college: you get more applicants because the incremental effort required to apply to college n+1 is low, but it drives down college admit rates as well as matriculation rates: the signal to noise ratio probably goes way down.
That is a feature. You look at what they worked on before to see if they are senior material. Then you do leetcode and system design to see if they are smart. A smart person would most likely learn whatever tech and problems they worked on, so if they do well on leetcode you just assume they did well in their past jobs. If they can't learn leetcode then you assume they did poorly in their past jobs. If they didn't practice leetcode then you got no signal on their smartness, so no-hire due to unpredictability.
So we get these possible results and what you do with it:
- Good leetcode, great and relevant experience: Hire at high level, this guy is smart so he likely learned a lot of things during his past experiences.
- Bad leetcode, great and relevant experience: no-hire, he might be smart but we don't know so we don't know if he actually performed in his past jobs.
- Good leetcode, no/minor expereince: hire at junior level, this guy will likely learn fast
- bad leetocde, no experience: don't hire, there are no positive signals.
This works really well actually. The only bad part is that some experts needs to brush up on this when they want to find a new job, but for a well paying company that isn't a problem as millions of candidates gladly brushes up for the pay bump.
Yes, it's an easy signal, it makes an unpredictable and messy process feel neat and binary. However, it is a false signal, and no amount of personal comfort changes that it has about as much effect on future employee productivity as hiring by height or favorite color.
Unfortunately, software productivity is never going to be measurable. Therefore we will never ever have a reliable metric to even prove to ourselves that we are more or less productive than our coworkers, much less worth hiring. This fact should sober our prideful arrogance about hiring and introduce some humility. Our best practices are a farce with absolutely zero data to back up their validity. Ask yourselves where are the studies that prove candidates who crush leetcode perform better than those who couldn't? Where is the double blind research?
We need a heaping dose of recognition that most of our best practices are there to validate the egos of the interviewers and give a neat binary decision without any mess. We have a toxic fraternity hazing ritual to make candidates bend the knee, with exactly zero evidence it works.
We have the gall to call ourselves scientists, and we don't even measure a single thing.
Want to add agreement that grinding leetcode is a terrible metric. The ideal candidate probably already has a full time job and if you add in a family and other responsibilities, they don’t have time to focus on leetcode. Heck, they probably are tired of thinking at work all day and don’t have the mental capacity to task switch into leetcode after work (I know I don’t!)
I think leetcode is an antisignal
I think there is something similar to art in GOOD software development, as somebody who does both. The primary reason I don't like coding for a living is the same reason I don't like writing fiction for a living: I really dislike being creative on demand and on a schedule. I find it stressful. I can't make myself get a story idea any more than I can make myself find an elegant solution to a coding issue. Sitting and staring at the screen more isn't going to solve the issue for me.
Software development is to math/CS what painting is to color theory. There is science, but there's also creation. That's why it's so hard to measure and hire for. Imagine trying to judge 1000 newly graduated oil painters who had all taken the exact same classes. How would you decide who was 'best'?
There are way too many developers out there and too few jobs
Google has data on this, when I worked there anyone working there could view it. Performance on their technical interviews is the strongest signal they have that someone will perform well at the job. It is stronger than every other signal. They have tried a bunch of different things, Google HR hates these technical interviews since it is hard for them to game so they work hard to remove it, but they can't since they can't come up with anything with a stronger signal.
Performance on a single technical interview isn't a strong signal, but the combined performance on 5 interviews is a strong signal. If you only run a single of these instead of many with many different people then you will hire a lot of bad people anyway. Also if you pay less than Google and similar then you will get their rejects, there is probably something wrong with the people who are good at algorithms but get rejected by FAANG, not sure why you would want to hire those.
I do believe that performance in a technical interview corresponds closely with overall performance, but I'm very curious how they measure job performance, considering tech job performance is mostly a subjective popularity contest. I'm not convinced that is a valuable signal either. This is what I was getting at, we can't measure productivity _of employees_ with any rigor, so how can we know if our interviewing process works. Since job performance is so subjective, it's not very surprising to me that people hired with a different process aren't as well liked. The one thing I know about people is they like people like them. People who can and will grind leetcode are a very similar group indeed, so a person who didn't do that likely sticks out like a sore thumb.
Perhaps all this is accounted for, in which case I'd love to see it.
I do believe you can do better than Google. But I don't believe you can do much better than Google at Google's scale with that many people if the goal is to hire people who can solve hard technical problems. In a small company you can have the founder or equivalent do the hiring, like early Google did, Larry Page reviewed every single candidate until they had thousands of employees. This is what they came up with after that. Also now instead of having Larry review everyone, they have the vice presidents review every hire in their org as an extra last step to ensure the hiring process doesn't turn bad or get gamed.
Note: If you don't pay like Google, or your goal isn't to solve hard technical problems like Google has great need for, then you probably shouldn't try to hire like Google. Also if you just run a single algorithm question that is easy to look up online, then you are not copying their process, since Google forbids such questions and require multiple questions. Also if you just run an automatic algorithm challenge online then you miss all the soft skills part that comes up when you talk to a person when solving the problem, so not their process either. Also a person who isn't at least decent at algorithms cannot administer an algorithm interview properly. They need to come up with an original problem, and they need to identify when the candidate has solved the problem even if the candidate did it in a way you didn't expect. Most engineers at Google can do that since they hired for that from the start. Most engineers at most companies can't. Also if you have managers make the hiring decision instead of an independent committee then you aren't copying their process. And if your executives are not constantly checking that the process is in line with expectations and doesn't bring in a lot of bad candidates then you aren't doing their process. Etc etc.
They do so many things for this to work, if you do all of that and wants the same thing as Google then it will work for you as well. But almost no companies wants to do all that work to get this working. They just say "Google does algorithm interviews, so go and ask the candidate an algorithm question and that is our process".
Also some caveats about this process and why you hear so much bad things about it:
- Managers at Google don't like it. They would prefer to hire the candidates they want, but as is it can be hard to even fill headcount you have been allocated if you aren't an attractive team, they would love to lower the bar to fill their seats. And the few people unattractive teams manage to hire often leave the team as soon as they can for better teams. I did that, I was first on a dying project and then I did well and joined a cool project working on cutting edge machine learning infrastructure. So this process isn't pushed by managers. It is the executives who want a well run organisation who wants this process. Which is why the executive review is mandatory, this process is first and foremost for their sake. And from heir perspective, hiring really smart people, put them in a poor project to test their ability and then get the best of those into the teams that matter is how you get great engineers to those important teams. There is no reason to lower the bar for poor maintenance projects even if those managers would love to do it.
For Google the most important thing is that their search and advertisement algorithms become as great as possible, and that their infrastructure costs gets optimized down as much as possible. They need the best engineers in the world to work on those things becaus...
wat? how did you draw that crazy conclusion?
even competitive programmers says that there's probably negative correlation between good engineer and competitive programmer
and I'm saying this as a person who doesn't mind having to solve algo questions
They don't. I've worked with some of the best competitive programmers in the world, they do think it is a positive and look for people with a strong competitive programming background.
However, if your interview process only selects for ability to solve algorithm problems, then people with a lot of practice solving algorithm problems will over perform relative their level of talent, so when you run a dumb regression analysis it might look like competitive programming experience is a negative signal. So if someone has competitive programming experience then you want to bring them in for an interview, but you should probably be harder on them in the interview than regular candidates to account for this effect.
That's fair
I'm unable to find what I've been reading, probably some thread on spoj (edit I found this [1]) or this https://codeforces.com/blog/entry/17302
for now I only found this
https://www.youtube.com/watch?v=F4rykKLcduI&t=435s
by https://codeforces.com/profile/Errichto
_____________________
[1] - https://discuss.spoj.com/t/spoj-a-praca-i-zycie/26380
https://en.wikipedia.org/wiki/SPOJ
Title: SPOJ & work and life
It's written in Polish, but as I see that was about kinda different thing
>How were SPOJ and programming skills useful in the context of your studies?
>Virtually not at all. I will say more - these skills were an obstacle in obtaining the title of engineer.
><rant about higher edu institution>
>...
>How useful are SPOJ and specific, classic algorithms at the programmer's work?
>Wandering around the labor market, I got the impression that it was easier to become a programmer with a vague idea of algorithms and extensive experience in creating specific projects than being a master of all kinds of algorithms, but without experience in coding something bigger.
So the point was kinda different than I remember
_____________________
This is the only relevant part of your comment. 90% of the tech companies out there should not rely on leetcode.
Another thing I noticed is that they stopped asking questions about concurrent code or writing recursive-decent parsers. They seem to be strangely fixated on a narrow and not particularly practical subset of CS body of knowledge. We don't discuss OOP or its alternatives anymore. Companies don't seem to care about my actual hard-won real life experience even when it directly applies to the kind of domain/system they are building.
Don't get me wrong, I expect people to know how to perform depth-first traversal or the general idea behind quick sort. But we are long past that stage even in small companies with average pay at least here in the SFBA.
Edit: I am not a great coder by any means. I just don't like to waste my time when it's clear I am not a fit for the company or the role.
It's not really a pattern of interview I expect most people can implement immediately, but it really gives a good idea of fluency. I just have to home in on what they do know. The easy questions get some momentum going and provide context for the follow ups. When they don't know an answer I'll tell them the answer. Sometimes this leads to them adding some depth beyond that answer as they know what's going on but haven't worked with it formally, or have worked with it in passing, or maybe know it by a different name.
I also suspect it makes candidates more likely to accept because you can develop a bond and have a relatively enjoyable time in interviews like this.
Several times I've had the interview process be so unpleasant I lost all interest in working for the company. I try to be considerate and withdraw so I don't waste everyone's time but once I did get an offer from a company like this.
"You've just created a new Rails app and you change into that directory. What are some of the directories and files you find at that top level?"
"What do you find in Gemfile? Gemfile.lock? What's the difference between the two?" Maybe some discussion of resolving dependencies/versions.
There are numerous "right" answers to which files/directories are at the top level and I don't care which ones they get, but they'll get some and it gets the flow going. Most of them can be taken to greater depth. If I want to guide it away from views into models, at least they know the level of detail we're working at.
Rails is a problem interview, too, because there is so much which is take it or leave it. There are Rails apps with no front end, and someone may have not worked with views for years, or the backend may be Api based and they haven't worked with models in the same way. I've worked on a project where 95% of the work was in the workers, and the user facing code was just nudging things between work states. So I use this to guide the conversations and draw out examples of their experiences when possible.