nope. tech is so fast moving if you're not at least experimenting with your own code then you're not going to be a good fit. we tend to also hire engineers who have contributed to open source.
No kidding. I'd be happy to walk through the stuff I have on github but I also don't want my coworkers measured by whether they do the same.
If anything, personal projects can end up being distractions from focusing singularly on work. Even though I'm entirely in favour of them, I also remember when I had a newborn at home and a family member in hospital, and such projects were not at all feasible.
there's also the issue where the stuff i put on github is for hacky throwaway stuff and not representative of my professional standards for a "real" job. it'd be like interviewing a chef and judging their abilities based off of what they made themselves for dinner last night rather than what they cook in a restaurant
I got irrationally angry about this at a previous job. Everyone was all excited management was having us all do a "hackathon" and I am loud and grumpy about how this should just be part of the normal course of business.
We found solutions and tools that better solve problems but to this day have still not been implemented.
Not everything needs a novel solution but not making room for innovation because the company is operating as a feature factory is boring.
I had to learn cobol last year. Not hard if you have read lots of books from the 70s and 80s and have a strong background in algorithms. It is expensive and annoying to maintain these systems, but until there's a business case to replace the underlying application, he's safe
For those of us already putting in 10-14 hours per day at our day jobs, or who work in regulated industries, this is a nonstarter. My experience is that the people who are really good are kept pretty busy by their employer
I avoid the issue by asking for “a page of code you want to have a conversation about”. If needed, I clarify it can be code they wrote, code they use or just code they are curious an about. Failing to bring something, anything, is obviously a problem (and happens occasionally). The discussion quickly illuminates where the candidate is in their career.
You're missing out on lots of amazing people who treat programming as work and have other hobbies. Or people in temporarily different situations (like young kids). Or people with more important things happening in their life. You're effectively filtering for mostly people in their 20s.
All the good programmers I know have side projects and live and breathe code. None of the mediocre/bad ones I know do.
Think of it this way: what's likely to make you a better programmer: spending half (0.5x) of every day on (a work-mandated subset of) code or spending most (1x) of every day on code? The answer is obviously the latter.
And it sounds weird, but most of my programming knowledge comes from outside of work, even the knowledge that I apply at work. Maybe it's because work naturally discourages exploration since you're focused on the company's priorities. For example, I'm never implementing a collection in C at work (I use `std::vector` and stuff). Yet doing that in my own time taught me why certain operations invalidate iterators -- a thing senior coworkers of mine didn't understand and which helped us catch a bug. :D
Sure, but a good employee is not just a good programmer. You don't want an obsessive, you want someone who is stable and has other things going on in his life.
Tangentially related. I was trying to describe to a friend of mine how being discerning after a few years of experience means I do less work (or toil) for a better outcome, and why it's worth the higher salary I command.
Young me would've enjoyed the process of building a thing, ignoring problems like maintenance burden, how fragile or brittle the new tool is, lessons learned from the old tool, etc. etc. (you know how it goes)
Current me will take a task and mull on it. No pen to paper for _days_, preferring a minor adaptation to the existing process, or discovering that the desire was misguided in the first place. The work not done and wisdom to not do it is worth its weight in gold.
Being really smart and inquisitive for 30 years .. I've been lucky that people paid me a lot of money to do whatever it took to make lots of different products work, at levels from extremely low to high. Just because people don't post their work to public archives doesn't mean they aren't working and learning all day. So this filter tends to catch people who are at second tier employers and are a bit bored
The good programmers I know have hobbies outside of coding too (gardening, retro computing, skiing), but coding is still one of them. If it's none of them, that's a red flag because it's never the case for any of the good ones I know.
Have you ever considered that people who have a single minded focus on software, might be solely good at software to the detriment of other skills?
8 hours of software development is more than enough practice for a developer. Having an employee with a well balanced life is superior to one who is unable to detach from his work.
I very rarely see anyone try anything different in a 9-5 setting. People leave university, join their first job, and spend 3 years learning how to develop in one way, and only one way.
At-work choices are rarely choices, because you just do the next thing the same way as you did the last thing. If you step out of line, management will reel you back in. A 'big' change is switching from Spring to Micronaut (or vice versa).
Some examples: I have pretty strong opinions about ORMs vs SQL. It would be interesting to discuss that on the job with someone who knows both well. But the guy you'll be talking to won't have tried SQL (beyond that academic thing they learnt at uni).
Discussions about languages and types? "Java is good because it has types and JS doesn't" was the level of discourse I got at my first serious job.
Having side projects at least indicates that they can try things out first-hand, own their own feedback loop, experience some surprises, and not just mindlessly cargo-cult "best-practices".
> Q: Assume that you have an infinite stream of data that is coming in from multiple threads in an unordered fashion. Write the stream items to the console in order.
If I were the interviewer I would be interested in what clarifying questions the candidate would ask and/or how they would highlight parameters and assumptions needed because the question is vague to the point of being unanswerable otherwise.
Proof: suppose there is only 1 infinite stream which is always descending. You will never find a lowest value, so you cannot rewrite the stream "in order".
At one job interview I was deliberately given a problem which had no efficient solution. The idea was to find a heuristic which would get close to the optimal solution and run in a reasonable time.
I got one of those at a FAANG interview. I was a bit less experienced and less confident at the time so I left the interview thinking I was misunderstanding some fundamental CS laws.
Sorry, I was paraphrasing the question and not giving every detail that I received. They also stated that there are no gaps, the values increment by one, and start at zero.
The data itself is ordered (which is why the task is to print them in order), but the order you receive the values is can happen in any order (ie the processing was split into multiple threads and each thread is posting back the results from the work).
Ah then that's the classic "merge k sorted streams" question. It's a good question and easy to solve in a coding interview. Good candidate should be able to solve in about 30 minutes. My favorite solution goes something like "put values in a heap and then read them back out" because you only need to read 1 value from each stream at a time.
Depends. If there is a minimum value, every value will appear, and you don't want dups, then this is doable. Say the streams are of natural numbers starting with 1, and every natural number will appear within each stream. The obvious thing to do is to not even bother doing anything with the streams except to drain them, and just write the natural numbers in order to stdout! Yes, I'm chuckling.
It's a trivial toy problem. It's not a useful problem to solve. But the question may be there just to get you to think of just this, and to ask questions.
The coding interview looks different when you view it for what it would be called in other industries: a licensure examination. It looks particularly insane to relicense for every single job you apply to. It also looks supremely unfair to have proctors for this exam with varying expectations and training to actually correctly administer it.
Yeah, but the variation can be good. If I had to imagine what the hypothetical licensing process would be, I imagine it would be something easy that admits tons of mediocre devs. Plus, would it come in different language/subfield flavors to account for different roles? One job would ask me to implement a custom allocator, and I'd pass; another would ask me how to add Frondle to Artifactory, and I'd fail miserably.
Actual engineering licenses in the US have kind of solved this.
There’s the easy exam that pretty much everyone passes eh e they get their degree (the FE), and then there’s the hard one that not even everyone attempts after a couple years of experience (the PE).
And within each level, you specify your discipline (civil, mechanical, etc) and then are required to have deeper knowledge of several subfields within that discipline.
Difficult to apply a lot of that, when in reality there are nearly infinite combinations of domain knowledge, software knowledge, architecture knowledge with languages and platforms. Some requiring more or less depth than others.
Software is a craft discipline... it would be better organized as a guild with reputation at stake in concert with endorsements. But then you risk what is effectively nepotism and politics.
> I imagine it would be something easy that admits tons of mediocre devs
Given that Medical boards and the Bar strike fear into the hearts of students and demand immense preparation, I think we could do pretty well. The test prep mentality isn’t altogether different from the “grinding leetcode” that happens today anyway. The difference is that it would at least be fair.
- licensing ensures only a minimum level of quality
- people with licenses still do interviews, often just as grueling
- licensed careers with high performers (lawyers, doctors, ib, etc) have other forms of filtering which are much more painful, like years of low pay internships
Studying for a few weeks to solve fun puzzles to make 400k sounds like a deal to me.
Hearing from people that did internships in other fields for almost nothing, during which CS students make more than most workers really drove that point home for me.
But being able to program is table stakes to be an effective developer. There better be a lot more to the interview if you are hoping to find rounded developers with proven experience at the level you are hiring for.
At entry level fore sure, but not everyone is going to be equal after 5-10 years of experience. They'll all be able to code of course, but some will have gained valuable experience and moved on in capability and some won't, which of course is why you interview to be selective - to find those who can do more than just regurgitate LeetCode solutions.
"Programming" is only half the battle. E.g. doesn't overcomplicate the codebase with weird one-off solutions, can actually work on features, some of which (God forbid) involve a UI or existing code.
In 20 years, most leetcode style genius types I've worked with really aren't great at this type of SaaS work.
What you want is someone who can break down a problem to the point where they can use the scientific method to come up with an algorithm on their own.
What you get is someone with canned responses in their head. And resultant mass layoffs. We're trying to automate picking just the right puppy instead of going out and playing with a few of them.
As an addendum to "communicating" is documentation. That said, plenty of those who do document/communicate well can still suck at teaching/training at a higher level.
Even in the US, "$400k" is almost certainly an outlier and not the typical case. Ever notice how people making these claims never provide data about actual comp distributions? I've been an engineer for almost 10 years and never had an offer come close to $400k per year.
It's the typical case for Bay Area-HQ tech companies, at essentially all levels for Tier-1 companies, higher levels at Tier-2/3 companies, and specialist roles beyond that.
"software engineering" doesn't print money any more than being "in finance" does, you'll make more or less depending on what company you do it for.
It probably depends. I know at a very senior level in a lot of companies, you can hit a base salary around half that for Staff/Principal positions and some Architect roles, especially in higher paying areas. The value of stock offerings and other compensation, bonuses etc will vary a lot though. You may grind out 5-10 years to reach that level, then 5 years to fully vest at half that pay to see a $1M stock payout at 5 years bringing the average to $400k. Who knows. Different arrangements work differently.
As an aside, when the market is relatively good, don't be afraid to straight up ask where compensation is on given roles when recruiters reach out, or to ask for more than you think you might get. You're pretty unlikely to get $400k or anywhere near it as a base... but you'd be surprised how reachable say $150k+ is as a base salary for a remote position when you aren't in SF or another major/expensive city.
Everyone on HN knows someone whose brother's uncle's girlfriend's nephew's former roommate works at Facebook and made $400K as a developer. And they'll point to that one person and say "See, it is possible to make $400K in tech." Yes, it is technically possible, just like it's possible to do plenty of difficult things. That doesn't make these salaries common. For every 1 person making $400K-800K at FAANG in the Bay Area, how many dozens are working outside the Bay Area and/or for some no-name company making $120K?
I dunno man. A good third of my class seems to be working at FAANG and we're all from an unremarkable university in Eastern Europe. I'd imagine the prospects are much better if you start in the States already.
The official estimate of software developer positions in the United States was 1,656,880 in 2023.[1] 10% of that would be a high estimate of FAANG software developers in all countries. And most in those companies make less.
The numbers are real. I was in a similar incredulous position 4-5 years ago. Hiring is still relatively weak, but you can always try to get an offer yourself. Or even just talk to and ask recruiters; they have no reason to lie. You don't have to take your cousin's girlfriend's uncle's word for it.
A giant proportion of developers have no idea wtf “levels.fyi” is and are quietly writing Java or some shit in some suburban office park in a city you wouldn’t ever bother to visit, for $80k-160k/yr.
Perhaps they meant to say "studying for a few weeks to solve fun puzzles, after getting a CS degree from Stanford, moving to a city where a basic apartment costs $1 million, and getting 8-10 years FAANG experience, also like half that $400k is stock not cash"
None of these things are true of me actually. Skill based interview is an equalizer which overcomes status education. And fine divide the salary by 2 and it’s still better than most “licensed” careers
Not to mention those of us who are self-taught and don't do as well in formal learning environments. One can be higher functioning, but not learn the same as others. It's harder to work around at times, and I do have to use various coping mechanisms in practice (same as setting lots of alarms for meetings through the day). I don't always do as well in converting literal leet code pre-screenings online though. I'd rather do a take-home type assignment (actually make a solution for X) or a person to person screening, where I can ask followup questions to better understand what I'm solving for in context.
Jobs in bay area with most tech companies with 5-10 years of experience at mid level gives you this. With zirp gone this may no longer be true but was the case 2 years ago
I assume the BLS median accounts for all software jobs. That would include software jobs at GM (the US equivalent of BMW), banks, insurance companies, and the rest of the economy. Does it control for industry?
Going back to the phrase "software companies", the closest category-breakdown looks to be "Software Publishers", where the mean average [0] rises a bit to $149k. (I'm going to assume that the relative difference in means is similar to the difference in not-shown medians.)
You can see higher values, but I think they look suspiciously like generic labels slapped over large something-opolies such as Google ("Web Search Portals, Libraries, Archives, and Other Information Services", mean $215k) or Uber and Lyft ("Taxi and Limousine Service", mean $187k.)
You can make double or triple that in total compensation at the staff or principal level in the Bay Area.
I'm not saying these jobs are easy, or easy to get, but yes, they exist, even in this market.
The reality is the biggest drop-off in recruiting is at the very top of the funnel. Your basic phone screen and first technical screen -- before you get to a full panel. Once there you have way more, qualified, applicants than roles. I'm not sure it really matters which person gets the job, and this is one way you can make a fairly arbitrary decision. It's nice because it's within your control to grind some leetcode lol. From the company's perspective it's fair and has limited negative selection risk, and doesn't lead to as many regrettable hires.
If you've got 100 great candidates and 1 role, does it really matter which great candidate gets the job, so long as the selection process is relatively fair and uniform? From the company's perspective, probably not.
Since there's no feasible way to select the local maxima, the second sort is consistency.
> You can make double or triple that in total compensation at the staff or principal level in the Bay Area.
Sure, it is possible, but that is a small number of roles at a small number of companies, and the vetting process will be more intense. I think the coding interviews we're talking about in this thread are going to be at the junior to mid-level for the most part.
I think it's a lot easier to get a staff role than being a Hollywood actor. Staff+ comprises around 7-10% of a company's engineering team.
A quick Google leads me to believe there's usually about 15-20 A-list actors at any given time and 130,000 members of SAG-AFTRA.
I think it's certainly achievable if you put in the time and effort (and of course if you're good at your job) -- the question is do you want to or not. It can be incredibly demanding and may not be intrinsically rewarding to you. It seems folks generally believe "senior" is an achievable level -- they should, it's usually the first "terminal" level at which it's ok to remain for the rest of your career -- then you shouldn't view staff as intrinsically out of reach.
Low level (L4) at big tech can clear $400k if the stock appreciates, otherwise L5 (“senior”, but really just mid level) will easily be there. Higher level ICs can make 7 figures.
Yes of course this is true but I was responding to this common idea that high salaries, say over 300k, don't actually exist in the US when they do and are very common for Engineers at large SV tech companies.
> - licensing ensures only a minimum level of quality… people with licenses still do interviews, often just as grueling
Agreed, I never asserted against these two points. The point of licensure is to make the first round, which is fairly routine at this point, more equitable and less susceptible to probabilistic effects. It also frees labor from administering this exam round to every candidate.
Other industries do not require repeating the license exam. Many have continuing education or other professional development requirements. These can be expensive and time consuming but not difficult generally.
How is income relevant to the claim interviews are often just as grueling in licensed careers?
Most software developers make <$150,000.[1] The median aerospace engineer makes 99% of the median software developer wage.[2] Probably the median licensed aerospace engineer makes more. The aerospace engineers I know think software interviews are insane. Lawyers and doctors I know think software interviews and their professions' filters are insane. I know nothing about investment banking interviews. But investment banking is known for hazing.
I personally think an industry-wide license would be an easier filter than requiring a degree: Schools don't often teach what you need to know, and sitting for an exam is a lot easier for the "I've been in the industry XX years without a degree."
It would also be easier for the industry to settle on some filters in the exam to block people who just SPAM job postings. For example, open jobs for doctors are only posted on websites that are available to doctors. There is no way for the general public to SPAM job listings for doctors.
The roughest outline for hiring as far as I can follow it is start with 100 resumes, filter it down to 10 by picking ones that are nicely formatted, then filter down to 1 by interviewing. Each level of filtering should be structured so that it biases towards technical competence.
It isn't really a licensure examination because, as you point out, the industry doesn't bother to put the resources in to make sure anything is systemically analysed. It is a process to balance supply and demand of jobs.
90% of the filter is resume formatting!!!? I'm open to hear your experience but that sounds like an arbitrary filter which limits your pool more than it selects for talent
It can depend, but yeah.. when you're filling 1-2 jobs and have a stack of 500 resumes, you're going to do quick filtering based on some pretty arbitrary decisions. Worse is that the recruiting companies will often do weird things to any formatting you have done. If even 50% of the applicants are technically qualified, dumping 90% at random is still likely to get you where you need faster.
It's worth taking the time to ensure your formatting is consistent, with no/few spelling and grammatical issues.
It will also vary by market/location. If you're looking at jobs local to you, it could be very different, especially for those jobs wanting domain experience. Different business sectors also congregate around adjacent businesses... so the choices and technology in one City will vary greatly to other Cities. As will the development process itself.
As someone with no formal degree, but approaching 3 decades of experience, this last (recent) job search was rather brutal. Not enough feedback to know if/what was going on. Could be lack of formal education, could be my age (legal or not).
I'm allergic to resumes that recite every single technology the candidate has ever used, leaving little said about what they accomplished using said technologies.
That's true and that's how I tried to have my resume but I've had quite a few recruiters saying something like "I cannot see a list of technologies you know". Which has been the opposite of what I've been told in the past, where impact and goals are more important.
In my experience, pure black-and-white text does extremely badly.
If someone is in a position where they have to read through 100 resumes, there is a limit to how often they can look at a paragraph-formatted WYSIWYG text edited black and white document that says "I did well at school, I worked some jobs afterwards, now I want to work here". They blend together something shocking.
It is worth having a tasteful dash of colour or maybe a formatted header/footer where possible. Low bar, but there is a big pool of serially unable-to-get-past-screening candidates who can't clear it and you want to stand out from them if at all possible.
You could view the trial-by-Leetcode that people undergo when they switch jobs every 4 years or so as a form of relicensing. One advantage that the current setup has over officially proctored examinations is that you get to try again repeatedly until you are successful.
Other industries do not require repeating the license exam. Many have continuing education or other professional development requirements. These can be expensive and time consuming but not difficult generally.
Real licenses do not expire without notice. Real license exams are more consistent, have clear pass criteria, and have higher pass rates.
Simple coding interviews are fine for filtering candidates.
Coding interviews with aha solutions or time bounds are just asking for people who memorize solutions. Forget about a problem solving engineering culture at these companies. Imo, the harder the leetcode nonsense, the worse the engineering culture.
Don’t overthink it, just talk about something you like about the language. The interviewer wants to hear relevant words that demonstrate you’ve actually worked in the space (and not just did a tutorial or read a blog post.)
Serde? Clap? It’s not the objectively best crate, just one you like a lot and think is well designed. Both of those are “wow, this is way better and yet simpler than every language I’ve used before” crates for me.
It's like when people ask you what your favorite movie or album is. They're not asking you to literally move to a desert island with only that, it just means one you like that comes to mind easily enough.
The modern FAANG Frankenstein interview is a mess. It was so/so at Google 15-20 years ago, it was so/so when initially FB but most everyone cargo-culted it 10-15 years ago. It’s become “grind leetcode” which is clearly a failure mode.
The trouble is it’s a hard problem, and it usually gives -some signal, so it’s sort of better than nothing? I guess? In cases where contract-to-hire make sense for both the company and candidate I generally regard that as ideal, but that’s not every situation.
Someone will solve this, and that person will be very well-loved.
Google did the "how many gas stations in the us" interviews for years before finally realizing that doing well on questions like this had no correlation to success on the job. Now, in reaction to that failure, everyone is doing the LeetCode thing instead, and presumably will eventually realize that this doesn't correlate either (especially irrelevant in this CoPilot era when ability to memorize and code up an algorithm is becoming about as relevant as ability to drive a stick-shift car).
There's really no substitute for interviewing based on candidate's experience.
Apparently the new job market trend is applicants firing off dozens/hundreds of AI generated applications, which are then screened by an AI on the other end.
Haha I’m not sure I agree about how useful CoPilot is in practice (there are certain tasks where it shines but all of the LLMs print invalid code routinely).
With that said you made me laugh out loud because I have a friend who knew two people trying to LLM some minor contract and it just went on forever. I definitely think two LLMs talking to each other with two people copy pasting and not realizing it is worth like, a Rick and Morty episode or something.
Yeah, I've even seen people in top ML jobs glowingly describe the main value of CoPilot as (just) smart autocomplete, but copying/regurgitation of algorithms ought to be something that even today's LLMs should be capable of.
Of course there's always Google search too if you are just looking for an algorithm, but LLMs help in the discovery process since you can describe what you need without knowing the name of it.
Using LLMs to find reasonable subset of algorithms for a problem sounds like a valid use case. Could even get naive implementation and lead to comparing them to pick right one.
If they take the LLMs advice were they astute or just lucky? You'd have to ask the candidate to go into details to get any signal, and if the algorithm is hard enough for them to have to consult an LLM, they might not be able to do that.
I was referring to LLM use on the job (or hobby), not as any part of interviewing process.
The reality is that most programmers can go an entire career without being algorithmically challenged, and on the rare occasion you need to do/optimize something and don't know what the best options are, then you can either just ask a colleague, or ask online, or Google, or nowadays discuss it with an AI!
It absolutely does not always happen. I was asked to do interviews twice with one employer. Both times, I requested the interviewee's resume. Both times I was refused because they didn't want me to "bias" my interview.
It’s not generally the responsibility of the person doing the technical screen to also obtain signal on past experience. At a company with recruiting process these fall on different people specialized in the specific interview types. I’ve done an average of 2-3 interviews a week over several employers for the last 12 years, so over a thousand interviews. I’ve also defined much tech interview loop for my current employer and usually interview from junior IC up to manager of manager roles.
I’ve never asked for candidate resumes because I don’t want to bias myself.
I’m not saying it will always, always happen, it’s just, it overwhelmingly will.
If you don't have a resume, do you still ask about candidates experience?
Many people can talk a good game and come across well but still be useless (e.g. hard fail on the simplest of white board tests).
I haven't done interviewing (on company's side of table) for a long time, but my main tack was always to go over projects on resume, focusing on last 5 years, and drill down to see if they could fluently talk about them, draw architecture diagrams, justify choices, etc, etc.
I you don't ask about past experience (= same as having resume), then how do you even know what the person's experience consists of? Useless cog, or tech lead? Highly agentic or passive, etc?
Maybe we were talking past each other a bit — for the specific technical portion of the interview where you are assessing coding ability (the thing you can leetcode to practice for), the resume doesn’t really add much value and can serve to bias the interviewer.
There is usually an experience and goals interview where a manager interviews the candidate to see if they would make a good fit for the team. That definitely involves a resume and a discussion of past work experience with the objective of confirm if they’re a cog or a tech lead :)
It can be important to split them up especially if you’re relying on more junior technical interviewers to validate coding skill. They won’t always know what they’re looking for yet along this axis.
Why do you think they haven't adjusted again? Is it possible it is actually (loosely) correlated to job performance?
I interviewed at Google and all the questions were practical, challenging, and not found on leetcode.
I've seen all kinds of interviews in my twenty years of experience, and while all types can be done poorly, (including DS&A ones), having some live coding is one of the best signals you can get in a short amount of time.
I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
Conversely, I've seen shy, humble candidates struggle to articulate their skills, and then crush harder and harder coding challenges.
> I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
You're making the opposite point you think you are. How do you know that you observed an inability to code as opposed to interview performance anxiety?
I've been coding for 25 years, but have bombed my share of very simple white-boarding exercises because I panic and my brain absolutely stops working. If it's experience-based or even take home, I generally knock it out of the park.
White-board coding works for FAANG because they have 100,000 applicants at any given time and it doesn't matter if it has an exceptionally high false negative rate. It does not work so well for shops that are struggling to find candidates.
>You're making the opposite point you think you are. How do you know that you observed an inability to code as opposed to interview performance anxiety?
I've done lots of interviews, and I know that if someone has seen a question recently, or if someone is rusty in interviews, or one of a million things can affect someone's live coding ability.
So I'm not looking for speed, I don't give more 'points' to someone who is calm and collected vs. someone who takes twice as long cause they're nervous. They both "pass" in my book.
I also don't particularly care if someone starts intermixing java and C++ syntax or forgets the string library and says len(var) instead of var.len() for example.
I'm even fine with giving hints. But there are literally people, with hints, who cannot write a basic recursive function or understand to check for null references. That isn't being nervous. That's just...being bad at the craft.
> It’s become “grind leetcode” which is clearly a failure mode.
I have no insider knowledge, but I always wondered if those kinds of cultures had a hazing / blind-leading-the-blind aspect to them. I.e. the people who got hired were the ones who jumped through some arbitrary hoops, so they doubled down on those hoops being the right hoops to choose the best candidates.
You see the same effect in the cottage industry built around "cracking the code" and getting hired in these sort of companies. They produce marketing/video content that endlessly repeats "grind leetcode and 3 other simple tips" for getting hired, and then they insist that this is the exact and only reason they got hired (rather than plain luck or some reference), so everyone outside looking in emulates and internalizes this idea as the exact and only way hiring could possibly be effective and "fair". Eventually those practices just become the norm and people are blind to alternatives.
What was once considered a test of "intelligence" and skill in coding (whether a true assumption or not) has just become a test of "how much is a person willing to struggle to earn entry into our hallowed halls". Actual potential ability in the role is secondary to getting the role.
This is a great point. As companies get bigger you can find HR actually codifies the behavior as part of their efforts to reduce unconscious bias in the hiring process. So you end up with developers, who know this is a flawed system, perpetuating it because the rules require it.
Hazing is exactly what it is. It's a club and if you want to join the club you're going to have to go through the rituals. I hate participating in the charade but I do it with as much humanity and compassion as I can muster.
Leetcode has the side-effect of filtering on “can code yes/no” which is a completely fair filter… I just wish it wasn’t difficult DSA problems we used.
The same screen can be done with much simpler problems. This light coding interview should also be more or less pass/fail. Do it before everything else to short circuit those who have no idea how to write a for loop.
Save the interesting insights for real problem solving, code reviews, systems design… DSA under pressure is not something that actually happens on the job IME.
Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
> Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
The same screen can be done with much simpler problems. Or direct questions.
I work at a FAANG (and obviously, I'm not a company spokesperson, just sharing my own experience). Those who are passionate about interviewing internally all seem to agree on not asking leetcode questions. I know leetcode questions get asked anyway, but there's pretty clear internal guidance and training for interviewers saying not to use them.
At least part of the problem is that leetcode questions are easy to ask, and most interviewers don't want to go through the hassle of coming up a question that scales well to the candidate's experience and knowledge.
We scrapped coding tests at our last round of hiring, but the interviewees were so bad they everyone’s time was wasted. We had to bring it back just to filter people out.
We talk about something like this now and then but the thing is, if fizzbuzz is still managing to eliminate more than half the candidates I don't think it's useless.
That's my impression. Anything more complicated and you're starting to filter based on your appreciation of the candidate's choices, style, etc. I.e. things that can be worked on or fitted to the company's work style.
I’ve been having trouble finding the scientific papers behind this theory, but one I’ve latched onto is that “filtering against known strong signals is helpful” but also that “additional filtering beyond that is more harmful than random choice of the remaining pool”.
Basically, it can probably be shown that if you hire a group of completely random sample of resumes vs. hiring a random sample of people who can produce a working fizz-buzz program, that the candidates from the latter group will perform better. But if you then filter the fizz-buzz group by “can they solve this negative base math problem?” you’ll be removing a disproportionate amount of candidates who would have turned out to be excellent for your organization, and your final “super-candidate” pool will actually be weaker (for your org / that position) than the average of all those who could solve fizz-buzz.
I think the research shows that you should filter based on what you know for sure provides a true signal, then select randomly from the pool which passed your known filters. If anyone can help me find anything relevant about this, I’d appreciate it.
Too many orgs treat hiring like the “secretary problem” but that requires the assumption that you can grade everyone accurately on a continuous scale. We can’t yet do that with software engineers - theres no way to say someone is “80% awesome” vs. “93% awesome”.
I don't agree. One part of coding tests is getting the right output, the other part is how they went about it - which is relevent even if they didn't solve the problem. You can tell a lot from the second part.
I'm not a huge fan of coding tests, and I have a lot of sympathy for those who refuse to do them, but it's a case of "It's not you, it's everyone else"
I believe it to be much better to see a candidate write some code quickly, have them write unit tests for it, discover that some cases were not handled properly and then they were.
This is how 99% of developers work, I don't believe anyone who says they write their code perfectly and under stress every single time.
• number theory for django dev: how many prime numbers are there.
• random brain teaser for django dev: infinite lasers pointed in space that turn at each other at the rotational speed of light- does the intersection travel faster than the speed of light
And questions that were very fair that I bombed:
• ms paint fill method on whiteboard. I think I checked diagonals when I wasnt supposed to - i cant remember but they were not happy with my answer.
• for a given phone number and a US dialpad what are all the possible letter combinations for the number? In python this is just the built in cartesian product. But I still struggle to write it from scratch with a recursive function and getting the accumulation correct.
> I have never heard of the negative base question
In what real world situation would you even use a new base system?
Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
I mean we use base-10 and base-2 both all the time. I'm using base-2 to talk to you over the internet right now!
> Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
The paint fill one is funny because the answer they’re probably looking for (demonstrate you’re comfortable with recursion) is actually the naive answer (blows up in practice.)
Your last example reminds me of a particularly absurd one years ago. I got asked to write a function that calculates all the subsets of a given set. The problem wasn't the question, though. It was that the person asking me this didn't even know what "yield" was in Python.
> I got asked to write a function that calculates all the subsets of a given set.
I very much dislike the interview questions which roughly translate to "do you know combinatorics/<insert-some-other-area-of-math>?". Even back when I was personally good at those questions (straight out of University), they were less interesting interview questions.
I'm sure there's some jobs where it's relevant, but for the vast majority of software jobs... not so much. I get why it's done - they can be small, self-contained problems, which don't require a lot of external context. And they can be easier to come up with.
> The problem wasn't the question, though. It was that the person asking me this didn't even know what "yield" was in Python.
That is a problem too. As an interviewer, I'm okay with someone using a language or lang feature I don't know, so long as they can explain it. Some interviewers are much less flexible. Hell, sometimes I'll feign ignorance over some lang feature just to see what their explanation is like, because it's a measure of signal on how well they can teach a feature to a more junior dev.
Thinking like an engineer, from the companies perspective, its just a filter.
While it could be the case that candidate A who passes the coding interview makes a terrible employee, while candidate B who does not pass the interview would make a better one, it doesn't matter. As long as the pool of candidates who pass the coding interview, fares better than the pool who does not, it will be a useful tool until someone comes up with a better one.
And the alternatives were worse. There was a time where unless you graduated from a top-flight school, or were top X% at one of the better state schools, or had some other "in" (e.g. knowing somebody) the cool kids at the time (Microsoft/Apple/Yahoo/Intel/etc.) wouldn't even talk to you
The goal is to view people as different "grades" like a commodity. They aren't measuring directly for skill, instead they're measuring for the odds you match a some "grade" of software developer.
To your point, if memorizing the first 1k digits of pi correlated with a higher "grade", then companies would use that instead.
> There was a time where unless you graduated from a top-flight school, or were top X% at one of the better state schools, or had some other "in" (e.g. knowing somebody) the cool kids at the time (Microsoft/Apple/Yahoo/Intel/etc.) wouldn't even talk to you
I'm a believer in Joel Spolsky's recruiting goal: "Smart and gets things done."[0]
Add "not a jerk" (which I find is part of "gets things done" on an ongoing basis) and everything else is either vanity or decision paralysis on the part of the interviewer.
The article is from someone whose final-boss interview was with...joel spolsky. And it was a pretty useless-in-the-real-world question he was given as a coding exercise (base -2 number conversion).
Yeah interview before Joel was base -2 conversion. Joel's interview was mostly a chat and we mostly talked about my dog (and other things I'm sure but I only remember talking about dogs).
Standardized licenses, per language / domain / etc, that expire and / or something analogous to professional engineering licensure in harder engineering disciplines.
Take the leetcode tests once every 5 years or so. Mentor for a few years with an experienced, and already licensed, engineer and get their approval.
Then the interviews can skip the drudgery and can be higher level and higher quality.
Standardised on what exactly? There are so many methodologies and/or tools, with endless possible combinations, each of which can reach the desired result in terms of functionality and even quality. And enforcing just one means no possibility of progress.
In physical engineering, things are very much limited by physical laws, which don't change (and even if our understanding on them might, that happens rarely enough to adapt). In software engineering, the foundational principles can and do change many times in an average engineer's career.
Not the poster, but I personally know folks who have gone
Assembly (for 6800 series Macs) -> C (PC on windows) -> Java (hosted on prem) -> Cloud hosted Java + JS/React.
Depending on your definition of ‘foundational principles’ either nothing has changed, or everything has. But one thing is clear, the day to day is completely different.
I'm talking about things like 8/16/32/64-bit processors, CISC vs RISC, from mainframes to personal computers, from isolated machines to ubiquity of modern Internet, quantum computing etc. Note that I'm talking about the fundamentals of software engineering, not software science.
- Don't let the hr 20-something non-engineers be your first filter - the hiring eng manager reviews resumes. Let HR review the work history, references, and W2 stuff later.
- Simple, practical coding questions related to your company need (so not algos/leetcode for 90% SaaS CRUD app companies)
- Engineer or manager talk to candidate about past work and projects.
If you are giant FAANG okay fine that won't scale. But most companies smaller than this will do just fine without the assumption of "everyone's a faker! Gotta prove you wanna work here!". This has worked fine for me in hiring past 20 years.
I've talked with applicants about their private projects. It gave me great insights. Also take home tasks with a discussion afterwards are a good thing and can be an alternative if candidates don't have private projects to talk about.
Theoretically, I agree these are a great way of generating some signal where there's not enough to be found by other methods. But, in practice, what I've found is that most of these types of exercises are way overscoped, and even those which are not put a hugely disproportionate time burden on the candidate with minimal return for most of them. In other words, nobody wants to take the time to properly review these things.
I've had much better experiences when people give me code and ask me to review it rather than when they give me a "spec" and ask me to write code based on it. One big red flag for the latter type of assignment is the phrase "production ready," as that can mean so many different things it's a borderline meaningless criterion.
I could really go on and on about this, but I'm going to stop here.
I absolutely agree with you. There are many take home tasks that are way overblown and expect too much. But I am convinced that you can scope it for say 1-2 hours and I think that's a reasonable time investment for an application.
And yes the interviewer has to review that stuff and ask good questions but nobody said this was easy. But for me it showed better outcomes than the usual live leet code interviews.. (I did them too).
Passing one significant whiteboard interview once is all you need, that's your "licensure."
Flip a coin whether I'll ace or fail a random leetcode question but after the one and only time my little monkey dance successfully got me into a "desireable" company, I started confidently rejecting other proposals to whiteboard me.
I've found that asking them to review some obviously bad code with glaring errors and problems is more informative than asking them to solve some random DSA problem.
Candidates who can code well can point out code that has obvious problems. Just ask if this is good or bad, and if it is bad, how they could improve it. This demonstrates competency and doesn't make the interview seem like a grind but instead more like a conversation.
"The martini may be simple, but it is not easy to make an excellent one. It's a very solid test of a bartender's skill because, unlike many drinks, ingredients alone cannot carry the cocktail. A piña colada for example, is mostly about ingredients (are you using a good coconut cream? fresh pineapple?) For the martini the chilling and dilution need to be just right. This tests the bartender's most important skill: mixing. Proper mixing of the beverage is ultimately what makes a martini."
[..]
"martinis are shockingly easy to fuck up. and this conversation is exactly the reason why the martini is a good test of a bartender's capability. being a bartender is more than putting fixed quantities of ingredients in a glass. how do you know when your martini is properly diluted, either by shaking or stirring? a good bartender will know. a bad bartender will not. a terrible bartender won't even realize dilution is crucial."
I don't really drink much and never had a martini in my life, but I thought it was pretty interesting.
True enough... Even in software, I was pretty amazed at how much of a filter of, here's a CSV, use one of N languages to load the data, do a check and output the valid entries to one file and the invalid inputs to an error file. You can use any libraries you like, please create a github repo and share with $ACCOUNT for your solution.
I know not everyone works with CSV necessarily, but there are dozens of libraries for a lot of languages. Even if focused on N being those supported in a given company/org. It should be less than an hour of work. Bonus points for any tests/automation, etc.
Last time I got hit with an interview question like that, my answer ultimately had to be "block the merge and counsel the person who wrote this about performance." I'm still not sure if that's the answer they were looking for (this was for a staff engineer position), but I'd stand by it 100% of the time.
I use the "launch ramp" technique. Ask a series of prompts (instead of a single prompt with a long answer). Explain the prompts that will get progressively more complex, aka the ramp will gradually get steeper and get very steep later. I can stop the interview quickly if the candidate can not find simple mistakes and how to remedy them. I can also jump ahead to complex issues to engage highly qualified candidates.
Just got be careful with the “how to improve” part. In my experience as an interviewee sometimes it becomes a regular algorithmic interview. A couple times I found some N+1 queries or inner loops and was asked to “fix it”, which might just turn into leetcode.
The best code review interviews are the ones where there is a healthy amount of actual code, with a handful of functions and classes, some badly named variables, bad comments, some misleading code paths, couple bad patterns, etc… the worst ones are a non-optimal solution and you’re asked to make it optimal. That’s just leetcode disguised as “code review”.
Leave it open-ended and include code with multiple levels of bad so it's not just a quiz. I used real C code that research scientists had given me. If they look at it and say there's no reason this should be in C and in a dynlang instead, that is fine. If I hand them C code where the entire program is a 1000-line main function, with lots of repetition, hard coded file names, and fixed-sized string buffers, and all they tell me is the indentation is icky: that's a negative signal.
> I've found that asking them to review some obviously bad code with glaring errors and problems is more informative than asking them to solve some random DSA problem.
I once had a coding interview like this, but the problem was that the code was so obviously bad, I couldn't even make sense of what the code was supposed to do if it were good. It felt like the interviewer had just come up with an example of bad code without any context of how the code would make sense if made "good". It was just totally artifical.
If someone had presented the bad code in some Stack Overflow question, I would have started by stepping back to ask, "What are your trying to do?" Except in this case, the interviewer wasn't actually trying to do anything except quiz me.
Identifying a bug in production code would be better, I think.
This is a good idea. This would also show someone's ability to read and contribute to existing code which is a large part of our day to day tasks. There are some that can only solve problems their way, which often means them trying to rewrite everything.
My approach has been giving candidates simple real world problems, e.g. extract the URLs from this file given a spec and a description of a few language builtins. I'll throw a few curveballs in depending on how they do but my goal isn't to stump them.
I'm mostly looking to filter out the candidates that flat out can't code or describe their thought process while coding. You'd be surprised how many candidates I've interviewed pass the resume check, get to the interview and can't reason out a problem that could be solved with two for loops and an if statement.
Honestly, as somebody who is hiring senior devs I can't imagine not doing a coding interview.
Unfortunately, it is very hard to judge somebody's coding ability in discussion alone. You can sort of get the idea whether they have or don't have experience and whether they have luck being asked about topics they know (although you can help your luck just knowing a lot of stuff).
I have seen a lot of candidates who were quite smooth talkers but could not code their way out of a paper bag. Mind I do not mean remembering some complex algorithm. The task is usually some relatively simple data manipulation so that I can see the person approaching the problem, asking questions, getting involved in some discussion, etc.
The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
The trouble is there is a lot of interviewers and many interviewers do not have a basic plan of what they want to achieve with the coding interview. What they want to learn about the candidates. Those interviews tend to suck.
What also sucks is candidates who come to the interview completely unprepared, unable to answer most of the questions or get any progress on the programming tasks and then spew misinformation on the Internet about how supposedly all coding interviews are stupid.
I guess if you can code you may have some misses, usually along with some hits. Sometimes you are out of luck. The point is not that you need to get every job that you apply to, it is enough to get one of them.
But if you can't code at all or you apply to positions that are clearly out of your league, you get rejected on all these interviews. And then what you do? Some people go to write on the Internet how all interviews are stupid without ever considering how they contributed to all those failed interviews, how real life works (ie. 90% of everything is crap including 90% of interviews) and how the situation might look from the other side.
I can't really code myself out of a paper bag either with a stranger watching and questioning me while I'm doing it. I just get so nervous and can barely think straight (in general I get rather nervous being watched). One time they asked me to do a function I literally exactly have on my public GitHub as part of a project (remove duplicate entries from an array without sorting; it's just a few lines and pretty straight-forward) and I couldn't really do it in the interview setting.
Of course I wasn't present in your interviews and I'm not saying some of those people really couldn't code at all. I've worked (well, "worked") with some of them so I know they exist. But I'm not so sure all of those who flunked couldn't code at all. In my experience I'm not a hugely unique individual.
If you don't want to accommodate that then fair enough; it's your interview process. But just saying your perspective is perhaps not entirely correct.
>Unfortunately, it is very hard to judge somebody's coding ability in discussion alone. You can sort of get the idea whether they have or don't have experience and whether they have luck being asked about topics they know (although you can help your luck just knowing a lot of stuff).
I disagree with your stated difficulty in judging coding ability by discussion alone. A skilled interviewer is easily able to ascertain breadth and depth of the candidate's experience provided the interviewer is also an expert and curious. And part of being a skilled interviewer is using all of the information at hand in their CV.
>The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
Would instantly pass on working with you based on this alone. The intentional ambiguity is deceitful. The expected dependence and expected deference is so patronizing and manipulative. I don't need to have an ego measuring contest in an interview. You win, you're the smart, awesome guy. Go enjoy all that success, bro.
I've had too many interviews like this where the dudes on the other side of the table have internalized their superiority at proctoring their simple tasks. These dudes think they are the great gatekeepers of... something. Like, the candidate just wants to cut code so they can pay rent and eat. They don't really care, nor should they, about your product, or your customers. It's an exchange of labor for a pittance of the value produced. And the dudes on the other side of the table are gate keeping it.
Actually, for the past two years I was using the coding task I got to do when I joined the company. This helps me appreciate how I felt when I have seen it for the first time.
> A skilled interviewer is easily able to ascertain breadth and depth of the candidate's experience [without a coding test]
And yet, many interviewers' experience shows that there are people that would pass a discussion-only interview and fail at basic coding tasks. There's plenty such people holding down jobs for a while, so even that may not be a sure indicator of skill.
> The intentional ambiguity is deceitful. The expected dependence and expected deference is so patronizing and manipulative.
I'm ~1.5years into my first job as a Backend Dev so I can't speak much about the industry, but based on my experience and what I hear from others, asking questions and clarifying unclear requirements is part of the job description. I almost never get my tasks in a precisely defined way and a lot of my job is gathering information and asking the right questions to build the right thing, often making my own choices and judgement calls. I assume that these skills are what GP comment is trying to test.
> They don't really care, nor should they, about your product, or your customers.
You can hardly blame a company for preferring someone who does. Or at least, pretends to.
I find a simple realignment would solve a lot of this interview angst: Interviewers seem to interview on the premise that they need to find out what the candidate can't do. But this is not useful. I can tell you what they can't do. To a first approximation, the answer is "everything". I am an experienced and skillful developer, yadda yadda yadda, and I've never touched React, have no game development experience, have never written kernel code, haven't touched matrices since school, have only dabbled in embedded development as a hobbiest, etc. etc. etc. Make a list of all the things I can't do and I look like an idiot. The list of possible things is too long.
The goal of an interview is to find out what the candidate can do.
If you interview someone, and they "fail all the questions we asked", it is not the candidate who has failed... the interviewer has failed. You ran an entire interview and all you know is what the candidate can't do. You have learned virtually nothing. The questions must be adjusted to what the candidate can do. Only for the absolute worst candidates should you ever exit an interview saying they failed at everything. (Sadly, such candidates do exist. For instance, consider the recent stories about AI fraud. If your level of programming skill is "I have ChatGPT", I'm going to have a hard time scoping a question down to you.)
If I had asked this question in an interview, or a related one, I would have stepped the problem down until the candidate could pick it up. (Odds are I'd have started lower and worked my way up anyhow.) If the candidate then sort of finds their footing and once going can start folding in the other requirements as we go, great. Who sits down and designs a system for all twelve adjectives ("reliable", "concurrent", "redundant", "runs on a Z80") we want anyhow? One can not help but design a system one adjective at a time, with the others only kept in mind to make sure we don't back into a corner. There's no reason not to interview that way too. (And I tell the candidate this is what we're going to do, so hopefully they don't feel like I'm out to get them or experience requirements whiplash without knowing why I'm doing this.)
Yes this, and to further this, the "can't do" thing that you measured is in fact extremely partial and incomplete: you've measured that the developer can't write an algorithm on a whiteboard with a felt tip marker in front of a stranger they just met. You don't even know if they can't write the algorithm. Just that they can't do it there.
Is that important to you? Maybe it is for some people. As a person who has been in team lead and hiring capacity before, it is not for me.
I trained for interviewing at Google twice, but never really chose to give interviews despite that being a highly pressured part of the job because I could not philosophically vibe with that process. But what some people who have adopted this process are missing is that Google etc does this only because they are swamped with resumes and have their pick of bazillions of quality engineers. They don't care about false negatives, more false positives.
Startups and smaller companies should care about false negatives. It's hard to find and retain good people. Smaller companies need to aggressively find and cultivate good people to make good teams in order to be/stay competitive -- and that means accepting a diversity of ways of working.
I'm glad whiteboard interviews are dead now that everyone does "virtual" on-site interviews. When I interviewed at Google, I requested a laptop because writing on a whiteboard all day hurts my hand. Every interviewer complained about me using a laptop.
What, exactly, was the substance of their complaint(s), if any? Was it just that you made them do their job in a way different from how they are used to?
The "point" of the whiteboard interview is to see how you think and converse and interact around your coding process, not (necessarily) how accurate/good you are at writing the code. You could write pseudocode that can't compile, and as long as you can explain the algorithm and talk about its complexity you could "pass" (in theory)
Screening interviews do take place in a shared doc or other editor, I believe.
I'm sorry, I don't see how that relates to what I was asking. To be clear, I agree, I just don't understand why you chose to write this in reply to what I wrote.
The complaint of the interviewers would (likely) be that they want to see the candidate go through the whiteboard process, because of the reasons I gave. It's just part of the process. Writing in an editor takes away their ability to probe that process.
Oh, I see. There's no literal reason why that discussion can't take place over a laptop screen, either. So, I suppose it more or less does boil down to "you are making us do our job differently from how we are used to, and we don't like it." That is supremely silly.
I've never had any company let me write pseudocode. I once tried to write "vec" instead of "std::vector" and the interview told me to write valid code on the whiteboard. (And I did explain that I was to avoid writing "std::vector" repeatedly).
I’ve managed to mostly avoid these kinds of interviews—is this for real?
Is this what lots of the folks mean when they claim tons of candidates “can’t even code?” Syntax or some names aren’t right for some particular language?
Shit, I forget the syntax for “for” loops or what the right way to get an array’s length is or what this language’s way of declaring a constructor looks like in languages I’ve written several hundred of lines of in the last few days, routinely, while under no real pressure at all. I crib off my own surrounding code for syntax hints constantly.
I’ve been doing this north of 20 years, and tend to end up as the guy to go to for tricky problems wherever I work, but I would fail 100% of whiteboard code-writing tests that gave any fucks about it being remotely correct in any particular language. I think the last time I could maybe have done it was when I knew only one language, and it was Perl, and I mostly wrote it in notepad—so, like, the first year or so after started writing code.
This was years ago (pre-covid) but what I remember was a mix of:
- "Oh, you're using a laptop"
- "Do you really need to use that?"
- "Just write it on the whiteboard"
- "It would be easier to see on the whiteboard"
They also didn't give me a mouse, so using the trackpad was slow.
What I hate most about whiteboards is how you can't easily insert lines so you have to leave lots of space. I tend to write code like an onion rather than top down.
Here's my take on it: the interview is testing your ability to "show" that you absorbed the content and process of your 4th year data-structures and algorithms classes. The whiteboard process is all about showing them that you can regurgitate what those courses taught you, likely in the manner that the prof taught you. You're essentially playing "professor" for them in front of a "class", for a short period of time. Not showing them that you can write code. The code you write on the whiteboard will look nothing like anything you'll ever write at Google.
I think one way to do well in that interview, is to pretend you're the professor and they're a student, and you're working through course material.
In the end, it's just showing them that you passed the socio-cultural hurdles they think are necessary, even if they no longer explicitly check GPAs and which school you came from at interview time.
Why?
Google is more often than not the first job these employees have after school, and many stay there almost forever after. A large percentage are masters or PhDs.
Google is founded by two Stanford grads, both children of academics, who never worked in the industry outside of Google.
Academia is hence the reference point against which Google measures things.
Google is structured in many ways just like a university. Publish or perish (Design docs, PRDs). Thesis committees (perf "calibration" committee, interview committees, etc) and review (intense code review). Even down to the physical "campus" structure. On site cafeterias, even housing/dorms (GSuites, etc)
It's something that was very foreign to me having worked in the industry for a decade before, without a degree.
> I think one way to do well in that interview, is to pretend you're the professor and they're a student, and you're working through course material.
If that's what they want, they should give out the questions in advance, and have the candidate prepare a slide deck. No professor conducts class by having students show up and fire random questions at them all semester.
More explicitly, if that's what they want, they should say that.
That's one of the higher leverage activities for an instructor. Anyone can go read a book, watch a video, or do some practice problems, and lecturing at a bunch of students isn't a great way to convey information, either in absolute terms or compared to those alternatives. The instructor's job is to tell you which books to read, why the material matters, and contextualize the course against other things you care about. The particular road they tell you to travel depends on where you currently are, and a 2-way conversation is critical to getting good results.
When I interviewed at Google I was making iOS apps using Objective-C at my current job, so I decided to stick with that for the whiteboard so I wouldn't blank out on some syntax at an inopportune time.
Big mistake. The method names get so long I was kept running out of space on the whiteboard and it took forever to write it.
I don't remember exactly what I wrote anymore (it was over a decade ago), but this site[1] has some examples of really long property and method names in Cocoa (its framework), like:
Anyway, for that and other reasons I didn't get the job (I apparently didn't prepare quite the right things, despite preparing for a few hours every night for the two weeks prior, which is how long the recruiter gave me before flying me to Mountain View, so my performance for at least two of the six interviews I had that day were a bit weak).
In some ways I'm grateful for Covid's impact on the job market because I now have many alternatives to physically drawing on a whiteboard. I can use TikZ as fast as my professors can draw, and I can put that on a monitor in a conference room. I've even used it on exams before.
I would probably struggle for reasons unrelated to my actual competence as a programmer if I was forced to use a physical whiteboard during an interview. Your comment made me appreciate that I'm graduating into a different world than 4 years ago.
I just (barely) failed an automated interview by some company who posted on the last Hiring thread. Their approach was interesting but they definitely measure the wrong thing IMO. They were looking for senior Python devs yet assumed that every senior Python dev knew X, Y or Z Python API/feature by heart. I failed a single question, precisely one where I wasn't familiar with the particulars and couldn't "figure it out" live due to the nature of the platform.
I've worked on a Python interpreter (not CPython) for many years now, I just never used that particular module (because we do it differently in our implementation). I suppose businesses try their best, but aren't always measuring what they think they are...
Related to not knowing everything, there's also this mal-assumption that if the candidate doesn't know something simple (and obvious to the interviewer) that they will never be able to figure it out on their own.
I have 30+ years of experience on my CV and I still am being asked to do coding interviews... I flatly refuse every time, because they have no connection to the actual job. The hiring process disregards experience and treats everyone in the same way. Ours is a stupid industry.
I've had people apologize to me for trying to make me do one. The interviewer was smart enough to realize that I was the senior in the room. I would have gotten that job but I declined it for another, more interesting one.
The mistake many companies make with attempting to hire senior developers is losing out on the opportunity to hire the good ones by assuming the candidate will fall on their knees to get the job and do the silly test and be subjected to some prolonged process. The better ones will simply not do that and lose interest the second you say "coding interview". They are only on the market for limited amount of time and all your competitors will be eager to hire them.
Hiring senior developers is mostly a sales job. You need to do your homework (i.e. read the CV, look at the Github) and really sell the notion how amazing it would be for the candidate to work for you and what a great fit they are. I speak from experience; having hired and built a few teams. If there's a lot of doubt after you did your homework, the interview needs to be about building a case that the candidate still has some redeeming features. If there isn't the interview needs to be about quickly confirming key points and then moving onto sales. If after all that you still doubt the person can code, then don't hire them.
"I just don't do online coding exercises. They typically bear no relevance to the project and if you expect me to be solving basic problem in CS while being watched and judged in real time when I join then I don't think I want to join. Also, if you cannot judge my experience based on my CV then I don't think you know who you are looking for."
I agree with the sentiment that the algorithms and data structure coding interview practices are a terrible measure of a software engineer candidate. There's so many other factors that make a good software engineer and to dismiss a candidate because they can't solve a single problem on the fly is wrong.
In my experience, I've never got an offer after bombing the coding portion of an interview even though I shined in system design and behavioral interview rounds.
We’re super proud of our content and just recently 2 people have landed Amazon with us. People actually feel ready for interviews with us. Give it a go :).
The ways every other field on earth interviews people? Do Surgeons need to perform mock surgeries before they are hired? Do Accountants need to complete a test audit? Do Lawyers perform a mock trial?
> Do Surgeons need to perform mock surgeries before they are hired?
They must have a degree from an accredited institution before they can begin practicing, and part of earning the degree is operating on cadavers, so, yes.
> Do Accountants need to complete a test audit?
In order to be a Certified Public Account in the US, you must pass the Uniform Certified Public Accountant Examination, which has a section on auditing and has "task-based simulations", which are:
"CPA Task-Based Simulations are scenario-based questions on the CPA Exam. They are a large part of what makes the CPA Exam so difficult. Each one will introduce a situation, provide data in the form of charts, memos, and emails, and require you to answer a series of questions."
So, I think yes?
> Do Lawyers perform a mock trial?
In order to be a lawyer in the US, you generally need to pass a state's bar examination. Most of those include "performance tests" which require the testee to simulate part of the job of being a lawyer. I don't know think mock trials are part of that, but writing a legal brief or doing other typical lawyer work is.
My understanding is that going to trial is a small fraction of what most lawyers do and lawyers going in that direction will gain that experience as junior members of a law firm.
>They must have a degree from an accredited institution before they can begin practicing, and part of earning the degree is operating on cadavers, so, yes.
You know what I meant. You have merely created a series of strawmen. They don't need to perform a trial surgery every time they interview for a new job. The same goes for all my examples so nice try.
In every profession that is relatively high stakes, there is a pretty concrete chain of validation that the person can do the thing they claim to be able to do and that chain of validation does indeed rest on demonstrated performance at that capability.
Yes, a surgeon doesn't need to perform traial surgeries every time they change jobs. But, also, they only work inside certified, heavily-regulated institutations that are able to vouch for their previous performance.
No such institutions exist for software engineering.
Many accountants are not Certified Public Accountants. Many engineers are not Professional Engineers. Many legal workers are not lawyers. And references in those fields are not more concrete.
Yea, OP's questions show that what we really need is an equivalent to the bar exam for software developers, if we want to move beyond these fizzbuzz interviews. Something that would at least assure a base level of programming skill. You'd still have to test for specific domain knowledge, but a comprehensive "programming bar" test could at least show someone met the minimum.
You ask them about their experience. What projects they've worked on. What things they've created. Then you drill down into the details with them. What was hard? What was surprising? How did they solve it? How would they do it differently?
Or you do a mock design exercise with them and hear them talk through their process. Are they going to use a relational DB or NoSQL? Do they need a queue or an event stream? What language do they want to use? Why?
Or worst case scenario you give them a take home exercise (compensate them!) and then for the follow up you talk with them about their code and the compromises they had to make.
Or hell, if they like white-boarding and want to do it, then go for it. But a lot of people (like me) have a tendency to lock up in those situations.
Classic mistake of overthinking it and failing to realize what interviewer really wants - which is to make sure the candidate can actually write code, like at all. The question itself doesn't really matter as much as it's just a pretext. I actually asked a variation of this question for many years at Google and it was clear within first 5 mins who has been writing code day-to-day and who's been mostly "brining key stakeholders into conversations at appropriate time".
I love the typo at the end. I have definitely worked with some stakeholders that I would have loved to stuff in a jar of pickling spices and leave on a shelf for several years to ferment.
When you feed people confabulated information, it acts as an encompassing medium, a reality buffer, effectively making them more inert, less engaged, in actual reality. Brined.
Exactly this. In the interviews I give I care about whether the candidate can write code, yes, but also talk and think about code.
The conversation is the most important part of the interview, and the thinking (and communication) is the most important thing I'm trying to judge after basic skills.
Like you said, you can get a good sense within the first few lines of pseudocode if someone's at least competent at writing code. But that's just one motivation behind coding questions.
It's also very difficult to talk about code, algorithms, and solving problems without a concrete problem and concrete code in front of the candidate and interviewer. So both the question and the code the candidate writes are mainly context for the conversation where I try to see how the candidate thinks.
These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments.
We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
That's a shame because coding questions get such a bad rap, but I'm not really aware of a better options. Take-home problems and looking at GitHub are unfair to many people. A well-run technical interview should give lots of people a chance to succeed.
Coding tests are an awful place to test someone’s conversational skills. I don’t talk while I code. You don’t either. Honestly I can’t even remember the last time I talked to anyone about the code itself outside of a PR. People talk about architecture and database migrations and why their containers aren’t behaving locally. Nobody ever tests for that stuff.
For me, it's not at all "I'm incapable of explaining this to you given a few minutes to collect my thoughts." and very much "You either get Coding Mode, or you get Conversation Mode, but not only do you never get both at the same time, I need time to context switch from one to the other.".
And I've been programming professionally for quite a long while now, so this quirk of mine doesn't seem to have made it difficult for me to work at programming shops.
As I've gotten older I've discovered that I really can't judge normalcy based on what I find natural to me. Every brain is different and abilities range. Some people visualize things in their head; some people have to talk out their thoughts; some people are more optimistic. If we are only grading on one very particular brain type then we are missing out on the opportunities for diverse thought patterns producing something even better than expected.
This provided me with a fascinating, albeit somewhat familiar, piece of insight: which is that I don't really ever hear my inner monologue. I'm not sure I have one! I'm either typing out my thoughts as I have them or speaking them as I have them.
I struggle in coding interviews precisely because of this: either I end up vocalizing my emotions and insecurities instead of coding, or I end up coding instead of talking about what I'm trying to accomplish. Often I will see many alternate pathways branching out before me, but if I try to start talking about them, I am no longer coding, and so my brain context-switches to "social and emotional."
Probably something I could get better at with practice, but I honestly end up commenting on places like HN simply because it "allows" me to think. If I could have a coding interview in the form of realtime text chat + code, that would be ideal for me.
I guess I have seen companies do things like "contribute to this open source project and work through an MR." I do find that quite appealing as an interview process.
Interestingly, it turns out a large number of people have no inner monologue. [1]
Some studies indicate that it's as low as 30% of people who do (so 70% don't have an inner monologue), while others show the opposite, implying around 75% of people have some amount of inner monologue while 25% do not. It's a difficult subject to test and study since we don't have direct access to people's minds and asking someone what they're thinking about literally forces their thoughts through the filter of language.
This is a fairly common condition, along the same lines as aphantasia (lack of inner-picture, rather than inner-voice). I do not believe there is any “cure” for it.
I don’t have a sense of a persistent inner voice either, but I can verbalise my thoughts just fine. It feels to me like the part of my brain that turns thoughts into English sentences automatically goes to sleep when it’s not being used. And that brain power can be used for something else.
When I’m doing particularly hard programming work, I can’t have words said near me. Even music with lyrics messes me up. I think because it wakes up my “thoughts to English” pathway and that gets in the way of my “thoughts to code” pathway.
Anyway, I don’t want to be cured. There’s nothing wrong with my mind. If anything I feel sorry for people who can’t turn off their inner dialogue, because it means they can never use those neurons for other tasks - like maths or programming.
Personally I can talk while programming if an interviewer wants that, but I’ll be a bit dumber at the keyboard than if I sat in silence.
i am the same except i use music (lyrics and all) to drown everything else out. i don’t. price the lyrics at all, but my brain is processing them because every now and then i suddenly break out of focus and am like “wait, what the hell did that song just say?” usually on comedy songs
Personally I only ask super easy questions because you should at least be able to talk about something trivial. Yet unfortunately the question “find the second largest number in an array of numbers” has a high failure rate as the first question, because there are a lot of people lying on their resume just throwing spaghetti at the walls.
It's not a test of someone's conversational skills, its a test of their technical communication. Conversational skills get tested in the small talk and introduction phases at the start and end of the interview.
That's quite an assumption. You've never heard of pair programming? You've never asked for help on a bug in your code? You've never talked through alternate approaches to a piece of code with a coworker? You've never hashed out an interface or a method signature or some pseudocode while talking through the problem? You've never walked through a calculation with an SME? All of these are "code and talk at the same time" exercises.
If I'm being brutally honest, I have a deep-seated suspicion that everyone who says they can't talk and code at the same time also just cannot code at all. I don't know you, of course, and I'd love to be proven wrong. My sample size is small, but the few people I've met who cannot talk-and-code also simply could not code.
Here's my brutally honest take on pair programming: Usually 1 person wants to do it more than the other, and usually that person is being unnecessarily assertive.
The only scenario I think pair programming is socially acceptable to force on developers is a senior type onboarding a new developer out of necessity - might screen share and direct them around some places to show the ropes.
Of course if you love to hang out with someone else while you write code for some reason - more power to you, have fun. For me it's a private thing, even after 20+ years. If anything the LLM is a much more useful sidekick to figure things out.
> Can't talk and code means can't code at all
I disagree with that, I can't even have lyrics in my music really if I'm working on something super hard especially outside my normal wheelhouse. It would at least be disruptive.
The last time "hanging out and coding" was a thing was learning it for the first time - I used to hang out with friends as a kid and we would all try to figure out what Visual Basic was lol and I remember hanging with a friend learning JavaScript during the early web days, drinking coffee through the night, good times.
These days it would feel forced and can't imagine why anyone would regularly pair program, especially now with LLMs.
Lots of people in tech have never heard of pair programming, because it's an absurd idea. This site isn't just silicon valley. This is a tiny fraction of the tech universe.
You are the exception I think. Most interviewers care about the correct answer. Get it and maybe get the job. Fail and definitely don't get the job.
If the interviewer said at the beginning, "I don't expect you to solve this problem in the 40 minute nor to have an optimal solution. I just want to watch you write some code and hear the problems you foresee and how you'd solve them" then maybe I could relax and do that. But, generally the pressure is on "get this right in 40 minutes or you're rejected"
This is actually why I dislike these "coding interviews are useless" type articles. The issue has as much or more to do with bad interviewers than it does with the fact that it's a coding interview.
When I'm tasked with interviewing candidates and evaluating these basic algorithmic and coding skills, I have a 5-part problem (each building on the next, and only revealed when the previous one is complete) that is basically impossible to finish in the time allotted. I tell the candidate ahead of time that it's an ongoing problem that's designed to not be completable in the time: we're going to work through this problem and see how far we get. I've passed candidates who "failed" the actual problem, when the conversation and coding that were shown still gave me a good understanding of their capabilities.
I, personally, cannot _think_ and _talk_ at the same time. It's just a stream of half-sentences, many of which my brain has already moved on from because what I originally thought won't work.
After writing this article it became very apparent to me that I'm complete garbage at interviews, but I'll outperform and exceed at the actual job function.
In my work, if you literally cannot write any code while also discussing the code, and if you literally cannot express thoughts while also thinking them, then you actually wont exceed at the actual job function, at all. You're not the only programmer on the team. I don't know why people think communication skills are not required for programmers. You won't be coding the correct thing unless you can talk about what you're doing.
And that's all I ever ask in an interview. Ask questions and talk about what you're doing. The worst hires I've ever seen were all the ones who never asked questions and never talked about what they were working on. Half sentences are fine; moving away from the keyboard while we talk is fine; being unable to talk and think at the same time probably is not.
If I'm not cut out to work in your environment, that's fine. I do disagree with your other conclusions, however. I'm not bad at communication, I'm bad at verbal communication while simultaneously trying to solve a problem. I'm excellent at problem solving and simultaneously chatting in something like slack, however.
> In my work, if you literally cannot write any code while also discussing the code, and if you literally cannot express thoughts while also thinking them, then you actually wont exceed at the actual job function, at all.
Followed by
> You're not the only programmer on the team.
It sounds like you're implying some connection between the two, whereas most successful teams don't require the behavior your team is demanding. Including the ones with good communication skills.
I can write code well. I can discuss it well. I simply don't need to do both at the same time. Unless people are in a pair programming session, they don't need to openly discuss the code while they're thinking about them and writing them. They can discuss the problem before and after. Why do they need to discuss it while coding?
It's like telling journalists or authors "Hey, if you can't discuss your story with the editor while you are authoring it then you can't succeed here."
>I don't know why people think communication skills are not required for programmers
That so significantly fails to resemble the claims being made that it strains credulity that it could be a good faith interpretation of the conversation.
>You won't be coding the correct thing unless you can talk about what you're doing.
Maybe, but that has no bearing on whether they need to be done at the same time, which they do not in just about work environment. I guess there's probably somewhere that does mandatory pair programming for everything, but I've certainly never seen it.
Shit, I can’t take notes on a meeting and also participate—like, at all. Decent odds I’ll reach the end and struggle to give you even the gist of what happened, without reading my own notes. If I’m trying to take notes and someone addresses me I’ll be all kinds of confused about what the context is.
And that’s English and mostly just writing what people are talking about, not thinking up novel things to write.
The vast majority of engineering design happens async - by typically a single engineer, puzzling/experimenting over possible solutions and then creating a design doc. Discussion then happens synchronously. Solving a complex design problem on the spot is not the norm.
I personally find system design interviews pretty tough - it's not a mode of operation I ever experience on the job. To solve them at Big Tech, you pretty much have to memorize many design patterns and be able to regurgitate them on the spot. Like algo questions, it's testing your ability to work hard to prepare more than anything else.
Not to say this doesn't have value as a filter, it's just not testing the thing you think it is.
You are not alone! This is also how I run coding interviews - I have absolutely passed candidates who did not actually complete the described problem. I always inform my candidates that we want to have a conversation about the problem and its solution. I also deliberately pick questions that are going to need thought and some design, specifically to spark that conversation - talking about reversing a list gets boring pretty fast.
The issues people have with coding interviews are more about the interviewers than the questions, honestly.
Adding another comment here, because this is part of the reason why I wrote this article.
> These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments.
> We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
I understand all of those things. I've written the same before[1]. However, as clear as your instructions are and as well meaning you may be, it may not help. I can logically understand every word you say, but as soon as that question rolls out, I will now be dealing with stress hormones and 30 years of learned behaviors from thousands of experiences, whether I choose to or not.
So while I applaud your methodology and wholeheartedly agree, just telling people that doesn't guarantee that it's not still an issue because humans are complex organisms.
Meh. I’ve met dozens like you. People who swear they just want to see “how you think.”
Then, in the post interview roundup we talk about the candidates and you’re a bit disheartened that they didn’t complete the exercise, so you give a pass even when every other person in the room gives a thumbs up.
My favorite question to ask these "see how you think" types when it's my turn is:
What is the most impressive thing you've ever done?
Just watch how they struggle with this one. I've never met anyone slinging code tests who is so curious in "how we think" that has ever made anything interesting whether it's code, design, music, a company, anything. It's just a pretentious statement made by gatekeeping noobs who love to interview, nothing more.
This is pretty much how it worked at the robotics company I worked at.
We would give them a whiteboard problem, but:
a) it was a simple, stupid problem in C (C++ was our implementation language, so thinking at byte level was an important skill)
b) we were very generous about minor mistakes like missing semicolons, etc.
c) we were very generous about "points for effort"; if they didn't make it through the problem but we saw that they were on the right track we might pass them. Total frauds outed themselves very early; they would produce between jack and squat in terms of actual code (a lot of bloviation though).
But again, most companies aren't that company, or your company. For most screening coding exercises, a correct answer (and even something like optimal algorithm complexity) is a must to pass the candidate.
The problem is that the candidate has been assured that they will be asked 'leet' code questions where solving the problem isn't enough, they will also be asked about O notation and how the code can be optimized and whether to use memoization or recursion. This is what the books will tell you, this is what YouTube will tell you, this is what 'helpful' recruiters will tell you.
And IME this is what most interviewers have been taught. They've got a list of sample questions, they've been told that if they give the knapsack problem and the interviewee doesn't immediately call out 'dynamic programming' than the interviewee is a pass.
If you only want to see working code than you are the exception rather than the rule.
I will ask all of those questions. But I don't expect perfect answers. You should at least know what big O is. I would really like it if you can tell an O(n^2) algorithm from a linear one. (That is often really important in real-world code). I would like you to consider different ways you can optimize the code.
I don't expect you to quickly crank out a novel optimal algorithm. But I like to see that you can think about how to solve problems programmatically. I would like to see you can identify differences in different algorithms and what tradeoffs there are. Considering different approaches that may be faster also shows that you didn't just memorize an algorithm from somewhere but that too took time to actual understand the problem and build a mental model in your head.
I have given people great reviews when they couldn't come up with a working algorthim because the clearly knew how to think like a programmer and considered all of the important things.
As an electrical engineer who has basically always worked as an embedded programmer, I don't know what big O is. But, I'm probably not applying for your open positions.
It definitely depends on the domain. In embedded programming you can often know that your n is small for your algorithms. If you are making a desktop app or web service it is very important to prepare for people using your service in unexpected ways and a super linear algorithm can result in terrible performance or downtime if not managed correctly.
They only want to see somebody who can get working code and a glimpse of their thought process. But from 100s of mediocre examples, the better coder will have a "better thought process."
Same goes for dating. Of course people will swear up and down they "only consider personality." Turns out, they've met 10 other people with a better personality than you.
Just because they're "only looking for x" doesn't mean they'll accept anybody that clears the bar.
The ultimate read between the lines though is that "oh I'm only looking for xyz, nothing superhuman" in a process where you have 10,000 competitors and applicants will still require high performance on your part. It's just a nicety, a meaningless phrase.
I disagree, I think it's a failure of recruiting if you got someone into an interview who can't code "at all". That must mean nobody at the company is technical "at all" either then, if you are getting people that far along without having any clue if they even write code at all.
I don't have this problem - I can easily tell who is good and who is not good by looking at their stuff online, which repos they are contributing to and what their contribution is. I look at personal projects - I can easily tell what parts they wrote vs didn't because it's usually specific to the project.
I can tell from their blog posts and comments, especially GitHub comments - I can even see if they're pushing features at 11PM on a Friday if the obsession piece is crucial to the hire.
People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Though I have given out 50+ code tests in my career because I had to, I would never choose to do this, if given the chance when hiring someone I would never give them a code test. I think it's an amateur move and wastes everyone's time. At its best (as in the case of CTCI interviews) it's an exclusivity filter for academics who memorized the optimal data structures for algorithms as taught in school, but the candidate might not have any of the skills needed to build app features, perform DevOps, etc. or even operate a Terminal - CTCI doesn't cover anything async, nothing about UIs, APIs, databases, services, git, design, file formats, etc. it's purely academic sport. And like I said, a good developer's work should be highly visible anyway - skip the random code test.
I would spend the recruiting effort finding specific developers using specific technologies that aligns with the role and making them excited about the opportunity rather than canvasing 1000 code tests out to anyone who applies.
> That must mean nobody at the company is technical "at all" either then, if you are getting people that far along without having any clue if they even write code at all.
Lots of people skate by in technical roles by barely doing anything technical. Lots of people also overinflate their achievements in resumes and conversations aka lying. How is non technical recruiter supposed to evaluate their coding chops?
> People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Lol, great method! I have a better one - just hire acm winners. No need to test them
Obviously taken into account - I still have no problem whatsoever identifying great developers on GitHub and I'm sure many other developers who actually code often could too. You have bigger problems if you can't tell if someone is lying or not about their abilities when all their work is visible to you. You should be able to easily tell what is theirs vs not.
> Lol, great method!
Yes people inflate their own egos and abilities - especially those who spend all their time interviewing others instead of building.
I prefer demonstrable experience for an engineer over standardized tests which tell you nothing about any real world experience: App architecture, async programming, APIs, UI, DOM, git, unit/e2e tests, any known framework or library, etc. A person who knows all those but hasn't memorized CTCI is a lot more useful than the CTCI memorizer with no evidence of work ever performed.
> While you're spending all your time and the company's money conducting maniacal quizzes and tests your competition is building features and teams and making progress
If your research on your candidates' profiles is of the same quality as you demonstrated nosing around my profile, I don't think any of your competition should be worried at all ;-)
No way to know for sure, but I'd guess the number of employed developers writing blogs and having personal projects is in the single digits. For developers with more than 20 years experience, I'd be surprised if it's even 1%. I've got more experience than that, which means I have an extensive network of former colleagues at a similar experience level and I don't know one single person who blogs about tech or works on personal projects. We all have busy family lives and plenty of money so absolutely no need to spend any personal time doing work related stuff.
Not to mention github profile eval would be much easier to game in the age of LLMs if it were to become mainstream. You can already kinda see the effects of these with people sending you mindless PRs fixing grammar and whatnot if you maintain any popular projects.
My go-to first screening question, regardless of where I work, is to present some JSON from a public API relevant to the domain (nothing too crazy, maybe 2 or 3 levels of nesting max), then ask the candidate to do a filter plus a sum or max - then for bonus, analysis or refactoring of their choice.
Way too often, this results in a painful slog, lots of hints from me, and, nowadays, snippets of AI generated code copied in that actually solve part of the problem but that they don't read and therefore end up breaking.
Yes. I've filtered out countless candidates who can't code like this. It's amazing how people will fire off applications to jobs they can't do. You might doubt this because you wouldn't do it, but just wait until you get to sift through a fresh wave of applicants.
Has everyone I hired got "perfect marks" on the test? Of course not! It's not about that. It's about seeing how they react to a problem, watching them break it down, ask questions, and, ultimately, get on with it. If the job is too sweep floors you need to be able to hold a broom. It's as simple as that.
Yes, unless you’re in a niche area or only hiring through a network, you’re going to sift through mountains of unqualified candidates because those are mostly the ones on the market (see Market for Lemons[0])
>Classic mistake of overthinking it and failing to realize what interviewer really wants - which is to make sure the candidate can actually write code, like at all.
What about the lad who develops homebrew who got rejected from Google because he wasn't able to invert a binary tree on the spot? Many Googlers use his software internally and externally. If the purpose of the interview is to make sure he can code why did he fail?
Why does "Many Googlers use his software internally and externally" mean he would be a good hire for Google?
From all his public complaining about failing an interview it seems Google did the right thing not hiring him, he has a massive ego and it's very possible that "writing homebrew" is less useful to google than "inverting a binary tree"
It means that he created a tool, with his skills and capabilities, that is a force multiplier for other Google engineers. This is a straight up undeniable example that his capabilities _already_ brought value to Google and their stacked deck of genius non-egostical binary tree inverters.
There's not a more pragmatic measure of whether somebody can code than a track record of a successful code project used by other coders.
Here's another way of phrasing it -- if Linus Torvalds went for an interview with Google would he have to invert a binary tree and if he failed to do so (maybe he misconstrues the question and messes up or him being Linus and just refusing) would that be a good reason to reject him? Linus also can be equally or far more egotistical than Max Howell.
I find the idea that just because someone is an excellent software engineer they are therefore guaranteed to be a good fit for a particular role at Google a bit weird
I'd say that if Linus applied to be a software engineer at Google they should be prepared to invert binary trees or do $generic_leetcode type things because that's the expectation for that role
If they applied to be Google Fellow or some other lofty position then I wouldn't expect them to need to do any coding at all in the interview
You should really read your own link - the part where he admits he made that whole story up. Im gonna guess failing some basic coding is not what tanked him. He still thinks they owe him the job though because he’s kinda famous and did a popular project in a language that Google doesn’t use, so that’s cute.
I also dont see how this one anecdote (even if it was true) invalidates anything I said above. You’re gonna have false negatives in any system you chose unless you just wave everyone in
Not even Max Howell thinks Max Howell has a great attitude. He's often a dick in his own words. Maybe Google would have found a different job for him if he wasn't.
He said 90% of Google engineers used Homebrew. Google engineers said it wasn't true.[1]
He said Homebrew using OS libraries saved a lot of pain. He presented it as an example of why Google should have hired him. Actually it caused enough pain Homebrew stopped it.
If the interviewer want to know if someone can write code at all, then why he/she expects much more than that in an interview?
I believe there is no single declaration about 'what the interviewer' wants.
Especially that several times they do not know themselves, just try to mimic what is expected from them or seen in some random situation they came across. Sometimes, of course.
---
"brining key stakeholders into conversations at appropriate time"
This is some very good quality euphemism here, two thumbs up! : ))
Sounds a bit delusional to me that you can just "teach someone data structures in 1 hour".
Also the role specifics matter here. It might be ok to bring in a junior who needs lots of mentoring onboard. For a tech lead who is going to be driving the direction of a team, I'd expect them to be up to speed and autonomous very quickly.
It's not "teach someone data structures in 1 hour", but teach them the approach needed for that problem in 1 hour.
And it's a pretty insightful comment, because being able to be taught a practically useful algorithm, and use it on a question, in just an hour is a sign of a good candidate. Interviews should often be run just like that.
I don't think "I can teach someone data structures in 1 hour" is a good faith interpretation of what OP meant. I would expect it's more like "I could teach someone about a data structure that could be used to solve this problem efficiently and simply in 1 hour" is closer to the mark.
Training is not billable. Managers decided they only like billable hours. Kind of like when you decide that you only like winning the race and you hate training.
I dropped out of school (English literature) because I needed to make money and pay bills due to a life change, and i applied at a place that does internet stuff to do tech support because computers were a big part of my hobbies. I moved up over the years learning as I went and now I've occupied a few 'engineering' roles. I used to feel somewhat embarrassed to hold an engineering role with no engineering degree.
But I stopped worrying the more I worked with software engineers. They were just people, some sucked, some were great, their education almost never had anything to do with the bracket they fell into. The worst one I've ever met has a PhD. The best one I've met also has a PhD.
Then I found more people in these positions that have degrees or no degrees, and the people that work well both with degrees and without had something in common. They can more or less be taught to do anything, they can extrapolate and apply knowledge outside of the specific example initially given, and they understand the big picture/desired end results. I hate to say it so crass, but they are good at identifying the trivial bullshit and addressing it or cutting through it rather than sitting in it. They "Get it." From the ideal to what the business demands, they just get it.
Training competent people is a boon. I don't know how you seek out interested/curious competency and then train it, but if we can figure that out, it'd be cool. I'm really tired of working with people who have degrees and jobs because their mothers told them it pays well. Computer Scientists with no interest or curiosity about computers.
Training and teaching is hard. "One hour of interviewing could be avoided with one hour of training" is wildly optimistic.
It also ignores the breadth of knowledge you likely want a candidate to have (and are just trying to sample at through a short few interviews). How many hours of training are you willing to sign up for? How confident are you that the knowledge will "stick" for any given candidate? Are you going to fall back to GPA and school prestige to measure how well they've retained knowledge in the past? That seems like a step in the wrong direction.
I don't think it's wildly optimistic, but perhaps we are thinking about it in different ways. I don't think I could teach someone how to implement each data structure in an hour, but I could easily go over maps/queues/stacks/hashtables and tell you when to use each in an hour. I know this because I do that very thing in less than an hour in code reviews.
I do agree that the "stickiness" is iffy, but it usually sticks pretty well. You then have a second problem that can be described as "when you have a hammer, everything is a nail" as they use their new fancy hashtable everywhere.
I refuse to do code tests. I don't mind doing a small 2-3 hour project that I will walk the interviewer through to explain my reasoning for doing things the way that I did, but I cannot code in front of someone. I can't even type correctly on my keyboard if someone is looking over my shoulder.
I've never touched leetcode and I interviewed with Nvidia a few years ago for a position I was an absolute shoe in for, unfortunately they wanted me to do a live leetcode.
I think people get confused with "the best" rather than "good enough". You'll never find "the best", you will find a number of people who will suffice.
Interviewing should be more about avoiding bad candidates than finding the best candidate.
This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
I've given coding interviews, all the questions I've given have been "leetcode easy" level at worst. In person, I usually try to get the person to write up an implementation of Towers of Hanoi. One of the example recursion problems. The interview is not an adversarial process, it's a cooperative one.
I want to see them think and I want to see them come up with code on the fly. Because while we can look up things on the job, at some point, it also requires original thought.
> Interviewing should be more about avoiding bad candidates than finding the best candidate.
> This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
Good points, but I should say that the people that didn't work out weren't always because of technical abilities. The company that had the worst success rate was fully remote, but I don't know if there's any interview process that can help with that.
We used four basic exercises during the first remote interview. Simple things like, "there is a number on each line of this text file. Find the sum of them."
This was an effective method to screen out applicants who didn't have the basic coding skills to align with their stated resume experience. And there were a decent number of these.
We did further development project exercises later in the process that took about 15 hours. We paid the candidates for this time, even those that didn't pass. It was also an effective screening tool.
All our exercises were very "real world." In the candidates own development environment and having been given instructions on how to prepare. They also have access to whatever Internet resources they want while doing the exercises. If they can't do the exercises, they can't do the job.
I know there are mixed opinions on this and I feel for candidates who have to invest a ton of time in exercises like this. But I can't imagine trying to hire without visibility into how they execute relatively basic software development tasks.
I think employers can and should structure the process so the time investment is minimal upfront and only increases as both parties have gotten to know each other and want to proceed.
Depends on the pay. If more companies start doing this, then you can imagine having to slog through 10-20 of these as an applicant will increase the time to find a job significantly. Most people don't get hired off of their first successful round of interviews. If the company pays as much as the job salary itself then I am fine with this.
Adding up all the overhead and stuff that comes before and after, we're talking about an interview process that's 3 full work days. That seems a lot, even when you're paying people. People will either have to take days off their job or sacrifice weekends.
It is a time commitment. We did have four separate exercises and they could be taken individually. So someone could do them a few evenings in a row or all in one day. Whatever worked best for them.
Truth is, I'd much rather hire someone who is interested in working for us for 2-4 weeks as a contractor instead of the normal interview process. But that usually doesn't work for the candidate.
For those wondering, we paid $50 an hour. It wasn't quite wage equivalent, but we felt it was reasonable to consider some of that "gap" as investment on the part of the applicant.
678 comments
[ 2.9 ms ] story [ 413 ms ] threadMay I ask what industry you work in?
I thought he was going to say that it was something like a foundational R&D lab or cybersecurity firm
If anything, personal projects can end up being distractions from focusing singularly on work. Even though I'm entirely in favour of them, I also remember when I had a newborn at home and a family member in hospital, and such projects were not at all feasible.
Why would being up to date matter?
Huh? What kind of stuff ya do?
And if you don't let people experiment during work, but only hire people that do, then that's a bad sign.
We found solutions and tools that better solve problems but to this day have still not been implemented.
Not everything needs a novel solution but not making room for innovation because the company is operating as a feature factory is boring.
I recently meet a programmer in his 50s, still working on Cobol. Sure, you aren't gonna hire him, but do you think he has any worries about his job?
Oh heck nah.
I mean, it's your choice, but your loss too.
Or people who used to be in their 20s.
All the good programmers I know have side projects and live and breathe code. None of the mediocre/bad ones I know do.
Think of it this way: what's likely to make you a better programmer: spending half (0.5x) of every day on (a work-mandated subset of) code or spending most (1x) of every day on code? The answer is obviously the latter.
And it sounds weird, but most of my programming knowledge comes from outside of work, even the knowledge that I apply at work. Maybe it's because work naturally discourages exploration since you're focused on the company's priorities. For example, I'm never implementing a collection in C at work (I use `std::vector` and stuff). Yet doing that in my own time taught me why certain operations invalidate iterators -- a thing senior coworkers of mine didn't understand and which helped us catch a bug. :D
Young me would've enjoyed the process of building a thing, ignoring problems like maintenance burden, how fragile or brittle the new tool is, lessons learned from the old tool, etc. etc. (you know how it goes)
Current me will take a task and mull on it. No pen to paper for _days_, preferring a minor adaptation to the existing process, or discovering that the desire was misguided in the first place. The work not done and wisdom to not do it is worth its weight in gold.
I mean I do have side projects, but those are all in a language I'm not specialized in and often single class small helper programs.
And the best developer I ever worked with had zero public repositories in Github. Now that gave me some perspective.
8 hours of software development is more than enough practice for a developer. Having an employee with a well balanced life is superior to one who is unable to detach from his work.
At-work choices are rarely choices, because you just do the next thing the same way as you did the last thing. If you step out of line, management will reel you back in. A 'big' change is switching from Spring to Micronaut (or vice versa).
Some examples: I have pretty strong opinions about ORMs vs SQL. It would be interesting to discuss that on the job with someone who knows both well. But the guy you'll be talking to won't have tried SQL (beyond that academic thing they learnt at uni).
Discussions about languages and types? "Java is good because it has types and JS doesn't" was the level of discourse I got at my first serious job.
Having side projects at least indicates that they can try things out first-hand, own their own feedback loop, experience some surprises, and not just mindlessly cargo-cult "best-practices".
If you want to be able to experiment and change things, go to a startup.
If I were the interviewer I would be interested in what clarifying questions the candidate would ask and/or how they would highlight parameters and assumptions needed because the question is vague to the point of being unanswerable otherwise.
Proof: suppose there is only 1 infinite stream which is always descending. You will never find a lowest value, so you cannot rewrite the stream "in order".
At one job interview I was deliberately given a problem which had no efficient solution. The idea was to find a heuristic which would get close to the optimal solution and run in a reasonable time.
If the input is infinite and unordered and the task is to produce output in order, how do you know when it’s OK to start writing output?
It's a trivial toy problem. It's not a useful problem to solve. But the question may be there just to get you to think of just this, and to ask questions.
There’s the easy exam that pretty much everyone passes eh e they get their degree (the FE), and then there’s the hard one that not even everyone attempts after a couple years of experience (the PE).
And within each level, you specify your discipline (civil, mechanical, etc) and then are required to have deeper knowledge of several subfields within that discipline.
Software is a craft discipline... it would be better organized as a guild with reputation at stake in concert with endorsements. But then you risk what is effectively nepotism and politics.
Given that Medical boards and the Bar strike fear into the hearts of students and demand immense preparation, I think we could do pretty well. The test prep mentality isn’t altogether different from the “grinding leetcode” that happens today anyway. The difference is that it would at least be fair.
- people with licenses still do interviews, often just as grueling
- licensed careers with high performers (lawyers, doctors, ib, etc) have other forms of filtering which are much more painful, like years of low pay internships
Studying for a few weeks to solve fun puzzles to make 400k sounds like a deal to me.
The number of people who "aced the interview" but are borderline worthless for real product development is not trivial in my experience.
I promise you that's not the case, lol.
In 20 years, most leetcode style genius types I've worked with really aren't great at this type of SaaS work.
What you get is someone with canned responses in their head. And resultant mass layoffs. We're trying to automate picking just the right puppy instead of going out and playing with a few of them.
It leaves out:
- communicating
- teaching
- dealing with ambiguity
- navigating politics
- working cross-functionally
Most high level individual contributors at large tech companies don't even code.
"software engineering" doesn't print money any more than being "in finance" does, you'll make more or less depending on what company you do it for.
What are the tiers? It's close to typical for a Bay Area Google L5 according to levels.fyi.
Google would be a tier1-2, Uber/Lyft/Netflix would be tier 1, Adobe would be tier 2-3, etc.
What makes Lyft higher quality than Google?
If not it's unsurprising. But these actually are typical offers at those companies including Base Salary + Stock + Bonus.
As an aside, when the market is relatively good, don't be afraid to straight up ask where compensation is on given roles when recruiters reach out, or to ask for more than you think you might get. You're pretty unlikely to get $400k or anywhere near it as a base... but you'd be surprised how reachable say $150k+ is as a base salary for a remote position when you aren't in SF or another major/expensive city.
[1] https://www.bls.gov/oes/current/oes151252.htm
But no licensing!
Also you can get this pay outside the bay area, still in high cost of living areas, but for that pay it's really not an issue.
$40k is extremely low for a senior role. Even in places like India or China.
A few very rare jobs at software companies in the US.
The BLS median for 2024 is way lower, at $110k. (Yet at the same time, still much higher than parent-poster's $40k figure.)
You can see higher values, but I think they look suspiciously like generic labels slapped over large something-opolies such as Google ("Web Search Portals, Libraries, Archives, and Other Information Services", mean $215k) or Uber and Lyft ("Taxi and Limousine Service", mean $187k.)
[0] https://www.bls.gov/oes/current/oes151252.htm#ind
I'm not saying these jobs are easy, or easy to get, but yes, they exist, even in this market.
The reality is the biggest drop-off in recruiting is at the very top of the funnel. Your basic phone screen and first technical screen -- before you get to a full panel. Once there you have way more, qualified, applicants than roles. I'm not sure it really matters which person gets the job, and this is one way you can make a fairly arbitrary decision. It's nice because it's within your control to grind some leetcode lol. From the company's perspective it's fair and has limited negative selection risk, and doesn't lead to as many regrettable hires.
If you've got 100 great candidates and 1 role, does it really matter which great candidate gets the job, so long as the selection process is relatively fair and uniform? From the company's perspective, probably not.
Since there's no feasible way to select the local maxima, the second sort is consistency.
Sure, it is possible, but that is a small number of roles at a small number of companies, and the vetting process will be more intense. I think the coding interviews we're talking about in this thread are going to be at the junior to mid-level for the most part.
A quick Google leads me to believe there's usually about 15-20 A-list actors at any given time and 130,000 members of SAG-AFTRA.
I think it's certainly achievable if you put in the time and effort (and of course if you're good at your job) -- the question is do you want to or not. It can be incredibly demanding and may not be intrinsically rewarding to you. It seems folks generally believe "senior" is an achievable level -- they should, it's usually the first "terminal" level at which it's ok to remain for the rest of your career -- then you shouldn't view staff as intrinsically out of reach.
Are you in the EU? Salaries are lower across the board there for engineers.
In the US for Silicon Valley Type large Tech Companies the starting salary for New Graduate SWE's is 150k USD on the low end.
For a senior engineer at FAANG it can easily get above 500k USD.
Most jobs are not FAANG or FAANG adjacent.
This sounds interesting--what did you work on?
Agreed, I never asserted against these two points. The point of licensure is to make the first round, which is fairly routine at this point, more equitable and less susceptible to probabilistic effects. It also frees labor from administering this exam round to every candidate.
My concern would be skill atrophy between relicensing and then you'd still want to do the same sorts of interviews.
The licensed engineers I know think software interviews are insane.
Most software developers make <$150,000.[1] The median aerospace engineer makes 99% of the median software developer wage.[2] Probably the median licensed aerospace engineer makes more. The aerospace engineers I know think software interviews are insane. Lawyers and doctors I know think software interviews and their professions' filters are insane. I know nothing about investment banking interviews. But investment banking is known for hazing.
[1] https://www.bls.gov/oes/current/oes151252.htm
[2] https://www.bls.gov/oes/current/oes172011.htm
It would also be easier for the industry to settle on some filters in the exam to block people who just SPAM job postings. For example, open jobs for doctors are only posted on websites that are available to doctors. There is no way for the general public to SPAM job listings for doctors.
It isn't really a licensure examination because, as you point out, the industry doesn't bother to put the resources in to make sure anything is systemically analysed. It is a process to balance supply and demand of jobs.
It's worth taking the time to ensure your formatting is consistent, with no/few spelling and grammatical issues.
It will also vary by market/location. If you're looking at jobs local to you, it could be very different, especially for those jobs wanting domain experience. Different business sectors also congregate around adjacent businesses... so the choices and technology in one City will vary greatly to other Cities. As will the development process itself.
So one can never win it seems :D
If someone is in a position where they have to read through 100 resumes, there is a limit to how often they can look at a paragraph-formatted WYSIWYG text edited black and white document that says "I did well at school, I worked some jobs afterwards, now I want to work here". They blend together something shocking.
It is worth having a tasteful dash of colour or maybe a formatted header/footer where possible. Low bar, but there is a big pool of serially unable-to-get-past-screening candidates who can't clear it and you want to stand out from them if at all possible.
Those are your top two problems? When was the last time you saw an exam where the proctor was expected to hold a conversation with the examinees?
Real licenses do not expire without notice. Real license exams are more consistent, have clear pass criteria, and have higher pass rates.
Coding interviews with aha solutions or time bounds are just asking for people who memorize solutions. Forget about a problem solving engineering culture at these companies. Imo, the harder the leetcode nonsense, the worse the engineering culture.
I still don’t know what my favourite Rust crate is. I feel I should have one after being asked more than a couple of times.
I don’t have one, though.
Much like I don’t have a favourite number or colour any more. I have favourite sets of numbers and colour palettes!
https://blog.burntsushi.net/regex-internals/
It's like when people ask you what your favorite movie or album is. They're not asking you to literally move to a desert island with only that, it just means one you like that comes to mind easily enough.
The trouble is it’s a hard problem, and it usually gives -some signal, so it’s sort of better than nothing? I guess? In cases where contract-to-hire make sense for both the company and candidate I generally regard that as ideal, but that’s not every situation.
Someone will solve this, and that person will be very well-loved.
There's really no substitute for interviewing based on candidate's experience.
Apparently the new job market trend is applicants firing off dozens/hundreds of AI generated applications, which are then screened by an AI on the other end.
With that said you made me laugh out loud because I have a friend who knew two people trying to LLM some minor contract and it just went on forever. I definitely think two LLMs talking to each other with two people copy pasting and not realizing it is worth like, a Rick and Morty episode or something.
Of course there's always Google search too if you are just looking for an algorithm, but LLMs help in the discovery process since you can describe what you need without knowing the name of it.
The reality is that most programmers can go an entire career without being algorithmically challenged, and on the rare occasion you need to do/optimize something and don't know what the best options are, then you can either just ask a colleague, or ask online, or Google, or nowadays discuss it with an AI!
That always happens in an interview loop too though.
So, yeah, go figure.
I’ve never asked for candidate resumes because I don’t want to bias myself.
I’m not saying it will always, always happen, it’s just, it overwhelmingly will.
Many people can talk a good game and come across well but still be useless (e.g. hard fail on the simplest of white board tests).
I haven't done interviewing (on company's side of table) for a long time, but my main tack was always to go over projects on resume, focusing on last 5 years, and drill down to see if they could fluently talk about them, draw architecture diagrams, justify choices, etc, etc.
I you don't ask about past experience (= same as having resume), then how do you even know what the person's experience consists of? Useless cog, or tech lead? Highly agentic or passive, etc?
There is usually an experience and goals interview where a manager interviews the candidate to see if they would make a good fit for the team. That definitely involves a resume and a discussion of past work experience with the objective of confirm if they’re a cog or a tech lead :)
It can be important to split them up especially if you’re relying on more junior technical interviewers to validate coding skill. They won’t always know what they’re looking for yet along this axis.
I interviewed at Google and all the questions were practical, challenging, and not found on leetcode.
I've seen all kinds of interviews in my twenty years of experience, and while all types can be done poorly, (including DS&A ones), having some live coding is one of the best signals you can get in a short amount of time.
I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
Conversely, I've seen shy, humble candidates struggle to articulate their skills, and then crush harder and harder coding challenges.
You're making the opposite point you think you are. How do you know that you observed an inability to code as opposed to interview performance anxiety?
I've been coding for 25 years, but have bombed my share of very simple white-boarding exercises because I panic and my brain absolutely stops working. If it's experience-based or even take home, I generally knock it out of the park.
White-board coding works for FAANG because they have 100,000 applicants at any given time and it doesn't matter if it has an exceptionally high false negative rate. It does not work so well for shops that are struggling to find candidates.
I've done lots of interviews, and I know that if someone has seen a question recently, or if someone is rusty in interviews, or one of a million things can affect someone's live coding ability.
So I'm not looking for speed, I don't give more 'points' to someone who is calm and collected vs. someone who takes twice as long cause they're nervous. They both "pass" in my book.
I also don't particularly care if someone starts intermixing java and C++ syntax or forgets the string library and says len(var) instead of var.len() for example.
I'm even fine with giving hints. But there are literally people, with hints, who cannot write a basic recursive function or understand to check for null references. That isn't being nervous. That's just...being bad at the craft.
I have no insider knowledge, but I always wondered if those kinds of cultures had a hazing / blind-leading-the-blind aspect to them. I.e. the people who got hired were the ones who jumped through some arbitrary hoops, so they doubled down on those hoops being the right hoops to choose the best candidates.
What was once considered a test of "intelligence" and skill in coding (whether a true assumption or not) has just become a test of "how much is a person willing to struggle to earn entry into our hallowed halls". Actual potential ability in the role is secondary to getting the role.
The same screen can be done with much simpler problems. This light coding interview should also be more or less pass/fail. Do it before everything else to short circuit those who have no idea how to write a for loop.
Save the interesting insights for real problem solving, code reviews, systems design… DSA under pressure is not something that actually happens on the job IME.
Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
The same screen can be done with much simpler problems. Or direct questions.
At least part of the problem is that leetcode questions are easy to ask, and most interviewers don't want to go through the hassle of coming up a question that scales well to the candidate's experience and knowledge.
Basically, it can probably be shown that if you hire a group of completely random sample of resumes vs. hiring a random sample of people who can produce a working fizz-buzz program, that the candidates from the latter group will perform better. But if you then filter the fizz-buzz group by “can they solve this negative base math problem?” you’ll be removing a disproportionate amount of candidates who would have turned out to be excellent for your organization, and your final “super-candidate” pool will actually be weaker (for your org / that position) than the average of all those who could solve fizz-buzz.
I think the research shows that you should filter based on what you know for sure provides a true signal, then select randomly from the pool which passed your known filters. If anyone can help me find anything relevant about this, I’d appreciate it.
Too many orgs treat hiring like the “secretary problem” but that requires the assumption that you can grade everyone accurately on a continuous scale. We can’t yet do that with software engineers - theres no way to say someone is “80% awesome” vs. “93% awesome”.
I'm not a huge fan of coding tests, and I have a lot of sympathy for those who refuse to do them, but it's a case of "It's not you, it's everyone else"
This is how 99% of developers work, I don't believe anyone who says they write their code perfectly and under stress every single time.
My worst interview questions that were silly:
• number theory for django dev: how many prime numbers are there.
• random brain teaser for django dev: infinite lasers pointed in space that turn at each other at the rotational speed of light- does the intersection travel faster than the speed of light
And questions that were very fair that I bombed:
• ms paint fill method on whiteboard. I think I checked diagonals when I wasnt supposed to - i cant remember but they were not happy with my answer.
• for a given phone number and a US dialpad what are all the possible letter combinations for the number? In python this is just the built in cartesian product. But I still struggle to write it from scratch with a recursive function and getting the accumulation correct.
In what real world situation would you even use a new base system?
Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
> Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
That's base-1, aka counting
I very much dislike the interview questions which roughly translate to "do you know combinatorics/<insert-some-other-area-of-math>?". Even back when I was personally good at those questions (straight out of University), they were less interesting interview questions.
I'm sure there's some jobs where it's relevant, but for the vast majority of software jobs... not so much. I get why it's done - they can be small, self-contained problems, which don't require a lot of external context. And they can be easier to come up with.
> The problem wasn't the question, though. It was that the person asking me this didn't even know what "yield" was in Python.
That is a problem too. As an interviewer, I'm okay with someone using a language or lang feature I don't know, so long as they can explain it. Some interviewers are much less flexible. Hell, sometimes I'll feign ignorance over some lang feature just to see what their explanation is like, because it's a measure of signal on how well they can teach a feature to a more junior dev.
While it could be the case that candidate A who passes the coding interview makes a terrible employee, while candidate B who does not pass the interview would make a better one, it doesn't matter. As long as the pool of candidates who pass the coding interview, fares better than the pool who does not, it will be a useful tool until someone comes up with a better one.
And the alternatives were worse. There was a time where unless you graduated from a top-flight school, or were top X% at one of the better state schools, or had some other "in" (e.g. knowing somebody) the cool kids at the time (Microsoft/Apple/Yahoo/Intel/etc.) wouldn't even talk to you
The goal is to view people as different "grades" like a commodity. They aren't measuring directly for skill, instead they're measuring for the odds you match a some "grade" of software developer.
To your point, if memorizing the first 1k digits of pi correlated with a higher "grade", then companies would use that instead.
I don't think this has changed.
Add "not a jerk" (which I find is part of "gets things done" on an ongoing basis) and everything else is either vanity or decision paralysis on the part of the interviewer.
[0]https://www.joelonsoftware.com/2007/06/05/smart-and-gets-thi...
Seriously, what are the alternatives?
Take the leetcode tests once every 5 years or so. Mentor for a few years with an experienced, and already licensed, engineer and get their approval. Then the interviews can skip the drudgery and can be higher level and higher quality.
In physical engineering, things are very much limited by physical laws, which don't change (and even if our understanding on them might, that happens rarely enough to adapt). In software engineering, the foundational principles can and do change many times in an average engineer's career.
I'd like to hear more.
Assembly (for 6800 series Macs) -> C (PC on windows) -> Java (hosted on prem) -> Cloud hosted Java + JS/React.
Depending on your definition of ‘foundational principles’ either nothing has changed, or everything has. But one thing is clear, the day to day is completely different.
The licence you value wouldn't be the licence I value.
- Simple, practical coding questions related to your company need (so not algos/leetcode for 90% SaaS CRUD app companies)
- Engineer or manager talk to candidate about past work and projects.
If you are giant FAANG okay fine that won't scale. But most companies smaller than this will do just fine without the assumption of "everyone's a faker! Gotta prove you wanna work here!". This has worked fine for me in hiring past 20 years.
Theoretically, I agree these are a great way of generating some signal where there's not enough to be found by other methods. But, in practice, what I've found is that most of these types of exercises are way overscoped, and even those which are not put a hugely disproportionate time burden on the candidate with minimal return for most of them. In other words, nobody wants to take the time to properly review these things.
I've had much better experiences when people give me code and ask me to review it rather than when they give me a "spec" and ask me to write code based on it. One big red flag for the latter type of assignment is the phrase "production ready," as that can mean so many different things it's a borderline meaningless criterion.
I could really go on and on about this, but I'm going to stop here.
And yes the interviewer has to review that stuff and ask good questions but nobody said this was easy. But for me it showed better outcomes than the usual live leet code interviews.. (I did them too).
Flip a coin whether I'll ace or fail a random leetcode question but after the one and only time my little monkey dance successfully got me into a "desireable" company, I started confidently rejecting other proposals to whiteboard me.
Candidates who can code well can point out code that has obvious problems. Just ask if this is good or bad, and if it is bad, how they could improve it. This demonstrates competency and doesn't make the interview seem like a grind but instead more like a conversation.
"The martini may be simple, but it is not easy to make an excellent one. It's a very solid test of a bartender's skill because, unlike many drinks, ingredients alone cannot carry the cocktail. A piña colada for example, is mostly about ingredients (are you using a good coconut cream? fresh pineapple?) For the martini the chilling and dilution need to be just right. This tests the bartender's most important skill: mixing. Proper mixing of the beverage is ultimately what makes a martini."
[..]
"martinis are shockingly easy to fuck up. and this conversation is exactly the reason why the martini is a good test of a bartender's capability. being a bartender is more than putting fixed quantities of ingredients in a glass. how do you know when your martini is properly diluted, either by shaking or stirring? a good bartender will know. a bad bartender will not. a terrible bartender won't even realize dilution is crucial."
I don't really drink much and never had a martini in my life, but I thought it was pretty interesting.
I know not everyone works with CSV necessarily, but there are dozens of libraries for a lot of languages. Even if focused on N being those supported in a given company/org. It should be less than an hour of work. Bonus points for any tests/automation, etc.
Last time I got hit with an interview question like that, my answer ultimately had to be "block the merge and counsel the person who wrote this about performance." I'm still not sure if that's the answer they were looking for (this was for a staff engineer position), but I'd stand by it 100% of the time.
The best code review interviews are the ones where there is a healthy amount of actual code, with a handful of functions and classes, some badly named variables, bad comments, some misleading code paths, couple bad patterns, etc… the worst ones are a non-optimal solution and you’re asked to make it optimal. That’s just leetcode disguised as “code review”.
I once had a coding interview like this, but the problem was that the code was so obviously bad, I couldn't even make sense of what the code was supposed to do if it were good. It felt like the interviewer had just come up with an example of bad code without any context of how the code would make sense if made "good". It was just totally artifical.
If someone had presented the bad code in some Stack Overflow question, I would have started by stepping back to ask, "What are your trying to do?" Except in this case, the interviewer wasn't actually trying to do anything except quiz me.
Identifying a bug in production code would be better, I think.
And the patch:
- does fix something but not the described bug.
- could do the same thing in a third of the added line count.
- has typos or other errors.
I'm mostly looking to filter out the candidates that flat out can't code or describe their thought process while coding. You'd be surprised how many candidates I've interviewed pass the resume check, get to the interview and can't reason out a problem that could be solved with two for loops and an if statement.
Unfortunately, it is very hard to judge somebody's coding ability in discussion alone. You can sort of get the idea whether they have or don't have experience and whether they have luck being asked about topics they know (although you can help your luck just knowing a lot of stuff).
I have seen a lot of candidates who were quite smooth talkers but could not code their way out of a paper bag. Mind I do not mean remembering some complex algorithm. The task is usually some relatively simple data manipulation so that I can see the person approaching the problem, asking questions, getting involved in some discussion, etc.
The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
The trouble is there is a lot of interviewers and many interviewers do not have a basic plan of what they want to achieve with the coding interview. What they want to learn about the candidates. Those interviews tend to suck.
What also sucks is candidates who come to the interview completely unprepared, unable to answer most of the questions or get any progress on the programming tasks and then spew misinformation on the Internet about how supposedly all coding interviews are stupid.
I guess if you can code you may have some misses, usually along with some hits. Sometimes you are out of luck. The point is not that you need to get every job that you apply to, it is enough to get one of them.
But if you can't code at all or you apply to positions that are clearly out of your league, you get rejected on all these interviews. And then what you do? Some people go to write on the Internet how all interviews are stupid without ever considering how they contributed to all those failed interviews, how real life works (ie. 90% of everything is crap including 90% of interviews) and how the situation might look from the other side.
Of course I wasn't present in your interviews and I'm not saying some of those people really couldn't code at all. I've worked (well, "worked") with some of them so I know they exist. But I'm not so sure all of those who flunked couldn't code at all. In my experience I'm not a hugely unique individual.
If you don't want to accommodate that then fair enough; it's your interview process. But just saying your perspective is perhaps not entirely correct.
I disagree with your stated difficulty in judging coding ability by discussion alone. A skilled interviewer is easily able to ascertain breadth and depth of the candidate's experience provided the interviewer is also an expert and curious. And part of being a skilled interviewer is using all of the information at hand in their CV.
>The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
Would instantly pass on working with you based on this alone. The intentional ambiguity is deceitful. The expected dependence and expected deference is so patronizing and manipulative. I don't need to have an ego measuring contest in an interview. You win, you're the smart, awesome guy. Go enjoy all that success, bro.
I've had too many interviews like this where the dudes on the other side of the table have internalized their superiority at proctoring their simple tasks. These dudes think they are the great gatekeepers of... something. Like, the candidate just wants to cut code so they can pay rent and eat. They don't really care, nor should they, about your product, or your customers. It's an exchange of labor for a pittance of the value produced. And the dudes on the other side of the table are gate keeping it.
When they themselves didn't have to solve it, since they came up with it.
And yet, many interviewers' experience shows that there are people that would pass a discussion-only interview and fail at basic coding tasks. There's plenty such people holding down jobs for a while, so even that may not be a sure indicator of skill.
> The intentional ambiguity is deceitful. The expected dependence and expected deference is so patronizing and manipulative.
I'm ~1.5years into my first job as a Backend Dev so I can't speak much about the industry, but based on my experience and what I hear from others, asking questions and clarifying unclear requirements is part of the job description. I almost never get my tasks in a precisely defined way and a lot of my job is gathering information and asking the right questions to build the right thing, often making my own choices and judgement calls. I assume that these skills are what GP comment is trying to test.
> They don't really care, nor should they, about your product, or your customers.
You can hardly blame a company for preferring someone who does. Or at least, pretends to.
The goal of an interview is to find out what the candidate can do.
If you interview someone, and they "fail all the questions we asked", it is not the candidate who has failed... the interviewer has failed. You ran an entire interview and all you know is what the candidate can't do. You have learned virtually nothing. The questions must be adjusted to what the candidate can do. Only for the absolute worst candidates should you ever exit an interview saying they failed at everything. (Sadly, such candidates do exist. For instance, consider the recent stories about AI fraud. If your level of programming skill is "I have ChatGPT", I'm going to have a hard time scoping a question down to you.)
If I had asked this question in an interview, or a related one, I would have stepped the problem down until the candidate could pick it up. (Odds are I'd have started lower and worked my way up anyhow.) If the candidate then sort of finds their footing and once going can start folding in the other requirements as we go, great. Who sits down and designs a system for all twelve adjectives ("reliable", "concurrent", "redundant", "runs on a Z80") we want anyhow? One can not help but design a system one adjective at a time, with the others only kept in mind to make sure we don't back into a corner. There's no reason not to interview that way too. (And I tell the candidate this is what we're going to do, so hopefully they don't feel like I'm out to get them or experience requirements whiplash without knowing why I'm doing this.)
Is that important to you? Maybe it is for some people. As a person who has been in team lead and hiring capacity before, it is not for me.
I trained for interviewing at Google twice, but never really chose to give interviews despite that being a highly pressured part of the job because I could not philosophically vibe with that process. But what some people who have adopted this process are missing is that Google etc does this only because they are swamped with resumes and have their pick of bazillions of quality engineers. They don't care about false negatives, more false positives.
Startups and smaller companies should care about false negatives. It's hard to find and retain good people. Smaller companies need to aggressively find and cultivate good people to make good teams in order to be/stay competitive -- and that means accepting a diversity of ways of working.
Screening interviews do take place in a shared doc or other editor, I believe.
Don't get me wrong I think it's silly, but
Or even using V = std::vector<T>;
for a concrete type T
Solves the problem. If someone denies you this, you don’t want to work there :)
Is this what lots of the folks mean when they claim tons of candidates “can’t even code?” Syntax or some names aren’t right for some particular language?
Shit, I forget the syntax for “for” loops or what the right way to get an array’s length is or what this language’s way of declaring a constructor looks like in languages I’ve written several hundred of lines of in the last few days, routinely, while under no real pressure at all. I crib off my own surrounding code for syntax hints constantly.
I’ve been doing this north of 20 years, and tend to end up as the guy to go to for tricky problems wherever I work, but I would fail 100% of whiteboard code-writing tests that gave any fucks about it being remotely correct in any particular language. I think the last time I could maybe have done it was when I knew only one language, and it was Perl, and I mostly wrote it in notepad—so, like, the first year or so after started writing code.
What I hate most about whiteboards is how you can't easily insert lines so you have to leave lots of space. I tend to write code like an onion rather than top down.
I think one way to do well in that interview, is to pretend you're the professor and they're a student, and you're working through course material.
In the end, it's just showing them that you passed the socio-cultural hurdles they think are necessary, even if they no longer explicitly check GPAs and which school you came from at interview time.
Why?
Google is more often than not the first job these employees have after school, and many stay there almost forever after. A large percentage are masters or PhDs.
Google is founded by two Stanford grads, both children of academics, who never worked in the industry outside of Google.
Academia is hence the reference point against which Google measures things.
Google is structured in many ways just like a university. Publish or perish (Design docs, PRDs). Thesis committees (perf "calibration" committee, interview committees, etc) and review (intense code review). Even down to the physical "campus" structure. On site cafeterias, even housing/dorms (GSuites, etc)
It's something that was very foreign to me having worked in the industry for a decade before, without a degree.
If that's what they want, they should give out the questions in advance, and have the candidate prepare a slide deck. No professor conducts class by having students show up and fire random questions at them all semester.
More explicitly, if that's what they want, they should say that.
That's one of the higher leverage activities for an instructor. Anyone can go read a book, watch a video, or do some practice problems, and lecturing at a bunch of students isn't a great way to convey information, either in absolute terms or compared to those alternatives. The instructor's job is to tell you which books to read, why the material matters, and contextualize the course against other things you care about. The particular road they tell you to travel depends on where you currently are, and a 2-way conversation is critical to getting good results.
Big mistake. The method names get so long I was kept running out of space on the whiteboard and it took forever to write it.
I don't remember exactly what I wrote anymore (it was over a decade ago), but this site[1] has some examples of really long property and method names in Cocoa (its framework), like:
splitViewControllerPreferredInterfaceOrientationForPresentation
or
initWithBitmapDataPlanes:pixelsWide:pixelsHigh:bitsPerSample:samplesPerPixel:hasAlpha:isPlanar:colorSpaceName:bitmapFormat:bytesPerRow:bitsPerPixel:
Anyway, for that and other reasons I didn't get the job (I apparently didn't prepare quite the right things, despite preparing for a few hours every night for the two weeks prior, which is how long the recruiter gave me before flying me to Mountain View, so my performance for at least two of the six interviews I had that day were a bit weak).
[1]: https://github.com/Quotation/LongestCocoa
https://en.wikipedia.org/wiki/Dysgraphia
In some ways I'm grateful for Covid's impact on the job market because I now have many alternatives to physically drawing on a whiteboard. I can use TikZ as fast as my professors can draw, and I can put that on a monitor in a conference room. I've even used it on exams before.
I would probably struggle for reasons unrelated to my actual competence as a programmer if I was forced to use a physical whiteboard during an interview. Your comment made me appreciate that I'm graduating into a different world than 4 years ago.
Wait, is Z80 seriously something that you want/design for?
https://tvtropes.org/pmwiki/pmwiki.php/Main/ArsonMurderAndJa...
I've worked on a Python interpreter (not CPython) for many years now, I just never used that particular module (because we do it differently in our implementation). I suppose businesses try their best, but aren't always measuring what they think they are...
The mistake many companies make with attempting to hire senior developers is losing out on the opportunity to hire the good ones by assuming the candidate will fall on their knees to get the job and do the silly test and be subjected to some prolonged process. The better ones will simply not do that and lose interest the second you say "coding interview". They are only on the market for limited amount of time and all your competitors will be eager to hire them.
Hiring senior developers is mostly a sales job. You need to do your homework (i.e. read the CV, look at the Github) and really sell the notion how amazing it would be for the candidate to work for you and what a great fit they are. I speak from experience; having hired and built a few teams. If there's a lot of doubt after you did your homework, the interview needs to be about building a case that the candidate still has some redeeming features. If there isn't the interview needs to be about quickly confirming key points and then moving onto sales. If after all that you still doubt the person can code, then don't hire them.
In my experience, I've never got an offer after bombing the coding portion of an interview even though I shined in system design and behavioral interview rounds.
-organized intelligently
-has simpler explanations than you find online
We’re super proud of our content and just recently 2 people have landed Amazon with us. People actually feel ready for interviews with us. Give it a go :).
They must have a degree from an accredited institution before they can begin practicing, and part of earning the degree is operating on cadavers, so, yes.
> Do Accountants need to complete a test audit?
In order to be a Certified Public Account in the US, you must pass the Uniform Certified Public Accountant Examination, which has a section on auditing and has "task-based simulations", which are:
"CPA Task-Based Simulations are scenario-based questions on the CPA Exam. They are a large part of what makes the CPA Exam so difficult. Each one will introduce a situation, provide data in the form of charts, memos, and emails, and require you to answer a series of questions."
So, I think yes?
> Do Lawyers perform a mock trial?
In order to be a lawyer in the US, you generally need to pass a state's bar examination. Most of those include "performance tests" which require the testee to simulate part of the job of being a lawyer. I don't know think mock trials are part of that, but writing a legal brief or doing other typical lawyer work is.
My understanding is that going to trial is a small fraction of what most lawyers do and lawyers going in that direction will gain that experience as junior members of a law firm.
You know what I meant. You have merely created a series of strawmen. They don't need to perform a trial surgery every time they interview for a new job. The same goes for all my examples so nice try.
Yes, a surgeon doesn't need to perform traial surgeries every time they change jobs. But, also, they only work inside certified, heavily-regulated institutations that are able to vouch for their previous performance.
No such institutions exist for software engineering.
Or you do a mock design exercise with them and hear them talk through their process. Are they going to use a relational DB or NoSQL? Do they need a queue or an event stream? What language do they want to use? Why?
Or worst case scenario you give them a take home exercise (compensate them!) and then for the follow up you talk with them about their code and the compromises they had to make.
Or hell, if they like white-boarding and want to do it, then go for it. But a lot of people (like me) have a tendency to lock up in those situations.
When you feed people confabulated information, it acts as an encompassing medium, a reality buffer, effectively making them more inert, less engaged, in actual reality. Brined.
The conversation is the most important part of the interview, and the thinking (and communication) is the most important thing I'm trying to judge after basic skills.
Like you said, you can get a good sense within the first few lines of pseudocode if someone's at least competent at writing code. But that's just one motivation behind coding questions.
It's also very difficult to talk about code, algorithms, and solving problems without a concrete problem and concrete code in front of the candidate and interviewer. So both the question and the code the candidate writes are mainly context for the conversation where I try to see how the candidate thinks.
These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments.
We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
That's a shame because coding questions get such a bad rap, but I'm not really aware of a better options. Take-home problems and looking at GitHub are unfair to many people. A well-run technical interview should give lots of people a chance to succeed.
The second paragraph in this comment is very, very, very close to how my brain operates: <https://news.ycombinator.com/item?id=40290732>.
And I've been programming professionally for quite a long while now, so this quirk of mine doesn't seem to have made it difficult for me to work at programming shops.
I struggle in coding interviews precisely because of this: either I end up vocalizing my emotions and insecurities instead of coding, or I end up coding instead of talking about what I'm trying to accomplish. Often I will see many alternate pathways branching out before me, but if I try to start talking about them, I am no longer coding, and so my brain context-switches to "social and emotional."
Probably something I could get better at with practice, but I honestly end up commenting on places like HN simply because it "allows" me to think. If I could have a coding interview in the form of realtime text chat + code, that would be ideal for me.
I guess I have seen companies do things like "contribute to this open source project and work through an MR." I do find that quite appealing as an interview process.
Some studies indicate that it's as low as 30% of people who do (so 70% don't have an inner monologue), while others show the opposite, implying around 75% of people have some amount of inner monologue while 25% do not. It's a difficult subject to test and study since we don't have direct access to people's minds and asking someone what they're thinking about literally forces their thoughts through the filter of language.
[1] https://science.howstuffworks.com/life/inside-the-mind/human...
When I’m doing particularly hard programming work, I can’t have words said near me. Even music with lyrics messes me up. I think because it wakes up my “thoughts to English” pathway and that gets in the way of my “thoughts to code” pathway.
Anyway, I don’t want to be cured. There’s nothing wrong with my mind. If anything I feel sorry for people who can’t turn off their inner dialogue, because it means they can never use those neurons for other tasks - like maths or programming.
Personally I can talk while programming if an interviewer wants that, but I’ll be a bit dumber at the keyboard than if I sat in silence.
One for you: Write a recursive function that finds the second largest number in an array of numbers.
That's quite an assumption. You've never heard of pair programming? You've never asked for help on a bug in your code? You've never talked through alternate approaches to a piece of code with a coworker? You've never hashed out an interface or a method signature or some pseudocode while talking through the problem? You've never walked through a calculation with an SME? All of these are "code and talk at the same time" exercises.
If I'm being brutally honest, I have a deep-seated suspicion that everyone who says they can't talk and code at the same time also just cannot code at all. I don't know you, of course, and I'd love to be proven wrong. My sample size is small, but the few people I've met who cannot talk-and-code also simply could not code.
The only scenario I think pair programming is socially acceptable to force on developers is a senior type onboarding a new developer out of necessity - might screen share and direct them around some places to show the ropes.
Of course if you love to hang out with someone else while you write code for some reason - more power to you, have fun. For me it's a private thing, even after 20+ years. If anything the LLM is a much more useful sidekick to figure things out.
> Can't talk and code means can't code at all
I disagree with that, I can't even have lyrics in my music really if I'm working on something super hard especially outside my normal wheelhouse. It would at least be disruptive.
The last time "hanging out and coding" was a thing was learning it for the first time - I used to hang out with friends as a kid and we would all try to figure out what Visual Basic was lol and I remember hanging with a friend learning JavaScript during the early web days, drinking coffee through the night, good times.
These days it would feel forced and can't imagine why anyone would regularly pair program, especially now with LLMs.
Speak for yourself
If the interviewer said at the beginning, "I don't expect you to solve this problem in the 40 minute nor to have an optimal solution. I just want to watch you write some code and hear the problems you foresee and how you'd solve them" then maybe I could relax and do that. But, generally the pressure is on "get this right in 40 minutes or you're rejected"
When I'm tasked with interviewing candidates and evaluating these basic algorithmic and coding skills, I have a 5-part problem (each building on the next, and only revealed when the previous one is complete) that is basically impossible to finish in the time allotted. I tell the candidate ahead of time that it's an ongoing problem that's designed to not be completable in the time: we're going to work through this problem and see how far we get. I've passed candidates who "failed" the actual problem, when the conversation and coding that were shown still gave me a good understanding of their capabilities.
After writing this article it became very apparent to me that I'm complete garbage at interviews, but I'll outperform and exceed at the actual job function.
And that's all I ever ask in an interview. Ask questions and talk about what you're doing. The worst hires I've ever seen were all the ones who never asked questions and never talked about what they were working on. Half sentences are fine; moving away from the keyboard while we talk is fine; being unable to talk and think at the same time probably is not.
Followed by
> You're not the only programmer on the team.
It sounds like you're implying some connection between the two, whereas most successful teams don't require the behavior your team is demanding. Including the ones with good communication skills.
I can write code well. I can discuss it well. I simply don't need to do both at the same time. Unless people are in a pair programming session, they don't need to openly discuss the code while they're thinking about them and writing them. They can discuss the problem before and after. Why do they need to discuss it while coding?
It's like telling journalists or authors "Hey, if you can't discuss your story with the editor while you are authoring it then you can't succeed here."
That so significantly fails to resemble the claims being made that it strains credulity that it could be a good faith interpretation of the conversation.
>You won't be coding the correct thing unless you can talk about what you're doing.
Maybe, but that has no bearing on whether they need to be done at the same time, which they do not in just about work environment. I guess there's probably somewhere that does mandatory pair programming for everything, but I've certainly never seen it.
And that’s English and mostly just writing what people are talking about, not thinking up novel things to write.
I personally find system design interviews pretty tough - it's not a mode of operation I ever experience on the job. To solve them at Big Tech, you pretty much have to memorize many design patterns and be able to regurgitate them on the spot. Like algo questions, it's testing your ability to work hard to prepare more than anything else.
Not to say this doesn't have value as a filter, it's just not testing the thing you think it is.
The issues people have with coding interviews are more about the interviewers than the questions, honestly.
> These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments. > We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
I understand all of those things. I've written the same before[1]. However, as clear as your instructions are and as well meaning you may be, it may not help. I can logically understand every word you say, but as soon as that question rolls out, I will now be dealing with stress hormones and 30 years of learned behaviors from thousands of experiences, whether I choose to or not.
So while I applaud your methodology and wholeheartedly agree, just telling people that doesn't guarantee that it's not still an issue because humans are complex organisms.
[1]: https://darrenkopp.com/posts/2016/02/25/always-learn-somethi...
Then, in the post interview roundup we talk about the candidates and you’re a bit disheartened that they didn’t complete the exercise, so you give a pass even when every other person in the room gives a thumbs up.
Nah. Re evaluate yourself and your biases.
What is the most impressive thing you've ever done?
Just watch how they struggle with this one. I've never met anyone slinging code tests who is so curious in "how we think" that has ever made anything interesting whether it's code, design, music, a company, anything. It's just a pretentious statement made by gatekeeping noobs who love to interview, nothing more.
I tried to save your comment lol
We would give them a whiteboard problem, but:
a) it was a simple, stupid problem in C (C++ was our implementation language, so thinking at byte level was an important skill)
b) we were very generous about minor mistakes like missing semicolons, etc.
c) we were very generous about "points for effort"; if they didn't make it through the problem but we saw that they were on the right track we might pass them. Total frauds outed themselves very early; they would produce between jack and squat in terms of actual code (a lot of bloviation though).
But again, most companies aren't that company, or your company. For most screening coding exercises, a correct answer (and even something like optimal algorithm complexity) is a must to pass the candidate.
And IME this is what most interviewers have been taught. They've got a list of sample questions, they've been told that if they give the knapsack problem and the interviewee doesn't immediately call out 'dynamic programming' than the interviewee is a pass.
If you only want to see working code than you are the exception rather than the rule.
I don't expect you to quickly crank out a novel optimal algorithm. But I like to see that you can think about how to solve problems programmatically. I would like to see you can identify differences in different algorithms and what tradeoffs there are. Considering different approaches that may be faster also shows that you didn't just memorize an algorithm from somewhere but that too took time to actual understand the problem and build a mental model in your head.
I have given people great reviews when they couldn't come up with a working algorthim because the clearly knew how to think like a programmer and considered all of the important things.
> which is to make sure the candidate can actually write code, like at all
It's also still a terrible approach and only gets leet code crammers.
They only want to see somebody who can get working code and a glimpse of their thought process. But from 100s of mediocre examples, the better coder will have a "better thought process."
Same goes for dating. Of course people will swear up and down they "only consider personality." Turns out, they've met 10 other people with a better personality than you.
Just because they're "only looking for x" doesn't mean they'll accept anybody that clears the bar.
The ultimate read between the lines though is that "oh I'm only looking for xyz, nothing superhuman" in a process where you have 10,000 competitors and applicants will still require high performance on your part. It's just a nicety, a meaningless phrase.
I don't have this problem - I can easily tell who is good and who is not good by looking at their stuff online, which repos they are contributing to and what their contribution is. I look at personal projects - I can easily tell what parts they wrote vs didn't because it's usually specific to the project.
I can tell from their blog posts and comments, especially GitHub comments - I can even see if they're pushing features at 11PM on a Friday if the obsession piece is crucial to the hire.
People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Though I have given out 50+ code tests in my career because I had to, I would never choose to do this, if given the chance when hiring someone I would never give them a code test. I think it's an amateur move and wastes everyone's time. At its best (as in the case of CTCI interviews) it's an exclusivity filter for academics who memorized the optimal data structures for algorithms as taught in school, but the candidate might not have any of the skills needed to build app features, perform DevOps, etc. or even operate a Terminal - CTCI doesn't cover anything async, nothing about UIs, APIs, databases, services, git, design, file formats, etc. it's purely academic sport. And like I said, a good developer's work should be highly visible anyway - skip the random code test.
I would spend the recruiting effort finding specific developers using specific technologies that aligns with the role and making them excited about the opportunity rather than canvasing 1000 code tests out to anyone who applies.
Lots of people skate by in technical roles by barely doing anything technical. Lots of people also overinflate their achievements in resumes and conversations aka lying. How is non technical recruiter supposed to evaluate their coding chops?
> People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Lol, great method! I have a better one - just hire acm winners. No need to test them
Obviously taken into account - I still have no problem whatsoever identifying great developers on GitHub and I'm sure many other developers who actually code often could too. You have bigger problems if you can't tell if someone is lying or not about their abilities when all their work is visible to you. You should be able to easily tell what is theirs vs not.
> Lol, great method!
Yes people inflate their own egos and abilities - especially those who spend all their time interviewing others instead of building.
I prefer demonstrable experience for an engineer over standardized tests which tell you nothing about any real world experience: App architecture, async programming, APIs, UI, DOM, git, unit/e2e tests, any known framework or library, etc. A person who knows all those but hasn't memorized CTCI is a lot more useful than the CTCI memorizer with no evidence of work ever performed.
If your research on your candidates' profiles is of the same quality as you demonstrated nosing around my profile, I don't think any of your competition should be worried at all ;-)
Doesn't really happen reliably in the real world.
> You have bigger problems if you can't tell if someone is lying or not ...
Whether or not I have bigger problems is independent from whether I need to recruit more developers.
Way too often, this results in a painful slog, lots of hints from me, and, nowadays, snippets of AI generated code copied in that actually solve part of the problem but that they don't read and therefore end up breaking.
Has everyone I hired got "perfect marks" on the test? Of course not! It's not about that. It's about seeing how they react to a problem, watching them break it down, ask questions, and, ultimately, get on with it. If the job is too sweep floors you need to be able to hold a broom. It's as simple as that.
[0] - https://en.m.wikipedia.org/wiki/The_Market_for_Lemons
What about the lad who develops homebrew who got rejected from Google because he wasn't able to invert a binary tree on the spot? Many Googlers use his software internally and externally. If the purpose of the interview is to make sure he can code why did he fail?
https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
He seems to have a great attitude and would fit right in but it's clear Google is optimising to keep people out rather than find great software Devs.
From all his public complaining about failing an interview it seems Google did the right thing not hiring him, he has a massive ego and it's very possible that "writing homebrew" is less useful to google than "inverting a binary tree"
There's not a more pragmatic measure of whether somebody can code than a track record of a successful code project used by other coders.
I'd say that if Linus applied to be a software engineer at Google they should be prepared to invert binary trees or do $generic_leetcode type things because that's the expectation for that role
If they applied to be Google Fellow or some other lofty position then I wouldn't expect them to need to do any coding at all in the interview
So the higher the role in Google the less the requirements?
I also dont see how this one anecdote (even if it was true) invalidates anything I said above. You’re gonna have false negatives in any system you chose unless you just wave everyone in
Where does it say that?
He said 90% of Google engineers used Homebrew. Google engineers said it wasn't true.[1]
He said Homebrew using OS libraries saved a lot of pain. He presented it as an example of why Google should have hired him. Actually it caused enough pain Homebrew stopped it.
[1] https://news.ycombinator.com/item?id=23844936
I believe there is no single declaration about 'what the interviewer' wants.
Especially that several times they do not know themselves, just try to mimic what is expected from them or seen in some random situation they came across. Sometimes, of course.
---
"brining key stakeholders into conversations at appropriate time"
This is some very good quality euphemism here, two thumbs up! : ))
> What I do know, however, is that for every 1-hour interview where I evaluated if someone knew their data structures, I could have just taught them.
I don't really hear much about training. I doubt it's because we don't do it, but maybe it's not an interesting topic for discussion.
Also the role specifics matter here. It might be ok to bring in a junior who needs lots of mentoring onboard. For a tech lead who is going to be driving the direction of a team, I'd expect them to be up to speed and autonomous very quickly.
And it's a pretty insightful comment, because being able to be taught a practically useful algorithm, and use it on a question, in just an hour is a sign of a good candidate. Interviews should often be run just like that.
But I stopped worrying the more I worked with software engineers. They were just people, some sucked, some were great, their education almost never had anything to do with the bracket they fell into. The worst one I've ever met has a PhD. The best one I've met also has a PhD.
Then I found more people in these positions that have degrees or no degrees, and the people that work well both with degrees and without had something in common. They can more or less be taught to do anything, they can extrapolate and apply knowledge outside of the specific example initially given, and they understand the big picture/desired end results. I hate to say it so crass, but they are good at identifying the trivial bullshit and addressing it or cutting through it rather than sitting in it. They "Get it." From the ideal to what the business demands, they just get it.
Training competent people is a boon. I don't know how you seek out interested/curious competency and then train it, but if we can figure that out, it'd be cool. I'm really tired of working with people who have degrees and jobs because their mothers told them it pays well. Computer Scientists with no interest or curiosity about computers.
It also ignores the breadth of knowledge you likely want a candidate to have (and are just trying to sample at through a short few interviews). How many hours of training are you willing to sign up for? How confident are you that the knowledge will "stick" for any given candidate? Are you going to fall back to GPA and school prestige to measure how well they've retained knowledge in the past? That seems like a step in the wrong direction.
I do agree that the "stickiness" is iffy, but it usually sticks pretty well. You then have a second problem that can be described as "when you have a hammer, everything is a nail" as they use their new fancy hashtable everywhere.
I've never touched leetcode and I interviewed with Nvidia a few years ago for a position I was an absolute shoe in for, unfortunately they wanted me to do a live leetcode.
Interviewing should be more about avoiding bad candidates than finding the best candidate.
This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
I've given coding interviews, all the questions I've given have been "leetcode easy" level at worst. In person, I usually try to get the person to write up an implementation of Towers of Hanoi. One of the example recursion problems. The interview is not an adversarial process, it's a cooperative one.
I want to see them think and I want to see them come up with code on the fly. Because while we can look up things on the job, at some point, it also requires original thought.
> This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
Good points, but I should say that the people that didn't work out weren't always because of technical abilities. The company that had the worst success rate was fully remote, but I don't know if there's any interview process that can help with that.
This was an effective method to screen out applicants who didn't have the basic coding skills to align with their stated resume experience. And there were a decent number of these.
We did further development project exercises later in the process that took about 15 hours. We paid the candidates for this time, even those that didn't pass. It was also an effective screening tool.
All our exercises were very "real world." In the candidates own development environment and having been given instructions on how to prepare. They also have access to whatever Internet resources they want while doing the exercises. If they can't do the exercises, they can't do the job.
I know there are mixed opinions on this and I feel for candidates who have to invest a ton of time in exercises like this. But I can't imagine trying to hire without visibility into how they execute relatively basic software development tasks.
I think employers can and should structure the process so the time investment is minimal upfront and only increases as both parties have gotten to know each other and want to proceed.
Truth is, I'd much rather hire someone who is interested in working for us for 2-4 weeks as a contractor instead of the normal interview process. But that usually doesn't work for the candidate.