> This is like asking a Ruby developer to debug PHP as a test of flexibility.
Sounds like an OK test to me. Great (senior) developers should be able to do that kind of thing. Categorizing yourself exclusively as "a Ruby developer" is a career trap.
being able to do that without significant time constraints isn't that bad imho, but inside of an interview?! That's borderline laughable, for me, as you're only going to be filtering out people based on whether they have any previous PHP exposure or not.
Is 7 figures even realistic? I was under the impression that even a senior FAANG job was still just under $300k typically. Does anyone outside of a CTO/CIO make $1mil+?
I know half a dozen of Senior Staff at FAANGs and they make I would say around 600k - 700k, more of course if stocks appreciate but that's hard to control.
Consistent total compensation offers of $1mil+ is probably reserved for roles above these, sometimes called Principal or Distinguished. I would say the rate of having one of these in an org are like 1 for ~150 engineers.
Isn't the average for software developers in the US ~100k(before taxes)?
Assuming that high earners are offsetting that to the higher end, most people aren't making 6 figures, and the bar isn't which language they're programming in.
That doesn't match my experience on both ends of the hiring table. And forgive me if forwarding the BLS statistics to candidates doesn't get them to accept offers, because I know it wouldn't help me when I can get paid a lot more elsewhere.
> That doesn't match my experience on both ends of the hiring table.
Congratulations, your experience is limited. The BLS stats represent the actual US salary data, not just your limited experience. If you want to make a claim about salaries in the US then look at data across the US and not just whatever is true within your limited bubble.
> And forgive me if forwarding the BLS statistics to candidates doesn't get them to accept offers
One of the few areas of reliable statistics about US software developer pay come from the US Government Bureau of Labor Statistics. The median wage as of May 2023:
$132,270
This means half of all full time employed devs are higher, and half are lower. The mean is more skewed by higher earners but is similar:
$138,110
It also varies quite widely by geographic location, from a mean high of $173,780 in California to only $125,890 in Texas, from $199,800 in San Jose to $132,500 in Austin to $98,960 in rural Kansas (where I have actually developed software before!)
The short of it is, the vast majority of software developers do not make the top salaries. Even L6 is rare within the top tier of tech. There is a lot of delusion in this field around pay, so it's important to be well informed. As a field we are still very well paid compared to most other jobs especially considering our safe working conditions and lack of needed credentials and education. Compared to most of the work on this planet, it's still a goldmine.
If you sample only from the 10,000 or so current Olympic athletes, you will draw similarly incorrect conclusions about the 8 billion planetary general population's athletic ability.
Come on! Everyone posting on HN makes $500K a year, has a vacation home in Tahoe and drives a brand new Ferrari. They think "L6 at Google in the Bay Area" is the median job level for the entire software industry.
Bro this comment is so out of touch it's ridiculous. A 6 figure salary is lucrative. 7 figures is a crazy pipe dream that basically nobody will ever experience.
In the short term. And then the entire industry experiences some kind of a technological shift, as it will for many more cycles, and you're jobless as early as the first wave.
It's ridiculous how developers mindlessly accept that you should constantly be learning to keep yourself relevant, but keep it shallow by just jumping from one tool to another, instead of encouraging deeper knowledge of generalizable patterns that stay relevant across waves of technological disruption.
Exactly. I managed to ride a nice maybe 8 year wave as an "OpenGL expert" but the industry moved on and I didn't go more general with 3D graphics to climb out of my niche. I haven't been in that industry since the OpenGL waterline crested and went back out to sea. Learned that lesson.
Be a generalist. Yes, deep specialists exist and yes, some of them have successful careers based on their deep specialty, but betting on specialization is like a high school kid planning to be a MLB baseball player.
that's what i am trying to explain in every job application. but almost every job description expects multiple years of experience with a very specific tech stack. so far being a generalist with senior level experience did not yield a single positive response, let alone an offer.
Ruby is so god awful yet so many successful companies use it as the bedrock language there's always going to be jobs maintaining the piles of crap it leads to.
That's either a fun or terrible exercise depending on who is administering and how. However, if you said it's a Ruby role and the candidate is good enough to be picky, you may scare them off when they think your description of the role was a lie.
Most of my development experience is in C/C++ and Python.
Know what I'd do if the interviewer asked me to debug PHP? Pretty much return the question:
"I've never used PHP. Are there logging macros/functions defined somewhere? Where do I see the output? syslog maybe? Is there a debugger of some sort I can use? How do I run each `piece` of code in isolation?"
(I am assuming the job listing did not explicitly mention PHP experience. If it did, both myself and the recruiter would absolutely deserve to fail me for this interview).
I agree. I've had this happen often enough on the job that it's not a totally made up example. And usually you'll be one of two or three people in the whole company who is able and willing.
Debugging old DSL vendor specific languages or code so old using, frameworks and standards long out of fashion and support, that they are half way to being a different language.
Adding support for some back ported features or patching security holes in an old client or legacy stacks.
Or at a big company we had some escrow code from a much smaller partner that we ended up becoming responsible for.
Often getting the environment setup for proper debugging is more work than anything.
I interviewed for a Scala role one time despite never having written it professionally. I suppose it's obscure enough that they couldn't afford to be too picky.
It was a pair programming exercise and so with some help from the interviewer and the IDE I was able to fumble through to a working result. I agree it was fun and educational.
Remember that in other fields like medicine, finance, academia, and law, getting in involves 5+ years of hoop jumping and commitment signaling that have nothing to do with the final job.
True, but someone with 10+ years in those fields doesn't face an interview asking them to prescribe treatment for a cold or explain the difference between civil and criminal law.
Yes, and the medical licensing board is in charge of that, not the employer, and every doctor is evaluated to the same standards. Also, doctors are only one of the professions mentioned.
Somehow in programming every interview starts from the assumption that the candidate must prove competence to the employer, and the employer decides what "competent" means.
Genuinely curious, knowing nothing about this field: does, for example a neurosurgeon with 15 year of experience, when looking for a new place to work, have to pass surgery "challenges", like doing a lumbar puncture in 15 min on a fake body to prove his experience?
No, but the medical profession has a ridiculous amount of gatekeeping, so you can't become a neurosurgeon with 15 years of experience in the first place (there are maybe a few thousand worldwide). On the other hand anyone with a computer and a curious mind is a software engineer.
Let me answer your unknowable question with another one:
Which doctor would you prefer do surgery on your brain - the one that jumped through all the hoops and went to 15yrs of med school - or the one that learned brain surgery through a YouTube tutorial?
I think you know the answer.
The problem is not gatekeeping in general it’s the detail and nuance of how gates are being kept and how to find the right amount of gatekeeping.
> Which doctor would you prefer do surgery on your brain - the one that jumped through all the hoops and went to 15yrs of med school - or the one that learned brain surgery through a YouTube tutorial?
My answer is that I don't have access to what I'd really like to base that judgment upon: their success rate, adjusted for the difficulty of the procedure.
Simpler, less-risky procedures would therefore have a lower skill/experience bar.
Basically, I'd want someone with decades of experiences to remove a tumor deep inside my brain, but I'd be much more willing to use someone who learned via YouTube this time last year if I'd suffered a TBI and was experiencing swelling and intra-cranial pressure.
It would be great if there were certain specialized mini-doctor programs that focused on simple things like treating the least risky problems, maybe even with AI to help filter which problems need a “real doctor” vs a mini specialist.
No, because the fact that a neurosurgeon with 15 years of experience is not in jail means they know what they're doing. Not the case for devs unfortunately.
No, but they'll ask for (and actually follow up on) references from their fellow surgeons, and check that they've passed their board exams, and that they're licensed in the state, and that they haven't had huge malpractice claims, and otherwise verify that the person is, in fact, actually a neurosurgeon who does neurosurgeon things in a hospital and is acceptably good at it.
It's not a perfect system, but it's a lot harder to fake being an experienced doctor than it is an experienced developer.
It’s not insane for companies to filter candidates. More people want to be highly paid software engineers than positions available.
The insane thing would be to expect someone to take you at your word for 150k*, when hiring managers know that 50-70% of people with your resume fail to write a nested for loop.
Dunno, all my "normal" engineering job interviews just revolved around my previous work, going through big picture design problems, and specific things related to the job.
Didn't have to solve PDE's on the whiteboard, or regurgitate integral transformations.
I have a couple family members who are doctors. Medical school and residency were nightmares, but after that they are golden. One of them works one week a month and makes 400k. Others have pretty cushy 9-5s and are taking home ludicrous salaries. They are also pretty recession proof and their salaries are fairly immune to economic pressures. I don't think mass doctor layoffs happened in 2008 or 2022.
With software you never have to stop proving yourself, and your skillset is always a few years away from being outdated. A doctor with 20 years of experience would be welcomed anywhere, but an engineer with 20 years experience is viewed with trepidation. The next "big thing" could roll out at anytime and suddenly crypto engineers are getting 800k job offers so everyone furiously tries to learn crypto stuff. A few years later that all dries up and now you have 100k engineers who are out of work and learned tech that no one cares about anymore. All the LLM engineers might be in the same position next year.
Of course. They paid an enormous cost and beat other competitors for that privilege which is carefully guarded by regulation and certification boards.
No doubt software has more instability, but the trade off is that you don’t have to do that multi year grind. Easier fire, easier hire. Similar pay.
> skillset is always a few years away from being outdated.
I strongly disagree with this. In your example you use “crypto” and “llm”. If this is your skill set you are fad chasing and needlessly increasing your exposure to changing markets.
Engineers solve problems by applying math and science expertise. This is a timeless skill.
Pretty common artifact that take-home challenges are being more widespread now that the labor pool is oversupplied with tech talent looking for jobs. Companies need new ways to filter for candidates that are willing to do whatever it takes and work the hardest. The easiest way to do so is give them a "three hour" take-home assignment and see if they are willing to do it.
Doesn't this approach self-select for people who
- value their time less,
- don't have other things to do (family, hobbies, side-projects),
- unemployed people
It's one thing where companies are applying this to 1000 applicants for an entry level position, but they also use this tactic when they reach out to you and tell you how excited they are about your background and resume.
In reality they didn't evaluate your background or resume and are just cold-calling you to fill some portion of their "funnel", then ignore all their excitement and reasons they reached out to you and have you jump through random hoops.
> When was the last time you had to debug an ancient codebase without documentation or help from a team?
I've had to do this at some level in every job.
Somebody left. The code was left to rot, whether it was a separate codebase or it was an entirely separate project. I got assigned to fix it. Sure, I had a team, but nobody knew more than I did, so it was just other devs who might have another perspective.
Collaboration and support might be standard, but mostly in the form of other smart people, not always documentation, up to date software, or people who know anything about the code.
I personally prefer these kinds of challenges as they mean I don't need to maintain two skillsets.
I'm fine with tests, but only if companies pay a standard fee (say, $100) for a dev's time. If a company doesn't respect your time during the interview process, it probably won't while you're on the job.
If someone gives me a take-home assignment that I spend 4 hours on, there is no commitment of time on their side. They may decide to not even look at the results. They may spend five minutes on it.
That's exactly my problem with take-home code challenges. If I go to an interview, and they waste my time, they have to waste their own. But with a take-home assignment, they can waste my time without wasting their own. That may lead them to be more wasteful of my time.
If that’s the case - sure. But that’s not inherent to code challenges. I evaluate results of challenges at work, they require 2-3 hours of candidate time and it rarely takes less than half of that time to fully evaluate and form a recommendation.
Agree, IMO even interviews should be compensated.
Most software companies want to schedule 3-4 interviews each 2 hrs long so they should minimally compensate 1 day pay.
If we assume 15 days off that leaves roughly 245 workdays per year and with a salary of $200k that would be close to $800.
A fair amount of compensation to spend half a weekend on a project would be $400.
The take-home challenges at my last jobs have been close enough to the work to be useful, and much more relevant than anything else I've seen in any interview.
So we should stop the coding challenges, stop LeetCode, stop whiteboarding, stop profiling candidates, stop asking for their GitHub page. How is hiring supposed to work then? Just post the contract online and the first one to mail it back gets the job?
I like live coding challenges, something like a ~2 hour pair programming session, ideally modifying an existing project. I invest as much time as each candidate, while we are both exploring whether we want to work together.
I personally hire people I’ve worked with before and I know what they can do.
If I can’t do that, I ask to see their prior work.
Good programmers have usually written a lot of code. Having them walk through and explain it usually gives a good idea of what they can do and how they think.
Sometimes you meet a programmer who only works on proprietary code and can’t share it.
In that case I ask them to explain the design of something similar, and write a modestly sized example of a component of that system. Watching them in their own dev environment for 30 minutes usually tells you all you need to know.
No, there are other ways, especially as AI coding assistants become more capable and developers can be more productive if they are able to leverage them.
One approach that I've encountered (with a YC company) is that the first interview was actually a code review. One of the founders asked me to review some SQL DDL, some backend API endpoints. The DDL was missing some indices that were needed for the queries. It was using an integer ID field. The API endpoints were missing input validation, error handling, etc.
I thought this was a GREAT way to start an interview that tested for depth of experience and platform/language knowledge.
This actually inspired me to build https://coderev.app because the tooling for this felt like it would be clumsy for both the interviewer and it was certainly for me as the interviewee.
But a lot of times, seeing a candidate's portfolio -- if they have one -- is probably even more insightful than any coding exercise. When I've been on the hiring side, one of my favorite things to do is to look through a candidate's GH and ask them questions about projects they've done, why they chose specific technologies, etc.
You could also, you know, talk to people, like we did before Google and the Silicon Valley bro-wanna-bes decided that coding interviews was the only solution.
I've hired plenty of people without having coding challenges or any form of live coding. I've been happy with all of those hires. A former co-worker did some take-home coding tasks, which they'd then talk to the candidate about during the interview. I feel like those hires where worse in may ways, they certainly didn't stay around as long. That may very well be completely unrelated obviously.
For years people have been complaining that exams aren't realistic, that some talented people just don't do well in an exam situation and we've mostly come to the consensus that this is correct and mistakenly filter out highly talented people. So why wouldn't we apply the same logic to hiring?
If you're hiring for a specialist position some coding exercise can absolutely be in order, but I can't think of any reason to have them for a junior position. So I wouldn't recommend dropping coding tasks completely, but I'd apply them more selectively, otherwise you risk missing a number of really good hires.
Part of it may also be that so many companies and interviewers are absolutely terrible at doing coding challenges, but do them anyway, because Google and Facebook do them.
I admit I've been hired once from just a friendly chit-chat, but I can't see how this could be reliable. There are so many bullshitters in the industry, who, during character creation put all their skill points into Charisma. They are smooth talking, good looking, and have all those charming Ivy League mannerisms of an Investment Banker or Enterprise Sales person. And while they might not be able to be technically deep enough to fool an engineer interviewer, they will absolutely fool the director or VP who does the final screen and has the final say. These kinds of people flock to companies who don't do robust technical screens.
I can see your point about the bullshitters being able to fool a VP, but in that case they wouldn't be doing the coding interview anyway.
On the other side we're also seeing bullshitters that can ace any leetcode challenge, but not actually design anything of value or work well with others.
There's probably not a one-size fits all, but if you're going to do coding interview do them well and understand many don't do well in these types of interviews because they don't see the value, or feel that their other skills are being ignored.
i had also good experiences with live coding. both as an interviewer and a candidate. you not only see a candidates skill, but also their way of communicating, and as a candidate if you are lucky to be interviewed by a future teammate or superior, you see how they will treat you when you need help working on something.
i had this approach work even in china where people tend to be very submissive and afraid to speak up. i expected them to have trouble in the interview, but most candidates told me they felt very at ease even though they had never worked with a foreigner before.
> What companies often ignore is the extra time candidates invest beyond the “suggested time” for these tests.
This is a feature not a bug. Companies are testing if you can focus and complete a hard uncomfortable challenging task, because at your job you’re expected to do things you don’t want to do, but will be rewarded for doing.
Requiring a huge time investment like that will filter out a lot of the non-desperate folks…probably exactly the folks they don’t want to filter out!
Also do they really want to set the tone that “I expect you to intuit what I want and I won’t tell you directly”? Sounds like an awful place to work, right out of the gate.
> Also do they really want to set the tone that “I expect you to intuit what I want and I won’t tell you directly”? Sounds like an awful place to work, right out of the gate.
I have only once had this be a problem and in a way, it was my fault for not noting the ambiguity in the submission. Every other time I have dropped a comment saying "you could have intended A, but also could have intended B so this is when I go back to product and request more information."
That's a perfectly fair and reasonable tone to set because that's how approximately 99% of non-junior jobs work. My boss says he has a problem he'd like me to solve. He might not have all the details so it's on me to investigate the options, identify the gotchas, and clarify the ambiguities.
I never deliberately snuck ambiguities into coding challenges. I've used it in oral sessions for senior and above devs though. "I'm a product manager who wants to add a new feature. I don't really know what's involved but I want you to implement it. Feel free to ask me a million questions and we'll talk it through!"
> probably exactly the folks they don’t want to filter out!
Maybe? non-desperate folks might waste the company's time. Usually companies can only give out one offer at a time. If the non-desperate person takes a while to accept, someone else in their pipeline may get an offer and flake.
We had a candidate agree to a start date and then cancel 2 weeks before because they decided to stay at their job.
> “I expect you to intuit what I want and I won’t tell you directly”?
They do tell you directly? "Do this hard uncomfortable task, where the task is study leetcode, completing a coding project."
Companies are backhandedly selecting for those who can invest the most time beyond the suggested time (read: single, childless people in their early 20s who live at home) and those who are willing to work insane hours. It's a pledge of fealty before you are graced with an interview. 15 pieces of flair is the minimum, but it's up to you whether or not you want to just do the bare minimum.
>When was the last time you had to debug an ancient codebase without documentation or help from a team?
Uh.. just about every company i've worked at in the last 20 years has had little to no documentation and the guy who last touched the code left already.
i think we need to come to terms with the reality of coding challenges in the interview process. i know i hate them personally, and dread having to interview again because i'll need to open leet code and remember how to do stupid shit like DFS on a graph, or manipulating linked lists. at the same time, a job opening for a SWE is opening soon in our company and we'll have to somehow filter people, and the job market is such that we'll get MANY applicants, most of them probably wrong for the job. i will probably end up giving them coding challenges (not necessarily leetcode, but some coding challenge for sure), because i need a way to grasp their problem solving and coding skills. i don't know a better way to do it in a condensed time frame of a 1 hour zoom call.
> build a mini-app from scratch in just a few hours
It depends on what kind of functionality we're talking about, but this kind of task is exactly what people at my current startup have been assigned at times. It is absolutely possible to build a CRUD web app with reactive UI using modern tools in a few hours.
> This is like asking a Ruby developer to debug PHP as a test of flexibility
Again it depends on what the debugging task is. At every startup I've worked at, it's expected that an engineer is able to jump into a task that they know very little about. Granted it becomes less reasonable the more niche the task, but PHP and Ruby are not particularly far apart in skillsets in the grand scheme of things. I would expect any web engineer to be able to do this.
> Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas
I agree with this. And, hiring should also focus on technical ability which does include working through difficult and unknown problems by oneself.
> build a mini-app from scratch in just a few hours
But how often do devs normally set up a project from scratch? We have like 3 new apps in a few years where I work now in addition to the long running ones. So on average one in like hundred devs here have been part of creating something from scratch.
Sure, when you know what you're doing it's quick. But the first time can be slow, no matter how good you are. So for a coding task that should take a few hours, you might have spent all the time just getting up and running, and then you actually start the task as over time.
> is absolutely possible to build a CRUD web app with reactive UI using modern tools in a few hours.
Yes, but it also leads to tons of bikeshedding. "Should I implement a linter? Should I write unit tests? Integration tests? Should I mock HTTP calls or use a mock server? How should I deploy things? How do I structure my CI?" All these choices are extremely subjective and context dependent and then somebody's gonna pass on you because they don't like that you used Cucumber.
Setting up a project is not something you do often, and if you do, you probably have templates and some team standards.
It's much better to give somebody a project structure and ask them to implement some basic task in it.
> When was the last time you had to debug an ancient codebase without documentation or help from a team?
> This is like asking a Ruby developer to debug PHP as a test of flexibility
> If the job requires specific tech skills, test those skills
Sounds like someone failed a coding challenge.
In all seriousness, you need to understand that companies rarely implement their hiring practices for the sake of it. It is usually the best their collective minds could come up with. Telling them to get rid of it without attempting to understand what problem they're trying to solve and without offering alternatives, is, to say the least, not a productive use of anyone's time.
I find in software development we try to reinvent the wheel all too often instead of borrowing practices from other professions. Let's have a look at what civil engineers have to go through to get hired at half the salary of a software dev, and let's incorporate that into our practice. That will make everyone a happy trooper !
> In all seriousness, you need to understand that companies rarely implement their hiring practices for the sake of it. It is usually the best their collective minds could come up with.
Unlikely. It's usually what the most senior person either read about or experienced in the past. In fact I'd say that it is highly unlikely that they did any introspection at all as to what they are actually trying to accomplish.
They just did it like everyone else because they believe exactly what you do -- that someone somewhere created the process with intention.
I feel like we've so lost the plot with tech hiring that we settled on what appears at best to be a local maxima.
Folks on this forum mostly won't remember but about 20 years ago the in-vogue way of interviewing software engineers was to ask "puzzle questions" (e.g. "I have 100 ball bearings and two are a different weight than the rest....") and "lateral thinking" questions ("why are manhole covers round?") because that's what they did at Microsoft and everyone copied Microsoft.
I'm told this is still the common style of interview for mechanical engineers, which says something about what it's like to work in that industry too.
> In fact I'd say that it is highly unlikely that they did any introspection at all as to what they are actually trying to accomplish.
Based on the observations I've made of hiring managers I've worked with, what they're trying to accomplish is to provide the appearance to HR that they at least attempted an unbiased hiring process. The result often resembles a "literacy test" and I suspect one has to be very intentional about avoiding that to practically avoid it.
> When was the last time you had to debug an ancient codebase without documentation or help from a team?
Yesterday! Working on a large project means there’s always some issue with a dark corner no one has looked at recently and because there’s no team to support, I get to go and bug hunt.
Many of us grew up when that was the norm, n00bs weren't given features. Coding jobs were like apprenticeships. You were mentored and free to explore. You learned what did and didn't work and how to leave the codebase a better, more maintainable place.
Today, privilege means starting from no real job experience to writing enshitified_prod_tokenizer() and laughing at the poor sap having to fix it in a few years.
> When was the last time you had to debug an ancient codebase without documentation or help from a team?
Right the fuck now. That's what happen when you get saddled with whatever monstrosity the last coder(s) who quit commited or had to maintain themselves.
And then you get the joys of trying to frankenstein some modern js "component" developed like only React exists on some vanilla / jquery front. Also a personal rant for those modern js script kiddies: forms action attributes exist and work.
> Dropping these absurd assignments and focusing on what really counts ...
These assignments are so much closer to a simulation of doing the actual job than the industry standard LC interview. The only thing closer is work trial which has the significant problem of requiring candidates to not currently have a job.
I had a very bad experience at a company that did not conduct any coding tests; there were just 2 or 3 interviews, and that was it.
The issues began when we started working together in a Data Engineering team. Most of our stack was based on Hadoop, Kubernetes, Ruby, Python, and several other technologies that required a basic understanding of cloud computing.
However, since we had such a wide range of work experiences and backgrounds, I often found myself not doing the work I was hired for but instead covering for others. I ended up doing a lot of "glue work" to compensate for the fact that some colleagues couldn’t even handle basic tasks, such as Python CLI packaging using Docker or inspecting jobs in Hadoop. After some time, I decided to leave—not because I thought I was special, but because I was fed up with spending 80% of my time providing support for people who were hired at the same level or even higher than me.
I recognize that the company had very poor hiring practices, but some form of basic technical testing is necessary.
When you say they "couldn't even handle basic tasks" do you mean they were completely unfit to complete their assigned tasks, or that they were getting hung up on simple tasks, while otherwise being independent and capable? The other key question is whether you had to explain the same things to the same people again and again. Being ignorant is excusable, but being unwilling to learn isn't.
> they were completely unfit to complete their assigned tasks, or that they were getting hung up on simple tasks
They were unfit to complete the tasks. But again was not their fault, I think was the hiring that failed.
One concrete example was the fact that our pipelines was quite straightforward for data engineers: packaged Ruby and Python CLIs that runs commands. The runtime was k8s. One of the biggest issues were when something broke in production, none of the people couldn’t go to the container, check the logs and understand the failure.
There’s one situation where I think makes sense to hiring without a test: if the company has the resources and money to provide levelling training for all new hires with no exceptions. I do not know how practical it is.
This happens when companies have too many applicants. It isn't the best way to evaluate candidates, but it is a cheap and easy way to do it and resumes aren't enough.
It's common for candidates who have an internal referral to skip all these steps, because the referral establishes a base line of credibility.
This is a friendly reminder that you are welcome to withdraw a job application as soon as you are offered a coding challenge. If this is an exchange with a recruiter over email or linkedin messages, you can simply reply with something like this:
"Thank you for sending that along. I would like to withdraw my application for this position."
Unfortunately, you often have to ask the interviewer (which is tough when it's a coding challenge and you have no way to ask for clarification on the rules). I know interviewers who will say "You didn't cover edge case A, B, and C, your solution is incomplete!" and other interviewers who will say "You are overthinking this, I'm just looking for an MVP."
That's assuming they don't pre-bikeshed before even looking at what you did.
Candidate: writes code matching current style as much as possible to show that you can adapt
Reviewer: Rejected. Didn't even use autoformatter, which takes literally zero effort, so clearly does not follow best practices.
Candidate: autoformats
Reviewer: Rejected. Changed more code than was really needed (separate commit or not), clearly showing they didn't try to keep the change as small as possible. If this were a real PR, they would be making life more difficult to the reviewers making it harder to spot the parts that were actually modified.
Candidate: keeps existing code as-is, but writes own changes following the currently accepted conventions
Reviewer: Rejected. They clearly can't follow an existing style and prefer to be a snowflake and "leave their mark" in the codebase.
Candidate: does any of the options above, and adds comment explaining the reasoning, including mentioning the other possible options that they thought about and their tradeoffs
Reviewer: Rejected. Candidate thinks too hard about trivial stuff, showing lack of focus on what really matters.
---
Granted, being rejected for those reasons above could mean that you dodged a bullet.
But yeah, like you mentioned in a different response, a live interview (coding or not) helps reduce these kinds of uncertainties to some extent, but of course these live interviews have other trade-offs compared to take-home tests.
494 comments
[ 3.3 ms ] story [ 320 ms ] threadIs this a joke?
My answer to that question: I'm doing it right now, and have had to do it at least occasionally in almost every job I've ever had.
Sounds like an OK test to me. Great (senior) developers should be able to do that kind of thing. Categorizing yourself exclusively as "a Ruby developer" is a career trap.
And a lucrative one at that.
No engineer that makes 7 figures calls themselves a ruby developer with the exception of DHH.
The key here is not to categorize as a “language developer”.
Consistent total compensation offers of $1mil+ is probably reserved for roles above these, sometimes called Principal or Distinguished. I would say the rate of having one of these in an org are like 1 for ~150 engineers.
Assuming that high earners are offsetting that to the higher end, most people aren't making 6 figures, and the bar isn't which language they're programming in.
Junior salaries go down much lower than $100k.
Congratulations, your experience is limited. The BLS stats represent the actual US salary data, not just your limited experience. If you want to make a claim about salaries in the US then look at data across the US and not just whatever is true within your limited bubble.
> And forgive me if forwarding the BLS statistics to candidates doesn't get them to accept offers
Did I ever even suggest such a thing?
> Did I ever even suggest such a thing?
My point is that the BLS doesn't set market rates or report on them.
$132,270
This means half of all full time employed devs are higher, and half are lower. The mean is more skewed by higher earners but is similar:
$138,110
It also varies quite widely by geographic location, from a mean high of $173,780 in California to only $125,890 in Texas, from $199,800 in San Jose to $132,500 in Austin to $98,960 in rural Kansas (where I have actually developed software before!)
The short of it is, the vast majority of software developers do not make the top salaries. Even L6 is rare within the top tier of tech. There is a lot of delusion in this field around pay, so it's important to be well informed. As a field we are still very well paid compared to most other jobs especially considering our safe working conditions and lack of needed credentials and education. Compared to most of the work on this planet, it's still a goldmine.
Source: https://www.bls.gov/oes/current/oes151252.htm
How many companies are out there paying $1,000,000/year for devs?
How many devs who can command that kind salary are going to put up with bullshit coding challenges?
Google alone has 4000 directors all making a million at minimum.
I have only worked at FAANG across my nearly decade long career. So, it’s a biased but very large sample.
That's not unheard of, but it's certainly rare. $1 million+ is not a "typical salary", even at Google.
Sounds like a good deal to me.
It's ridiculous how developers mindlessly accept that you should constantly be learning to keep yourself relevant, but keep it shallow by just jumping from one tool to another, instead of encouraging deeper knowledge of generalizable patterns that stay relevant across waves of technological disruption.
Be a generalist. Yes, deep specialists exist and yes, some of them have successful careers based on their deep specialty, but betting on specialization is like a high school kid planning to be a MLB baseball player.
And that’s complimentary with marketing oneself as Ruby/Rails dev or whatever else the market needs at the time.
Know what I'd do if the interviewer asked me to debug PHP? Pretty much return the question:
"I've never used PHP. Are there logging macros/functions defined somewhere? Where do I see the output? syslog maybe? Is there a debugger of some sort I can use? How do I run each `piece` of code in isolation?"
(I am assuming the job listing did not explicitly mention PHP experience. If it did, both myself and the recruiter would absolutely deserve to fail me for this interview).
Debugging old DSL vendor specific languages or code so old using, frameworks and standards long out of fashion and support, that they are half way to being a different language.
Adding support for some back ported features or patching security holes in an old client or legacy stacks.
Or at a big company we had some escrow code from a much smaller partner that we ended up becoming responsible for.
Often getting the environment setup for proper debugging is more work than anything.
But yes, it's a good test for a senior+.
Interviewer: Now reverse this array.
Me: OK, in Python that would be array.reverse(), or reversed(array). I bet JS has one of those, probably the .reverse method.
Interviewer: Great guess!
That was genuinely fun. I came out of it feeling like I'd learned a few things, and the other person got to see how I'd reason about a new problem.
It was a pair programming exercise and so with some help from the interviewer and the IDE I was able to fumble through to a working result. I agree it was fun and educational.
We are blessed.
Somehow in programming every interview starts from the assumption that the candidate must prove competence to the employer, and the employer decides what "competent" means.
Which doctor would you prefer do surgery on your brain - the one that jumped through all the hoops and went to 15yrs of med school - or the one that learned brain surgery through a YouTube tutorial?
I think you know the answer. The problem is not gatekeeping in general it’s the detail and nuance of how gates are being kept and how to find the right amount of gatekeeping.
No unnecessary gatekeeping, but maximum necessary quality standards (which I distinguish from gatekeeping).
Practically? No one has figured out how. The best we have are coding challenges.
My answer is that I don't have access to what I'd really like to base that judgment upon: their success rate, adjusted for the difficulty of the procedure.
Simpler, less-risky procedures would therefore have a lower skill/experience bar.
Basically, I'd want someone with decades of experiences to remove a tumor deep inside my brain, but I'd be much more willing to use someone who learned via YouTube this time last year if I'd suffered a TBI and was experiencing swelling and intra-cranial pressure.
It's not a perfect system, but it's a lot harder to fake being an experienced doctor than it is an experienced developer.
The insane thing would be to expect someone to take you at your word for 150k*, when hiring managers know that 50-70% of people with your resume fail to write a nested for loop.
Didn't have to solve PDE's on the whiteboard, or regurgitate integral transformations.
With software you never have to stop proving yourself, and your skillset is always a few years away from being outdated. A doctor with 20 years of experience would be welcomed anywhere, but an engineer with 20 years experience is viewed with trepidation. The next "big thing" could roll out at anytime and suddenly crypto engineers are getting 800k job offers so everyone furiously tries to learn crypto stuff. A few years later that all dries up and now you have 100k engineers who are out of work and learned tech that no one cares about anymore. All the LLM engineers might be in the same position next year.
Of course. They paid an enormous cost and beat other competitors for that privilege which is carefully guarded by regulation and certification boards.
No doubt software has more instability, but the trade off is that you don’t have to do that multi year grind. Easier fire, easier hire. Similar pay.
> skillset is always a few years away from being outdated.
I strongly disagree with this. In your example you use “crypto” and “llm”. If this is your skill set you are fad chasing and needlessly increasing your exposure to changing markets.
Engineers solve problems by applying math and science expertise. This is a timeless skill.
It's one thing where companies are applying this to 1000 applicants for an entry level position, but they also use this tactic when they reach out to you and tell you how excited they are about your background and resume.
In reality they didn't evaluate your background or resume and are just cold-calling you to fill some portion of their "funnel", then ignore all their excitement and reasons they reached out to you and have you jump through random hoops.
I've had to do this at some level in every job.
Somebody left. The code was left to rot, whether it was a separate codebase or it was an entirely separate project. I got assigned to fix it. Sure, I had a team, but nobody knew more than I did, so it was just other devs who might have another perspective.
Collaboration and support might be standard, but mostly in the form of other smart people, not always documentation, up to date software, or people who know anything about the code.
I personally prefer these kinds of challenges as they mean I don't need to maintain two skillsets.
That's exactly my problem with take-home code challenges. If I go to an interview, and they waste my time, they have to waste their own. But with a take-home assignment, they can waste my time without wasting their own. That may lead them to be more wasteful of my time.
$100 is insultingly low for a 4 hour assignment.
If we assume 15 days off that leaves roughly 245 workdays per year and with a salary of $200k that would be close to $800.
A fair amount of compensation to spend half a weekend on a project would be $400.
I like live coding challenges, something like a ~2 hour pair programming session, ideally modifying an existing project. I invest as much time as each candidate, while we are both exploring whether we want to work together.
If I can’t do that, I ask to see their prior work.
Good programmers have usually written a lot of code. Having them walk through and explain it usually gives a good idea of what they can do and how they think.
Sometimes you meet a programmer who only works on proprietary code and can’t share it.
In that case I ask them to explain the design of something similar, and write a modestly sized example of a component of that system. Watching them in their own dev environment for 30 minutes usually tells you all you need to know.
One approach that I've encountered (with a YC company) is that the first interview was actually a code review. One of the founders asked me to review some SQL DDL, some backend API endpoints. The DDL was missing some indices that were needed for the queries. It was using an integer ID field. The API endpoints were missing input validation, error handling, etc.
I thought this was a GREAT way to start an interview that tested for depth of experience and platform/language knowledge.
This actually inspired me to build https://coderev.app because the tooling for this felt like it would be clumsy for both the interviewer and it was certainly for me as the interviewee.
But a lot of times, seeing a candidate's portfolio -- if they have one -- is probably even more insightful than any coding exercise. When I've been on the hiring side, one of my favorite things to do is to look through a candidate's GH and ask them questions about projects they've done, why they chose specific technologies, etc.
I've hired plenty of people without having coding challenges or any form of live coding. I've been happy with all of those hires. A former co-worker did some take-home coding tasks, which they'd then talk to the candidate about during the interview. I feel like those hires where worse in may ways, they certainly didn't stay around as long. That may very well be completely unrelated obviously.
For years people have been complaining that exams aren't realistic, that some talented people just don't do well in an exam situation and we've mostly come to the consensus that this is correct and mistakenly filter out highly talented people. So why wouldn't we apply the same logic to hiring?
If you're hiring for a specialist position some coding exercise can absolutely be in order, but I can't think of any reason to have them for a junior position. So I wouldn't recommend dropping coding tasks completely, but I'd apply them more selectively, otherwise you risk missing a number of really good hires.
Part of it may also be that so many companies and interviewers are absolutely terrible at doing coding challenges, but do them anyway, because Google and Facebook do them.
On the other side we're also seeing bullshitters that can ace any leetcode challenge, but not actually design anything of value or work well with others.
There's probably not a one-size fits all, but if you're going to do coding interview do them well and understand many don't do well in these types of interviews because they don't see the value, or feel that their other skills are being ignored.
i had this approach work even in china where people tend to be very submissive and afraid to speak up. i expected them to have trouble in the interview, but most candidates told me they felt very at ease even though they had never worked with a foreigner before.
This is a feature not a bug. Companies are testing if you can focus and complete a hard uncomfortable challenging task, because at your job you’re expected to do things you don’t want to do, but will be rewarded for doing.
Also do they really want to set the tone that “I expect you to intuit what I want and I won’t tell you directly”? Sounds like an awful place to work, right out of the gate.
I have only once had this be a problem and in a way, it was my fault for not noting the ambiguity in the submission. Every other time I have dropped a comment saying "you could have intended A, but also could have intended B so this is when I go back to product and request more information."
I never deliberately snuck ambiguities into coding challenges. I've used it in oral sessions for senior and above devs though. "I'm a product manager who wants to add a new feature. I don't really know what's involved but I want you to implement it. Feel free to ask me a million questions and we'll talk it through!"
Maybe? non-desperate folks might waste the company's time. Usually companies can only give out one offer at a time. If the non-desperate person takes a while to accept, someone else in their pipeline may get an offer and flake.
We had a candidate agree to a start date and then cancel 2 weeks before because they decided to stay at their job.
> “I expect you to intuit what I want and I won’t tell you directly”?
They do tell you directly? "Do this hard uncomfortable task, where the task is study leetcode, completing a coding project."
That is an option if you have higher personal obligations and are struggling mentally.
Uh.. just about every company i've worked at in the last 20 years has had little to no documentation and the guy who last touched the code left already.
It depends on what kind of functionality we're talking about, but this kind of task is exactly what people at my current startup have been assigned at times. It is absolutely possible to build a CRUD web app with reactive UI using modern tools in a few hours.
> This is like asking a Ruby developer to debug PHP as a test of flexibility
Again it depends on what the debugging task is. At every startup I've worked at, it's expected that an engineer is able to jump into a task that they know very little about. Granted it becomes less reasonable the more niche the task, but PHP and Ruby are not particularly far apart in skillsets in the grand scheme of things. I would expect any web engineer to be able to do this.
> Hiring processes should focus on problem-solving, collaboration, and growth in relevant areas
I agree with this. And, hiring should also focus on technical ability which does include working through difficult and unknown problems by oneself.
But how often do devs normally set up a project from scratch? We have like 3 new apps in a few years where I work now in addition to the long running ones. So on average one in like hundred devs here have been part of creating something from scratch.
Sure, when you know what you're doing it's quick. But the first time can be slow, no matter how good you are. So for a coding task that should take a few hours, you might have spent all the time just getting up and running, and then you actually start the task as over time.
Yes, but it also leads to tons of bikeshedding. "Should I implement a linter? Should I write unit tests? Integration tests? Should I mock HTTP calls or use a mock server? How should I deploy things? How do I structure my CI?" All these choices are extremely subjective and context dependent and then somebody's gonna pass on you because they don't like that you used Cucumber.
Setting up a project is not something you do often, and if you do, you probably have templates and some team standards.
It's much better to give somebody a project structure and ask them to implement some basic task in it.
> This is like asking a Ruby developer to debug PHP as a test of flexibility
> If the job requires specific tech skills, test those skills
Sounds like someone failed a coding challenge.
In all seriousness, you need to understand that companies rarely implement their hiring practices for the sake of it. It is usually the best their collective minds could come up with. Telling them to get rid of it without attempting to understand what problem they're trying to solve and without offering alternatives, is, to say the least, not a productive use of anyone's time.
I find in software development we try to reinvent the wheel all too often instead of borrowing practices from other professions. Let's have a look at what civil engineers have to go through to get hired at half the salary of a software dev, and let's incorporate that into our practice. That will make everyone a happy trooper !
Unlikely. It's usually what the most senior person either read about or experienced in the past. In fact I'd say that it is highly unlikely that they did any introspection at all as to what they are actually trying to accomplish.
They just did it like everyone else because they believe exactly what you do -- that someone somewhere created the process with intention.
I feel like we've so lost the plot with tech hiring that we settled on what appears at best to be a local maxima.
I'm told this is still the common style of interview for mechanical engineers, which says something about what it's like to work in that industry too.
Based on the observations I've made of hiring managers I've worked with, what they're trying to accomplish is to provide the appearance to HR that they at least attempted an unbiased hiring process. The result often resembles a "literacy test" and I suspect one has to be very intentional about avoiding that to practically avoid it.
Yesterday! Working on a large project means there’s always some issue with a dark corner no one has looked at recently and because there’s no team to support, I get to go and bug hunt.
Today, privilege means starting from no real job experience to writing enshitified_prod_tokenizer() and laughing at the poor sap having to fix it in a few years.
Right the fuck now. That's what happen when you get saddled with whatever monstrosity the last coder(s) who quit commited or had to maintain themselves.
And then you get the joys of trying to frankenstein some modern js "component" developed like only React exists on some vanilla / jquery front. Also a personal rant for those modern js script kiddies: forms action attributes exist and work.
These assignments are so much closer to a simulation of doing the actual job than the industry standard LC interview. The only thing closer is work trial which has the significant problem of requiring candidates to not currently have a job.
The issues began when we started working together in a Data Engineering team. Most of our stack was based on Hadoop, Kubernetes, Ruby, Python, and several other technologies that required a basic understanding of cloud computing.
However, since we had such a wide range of work experiences and backgrounds, I often found myself not doing the work I was hired for but instead covering for others. I ended up doing a lot of "glue work" to compensate for the fact that some colleagues couldn’t even handle basic tasks, such as Python CLI packaging using Docker or inspecting jobs in Hadoop. After some time, I decided to leave—not because I thought I was special, but because I was fed up with spending 80% of my time providing support for people who were hired at the same level or even higher than me.
I recognize that the company had very poor hiring practices, but some form of basic technical testing is necessary.
They were unfit to complete the tasks. But again was not their fault, I think was the hiring that failed.
One concrete example was the fact that our pipelines was quite straightforward for data engineers: packaged Ruby and Python CLIs that runs commands. The runtime was k8s. One of the biggest issues were when something broke in production, none of the people couldn’t go to the container, check the logs and understand the failure.
There’s one situation where I think makes sense to hiring without a test: if the company has the resources and money to provide levelling training for all new hires with no exceptions. I do not know how practical it is.
This is pretty much my day job. OP is spoiled if they think this is an uncommon situation.
It's common for candidates who have an internal referral to skip all these steps, because the referral establishes a base line of credibility.
How is a coding challenge not problem solving?
Collaboration can be part of coding challenges.
How in the heck do you plan to measure growth in a hiring process?
Lol, I'm pretty sure that's been my job description for the past 20 years.
"Thank you for sending that along. I would like to withdraw my application for this position."
do you show them a quick'n'dirty solution that ignores edge cases but shows i'm a pragmatic and not going to overcomplicate things?
OR
do you show something fancy that you'd never actually do in an real codebase that shows off my depth of knowledge and where my ceiling is?
Candidate: writes code matching current style as much as possible to show that you can adapt
Reviewer: Rejected. Didn't even use autoformatter, which takes literally zero effort, so clearly does not follow best practices.
Candidate: autoformats
Reviewer: Rejected. Changed more code than was really needed (separate commit or not), clearly showing they didn't try to keep the change as small as possible. If this were a real PR, they would be making life more difficult to the reviewers making it harder to spot the parts that were actually modified.
Candidate: keeps existing code as-is, but writes own changes following the currently accepted conventions
Reviewer: Rejected. They clearly can't follow an existing style and prefer to be a snowflake and "leave their mark" in the codebase.
Candidate: does any of the options above, and adds comment explaining the reasoning, including mentioning the other possible options that they thought about and their tradeoffs
Reviewer: Rejected. Candidate thinks too hard about trivial stuff, showing lack of focus on what really matters.
---
Granted, being rejected for those reasons above could mean that you dodged a bullet.
But yeah, like you mentioned in a different response, a live interview (coding or not) helps reduce these kinds of uncertainties to some extent, but of course these live interviews have other trade-offs compared to take-home tests.