It doesn't look like a good match. Just walk away saying you're not interested.
As a company, you'll want to prevent this sort of scenario, but as the interviewee you really don't owe them anything so you can just say you're no longer interested.
If, for some reason, you feel compelled to improve their methods then you will have to succeed at their assessment in order to do so. It is a property of humans that to change a thing you have to be able to show you can handle a thing.
If you don't want to do the take home test, tell the company just that. You aren't going to hurt anyone's feelings. You don't have to be overly polite about it or think of excuses. No one there is going to care.
They will just move on and hire someone else. You can move on and apply somewhere else. That's all.
The purpose of the question is to avoid triggering "they will just move on and hire someone else". As you well know.
(and evidenced by "I will also state that they can check out some of the previous code I have written on my GitHub." where the candidate still wants the company to keep considering them after declining the test.)
Whether or not you think a take home test is appropriate, it would be unethical of the company to let some candidates skip it. Every candidate should be evaluated the same way.
Why would this be? The goal of the test is to have a minimal baseline. Considering outside proof of this baseline is not unethical. If maybe it shouldn't be used to rank candidates, but you'd probably do this anyway for the final consideration.
Even if it's not specifically that, if the company thinks you are the best available candidate (by whatever criteria they have) despite your not taking the test, I don't see how it's unethical for them to hire you.
Why would it be ethical that every candidate has to go through the entire evaluation process? It would be possible that a candidate can demonstrate mastery of skill enough that they don't need to go through the same evaluation as one unable to demonstrate the same mastery. A take home test for those who don't commit code to github doesn't seem unethical. Much like how most candidates likely don't even get the chance to take the take home test because they were already eliminated.
Perhaps your reaction to the test is part of the test too. If you do it, you'll be evaluated on your technical performance, and if you refuse you've simply answered another question they might have about you, albeit a non-technical one. Completing the test might signal compliance, which might be exactly what they're filtering for.
Right and if 90% of the candidates decline and the ultimate hire ends up being less than ideal, the company will (or they should) re-evaluate whether it's a good idea.
> Whether or not you think a take home test is appropriate, it would be unethical of the company to let some candidates skip it. Every candidate should be evaluated the same way.
Ask yourself how the company founders and initial employees got their jobs. Almost certainly not by going through their ex post facto, arbitrarily selected in-house hiring process. Was that unethical?
IMO there's actually nothing wrong with hiring the first qualified applicant who shows up, and then calling off the search. Is there an ethical obligation to hire "the best possible candidate", and why? See https://en.wikipedia.org/wiki/Satisficing
Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone? It it unethical to go with an internal candidate and not offer an open position for everyone?
Companies tend to waste way too much time, effort, and money on hiring. It's premature optimization, the root of all evil.
Uniformity is not the same as ethics. Otherwise, we'd be ethically obligated to eliminate special parking stalls for the disabled. Moreover, there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits. How do you judge the ethics of the criteria themselves, in comparison to other possible uniform criteria?
> Ask yourself how the company founders and initial employees got their jobs.
For founders this is a perk of being a founder. You are welcome to found your own company if you wish.
> Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone?
Yes, it can be. Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
Yes, but that didn't answer my question about ethics.
> You are welcome to found your own company if you wish.
I did!
> Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
How often do tech companies get a "representative" pool of candidates?
But this actually goes to the other issue I raised:
> there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits.
In particular, take-home exams that take many hours to complete tend to discriminate against certain kinds of candidates, those who don't have a lot of free time outside of work, family, etc.
It's important to note that these tests are primarily used by tech companies as a screening device, not as the method for making the final hiring decision, which tends to be more subjective. They don't simply add up test scores and choose the highest scorer.
Moreover, the tech industry seems to be obsessed with tests, but they're an outlier. Tons of other industries use more informal methods of hiring and don't give tests at all, much less take-home tests. Do you think they're all unethical?
If you care at all about bias and diversity, you may want to learn more about how to construct interviews that reduce both. No, every set of questions and every interview technique is not equal when it comes to bias. I'd suggest the book "Becoming the Evidence-Based Manager" by Gary Latham.
On the topic of hiring, a "Job Simulation" is one of the most objective measures, which is pretty close to what a practical test is. Questions about things candidates actually did is another important technique (e.g. tell me about a time you ...), and the third category of techniques that work is giving hypothetical situations that they need to explain how they would respond to.
Although informal conversations are common in industry, they are some of the worst techniques. For example, if you start with a "coffee chat" research has shown that most people more or less make up their mind in that first meeting then bias the rest of the interview results to unknowingly justify their decision.
I'd hope that every adult understands that unless you have any extra leverage in a situation you aren't going to get special treatment.
"I know that every other candidate for the job successfully did this exercise but I found it too difficult to even set up. Can you hire me over all of them?"
And I hope that every company understands that unless you offer a sane hiring practice, you aren't going to get the best developers.
"Every other candidate was desperate and rejected a thousand times so they're happy to waste their time on your pointless exercise that you won't even look at."
The good candidates are busy getting hired by companies that understand how to evaluate their work based on interviews, references, and several other things that don't waste their time.
Depending on the company and current economic situation, they might just have to swallow. With the layoffs right now, employers have the upper hand and candidates are well advised to just comply and do the test.
But there were also reports on HN that before the layoff wave, they'd get 2-3 applications per year for a senior position. It is of course any employers right to continue applying the same hiring standards as eg. FAANG/Big Tech would. But if I were the employer, a better strategy in such a situation would be to just swallow that I can't pick and choose if I have nothing to offer for leverage :)
I'm sure you're smart enough to realize that what I'm hinting at is that the job market is a _market_ and that employees don't have to put up with every bullshit depending on the market conditions.
I landed a job doing embedded Rust, and the company had been looking for someone to fill the role for 6 months with only one qualified applicant who bailed, and external contractors with C/C++ experience only. They surely were bad at looking, since neither they nor the recruiter consulted the local Rust meetup group. But there are starved companies looking for specialists out there even in an employers market.
Anecdata: I'm no one special and have told smaller companies I am not interested in a take home but would be happy to do something more timeboxed and gotten positive responses.
I would give you a positive response if you said that.
If you accepted the take-home, and failed to get started on it, and then asked for something else, I wouldn't. Because I have evaluated other candidates and they were able to do it.
I don't think it's reasonable to expect that result, no matter how polite the applicant is. Every company has their own process for vetting applicants, and it's vanishingly rare that you can decide not to follow their process and still be considered for the position.
I don't do these sorts of tests, personally (I even had one that they expected would take a full 40 hour workweek to complete!). If I'm presented with one, I know that in declining to do it, I am disqualifying myself for the position.
It's just life. We all decide what our limits are, and in deciding those limits, we exclude some possibilities.
Not revealing the time spent poses risk of impostor syndrome, risk of actually landing a very stressful job.
Refusing to solve the task indicates that they would rather post-pone finding out, and rather risk rejection if it means not being challenged on their solving speed disqualifying them, because they think that it too likely would disqualify them.
In a fantasy land that might be the case but being one of 10-1000 people in the early funnel, you as an applicant have zero leverage. No one has a vested interest in moving you forward when you go against their process. Heck, they’re probably happy to whittle down the pool because you had an “attitude” or some other nonsense reason.
Personally I prefer a take home over a full day technical whitebording on-site which is my idea of hell. But unfortunately many companies do both in the same loop instead of one or the other
Were they actually upset? Maybe the impression that things went great was really just that the candidate was diplomatic, and they were just waiting for the next excuse to exit the process. I've had phone interviews with some really bad interviewers that should have been a sign that the company may not have been a good place to work.
If there were a significant percentage of applicants behaving this way, that's a signal that something may have gone terribly wrong during the interviewing process that is causing the applicants to be upset.
If it were my company, I'd be taking a cold, hard look at that process to be sure that it's OK. Perhaps it's not really going as well as it seems.
Talking salary in between interview steps is kinda weird. Sometimes you discuss salary ranges at the start to make sure you're not completely misaligned, but otherwise it's something that normally comes at the very end when you're decided you want to hire the person and are making a concrete offer.
I suspect that your messaging during phase 2.5 is bad and you're misleading candidates into thinking that you've decided to hire them, and then when you follow that up with a take home they conclude that you're fucking with them or a disorganized mess.
Not saying this is the case with this particular company, but I've seen a lot of companies that don't realize that an interview is a two-way street, and the applicants are (or should be!) assessing the suitability of the employer just as much as the employer is assessing the suitability of the applicants.
Why do you do the take home test at the end? Seems odd. Take home tests are used because it saves time to the company, so you do it first. If the candidates passes the test, then the "real" interviews starts.
By doing it at the end, you are just telling the candidate "Oh, you are good. You passed phase I and phase II... but that's not enough you poor candidate!".
Interviewing is a two way street. It's perfectly likely that they did the two phases of interviews, met with everyone on the team, and decided the company wasn't right for them. Or a better offer came along.
It could happen simply because the candidate was also interviewing other companies (because who interviews just one company these days) and they got an offer from their preferred company.
Basically, assume that candidates can drop at any time.
That sounds reasonable, I could see someone (younger me) going through phase II to get a better sense of the company before investing actual brain cycles on a takehome test. Given the power imbalance between hirers and hirees, it doesn't feel remotely unethical to me - hirers are paid to do that, hirees are not. Only way an individual has at times is to lie. I have gotten the question, "if we give you this offer, will you accept it?" and the rational option is to say yes, then make up my mind. Nothing is binding until it is signed, and even then it can easily be rescinded by either side.
It will take OP more than 3 hours to compose and post question, read all the top rated answers and comments, reach a decision, and finally write a thoughtful email. Possibly OP wanted moral support in their self-deception as to the actual issue. It is clearly not OP's valuable time that is the issue.
My advice:
Your first step as a candidate when given an assignment (with specific time constraints that you feel are mis-estimated), is to guestimate what you think it will take. Remember, this is exactly what happens at work with task estimation. Are you going to 'politely quit' if your manager gives you a task that can't be done in a certain time window?
"I reviewed the task and my estimate is that this will take a few more hours than specified. I would like clarification if the time limit is an informative hard limit."
One benefit of asking for estimate confirmation (specially for people like me who tend to go for more general solutions) is that you may actually get useful feedback. If you get a firm 'yes' from a competent team then it means they are looking for absolute simplicity in solution, and your weekend project of pimping your take home task is not only a wasted effort, it may actually take you off the list.
Whatever response you get back it will be informative, sometimes even shedding light on what sort of organization you are applying to join.
> It will take OP more than 3 hours to compose and post question, read all the top rated answers and comments, reach a decision, and finally write a thoughtful email.
What? I like sokoloff's response[1]. It takes 3 seconds.
>Not sure what the purpose of the question is.
>
>If you don't want to do the take home test, tell the company just that.
Just a guess, but perhaps they're trying to provide some constructive feedback to the company to let them know that the time estimate they're giving to their candidates is potentially way off. (That said, maybe it's not way off for people with more experience, but the candidate doesn't get that.) It's probably futile, but I can understand the urge. If they company doesn't know a problem exists, they can't fix it (if it is an actual problem).
Just say no. I won’t participate in interview activities that don’t involve employees spending an equal amount of time with me or paying me an hourly rate. I’m also not bothered about seeming polite when someone tells me to spend several hours of my time doing their test. The request is inconsiderate itself.
Exactly. If they abuse you during the hiring process, they will probably abuse you after they hire you.
And yes, this is abuse. Demanding meaningless work, for free, for the chance to get hired? Because you can't figure out how to interview in a way that allows you to judge competence? Yeah, I think it's fair to call that abuse. At least exploitation.
I typically don't do "take home tests" anymore. I explain that in the past they have occupied a lot of my time and even when successful, usually things didn't work out. I point them to some of my recent foss work in return (with full git commit logs).
I would probably make an exception if they offered to pay me even a token per hour (or even donate the hourly to a charity - it's about incentives not money). I have made an exception when it was literally a 20 min code challenge for a promising lead.
On one hand, this does frequently stop things moving forward. On the other hand I never felt like I missed out.
One company I sent them my usual polite decline and got back an angry rant from the CEO! I responded with "Well, give me your product for free for 8 hours or so and I'll give you my time for free for the same length." He did not respond after that.
This is probably bad advice for very new engineers where a code sample can help differentiate you a lot, but I wish more senior+ engineers would refuse these dumb tests.
The handful of times I've dumped a bunch of time and effort into an at-home screening assignment, it took far longer to do well than whatever the allotted time was made out to be. Emphasis on the "do well" part.
My (unscientific) sense is that these things are really just screening for people willing to jump through unreasonable hoops.
I feel like this is a mismatch of delivery expectations rather than a problem with take-homes. I explicitly tell our candidates that I don't expect them to put together a polished solution in the couple of hours they have for the take-home. That's fine. The point is to merely get an idea of someone's ability.
Counterpoint: what idea can be had from rushed / time-constrained / deliberately careless work examples?
Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
Experiences vary, but every job I've taken and liked - no coding tests of any kind. More of a discussion about an example piece of code, for that part of the interview, but no requirement to come up with new code.
I'm a firm believer that you can evaluate exactly what you test. If you are asking someone to rush to deliver an algorithm, you are testing their ability to rush to deliver an algorithm. If you actually want to know whether they can deliver a polished solution, you asked the wrong question.
In my case, I actually do want to know how quickly people can pick up a new problem and get started writing some code, and that's how I evaluate what they deliver.
> Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
I have two thoughts about this.
1. If people can deliver, they can deliver.
2. If you come up with an original problem (rather than lazily copy and pasting some fizz-buzz type problem), you'd be surprised how many people fail to apply any basic problem solving.
> a discussion about an example piece of code
I do find that it is useful to keep code exercises short, but ask a few questions about their solution as a follow up. People often struggle to explain their solution if they just copy/pasted.
Take homes aren't great, they're just less bad than whiteboarding or live evaluations. I wish it was possible to judge coding ability through a resume, but you'd be amazed at how many 'senior engineers' can't write a line of code.
The best way for senior folks to get a new job is through referrals.
While there exist companies that overdo it and expect weird things like that, I believe the majority of companies expect something simple — if you’re applying to be a <rolls dice> Rails backend developer, you would probably be asked to build some simple API with a couple endpoints. And if you are applying for that position, you should know how to set up Ruby (and you should probably have it installed on your laptop, even if you aren’t doing much coding outside of work/for fun/in open-source projects), install Rails, create an app, and write the simple endpoints. The OOP seems to be struggling with something like that (and “virtual environment” smells very Pythony to me — even though Python packaging is a trainwreck, someone applying for a Python job shouldn’t need 3+ hours to set things up for a simple project.)
Sending a valid, minimal reproduction case in the highest possible score on the take-home in my book. The take home is designed to efficiently weed out the hopelessly unqualified (of which the applicant pool is unsurprisingly over-represented). You clearly passed that test and, by asking a second time, the company saved you a bunch of further wasted time.
I'll give you a counter example. I've worked extensively with the symfony framework (10+ years at this point), and clearly remember a company I've applied to a couple of years back guesstimating that the take home assignment is going to take 2-3 hours tops to complete.
And the requirements might be reasonable for an already setup project, but when you have to do it from zero; unless you're scaffolding new symfony projects every time a new release is out, you're going to waste at least an hour scaffolding/googling/installing/wiring up all the dependencies/auth/orm you might need for you flow. It's not as bad as within the NodeJS ecosystem, but all libraries/frameworks that have a yearly major release, like to break things along the way so you're never going to hit the ground running in a couple of minutes.
I vote with my labour by simply going elsewhere. Amazon requires interview candidates to show goverment ID to a webcam. I dont have a webcam and dont intend to be scrutinized in such an obtuse way. Happy to do an onsite, not happy to have to prove my citizenship to solve remote fizzbuzz.
I think that one is because people have their friend doing the coding interview for them. However, I understand why you’d rather keep your privacy as well.
Seems like such a bizarre thing to be worried about. What privacy are you losing by showing your ID on a webcam to an interviewer who already has screened you?
Are you applying anonymously? What privacy are we talking about, here? Put yourself in front of a blank wall if you're concerned about the interviewer gaping at your house. I don't think there's a better way to scream "I'm difficult to work with".
You shared it with the employer HR, not the interviewer. Say you're an attractive person and someone now becomes obsessed with you and has your address to stalk you. That's more what I was thinking about
I've never done a take-home test before meeting with the hiring manager and actually got an interview after doing the test. I've never done a test after interviewing with the hiring manager and not gotten a job offer. Therefore my rule nowadays is that I only do take home tests after meeting with the hiring manager.
Oh and I should also add that I've never had a recruiter tell the truth about how long one of these should take. They always lie with some ridiculous number of hours that I know only a tiny percentage of the entire programming population could perform at.
Which says the former are using it (and have no qualms about wasting your time) instead of actually winnowing by resume first. Smart to rule them out, they don't value your time whatsoever.
> I've never done a test after interviewing with the hiring manager and not gotten a job offer. Therefore my rule nowadays is that I only do take home tests after meeting with the hiring manager.
I agree, yet there's a bunch of people in this comment section claiming you should give people take-home tests first which makes no sense to me.
This isn’t “politely decline”, it’s “tell them I failed to do it (or at least spent the suggested amount of time for the task and only got my environment setup) but still want to be considered for the job”.
If it’s taking over 2 hours to setup the environment, then either their stack is so complex and poorly documented that it should be a red flag to bail out, or you grossly unqualified. Either way, doesn’t seem like a good fit for either side.
Maybe. It could also be some bug, or misconfiguration, version conflict, or whatever in their interview problem repository that they haven't fixed, because no interview candidates want to admit that it took them 3 hours just to get it running. Their actual production code could be an updated version of the same code, or different code altogether. In other words, this could be a perfectly good company, but the engineers don't spend time debugging the interview process when they aren't asked to.
I would agree that companies that give "2-3 hour" take home problems are a red flag for me — red as in, I won't do it anymore. That situation becomes an arms race quickly: if I can solve it in 3 hours, why not spend another 3 hours doing a second, more elegant solution that makes me look even smarter? I'm sure that's what the other candidates I'm competing with are doing. And if they're spending 6 hours, I should spend 8 hours at least, to get an edge over them. But they know I'm spending 8 hours, so they're probably spending 10...
>I would agree that companies that give "2-3 hour" take home problems are a red flag for me
I wouldn't say simply having at 2-3 hour take home by itself is red flag. But if the take home is a pile of hot garbage (poorly or inadequate documentation on how to setup a local env for example) then that is red flag.
My first hire (that wasn't someone I knew/previously worked with) I did not require that they complete a take home test. That hire turned out to be a complete disaster. I have since started requiring take home tests, and I will never again hire anyone that I don't know without a take home test. After requiring take home tests the candidates that have been hired have been much better.
Why the assumptions?
Not everything is a docker run.
I can think of several well documented setups that can take upward of 2 hours, especially if you dont have high-speed internet or not so powerfull computer at home.
>I can think of several well documented setups that can take upward of 2 hours
If ALL you have is "well documented" setups that take 2 hours, then that company sucks.
Look you don't have to have docker, but if you have a setup that is merely a Confluence page, or a pdf file, and don't have ANY shell scripts, or any way whatsoever to bootstrap setting up your environment locally, then that is a massive red flag.
Eh...I just finished a month of interviews and I ran into two companies who were early on in their hiring process and their take home test definitions/environment instructions were just...broken. Sure, I debugged it and got it working while going over the issues with the interviewers but you're always left wondering "is this part of the test" or "do these folks just suck?"
Or they're making the cardinal mistake of hiring developers who have N years of experience painting brown houses (per the link below) -- i.e. "Have you worked with exactly our stack, such that you have all the build steps and gotchas in your muscle memory?" (without a good reason for doing so) - rather than general technical and GSD skills, which usually.
Granted, sometimes fluency in the XYZ of their stack matters - but usually it just doesn't, and grilling people for this fluency shows a lack of sense for the broader picture, in my view.
It happens with groups of friends and recruiters that send batches of applicants to the same company too though. They all share intel that's passed on. For example, there's a company that makes you do a command line battleship game, everyone knows this and they expect you to spend a day on it but many of course take a week...
I give similar assignments to our candidates. Most of them complete most of the assignment, and we consider that a pass. Some people struggle to get started like this, and to be honest, this is an intentional part of the assignment; they wouldn't be successful at the company. The ability to pick up something up quickly is not something everyone has.
In my experience, companies never bend on their in-house hiring process, so just walk away. No need to explain, unless they ask you to explain, and even then I'm not sure you ought to explain.
Politely say something like "After some reflection, I've decided not to pursue the position at [company]. Thank you very much for the consideration. I wish you the best of luck."
It would be if I were to say it. I would not wish luck upon someone that behaved like this. It would mean they would have the potential for continuing this insanity on others.
Exactly. I'll do the take home all day long over standing in front of a white board and hoping I'm familiar with the random programming trivial question that I'm about to be asked.
Now apply to 10+ jobs when you're unemployed, and burdened by at least 2-3 take home assignments. Sure if you're in a role and prospecting for other jobs, 1 assignment every couple of weeks/months isn't much, but if you're applying in bulk, say goodbye to your days.
I'd still prefer, as leetcode needs "prep" and studying, even if you are senior. I don't have that time free compared to a few take homes in areas I already know
I'm genuinely surprised I had to scroll so far to see this answer, even more so by the downvotes. Especially for a junior candidate, I'd consider a pretty good sign if they were able to ask for help effectively and at the right time in their process.
Depends if it was a standard project format like npm or setup.py.
The risk is, by asking, you indicate you don't know how to use standard industry tooling.
"hey, your starter code is not running in my environment, have any documentation about how to set it up" is a better question to ask when it's some custom script they hacked together.
> The risk is, by asking, you indicate you don't know how to use standard industry tooling.
But if that's actually the problem, then it's in the best interest of both the employer and the applicant that they not be hired. The applicant isn't ready, and is likely to have a miserable time on the job.
> then it's in the best interest of ... the applicant that they not be hired
This kind of wordplay makes sense from HR's perspective. Ex: it "wouldn't be a fit for you anyway", there are "other jobs where you'll be able to grow more", etc.
The best interest of the applicant is always to get an offer (or stop the interview process of their own volition). Not get rejected by a screen and get zero feedback. Obviously this costs more for the business, which is why they have incentive to use that kind of reasoning.
Even if you shouldn't take it, competing offers can be used in various negotiations. And getting practice at interviewing is how you get better at interviewing (and at your actual job).
I don't think it's fair to call my comment wordplay. The only point that I was making is that if an applicant is unable to perform adequately in the job, it's in the best interest of the applicant and the employer that they not be in the job.
I don't think this is a controversial statement. If such an applicant gets the job, both the applicant and the employer will be unhappy and someone will either have to quit or be fired, which also sucks for all concerned.
It evokes a larger brand of similar wordplay which is basically the professional equivalent of "it's not you, it's me". The reason people don't like it is that it essentially tries to trick applicants into misunderstanding their economic interest, rather than just admit they're rejecting you because you're bad.
> If such an applicant gets a job...
Getting the job offer would not cause any of those problems. By getting it you would build valuable interviewing experience. Taking it is the thing that's not in your interest. That's all I'm trying to say.
1. It is near trivial for an experienced programmer in that language to do the task: You don't belong applying for this job. Example: Python. If you can't setup a python virtual env in about 30s... don't apply for a python job.
2. It is near impossible without their IDE settings. Java projects can be like this, at times. Though usually not to the tune of 3 hours, but I could see it getting there with a complex J2EE + database setup.
Know which case you are in, judge accordingly.
Personally: I never do take homes, unless I have a good feel for the company. If the take home is given by HR. Welcome to my circular file. This is a hard and fast rule at this point. If the hiring manager wants me to do it... That's different. But I want to talk to them and decide if this is worth my time first.
If a company get get 500 candidates to each spend an hour on a task, for 500 unpaid person-hours of work, to save themselves single-digit paid hours weeding through applications to narrow them down, seems like it's very bad in an absolute what's-best-for-society sense, but quite a deal for the company.
Your experience is probably in the minority of Python developers. There have been plenty of updates to popular packages + the core interpreter during that time, updates to package/env managers (e.g. poetry). It's one thing if you're using python to solve projecteuler problems, but it's another if you're writing any full-stack code that has a variety of dependencies. It's great when you can set up a dev environment and then "forget" about it for some time, but 5 years is an exceedingly long time to not touch an env setup, and it implies that you are not really working with external dependencies, or too many other coworkers.
(Talking about several-hour long take-homes, not FizzBuzz problems).
It is cruel, but it is numbers game. The take-home is easier for hiring managers given the macro environment and would help to cull out false positives (hence "numbers game"). Our industry has way passed the point of not playing these numbers game.
The most friendly ones I've seen is a "take-home" done onsite with generous time given, and an engineer can answer questions. It helps to simulate work environment as much as possible. The upfront cost is extreme for the companies though: you put up your engineers time for these onsites. You did end up with good hires, but the time limit is real (v.s. take-home), and that shoot up your false negatives through the roof. But it does respect candidate's time, and friendly.
For companies can afford (in the sense that good candidates just walk away), "take-home" portion also helps "alignment". If you spend extra time on it and really want to get in, you are probably ideal (or desperate, but still ideal, gives the company more leverage). These are given no cheating is happening though.
I have to admit that I never did good on these "take-home" tests. Setting up environment normally was not an issue, just without an engineer to clarify constraints, I often end up with less-than-commonly-expected solutions and make the interviewer hard to judge the end-result (most "take-home" from interviewers' perspective are very structured like exams, you get x points here, y points there, if you go off the expected path, it was hard to score these points). ChatGPT probably will do a better job than me at this these days though.
One of my most enjoyable past interviewing experiences was doing a take-home for some game company (probably Accolade). They shipped it as a zip file (late 90s), and there was a clear "todo" function. The task was to navigate an airplane across the map from a starting square to a destination square while avoiding mountain squares. The airplane always moved ahead 1 square each move and you could tell it to turn left 45°, stay straight, or turn right 45°. If you compiled it without writing the function, the airplane just flew straight.
Perfectly reasonable task, IMO; it's even relevant to the gaming domain; you could do a simple version in an hour; I spent about 2 hours as I wrote a test harness and let it run overnight keeping score of how many maps it succeeded on and how many it failed on.
The interviewer used my submission in the on-site as well to ask a few questions about it.
Your numbers game comment is spot-on. From the employers' end, the hiring market problem looks like:
There's high pay for software engineering roles.
There's no effective credential mechanism. (I think this good, but it's not helping employers here.)
Because of the combination of the first two, lots of people are motivated to try to land a job.
Because of that, lots of people who aren't really qualified enter the pipeline.
Because they're not really qualified, they keep applying over and over, while the qualified eventually get hired.
Because of that, the applicant pool has a massive adverse selection problem (the average applicant is *much worse* than the median employee) and companies have to find a way to filter out the worst-of-the-worst before applying significant manual effort.
Take home coding tests as a 0th round interview step can be extremely effective if they are short (30m-1h), relatively easy pulse checks (a step or two above FizzBuzz level). Any more involved than that and they are insulting to candidates, thus driving away talent.
I worked for a company that administered a simple half-hour take home coding challenge before any interviews. The problem was simple but not trivial — print Fibonacci numbers, take a string of brackets and print if they were properly nested (e.g. '()(())' is OK, '(()(' is not).
We had essentially zero attrition of candidates with good CVs who refused to take the test. On the other hand, we disqualified >80% of applicants from the test alone, saving an enormous amount of interviewer time. It would have sucked to have phone screened all those candidates and spent half an hour of company time on each one to discover they can't code at all.
It sounds like it is working for that company so more power to them I suppose, and I've certainly done my fair share of coding tests and take home assignments, however I consider it personally disrespectful to automate the initial interview round in this way. If you're asking me to put in work, even if it's an ostensibly short task, I want to talk to someone first.
Well, ok, everybody sees things their own way. Personally, I would be glad to do a short take home if it's clear that it will reduce the bullshit analysis of curriculum.
I'd be even happier if it replaced a whiteboard FizzBuzz, and would raise good flags if it was a mathy problem like the ones the OP talked about, instead of unmotivated terminal or memory juggling.
I agree, it's disrespectful. I always give people the opportunity to speak with a human about the job before requiring they invest any time on an assignment. Seeing if someone has the skills to do the job is secondary to determining if the job is a fit in plain-english conversation.
I would hope that your instructions are very explicit on what is expected if you are rejecting candidates for submitting _too complete a solution_ to the problem you gave them.
It depends. The snippet of code itself, yeah, it could take less than 30min. But then what, do you submit just as it is? No, you run "main.py/main.go/main.whatever". Do we need command line parameters? Maybe? Better to set them by default? Ah, well, then we probably need a "-help" so that the parameters are self documented... or do we also add them to the README.md? Because we are adding a README.md, right? And, do we setup the git repo (because we are submitting a git repo and not a zip file, right?) the "right way"? That means different things with different programming languages. Ah, what about the tests? Should we add an easy way to run the tests (e.g., Makefile?) or do we rely on the default way to run tests in the programming language we have chosen? But what if we choose a programming language the company I'm applying for doesn't use? Well, better to document that, right? Wait, do we need to print the results in the standard output just like that? The specification of the problem doesn't say much so I better e-mail them to ask for further instructions (that will show them that I don't assume anything and I always avoid doing the wrong thing! That probably gives me points at the end of the day). 1h has passed since I sent the email and no response from them so far... umm, should I just do it my way? Ummm
>hopefully none of your candidates need anywhere near 1/2 hour to code stuff that simple
IDK, is it that simple for everyone in tech to finish it in way less than 30 min? I think I would definitely need more than 30 min for the nested brackets validation problem and I've been in SW dev for 10 years and I'm about average in my area compared to everyone I know (non-faang/non-big-tech).
The thing is, I haven't touched algorithmic riddles like that since I graduated university over 10 years ago, and none of my daily assignments at my coding jobs since, were ever anything close to that, and nor do interviews in my area (Europe) ever require those kinds of problems to be solved, so I would definitely need some time to warm up, grab a piece of paper, and think of a solution for that one.
New-grad version of me who did a lot of algorithmics assignments, lab work and interview prep back in the days could definitely be faster, but this version of me who's head is filled with solving the issues assigned at work, not so much, and would need longer.
When I apply to jobs in my area, most companies here just check if you have domain knowledge related to the problems you'd have to solve if you got the job. Coding riddles like that seems more of a US, big-tech, VC-unicorn kind of thing that's missing in the companies I applied at. I wouldn't mind such riddles as an interview, but I definitely would fail them if they were strictly timeboxed.
I guess I'm not employee material for his company.
I bet you wouldn’t, it’s not something I’ve come across before, but having seen it now I’m absolutely stealing it because it has the wonderful property of seeming complex on the surface and actually being incredibly simple.
Do you have a language that allows you to easily create a stack? Push open brackets onto the stack, when you get a closing bracket, pop the stack and make sure it's the "right" one. When the input is exhausted, if the stack is empty, the input was balanced.
Since the only thing you are pushing onto the stack are open brackets you don't actually need the stack. Just keep a count of how many brackets would have been on the stack.
Increment the count when you see an open bracket. Decrement the count when you see a close bracket. If the count every goes negative there was a mismatch. If the count does not end up at 0 when you've processed all the brackets there was a mismatch.
It can also be treated as a text transformation problem. Every non-empty string of matched brackets must contain at least one '()', and deleting a '()' from a string does not change whether or not the string is balanced.
That gives an approach of repeatedly finding and deleting '()' from the string until there are no more '()'s to delete. If that gives you an empty string the original string was all matched. If the final string is not empty then the original string had a mismatch. Here's that as a shell script.
#!/bin/sh
S=`echo $1 | tr -c -d '()'`
P=x
while [ "$S" ]; do
if [ "$S" = "$P" ]; then
echo "Mismatch"
exit 1
fi
P=$S
S=`echo $S | sed -e 's/()//g'`
done
echo "All match"
exit 0
It depends on the problem definition (specifically on the definition of the word "brackets"). If you want to ensure the proper balancing of all of {}, [], and () [while excluding improper nesting], you need a stack [or equivalent]. If you only want to ensure () balancing, you can just do a count.
IMO:
({[]}) // has balanced brackets
({[}]) // has imbalanced brackets
I think your algo just checks if there are exact number of open and closed brackets, but it doesn't check that they actually match together in a valid way.
yes, the above is true. Kind of shocked that anyone could get through an intro CS course without having knowledge of this as it is a very common type of sequence.
It also generalizes, apparently "Fibonacci sequences" are defined by the initial 2 values, with the rule mentioned above. I thought this was weird once but upon checking it it is true that, for example, 4 5 9 14 23 37 would also be Fib{4,5} or something. Idk it's been a long time.
This is one of those things that you'd be amazed once you start looking at interviewer's ability to judge effort. Often times the questions asked or expected of is just excessive. They'll ask problems that if you didn't know the trick or (years of ) research behind it, you'd never solve it. They'll give a take home asking for a full service and full deployment on K8s and claim "it should only take 6 hours".
I fail to see the problem. If companies look for people who have years of experience in k8s or for people who don’t trip on something specific it is fair game.
If they look for full stack dev and then grill on some obscure details of one tech stack piece it is also their problem.
Someone can have lots of k8s experience but not have needed to set things up end-to-end. Most people generally are familiar with "X as used at my job" which is very different from "X as what happens when I use it from scratch". This is true not just for orchestration systems like k8s, but also programming languages: people are used to the libraries they have for solving common problems, which are not necessarily OSS / standard libraries.
We discuss "experience with" which for me is really vague.
Is company looking for experience like "someone who could connect to k8s and debug application"? Is company looking for "someone who can setup k8s and have networking up and running"?
How do deal with people claiming "experience with" where someone set something up for them and they were using stuff but never had to dig into details.
In the end it is also like driving - while driving is driving then "F1 driving" is so much more and while F1 driver is not an engineer you still expect him to understand how ICE engine works and bunch of other technical details. For everyday driver you expect no more than "put in fuel, turn the key, throttle and you drive, break you stop".
Context of this is:
Senior engineer expectation: Build an app that returns results for x problem. Deploy it [there are a lot of requirements there to make that at least a week long]
Staff: Create and deploy the app with a CI/CD+deployment platform. (K8s)
I can echo this and also push back against most of the other anti-test comments in the thread. I agree that an unbounded and abstract take home test is bad, but we use an online test provider (leetcode-esque) with some custom relatively easy questions.
We only do it for certain demographics (i.e. interns and fresh grads) while experienced candidates skip it, and it certainly saves a TON of interviewer time.
Even for the internship alone we get tens of thousands of applications and it's pretty intractable to review all of their resumes manually in a relatively small window of time, so there needs to be some type of automated system to discard a majority of candidates.
FWIW, I've had good success with these two approaches:
* code review: Screen share with the candidate a small code snippet in a language/framework that they are comfortable with that needs improvement. Have candidate talk through what feedback they would provide in the context of a code review to improve the code.
* code pairing: Pair candidate with an engineer on the team to improve some existing code (add a simple feature).
Both of these approaches have been very helpful at understanding how skilled a candidate is, how familiar they are with a language or framework, how they provide constructive feedback, etc.
Code snippets can be designed to encourage discussion of security practices (SQL injection, for example), capacity planning (5 transactions/day, /hour, /sec?), algorithmic complexity, etc.
I was recently looking for a job and a company I interviewed at had the first round as a take home test. You had to write a CLI program that called their stupid-simple rest API (takes a number, returns 1+ numbers streamed) in the language/runtime the company uses. They said it shouldn’t take over 2-3 hrs. I set a 2Hr timer, completed it and got partway through tests, and stopped (leaving unfinished tests or missing tests etc). I spent 30 min writing todo comments (“TODO: refactor this method for easier test-ability”) and documentation.
2.5hr total, then moved on to next in-person round. Their in person interview questions are all based on either my code or their server side code for this test project. It was great because I already knew about it from the take home.
They explicitly asked for a simple, working cli program in the language they use, in a reasonable time box. They said if you have time, consider test-ability, consider performance (the service streamed numbers so you needed to not store them in-memory), and other things a “real” program would need to consider. They gave themselves amble signal for a simple project, and set the time box low enough that most anyone can find that time.
I will absolutely be using a similar process for hiring at my current company.
Seems like a horrible design choice to implement something so error-prone (and not immediately, visually obvious) as a one-liner -- and involving both dynamic code generation, -and- the awkward and performance-killing use of exception handling as a loop control, no less.
Not really sure why any company would set a take home test these days. It not only sends a signal that your time as a candidate is worth less than the company's, it makes it easier to game (e.g. use ChatGPT).
This is a thing now when applying for developer jobs.
When I was looking for a job in 2020-2021, I think it happened everywhere I got past the original opening interview round.
Sometimes the task was trivial or even submitting code samples (which can be problematic if the relevant examples were done for a previous proprietary interest that would not be agreeable to share this code). Other cases were more elaborate, requiring a half-day of work or more.
In one instance, I had an assignment for a front end JS / UX page, completed the assignment then heard back from the prospective employer that according to their review of my "homework" submission, I was overqualified for the position.
I mean, you can say "no", but then in all probability the prospective employer is going to proceed to the next candidate.
I don't give take-home test tasks. I believe this is a waste of time for the company because it gives pretty much no useful information. A well written solution says nothing -- it could even potentially have been outsourced to somebody else. It also causes the candidate to waste as much time as possible because the candidate will feel pressed to spend every bit of time to further polish the solution.
I think if I was to refuse take-home task this would be the exact official reason: Because I believe take-home tasks are unfair and incentivise candidates to waste as much time as possible for no benefit for the company.
Instead, I do pair programming session debugging a piece of code and unit tests. It is not about the solution, it is about getting to know the candidate as he/she is working on a task.
> "The test was as much about seeing if the if the candidate could deal with manually setting up the build environment without assistance as much as it was about the coding problem itself. If the candidate couldn't get the tools installed on his own or felt insulted it wasn't his ideal IDE, then they figured the candidate would not do well as an embedded engineer on more complicated boards with even worse tool chains."
In general however a take-home test voids the primary rationale for the real-time on-site coding interview, which is that it's very easy these days to complete such tests without knowing much about the material via various routes, as academic CS programs continue to demonstrate. It's also true that academic CS programs tend to be terrible when it comes to learning about tools, toolchains, build systems, virtual environments, etc. There are some efforts to change this at least:
Even that's not that comprehensive. Searching "introduction to toolchains and complex build systems" reveals how difficult the issue is. It's the kind of thing that should probably be included in the job description.
Yes, I find that students straight out of school understand the concepts of coding, but have trouble applying their knowledge. Knowing how to write a function in Python is one thing, but if you can't install it on a computer and use it to solve a problem, you're not going to be productive unless we have other employees who will do those things for you. And some organizations do. Particularly the large organizations that hire lots of students straight out of school.
Giving people take home tests or leet code challenges is a backwards hiring process. Why as a manager are you wasting your time and the candidate's time "discovering" if they can code when there's millions of developers who are actively working in the open on OSS platforms like GitHub?
I don't give tests, I don't give technical interviews. I scout amazing developers and try to convince them to join my team. Not only do you get top notch people like that (who very much appreciate not being "tested"), you also save a ton of time.
Your job as a manager is to proactively search for the best people possible to join your team, hire them and keep them as happy as humanly possible.
You will certainly be able to find excellent developers that way. You'll also automatically rule out the vast majority of amazing developers that aren't "actively working in the open on OSS platforms".
If I can quickly hire as many amazing candidates as needed using this method, then I don't really have a need to try to find the needle in the haystack that are the great devs in the global community of all devs using technical interviews. That's a very wasteful process.
> Why as a manager are you wasting your time and the candidate's time "discovering" if they can code when there's millions of developers who are actively working in the open on OSS platforms like GitHub?
The number of developers who have something recent and meaningful on GitHub is extremely small.
I’ve interviewed and hired a lot of great developers. Very few of them had much of anything on GitHub. Often the most I could find were one-line patches to projects somewhere.
If you limit your hiring to GitHub OSS superstars, you’re going to miss out on a massive number of excellent developers.
The truth is that I've never had a problem sourcing candidates this way for the past decade+. There are tens of millions of developers on GitHub. You will find people who are very accomplished in the exact area you need, and it's easy to track them down by navigating through the projects.
Extra bonus points for hiring the maintainers of libraries/software you're actually using.
This is true. I have no code on GitHub or the like, personally. Although I have plenty of code that I can provide to a company if they want to see it. Some have, most haven't.
I don't think it's an issue that some companies want to use GitHub as a shortcut to vetting developers. It does limit the pool they're fishing in, but that's what all vetting processes do.
> there's millions of developers who are actively working in the open on OSS platforms like GitHub?
There's an order of magnitude more qualified developers who don't have the time/aptitude/patience for contributing to OSS. You're shutting yourself off from the bulk of the hiring market by limiting candidates to OSS contributors.
It's true that there are great devs that aren't doing OSS. There are so many devs doing OSS though that it's totally possible to hire exclusively this way. I know because I've done it for half a dozen teams, in all manner of different companies, to great success.
You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is.
I have seen software engineers getting lost trying to solve a trivial problem because they like to always write code that scales well or showcase their knowledge of software design.
Take home projects are great to learn about a candidate's decision making process given the use case and constraints of the final solution.
IMO that would be hard to assess by looking at a repository.
As far as having an entire team of top performers, IMO most companies don't need that.
Where I work most of the stuff we do never gets released as a final product, we may work on a new feature for six months and then gets killed, because the business requirement changed, because we partnered with another company that already provides that functionality or because our VP changed his vision.
Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
We do have a few way above average coders leading the way, but most of our team is made of good coders, not stars.
> You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is.
Starting a project from scratch, shipping it and supporting it is quite a bit more difficult than regular company drudgery. I've never had an issue with someone hired via this method struggling to perform once in the door.
> Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
You should always pay a fair salary, but you'd be surprised that when you use this method you can hire some amazing people for "regular" pay. Respect goes a long way with top notch developers. A lot of the best people won't even do the leetcode dog and pony show, even if it results in very high pay. People will usually compensate for what they view as a toxic environment by demanding a higher salary (and quitting a couple of years in).
This is fascinating. What is your process for scouting amazing developers? Do you just go to random repositories, look at random commits, and identify people who are writing clean code?
I start with anyone building with my company's OSS tools or contributing to them in a meaningful way. I also look at anyone working on libraries or OSS software that we're using in our projects.
Beyond that I scout for people who launch projects from scratch (large or small) as it's a great way to see how people solve problems and how good their product sense is. Sometimes I'll just browse through the top charts on GH in the language we're using. Between GH, Twitter, HN and Reddit I see dozens of projects a day (more than I have time to investigate).
One nice thing about doing this is it tends to snowball. Once you hire a high profile maintainer or several OSS enthusiasts, it makes it much easier to hire more. OSS developers will typically be skeptical of "business people", so it's best to either have an OSS presence as a manager or reach out through another OSS dev.
I also stress that people can keep working on OSS in their free time (and keep ownership over their personal code). I go into the first meeting prepared to offer the person a job and also offer them the opportunity to interview anyone else at the company they may want to talk to.
The results have been very positive, even if they don't take the job. They're usually flattered that you've done the research on their work and value it enough to offer them a job sight unseen. I would estimate that I have ~75% acceptance rate.
220 comments
[ 2.9 ms ] story [ 225 ms ] threadAs a company, you'll want to prevent this sort of scenario, but as the interviewee you really don't owe them anything so you can just say you're no longer interested.
If, for some reason, you feel compelled to improve their methods then you will have to succeed at their assessment in order to do so. It is a property of humans that to change a thing you have to be able to show you can handle a thing.
If you don't want to do the take home test, tell the company just that. You aren't going to hurt anyone's feelings. You don't have to be overly polite about it or think of excuses. No one there is going to care.
They will just move on and hire someone else. You can move on and apply somewhere else. That's all.
(and evidenced by "I will also state that they can check out some of the previous code I have written on my GitHub." where the candidate still wants the company to keep considering them after declining the test.)
How do you know that's the goal of the test?
Ask yourself how the company founders and initial employees got their jobs. Almost certainly not by going through their ex post facto, arbitrarily selected in-house hiring process. Was that unethical?
IMO there's actually nothing wrong with hiring the first qualified applicant who shows up, and then calling off the search. Is there an ethical obligation to hire "the best possible candidate", and why? See https://en.wikipedia.org/wiki/Satisficing
Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone? It it unethical to go with an internal candidate and not offer an open position for everyone?
Companies tend to waste way too much time, effort, and money on hiring. It's premature optimization, the root of all evil.
Uniformity is not the same as ethics. Otherwise, we'd be ethically obligated to eliminate special parking stalls for the disabled. Moreover, there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits. How do you judge the ethics of the criteria themselves, in comparison to other possible uniform criteria?
For founders this is a perk of being a founder. You are welcome to found your own company if you wish.
> Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone?
Yes, it can be. Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
Yes, but that didn't answer my question about ethics.
> You are welcome to found your own company if you wish.
I did!
> Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
How often do tech companies get a "representative" pool of candidates?
But this actually goes to the other issue I raised:
> there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits.
In particular, take-home exams that take many hours to complete tend to discriminate against certain kinds of candidates, those who don't have a lot of free time outside of work, family, etc.
It's important to note that these tests are primarily used by tech companies as a screening device, not as the method for making the final hiring decision, which tends to be more subjective. They don't simply add up test scores and choose the highest scorer.
Moreover, the tech industry seems to be obsessed with tests, but they're an outlier. Tons of other industries use more informal methods of hiring and don't give tests at all, much less take-home tests. Do you think they're all unethical?
On the topic of hiring, a "Job Simulation" is one of the most objective measures, which is pretty close to what a practical test is. Questions about things candidates actually did is another important technique (e.g. tell me about a time you ...), and the third category of techniques that work is giving hypothetical situations that they need to explain how they would respond to.
Although informal conversations are common in industry, they are some of the worst techniques. For example, if you start with a "coffee chat" research has shown that most people more or less make up their mind in that first meeting then bias the rest of the interview results to unknowingly justify their decision.
https://www.eeoc.gov/laws/guidance/employment-tests-and-sele...
"I know that every other candidate for the job successfully did this exercise but I found it too difficult to even set up. Can you hire me over all of them?"
What do you think the company's response will be?
"Every other candidate was desperate and rejected a thousand times so they're happy to waste their time on your pointless exercise that you won't even look at."
The good candidates are busy getting hired by companies that understand how to evaluate their work based on interviews, references, and several other things that don't waste their time.
But there were also reports on HN that before the layoff wave, they'd get 2-3 applications per year for a senior position. It is of course any employers right to continue applying the same hiring standards as eg. FAANG/Big Tech would. But if I were the employer, a better strategy in such a situation would be to just swallow that I can't pick and choose if I have nothing to offer for leverage :)
I'm sure you're smart enough to realize that what I'm hinting at is that the job market is a _market_ and that employees don't have to put up with every bullshit depending on the market conditions.
If you accepted the take-home, and failed to get started on it, and then asked for something else, I wouldn't. Because I have evaluated other candidates and they were able to do it.
I don't do these sorts of tests, personally (I even had one that they expected would take a full 40 hour workweek to complete!). If I'm presented with one, I know that in declining to do it, I am disqualifying myself for the position.
It's just life. We all decide what our limits are, and in deciding those limits, we exclude some possibilities.
They spent longer time than preferred.
And yes, they're looking for a shortcut.
There is no way around fulfilling the task.
Not revealing the time spent poses risk of impostor syndrome, risk of actually landing a very stressful job.
Refusing to solve the task indicates that they would rather post-pone finding out, and rather risk rejection if it means not being challenged on their solving speed disqualifying them, because they think that it too likely would disqualify them.
Personally I prefer a take home over a full day technical whitebording on-site which is my idea of hell. But unfortunately many companies do both in the same loop instead of one or the other
About a month ago I had this happen:
-Explained full interview process during the phase 1 call (15 min call, round table, and lastly a take-home code challenge)
-they were cool with all this
-they get to the end of phase II, after an hour long call with a bunch of people, and passed it/went great
-they then surprised us by refusing phase III/the take home test
As an employer I try to set expectations early, they acknowledged it and then proceeded to get upset when we did what we said we were going to do.
Also, why would you do a round table before a take home? Do you not value the time of your people?
I mean they made it through Phase 2 and sorta 2.5 where we confirm salary and they accepted. Everything seemed on track until the test was mentioned.
If they ghosted us after the round table interview then sure, but why come back and accept a salary just to bail at the last step?
If it were my company, I'd be taking a cold, hard look at that process to be sure that it's OK. Perhaps it's not really going as well as it seems.
I suspect that your messaging during phase 2.5 is bad and you're misleading candidates into thinking that you've decided to hire them, and then when you follow that up with a take home they conclude that you're fucking with them or a disorganized mess.
By doing it at the end, you are just telling the candidate "Oh, you are good. You passed phase I and phase II... but that's not enough you poor candidate!".
Every time I've been presented with a take-home test, it was done this way. I'm genuinely surprised to hear that some companies do this at the end.
Basically, assume that candidates can drop at any time.
My advice:
Your first step as a candidate when given an assignment (with specific time constraints that you feel are mis-estimated), is to guestimate what you think it will take. Remember, this is exactly what happens at work with task estimation. Are you going to 'politely quit' if your manager gives you a task that can't be done in a certain time window?
"I reviewed the task and my estimate is that this will take a few more hours than specified. I would like clarification if the time limit is an informative hard limit."
One benefit of asking for estimate confirmation (specially for people like me who tend to go for more general solutions) is that you may actually get useful feedback. If you get a firm 'yes' from a competent team then it means they are looking for absolute simplicity in solution, and your weekend project of pimping your take home task is not only a wasted effort, it may actually take you off the list.
Whatever response you get back it will be informative, sometimes even shedding light on what sort of organization you are applying to join.
What? I like sokoloff's response[1]. It takes 3 seconds.
1: https://news.ycombinator.com/item?id=35183583
Just a guess, but perhaps they're trying to provide some constructive feedback to the company to let them know that the time estimate they're giving to their candidates is potentially way off. (That said, maybe it's not way off for people with more experience, but the candidate doesn't get that.) It's probably futile, but I can understand the urge. If they company doesn't know a problem exists, they can't fix it (if it is an actual problem).
And yes, this is abuse. Demanding meaningless work, for free, for the chance to get hired? Because you can't figure out how to interview in a way that allows you to judge competence? Yeah, I think it's fair to call that abuse. At least exploitation.
I would probably make an exception if they offered to pay me even a token per hour (or even donate the hourly to a charity - it's about incentives not money). I have made an exception when it was literally a 20 min code challenge for a promising lead.
On one hand, this does frequently stop things moving forward. On the other hand I never felt like I missed out.
One company I sent them my usual polite decline and got back an angry rant from the CEO! I responded with "Well, give me your product for free for 8 hours or so and I'll give you my time for free for the same length." He did not respond after that.
This is probably bad advice for very new engineers where a code sample can help differentiate you a lot, but I wish more senior+ engineers would refuse these dumb tests.
The handful of times I've dumped a bunch of time and effort into an at-home screening assignment, it took far longer to do well than whatever the allotted time was made out to be. Emphasis on the "do well" part.
My (unscientific) sense is that these things are really just screening for people willing to jump through unreasonable hoops.
Counterpoint: what idea can be had from rushed / time-constrained / deliberately careless work examples?
Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
Experiences vary, but every job I've taken and liked - no coding tests of any kind. More of a discussion about an example piece of code, for that part of the interview, but no requirement to come up with new code.
In my case, I actually do want to know how quickly people can pick up a new problem and get started writing some code, and that's how I evaluate what they deliver.
> Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
I have two thoughts about this.
1. If people can deliver, they can deliver.
2. If you come up with an original problem (rather than lazily copy and pasting some fizz-buzz type problem), you'd be surprised how many people fail to apply any basic problem solving.
> a discussion about an example piece of code
I do find that it is useful to keep code exercises short, but ask a few questions about their solution as a follow up. People often struggle to explain their solution if they just copy/pasted.
The best way for senior folks to get a new job is through referrals.
The sdk was broken!
So I produced a minimum viable repro, wrote up a bug report and even included an analysis of where the bug was.
The company had the gall to still ask me to do a take home using the patched sdk!
If your take home test is broken, stupid or not trivial to setup, it's not the applicant who is at fault.
And the requirements might be reasonable for an already setup project, but when you have to do it from zero; unless you're scaffolding new symfony projects every time a new release is out, you're going to waste at least an hour scaffolding/googling/installing/wiring up all the dependencies/auth/orm you might need for you flow. It's not as bad as within the NodeJS ecosystem, but all libraries/frameworks that have a yearly major release, like to break things along the way so you're never going to hit the ground running in a couple of minutes.
Oh and I should also add that I've never had a recruiter tell the truth about how long one of these should take. They always lie with some ridiculous number of hours that I know only a tiny percentage of the entire programming population could perform at.
I agree, yet there's a bunch of people in this comment section claiming you should give people take-home tests first which makes no sense to me.
I would agree that companies that give "2-3 hour" take home problems are a red flag for me — red as in, I won't do it anymore. That situation becomes an arms race quickly: if I can solve it in 3 hours, why not spend another 3 hours doing a second, more elegant solution that makes me look even smarter? I'm sure that's what the other candidates I'm competing with are doing. And if they're spending 6 hours, I should spend 8 hours at least, to get an edge over them. But they know I'm spending 8 hours, so they're probably spending 10...
I wouldn't say simply having at 2-3 hour take home by itself is red flag. But if the take home is a pile of hot garbage (poorly or inadequate documentation on how to setup a local env for example) then that is red flag.
My first hire (that wasn't someone I knew/previously worked with) I did not require that they complete a take home test. That hire turned out to be a complete disaster. I have since started requiring take home tests, and I will never again hire anyone that I don't know without a take home test. After requiring take home tests the candidates that have been hired have been much better.
It doesn't have to be a docker run.
>I can think of several well documented setups that can take upward of 2 hours
If ALL you have is "well documented" setups that take 2 hours, then that company sucks.
Look you don't have to have docker, but if you have a setup that is merely a Confluence page, or a pdf file, and don't have ANY shell scripts, or any way whatsoever to bootstrap setting up your environment locally, then that is a massive red flag.
"It works on my machine!"
Granted, sometimes fluency in the XYZ of their stack matters - but usually it just doesn't, and grilling people for this fluency shows a lack of sense for the broader picture, in my view.
https://versastudio.com/blog/if-carpenters-were-hired-like-p...
Politely say something like "After some reflection, I've decided not to pursue the position at [company]. Thank you very much for the consideration. I wish you the best of luck."
If a company is asking you to use your free time before you're hired, that's a red flag for their expectations if you do end up being hired.
Instead of spinning your wheels and wasting your time on your own.
That's what I'd want an employee to do.
I care less that you are having issues than I do about a complete shut down or lack of communication.
Update: Already getting downvoted for this response and not even a comment to explain why... classic.
The risk is, by asking, you indicate you don't know how to use standard industry tooling.
"hey, your starter code is not running in my environment, have any documentation about how to set it up" is a better question to ask when it's some custom script they hacked together.
But if that's actually the problem, then it's in the best interest of both the employer and the applicant that they not be hired. The applicant isn't ready, and is likely to have a miserable time on the job.
This kind of wordplay makes sense from HR's perspective. Ex: it "wouldn't be a fit for you anyway", there are "other jobs where you'll be able to grow more", etc.
The best interest of the applicant is always to get an offer (or stop the interview process of their own volition). Not get rejected by a screen and get zero feedback. Obviously this costs more for the business, which is why they have incentive to use that kind of reasoning.
Even if you shouldn't take it, competing offers can be used in various negotiations. And getting practice at interviewing is how you get better at interviewing (and at your actual job).
I don't think this is a controversial statement. If such an applicant gets the job, both the applicant and the employer will be unhappy and someone will either have to quit or be fired, which also sucks for all concerned.
> If such an applicant gets a job...
Getting the job offer would not cause any of those problems. By getting it you would build valuable interviewing experience. Taking it is the thing that's not in your interest. That's all I'm trying to say.
1. It is near trivial for an experienced programmer in that language to do the task: You don't belong applying for this job. Example: Python. If you can't setup a python virtual env in about 30s... don't apply for a python job.
2. It is near impossible without their IDE settings. Java projects can be like this, at times. Though usually not to the tune of 3 hours, but I could see it getting there with a complex J2EE + database setup.
Know which case you are in, judge accordingly.
Personally: I never do take homes, unless I have a good feel for the company. If the take home is given by HR. Welcome to my circular file. This is a hard and fast rule at this point. If the hiring manager wants me to do it... That's different. But I want to talk to them and decide if this is worth my time first.
The old game of chicken: https://en.wikipedia.org/wiki/Chicken_(game)
As usual, the party with the best alternatives wins.
It would probably take me an hour to set it up again.
Still I produce quality Python code (almost) every day. To me, the skills are entirely orthogonal.
But I was commenting on this:
> If you can't setup a python virtual env in about 30s... don't apply for a python job.
It is cruel, but it is numbers game. The take-home is easier for hiring managers given the macro environment and would help to cull out false positives (hence "numbers game"). Our industry has way passed the point of not playing these numbers game.
The most friendly ones I've seen is a "take-home" done onsite with generous time given, and an engineer can answer questions. It helps to simulate work environment as much as possible. The upfront cost is extreme for the companies though: you put up your engineers time for these onsites. You did end up with good hires, but the time limit is real (v.s. take-home), and that shoot up your false negatives through the roof. But it does respect candidate's time, and friendly.
For companies can afford (in the sense that good candidates just walk away), "take-home" portion also helps "alignment". If you spend extra time on it and really want to get in, you are probably ideal (or desperate, but still ideal, gives the company more leverage). These are given no cheating is happening though.
I have to admit that I never did good on these "take-home" tests. Setting up environment normally was not an issue, just without an engineer to clarify constraints, I often end up with less-than-commonly-expected solutions and make the interviewer hard to judge the end-result (most "take-home" from interviewers' perspective are very structured like exams, you get x points here, y points there, if you go off the expected path, it was hard to score these points). ChatGPT probably will do a better job than me at this these days though.
Perfectly reasonable task, IMO; it's even relevant to the gaming domain; you could do a simple version in an hour; I spent about 2 hours as I wrote a test harness and let it run overnight keeping score of how many maps it succeeded on and how many it failed on.
The interviewer used my submission in the on-site as well to ask a few questions about it.
Your numbers game comment is spot-on. From the employers' end, the hiring market problem looks like:
I worked for a company that administered a simple half-hour take home coding challenge before any interviews. The problem was simple but not trivial — print Fibonacci numbers, take a string of brackets and print if they were properly nested (e.g. '()(())' is OK, '(()(' is not).
We had essentially zero attrition of candidates with good CVs who refused to take the test. On the other hand, we disqualified >80% of applicants from the test alone, saving an enormous amount of interviewer time. It would have sucked to have phone screened all those candidates and spent half an hour of company time on each one to discover they can't code at all.
I'd be even happier if it replaced a whiteboard FizzBuzz, and would raise good flags if it was a mathy problem like the ones the OP talked about, instead of unmotivated terminal or memory juggling.
I'm nowhere near elite...but hopefully none of your candidates need anywhere near 1/2 hour to code stuff that simple.
Usually it's a "Please write a program to solve this problem with the following rules" or "Please write a service that services x,y,x"
And so on...
IDK, is it that simple for everyone in tech to finish it in way less than 30 min? I think I would definitely need more than 30 min for the nested brackets validation problem and I've been in SW dev for 10 years and I'm about average in my area compared to everyone I know (non-faang/non-big-tech).
The thing is, I haven't touched algorithmic riddles like that since I graduated university over 10 years ago, and none of my daily assignments at my coding jobs since, were ever anything close to that, and nor do interviews in my area (Europe) ever require those kinds of problems to be solved, so I would definitely need some time to warm up, grab a piece of paper, and think of a solution for that one.
New-grad version of me who did a lot of algorithmics assignments, lab work and interview prep back in the days could definitely be faster, but this version of me who's head is filled with solving the issues assigned at work, not so much, and would need longer.
When I apply to jobs in my area, most companies here just check if you have domain knowledge related to the problems you'd have to solve if you got the job. Coding riddles like that seems more of a US, big-tech, VC-unicorn kind of thing that's missing in the companies I applied at. I wouldn't mind such riddles as an interview, but I definitely would fail them if they were strictly timeboxed.
I guess I'm not employee material for his company.
Increment the count when you see an open bracket. Decrement the count when you see a close bracket. If the count every goes negative there was a mismatch. If the count does not end up at 0 when you've processed all the brackets there was a mismatch.
It can also be treated as a text transformation problem. Every non-empty string of matched brackets must contain at least one '()', and deleting a '()' from a string does not change whether or not the string is balanced.
That gives an approach of repeatedly finding and deleting '()' from the string until there are no more '()'s to delete. If that gives you an empty string the original string was all matched. If the final string is not empty then the original string had a mismatch. Here's that as a shell script.
IMO:
I have no idea how to calculate Fibonacci numbers.
I'm a decent coder, have a bachelor of Software Engineering but never "loved" math.
I'm sure I could learn in a three minute Google.. but I certainly couldn't do it on a whiteboard with no Google access.
1 1 2 3 5 8 13 ...
That's it. Now you know.
It also generalizes, apparently "Fibonacci sequences" are defined by the initial 2 values, with the rule mentioned above. I thought this was weird once but upon checking it it is true that, for example, 4 5 9 14 23 37 would also be Fib{4,5} or something. Idk it's been a long time.
If they look for full stack dev and then grill on some obscure details of one tech stack piece it is also their problem.
Is company looking for experience like "someone who could connect to k8s and debug application"? Is company looking for "someone who can setup k8s and have networking up and running"?
How do deal with people claiming "experience with" where someone set something up for them and they were using stuff but never had to dig into details.
In the end it is also like driving - while driving is driving then "F1 driving" is so much more and while F1 driver is not an engineer you still expect him to understand how ICE engine works and bunch of other technical details. For everyday driver you expect no more than "put in fuel, turn the key, throttle and you drive, break you stop".
Staff: Create and deploy the app with a CI/CD+deployment platform. (K8s)
We only do it for certain demographics (i.e. interns and fresh grads) while experienced candidates skip it, and it certainly saves a TON of interviewer time.
Even for the internship alone we get tens of thousands of applications and it's pretty intractable to review all of their resumes manually in a relatively small window of time, so there needs to be some type of automated system to discard a majority of candidates.
* code review: Screen share with the candidate a small code snippet in a language/framework that they are comfortable with that needs improvement. Have candidate talk through what feedback they would provide in the context of a code review to improve the code.
* code pairing: Pair candidate with an engineer on the team to improve some existing code (add a simple feature).
Both of these approaches have been very helpful at understanding how skilled a candidate is, how familiar they are with a language or framework, how they provide constructive feedback, etc.
Code snippets can be designed to encourage discussion of security practices (SQL injection, for example), capacity planning (5 transactions/day, /hour, /sec?), algorithmic complexity, etc.
2.5hr total, then moved on to next in-person round. Their in person interview questions are all based on either my code or their server side code for this test project. It was great because I already knew about it from the take home.
They explicitly asked for a simple, working cli program in the language they use, in a reasonable time box. They said if you have time, consider test-ability, consider performance (the service streamed numbers so you needed to not store them in-memory), and other things a “real” program would need to consider. They gave themselves amble signal for a simple project, and set the time box low enough that most anyone can find that time.
I will absolutely be using a similar process for hiring at my current company.
const test = toTest => { try { new Function(`return [ ${toTest.replaceAll('(', '[').replaceAll(')', '],')} ]`)(); return true } catch (e) { return false; } }
When I was looking for a job in 2020-2021, I think it happened everywhere I got past the original opening interview round.
Sometimes the task was trivial or even submitting code samples (which can be problematic if the relevant examples were done for a previous proprietary interest that would not be agreeable to share this code). Other cases were more elaborate, requiring a half-day of work or more.
In one instance, I had an assignment for a front end JS / UX page, completed the assignment then heard back from the prospective employer that according to their review of my "homework" submission, I was overqualified for the position.
I mean, you can say "no", but then in all probability the prospective employer is going to proceed to the next candidate.
I think if I was to refuse take-home task this would be the exact official reason: Because I believe take-home tasks are unfair and incentivise candidates to waste as much time as possible for no benefit for the company.
Instead, I do pair programming session debugging a piece of code and unit tests. It is not about the solution, it is about getting to know the candidate as he/she is working on a task.
> "The test was as much about seeing if the if the candidate could deal with manually setting up the build environment without assistance as much as it was about the coding problem itself. If the candidate couldn't get the tools installed on his own or felt insulted it wasn't his ideal IDE, then they figured the candidate would not do well as an embedded engineer on more complicated boards with even worse tool chains."
In general however a take-home test voids the primary rationale for the real-time on-site coding interview, which is that it's very easy these days to complete such tests without knowing much about the material via various routes, as academic CS programs continue to demonstrate. It's also true that academic CS programs tend to be terrible when it comes to learning about tools, toolchains, build systems, virtual environments, etc. There are some efforts to change this at least:
https://missing.csail.mit.edu/
Even that's not that comprehensive. Searching "introduction to toolchains and complex build systems" reveals how difficult the issue is. It's the kind of thing that should probably be included in the job description.
I don't give tests, I don't give technical interviews. I scout amazing developers and try to convince them to join my team. Not only do you get top notch people like that (who very much appreciate not being "tested"), you also save a ton of time.
Your job as a manager is to proactively search for the best people possible to join your team, hire them and keep them as happy as humanly possible.
The number of developers who have something recent and meaningful on GitHub is extremely small.
I’ve interviewed and hired a lot of great developers. Very few of them had much of anything on GitHub. Often the most I could find were one-line patches to projects somewhere.
If you limit your hiring to GitHub OSS superstars, you’re going to miss out on a massive number of excellent developers.
Extra bonus points for hiring the maintainers of libraries/software you're actually using.
I don't think it's an issue that some companies want to use GitHub as a shortcut to vetting developers. It does limit the pool they're fishing in, but that's what all vetting processes do.
There's an order of magnitude more qualified developers who don't have the time/aptitude/patience for contributing to OSS. You're shutting yourself off from the bulk of the hiring market by limiting candidates to OSS contributors.
I have seen software engineers getting lost trying to solve a trivial problem because they like to always write code that scales well or showcase their knowledge of software design.
Take home projects are great to learn about a candidate's decision making process given the use case and constraints of the final solution.
IMO that would be hard to assess by looking at a repository.
As far as having an entire team of top performers, IMO most companies don't need that.
Where I work most of the stuff we do never gets released as a final product, we may work on a new feature for six months and then gets killed, because the business requirement changed, because we partnered with another company that already provides that functionality or because our VP changed his vision.
Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
We do have a few way above average coders leading the way, but most of our team is made of good coders, not stars.
So far that's working great for us.
Starting a project from scratch, shipping it and supporting it is quite a bit more difficult than regular company drudgery. I've never had an issue with someone hired via this method struggling to perform once in the door.
> Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
You should always pay a fair salary, but you'd be surprised that when you use this method you can hire some amazing people for "regular" pay. Respect goes a long way with top notch developers. A lot of the best people won't even do the leetcode dog and pony show, even if it results in very high pay. People will usually compensate for what they view as a toxic environment by demanding a higher salary (and quitting a couple of years in).
Beyond that I scout for people who launch projects from scratch (large or small) as it's a great way to see how people solve problems and how good their product sense is. Sometimes I'll just browse through the top charts on GH in the language we're using. Between GH, Twitter, HN and Reddit I see dozens of projects a day (more than I have time to investigate).
One nice thing about doing this is it tends to snowball. Once you hire a high profile maintainer or several OSS enthusiasts, it makes it much easier to hire more. OSS developers will typically be skeptical of "business people", so it's best to either have an OSS presence as a manager or reach out through another OSS dev.
I also stress that people can keep working on OSS in their free time (and keep ownership over their personal code). I go into the first meeting prepared to offer the person a job and also offer them the opportunity to interview anyone else at the company they may want to talk to.
The results have been very positive, even if they don't take the job. They're usually flattered that you've done the research on their work and value it enough to offer them a job sight unseen. I would estimate that I have ~75% acceptance rate.