>While I do have some ideas of how to improve the process, I do not have a suggested perfect hiring process. Partially because it is impossible to craft a single process that will work for all roles, for all candidates, for all companies. We need to balance being unbiased and impartial with respecting the experience and background of the candidate. It is relatively easy to do in one-off situations but very hard to do at scale.
The thing to do is to remove proxies for competence and intermediary parties. Talk directly to the candidate, about the actual work (don't try to "see how the handle" odd questions).
However, this requires decentralization of decision making which reduces control.
I think there's an inherent collective bias against making interviewing and hiring easier because that would make the hiring market more liquid, meaning their best people will be able to more easily find jobs elsewhere.
Individual Germans may not have thought of themselves as particularly bigoted against Jews, but that didn't stop them from participating in regimes that marginalized them.
Most of the people fighting against slavery owned slaves. Jim Crow persisted well into the sixties and it's probable that many of the people against slavery would have been for Jim Crow.
It's very possible for the hiring market to be biased against liquidity even when everyone wishes it were easier to hire.
I replied to you elsewhere but I did just remember that Google / Apple poaching deal. That isn't even a bias, but outright manipulation at a massive scale that hurt the market more than any bias probably could. I am just wondering where in the hierarchy things like this go vs. subtler market forces you are theorizing.
Well, things evolve over time. Market manipulation would be slavery, while bias is just Jim Crow. The whole world is operating under a biased, ugly regime, it just gets better little by little over time.
The market is made up of individuals acting in their own self interest. Why would any individual not make hiring easier if they thought they could gain from it?
Note that Jim Crow/Davis Bacon/etc was a legal regime, a "must discriminate" set of laws to prevent people from acting in their own self interest. What such mechanism exists today preventing me from acting in my own interest?
Speaking as a business owner actively hiring... the cost of a mishire is huge. It is difficult to understate mishire costs. This means you have to simply spend a lot of time with candidates. Even then you have a narrow window of time, relative to how long the business and employee have to live with one another, to figure out if it is a match. This means you will say no to a lot of people after investing significant time and effort into them (and them into your business). We owe it to our teams to hire well. If that means making it harder, so be it. Think about if it is easier for the wrong people to slip in under the radar. It can wreck culture and team quick. Most teams and businesses dont spend enough time. In Work Rules (Bock) they found the first four long interviews they did helped a lot at Google. After that interviews added one percent accuracy in judging a hire. It is a hard problem and I think the first and foremost goal in any hiring process that is well developed aims to make good hires the #1 priority.
I won't discount your theory of a bias, but I think it is a small influencer if it exists.
In my experience, hiring is the most political act a company does, meaning individual goals very often take priority over company goals.
Well above avoiding mishires, loyalty concerns are paramount in every organization I've worked for. You'd think that they'd treat hiring as the quintessential bet-the-company decision, and operate accordingly, but this simply doesn't happen. Leadership creates policies and procedures designed to introduce opacity in the process with the ostensible goal of avoiding mishires, but the real goal of preserving power.
Agree. The best thing a business can do is standardize their hiring practices to weed out intentional and unintentional bias and politics. It is hard, though, but possible. Guess that is why I love working in a small business. We keep it transparent and real. We just want to hack things and work with good people.
Absolutely agree and I'll add that you're shopping in a market for lemons. There are multiple adverse selection criteria against the pool of job seekers. (Excellent engineers are more likely to be retained/less likely to be fired. Excellent engineers are more likely to receive an offer when they do interview. Poor engineers are less likely to be retained, more likely to be fired, less likely to be hired.)
In steady state, the applicant pool is of lower overall average quality than the pool of employed engineers.
There was (who I feel was) a mishire at my job. I haven't seen him produce a single work-product since he started 6 months ago. I get the sense that he's worth more to the company (a large tech conglomerate) as a write-off than he is a producer to the team.
We recently hired an ops guy into a dev role and the guy just does not know what he's doing. Merging his own pull requests before code review or before tests have passed the build system. Downs development/staging servers because he's "testing something" and forgets to bring them back up.
Our place has a 12-month contract before being made perm. Once you are perm you are not getting let go unless it's part of larger cutbacks (government). He was made perm before 12 months and I was astounded. They didn't even ask me what I thought before they did it.
1. the project slips while you expect your new hire to work, and they don't deliver.
2. opportunity cost of actually hiring someone good - by the time you've realized you've made a mistake, a good person is off the market.
3. you actually have to pay a bad hire for their time. you can't just not pay them. that's cold hard cash out the door.
4. everyone on the team starts wondering, how the hell did this guy make it through this process? are we being run by idiots?
5. if the hire is remote in another state, you have to register your business with their tax authorities, and deal with that whole payroll rigamarole.
6. also health insurance bullshit.
7. setting up accounts, changing passwords/keys when they get canned.
8. actually firing them is not fun unless you are a total sociopath, or they are actively causing damage by destroying value (which is extremely rare -- most people will just silently not do a fucking thing for weeks on end).
9. worrying about a lawsuit because ironically, people who cause damage like to threaten these things.
this is on top of your already full workload of doing actual, productive things for actual paying customers and good employees.
It's virtually impossible for most companies to manage someone out in less than six months. They're earning full freight salary for each of those six months. Many of your overhead costs like e.g. recruiting fees, new machinery, training, come due within that period and are not refundable.
This only costs actual cash outlay, not opportunity cost of other team members (could be coding but are instead spinning up the new engineer), overhead from management salaries, cost to redo work of an other-than-productive employee, etc.
At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
"Managing someone out" is a large company concept. Once they realize the mistake, the owner of a small company can get rid of somebody immediately. It can take surprisingly long to realize there's been a hiring mistake, though.
>>At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
Sorry, WHAT?
>>pportunity cost of other team members (could be coding but are instead spinning up the new engineer)
Hey I've got news for you. Developers don't just waltz in and start coding from day 1, they need to be brought up to speed on the systems. Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
> Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
You're being unnecessarily defensive. Just because people are people does not change the fact that it costs companies money to onboard them. This whole subthread is "why is it expensive to hire people?" and one part of the answer is "because you have to pay them money while you train them, and pay their colleagues while they do it". It's not a reflection of any unrealistic expectations--it's the opposite.
> Sorry, WHAT?
Computer, desk, office space: $5k
Job ads: $500-3k per hire is not unrealistic
Time spent sifting through applications: Easily 5 hours per week. If the posting is up for a month, that's 20 hours. That translates to about $1.5k in fully loaded costs for an employee with a $100k salary.
Time spent conducting phone screens: Assume 50 phones screens for one position. 30 min per person (20 minute conversation, 10 minute notes/break). 25 hours. Approximately $1.8k for the same $100k employee.
Further interview rounds: Assume about 10 candidates per position. Assume at least 4 person-hours per candidate, plus at least 8 hours devising evaluations (programming tests or what-not). 48 hours = ~$3.6k. Add in $1k of travel costs per candidate = $10k.
Draw up an employment offer. Assume you've got a stock one, but the new hire would like to make an amendment. Run it by a lawyer. $500.
We're at about $25k. It's not $40k, but it's not free, either. It would be easy to spend more time on any of those steps. If you use a recruiter, it will be higher. If you calculate based on SV salaries and costs, it will be higher.
The grandparent post was pointing out that once you've decided to hire someone for one day they'll probably be around for at least 6 months. That's where the majority of the cost comes from.
If that's the figure, then maybe bad hires really aren't that bad. Presumably the company won't have gotten zero or negative value from a bad hire, and even if it was $0, $25k isn't going to make or break any but the scrappiest bootstrapper.
We are a small information security consulting boutique. We are interviewing and hiring more than one person. For a small team making good hires is a big deal (most teams are small, even if the organization is large).
I don't know how else to do it, especially when it is really rare for someone to have the skills we need out of the gate. That means testing (we have a testing process) and time talking with candidates. On the other side, streamlining things, we have made hiring missteps in the past, which had measurable and high cost to the business. For us at least, it isn't hot air. Hiring is a hard problem. We spent a lot of time trying to optimize it to find an ideal way that costs less time.
Consider this: every hour a consultant is interviewing they arent doing research or billable work. So we believe strongly enough in how we do things to put our money where our practices are. Maybe the economics for the Googles of the world are different, but for us it is a real decision to be made.
The cost of a mishire is huge and obvious, but don't discount the cost of passing up on a good hire, which is much less visible. But every time a good candidate is passed on, the odds of a mishire go up. There isn't any way to determine with certainty whether a hire will work out or not before they take the job.
Also, take care of the good hires you already have. Everybody takes their people for granted.
I don't think I've ever heard a manager even say the phrase 'turnover' until they have a few ex-employees that they really couldn't afford to have lost, and a bunch of disgruntled people still there. Once the horse has left it's too late to close the barn.
> Use homework assignments to assess the candidate’s engineering skills
while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound. secondly, they are much more difficult to setup and evaluate: do you provide a spec? code to modify? third, asking people to do work for free in order to explore the potential relationship is a big ask. lastly, they are very time-consuming for the company.
in grad school i learned the hard way to always ask for in-class exams instead of take-homes :-). time and scope constraints can be favorable both for the company and the candidate.
> Enough with the graph traversal problems.
the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill. it is that they are a way to gain information about the candidate's problem-solving ability in a limited amount of time. unavoidably, these questions test for speed and familiarity along with problem-solving. that makes them a lot less than perfect, but i've generally found them valuable.
The issue with tricky algorithms questions is that they often become tests of how well you can keep your cool. I've (and I'm sure others have also) bombed relatively easy questions due to nervousness compounding. I've also done much better than expected on other interviews because I remained calm and kept the interview flowing smoothly. This is the real skill under test, IMO. Of course, your ability to solve problems is still important, and a subset of what's being tested. The issue is just that there's a lot of (possibly inherently random) factors that cloud that signal.
My favorite example of this is when an interviewer asked me about an algorithm. In the course of the problem it occurred to me, what I needed was to break down the input integer into two parts--two equal parts that could be multiplied together. I kept referring to these two parts as such.
After the interview he said I was clearly weak on math because I didn't know the definition of a square root. Something I did know (10 or so college math courses in linear algebra, calc etc) and was describing, but was at a loss for words.
I find this point interesting, as to me, this would be an argument for, not against. Time constraints on homework assignments are a lot closer to the constraints you'd have programming for a job than "whiteboard coding" would be. I'd hazard a guess that doing a good job on an in-depth, open-ended problem given a few days to a week is a better indicator of ability to be a successful software engineer than being able to solve a logic puzzle on a whiteboard in fifteen minutes.
If the interview time is paid, sure. But if it's not, agreeing to do unbounded unpaid work for a chance of being hired for a paid gig, especially when it's not even clear the chance is particularly strongly correlated to effort put in, gets worse as a proposition the longer you spend on it.
I was speaking more from the side of the company; from the side of the interviewee, it's less clear-cut; if you're the type that doesn't tend to do well in interviews, it might be preferable, but if you don't feel like you'll do a better job on a homework-like problem, then it's definitely worse.
By design. In the candidate pool, there will be some who see the odds as too steep. For others the cost of completing assignments is lower, making the proposition better. Not good, but comparatively better (less bad) than for the former kind. And that's how we know whom to hire.
i agree that it seems easier to simulate working conditions with a homework assignment. (it does add a lot of work for the company to do homework problems well, but let's leave that aside)
the downside is that the candidate has a limited amount of time to talk to you. they often have a full-time job going on top of the job hunt. secondly, are you really game to spend 0.5-2 days of unpaid work just for the chance of being invited to interview further?
Also mentioned in sibling comment, but I was thinking more from the perspective of a company trying to accurately gauge a candidate's caliber. From the side of an interviewee, it definitely isn't a clear win; the only case where I think a candidate would prefer this would if they tend not to perform well in interviews (which I don't think is necessarily a good indicator of whether or a not a candidate would perform well for a job).
An additional perspective I didn't think of until now is whether this type of recruitment process would be more beneficial to the industry as a whole. An argument could be made that we'd all be better off as a whole if interviews tended to more accurately assess candidates and candidates were able to get a more realistic sense of the type of work they'd be doing for a company.
Also it doesn't scale for the candidate. If you need 0.5-2 days of unpaid work for _each company_ and you're applying at 20 companies, you're spending more time applying for jobs than you would be working 10-40 hours of unpaid work just to get to the next step.
>>while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound.
Um, this is a good thing, not a bad thing. Research shows that stress significantly reduces cognitive ability, and time pressure definitely adds to stress - sometimes substantially. What this means is that when you ask whiteboard coding questions during the interview, you're not measuring the candidate's on-the-job ability. You're simply measuring how well they perform under severe pressure. If that's fine with you, as in it's representative of your work environment where people have to churn out code around the clock, fine. Otherwise, homework questions are much better.
i agree with all that -- again, the i don't think that whiteboard interviews are perfect.
but also, keep in mind that the candidate has a limited amount of time, and often a full-time job on top of the job hunt. the homework questions are only good if the candidate has time and desire to them.
Also the stress of someone standing over your shoulder watching you work. At work, I enjoy hashing out problems with a coworkers at a whiteboard, but in an interview it's paralyzing to have someone watching me. As a student, I'd totally freeze up in the middle of a test whenever a teacher started walking down my aisle.
They can absolutely be timebound, unique URL time stamped at the download, with however many number of hours from then to send in whatever you have, completed or not.
I've only experienced this once and it was stressful as could be, but was from a person I absolutely respected and still do. No-bs, either right or wrong and plenty to discuss in the followup conversation.
Did it from home, pulled the file at around 11pm right when I knew my best time to work was. Would happily do as part of an interview any time.
> asking people to do work for free in order to explore the potential relationship is a big ask
Asking to work somewhere without demonstrating that you can do the work is asking a lot. Unfortunately, job history alone is not enough to weed out people that cannot program (let alone well.)
> they are very time-consuming for the company.
Not if you implement a simple test spec or scoring system. We're engineers!
> the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
So don't ask the generic questions. Ask someone to implement something that they would need to do on the job. Obviously don't ask them to implement a feature that takes 12 hours, but a basic screen that is a snapshot of 45 minutes of on-the-job experience isn't crazy.
We also do a take-home exercise first to see how you think without us staring at you. Day-to-day at Seneca Systems we aren't staring at you, so it feels more relevant.
> Asking to work somewhere without demonstrating that you can do the work is asking a lot.
Which other professions have this attitude? I've either worked or supported a pretty wide variety of other jobs, and I can't off the top of my head think of a single one that does.
Even if some examples are brought up (and I'm genuinely interested in what they'd be), it's far from the usual approach in most hiring situations.
Huh? Your answer surprises me, as I can think of tons of examples. Designers are often asked to provide a portfolio to show the work they are capable of. Same thing for photographers. In a previous life I was a ballet dancer, and the whole purpose of auditions is to demonstrate that you are a competent dancer.
I generally prefer an existing portfolio of code (e.g. github repos) to see the kind of code someone produces, but if you aren't able to show some fairly extensive examples of your work, homework is usually a bunch better assessment of your skills than what can be accomplished in a ~45 min interview.
Fair. I should've thought of designers and photographers. Ballet makes sense, since there are trials for prestigious orchestras and the like, but that isn't something I knew about.
When I wrote my comment, I was thinking of the medical profession, dentistry, auto & other mechanics, appliance and other device repair, construction ("there's a pile of lumber in the back, you have an hour to build a shed and then we'll consider hiring you"), plumbing, civil engineering ...
Most of those other professions you listed have much more meaningful and standardized certifications than software. A board certified cardiologist has already demonstrated real proficiency by going through the process to get certified. In software, the whole reason for these complicated hiring processes is that no one really trusts software certifications, and even company pedigree is not always a reliable signal.
Continuing: Imagine a guitarist wanted to join your band, but they refuse to let you hear them play until they're hired, saying they won't work for free.
Why is your company so special that I'm the one doing all the proving?
There are a lot of clowns out there, on both sides of the table. At this point I've had my quota of companies that sounded good in the interview process but ended up being like a bad second date. Full of chaos and not anything I needed in my life.
One place drilled me on testing, and I mistook that for caring about tests. Nope. It turned out that not only did they not have any test automation, but their code base was such a ball of mud that you couldn't have many of them if you wanted to. I had absolutely no interest in pushing for a renewal on that contract.
That's not my experience, being on both sides of it. (devops kind of tests) Given no bounds, you will not be able to produce a 100% solution from the recruiter side of view. Something will always look weird. There's always another approach, generic parts possible to extract, places that can be left available for future expansion, etc. I'd go as far as, if you can finish the test in 2h, then it doesn't matter if you spend 4h or 8h on it, because you'll just do more of the same in most cases.
For example you can document one module and say "the rest of the docs are TODO due to time" - it shows you know how to document and you can move to another bit. My favourite submissions did 5% of many things to show they are aware of those areas, rather than super-polished trivial solutions of just the basic question.
On the contrary, I'd say that having unbounded time tells the interviewer a lot about how you go about building things. Do you know how to build something that works and stop at an appropriate level of abstraction, documentation, and testing for the type of problem at hand, or do you go off into the weeds and build FizzBuzzEnterpriseEdition (https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...)?
>the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
Actually, my codebase has an honest to god graph traversal in it, on the front-end no less. Basic manipulation of graph relationships without much friction should be standard.
From what I've seen the best hiring process seems to be Automattic's. They do a short interview over chat then give them a project if they pass the interview where they pay them $25/hr and then evaluate at the end. Minimal time wasted for both parties.
My current company used a "homework assignment" and they bought a lot of my trust with it.
1. The "assignment" took me about 4 hours, which they expected and told me off the bat.
2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.
Additionally, most of us are more willing to do interview work in our off time. Taking a vacation day to interview is a direct financial investment. Spending a Saturday doing a 4 hour home work is a sacrifice, but doesn't require taking vacation time. Taking an early afternoon off to go talk with 3-4 engineers for 1-2 hours is something that can much more easily fit into ones schedule.
EDIT: I want to provide an anecdote around this. If your salary is $150k, it may cost you roughly $400 in gross pay to interview somewhere. Sure, you could take paid vacation, but vacation time is paid out in cash when you leave so it still costs you. Companies with unlimited vacation are a different story.
At my previous company, I was the hiring manager and gave a homework assignment that was targeted a little shorter than you describe, at right about 2-3 hours. I was clear upfront that it would take about that long, though if you wanted to take longer, you could. Within reason, I didn't hold the time against the person if they took longer.
The crazy thing was that over half of the applicants that passed a resume screen and phone interview could not complete the homework assignment. Over half. Some couldn't get it at all (either gave up or had compile errors), some got it to run but it didn't work, a few got it to mostly work but their programs had serious bugs, and the last few actually got it right.
The assignment was to read a file containing a list of numbers (some formatted incorrectly, so there was some very simple parsing logic involved), call an API using each correctly formatted number as a parameter, and store what the API returned to a file.
I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
Also, credentialling. Even if you can't do the work and only stay a few months, Google or Goldman or another big name on your CV will still open many doors.
In Germany there is a culture of not changing jobs very often (at most every few years, but even this should not happen too often). So if you have a "well-known" name such as Google or Goldman Sachs on your CV, but worked there only for few months, it is a clear sign that you were fired and about every future employer will be very cautious.
1. What is most important in your CV is the last employer. If it seems that you were fired, this can cause problems with getting a (decent) job afterwards. If you were poached, it is because the other employer wants you badly. So in your example your new "last" employer becomes JP Morgan or Facebook. So this kind of problem does not occur here.
This does not contradict the fact that it still leaves/can leave a bad impression on impression if your were willing to be poached without a very good reason, since can be considered as a strong sign that your are not loyal to your employer and only hunting for money.
2. Poaching is rather uncommon in Germany. As I outlined in 1 being willing to be poached multiple times is a sign that you are not very loyal to your employer. Thus there is a strongly reduced incentive for employees to be poached.
Also for poaching to work, you have to offer lots of money - I'd be really surprised if there are lots of employers that are willing to start such an "arms race with money". Why not simply offer decent money in your regular job advertisements and get good people without any arms race?
Poaching might make sense if there you want to get "rockstars" that are not available on the "free job market", but can only be poached. But I'm also not aware of any industry sector in Germany where there exist employees that are treated as rockstars. :-(
Your vision is clearly limited to Germany, where poaching doesn't exist because there is no jobs and no strong companies competing.
Just imagine a random company in a tech hub. They host a meetup once... the days after the meetup, most of their employees have been contacted by recruiters and other companies who attended.
If they want anyone in particular and/or if any employee's salary is not on-par with their value, here they go! :D
Outside the Valley and startup culture most companies looking for software developers in the US view a tenure of less than two years as being short, and a series of them indicates "job hopping."
I'm not commenting on whether that's a good or bad thing, because good or bad it's simply the way it is.
This. If people didn't lie (or stretch the truth) or cheat, we would only need resumes to know whether someone can do the technical aspects of any job.
This has happened at my company. We give a similarly near-trivial hw problem, and sometimes people come in and literally "cannot remember" any single thing about the assignment, including the names of the columns in the data or the approximate objective of the HW. This is after performing plausibly well on a phone screen etc.
From my recruiting experience people cheat even during live-coding interview. I was pretty amused seeng a person whispering to a buddy outside of the video pickup area trying to get _any_ code written.
> I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
I once asked a candidate to write a for loop in C#[0], using my laptop and Visual Studio and watched for a solid 5 minutes while he struggled -- And I don't mean jitters around syntax, I mean, failing to declare a variable properly and while he knew what a for loop was, he had no idea how to write one in C#. His resume touted a series of projects that we estimated put him a little above a Junior developer and the only language he referenced was C#. I don't normally do these kinds of "on the spot tests" because I, personally, hate them, but I was cheering when I read Coding Horror's "Fizz Buzz" article[1] on the subject.
This was the most foul example and my only experience is with developer and ops/security positions, but if you're getting half of developers who are in the same time zone as the title[2], you're doing well. Does this happen in other parts of the business world? Some of the folks we get in would be like a Walmart cashier applying to run a finance department because "they've handled money before". And what if they got the job, do they think they'd come close to being successful? I'm all for the "fake it until you make it" and I'm very willing to overlook weaknesses around experience/education if the candidate is really passionate and I can get enough of a comfort level that they'll be a quick study and love the job, but you have to be realistic about your deficiencies.
A lot of the problem, I've found (being on both sides of this in the past) is with recruiters, as well. They tend to operate on a "throw darts at the wall, blindfolded" approach, with the assumption that if they send out a million candidates for a million jobs without regard for qualifications, that they'll get lucky, one will land somewhere and they'll get a commission from sheer volume on bad odds. This serves nobody well -- it causes companies to put up multi-step gates that involve people who aren't close enough to the field to discern a good candidate from a bad one via resume performing a filter operation that disproportionately disqualifies the really passionate but "not quite a perfect fit on the qualifications side" and it causes job seekers to take a bunch of interviews for positions they won't like/aren't qualified for and won't land -- causing them to have a miserable experience interviewing (and probably resulting in some dropping out of the job market entirely).
[0] The exercise was, specifically, write a for loop with an int value that is incremented by one, adds that int value to another int variable, and multiplies the result against the counter, assigning the value to a third variable. Print the result to the console. This was before "Fizz Buzz" became all the rage, I did my best to give a blindingly simple problem to just validate that the person could do something in the language.
[2] Never mind the details "Must have 4 years of experience in Angular", nonsense -- a topic for another post, but the way we write job requirements is about stupid. I mean, I've seen experience requirements that exceed the length of time the technology was in existence, and I'd rather see a candidate demonstrate an ability to adapt and learn new things than be overly concerned with whether or not they've used a framework that is really similar to a bunch of other frameworks.
Hiring at the company worked like this, provided the resume wasn't passed directly from a member of staff close to that team:
(1) Applicant applied through web/recruiter/e-mail. (2) Resume was reviewed by a generic hiring team who covered every kind of job the company had and none of whom had even the most basic understanding of what a developer's qualifications should be[0]. (3) This same team does a myriad of phone screens to further whittle the pile. (4) Team's manager reviews remainder and often didn't perform a second screen since one was already done[1]. (5) Candidate arrives.
[0] These same people provided many of the "templates" used for the job descriptions, so you can imagine these were similarly bad. This practice resulted in the company, for years, refusing all resumes where the candidate did not complete a 4-year degree program. I pushed for an end to that for dev jobs after two candidates I hired in who had limited/no college experience turned out to be excellent employees.
[1] And in some cases, the manager knew nothing about software development and was put there from some part of the business, so s/he'd be about as effective at a phone screen as the hiring team.
Bad jobs are constantly in need of filling: people quit very frequently.
Bad employees are constantly in need of placement: people get fired very frequently.
If you're hearing about a particular job or candidate, the odds are they are bad. Most jobs are filled internally or by referral. IIRC, it's only about 20% of jobs that are filled by recruiters and/or cold job applications.
You're right about bad employees, but I've not found it difficult to weed those candidates out. At the same time, I don't fault a candidate for being laid off provided they have a good explanation[0] and/or they're otherwise a good fit.
As for bad jobs, even bad jobs need someone who is functionally capable of doing them. Sure, you have to be vigilant about filling those positions, but it's rare to encounter a "bad job" in this field that would be fine to have a terrible candidate in.
Regarding jobs being from internal candidates: It's interesting, because I hear that statistic rather frequently and often wonder where it originates or whether or not it has the meaning that is often attributed to it. Again, pulling from my anecdotal experience, though my resume was passed in from an internal employee for the two jobs I was most interested in[1], in one case my resume was handed to a sales person who reached out to me with a question about the company I had just been laid off from -- I had never talked to him or met him before and he lived about 100 miles away and in the other case, my sister handed my resume off to a friend of hers who worked there[2]. This friend and I met for the first time on my second day of work there. I don't even know if the guy is a software developer or not -- and that's the crux of the issue. If the recommendation is coming from someone doing the same/similar work on the same/similar team, that candidate is probably a good fit. If it's from someone doing wildly different work or who has no idea what a good candidate would look like, it's not much better than the blind solicitation from a recruiter. This is further complicated by the fact that many/most companies grant a referral bonus, incentivizing staff to solicit and (possibly) lowering the quality of candidates coming through that channel[3]. I'm really curious about others' thoughts on this, particularly those who've hired for their own companies.
The last issue with the 20% rule is that in companies that are dedicated to filling a diversity standard of some kind, it's hard to get there from internal employees unless you already have a very diverse staff. Reality being non-ideal, you tend to hang out with folks who are similar to you[4].
[0] There are plenty of reasons people get laid off/lose a job that have nothing to do with the competency of the employee. Companies go completely out of business, shedding all jobs or merge with companies where the whole department gets eliminated. Less "great" excuses are role elimination, though, this happened directly to me in January but I was offered a position for every job I interviewed for, so given appropriate context it can be fine as well.
[1] I interviewed at four places, all four offered me a position, two of the places were the highest on my list and I took one of my top choices. The two that were high on my list technically had my resume passed in from someone who already worked there.
[2] That it was my sister certainly helped -- she's been the CFO of a very well known global company and is currently the CEO of a mid-sized company -- her recommendation is trustworthy and people who know her know that she wouldn't embellish anything just because of family relation.
[3] I'm on the fence about this, though, personally. On the one hand, I'm less likely to recommend anything but the absolute best candidate for a role at company I work for if there's not a financial incentive -- it's nearly all downside otherwise. If the candidate works out, it's unlikely anyone is going to remember you sent that resume in. If the candidate is a mess, you get a ding on your record for recommending a fool and a subconscious ding to your own reputation that you associate yourself with fools (unfair as that is, I've seen it). If there's a financial incentive, there's always the "...
Okay. Now you're sitting with an interviewer to talk about your home asssigment. Turns out there were some changes in a system and you program needs to evolve accordingly: actual format is protobuff struct and API is asynchronous — you send a bunch of tasks, server queues them up and gives you ids, after some time, results are pushed to a client as they come up. You must store data in SQLite and account for HTTP or REST errors (retry logic, etc.).
Your one liner is neat and all, but now you have to do 2-3 hours of initial work + w/ever time it takes for adjustments in 2 hours scheduled for an interview.
You could've thought out this toy architecture up front with initial simple implementations or at least discuss what they expect you to provide, but instead you wrote a shell script for some nefarious reason, even though you saw obvious disconnect between estimated time and time you actually spent.
Nah, i'd hire the guy with the 1 liner over the guy with the robust architecture any day. A robust architecture is great and all, but I think it should be built after a need for it has been proven.
When I was a junior developer, I was just trying to get code to work. When I learned design patterns, and had some bad experiences, then I started architect "robust" architectures for everything. After perhaps 6 or 7 years of doing this, I started to question my insanity. Sure there were several instances where I was like "Damn, really glad I built it like that" but there were an equal amount where it just ended up turning a simple problem into a complex problem. There's no good excuse for that.
So now, I go high school simple first, and college hard later. If it means I have to do some rework, fine. I'm okay with a little rework, because in the end it aggregates to a simpler platform that took less time to build.
This is ridiculous. There are so many changes in requirements there (asynchronous with push results being the biggest), that everyone is going to have to throw away what they wrote an start over.
Anything beyond 10-20 lines for the solution to this problem is pointless levels of abstraction and would be a huge red flag.
Sure, but imagine all you knew was C++, which is true for many people. Several hundred lines of code right there. I'm a hiring manager, given questions like this and seen that exact result. Remember that many people cannot solve fizzbuzz.
Then your choice is whether you want to hire people who only know C++. For some jobs, you don't. But if you do, you need to have multiple tests and choose one based on the candidate's skills.
We have an in office test--the dev sits down at a laptop with internet access, gets clear instructions and scaffolding code. We know that the tasks take n minutes because we see the time that candidates take.
Often the assignments are more complex than described. I wouldn't be at all surprised if this was the case here. Also 2 hours is actually a really short amount of time, it can take 30 minutes or more just to understand what the person is looking for if the instructions are not clear.
The last two Homework assignments I got took about 50% longer than the hiring person suggested they would take.
I'm perfectly competent at writing complex code. In fact I'm usually the guy who fixes bugs or perf issues that nobody else believes exist.
A problem you came up with on your own is going to seem simpler to you than to everyone else. Why? Because you thought you picked it at random but your subconscious picked the one it thought was good. In other words, you picked the one you were already primed to answer, and it seemed simple.
So either people are going to work at it longer, the timid will give up (there's nothing wrong with timid coworkers as long as they have someone to listen to them), and the brash will get it done in the time allotment by cutting corners.
Are those the people you were trying to hire? Because that's who you're gonna get.
I'm sorry but the problem is a trivial one. Even if you've never read a file, never made a http call in code or written to a file, these are all things a 5 minute stack overflow away.
I absolutely would not hire someone who failed that test. Your defence of it seems bizarre to me.
chillydawg, I take hinkley's assertion to be more of a warning to be careful about biases that may have inadvertently been introduced through the testing methodology. I think it's impossible for us to know without seeing the test you use, but it's always possible that the requirements aren't as clear as you think they are, or potentially what you deem as trivial may be biased from recent direct experience on a problem that you've then turned into a test.
I think these tests are a great tool, but like anything we all just need to be aware and careful of the biases introduced. Also, we need to be mindful that really good developers who are employed, have families etc, may not have an over abundance of time to spend on these, so if 10 companies each send a challenge that takes 4 hours, and it's unclear if the assessment will lead to further interviews, some better candidates may give up on the process or be selective about the ones they complete.
I think this creates an additional bias, that the people who complete your assessments may be the ones with the most time to invest in the process (ie the currently unemployed).
At my current job, we have an "onsite" challenge that we used to hire some Cuban devs. We went to Cuba (from Mexico) and gave the candidates a codebase for Tetris in an interpreted language they did not know. They have 2 hours to transform that Tetris into a Nibbles (snake) game.
The catch? There is no internet in Cuba, so they had to do it using a book, and asking questions to a programmer who was helping them there.
I think this is a good point. There's a real risk that a homework you think is simple is a little more complicated because you're making assumptions. I think it's good to have a few current team members actually code it up (not just whiteboard it), and also to talk with a few candidates to see how long it took them, to calibrate this.
It's also better to aim small (a la FizzBuzz) and walk up the complexity slowly, as opposed to starting with "Solve this ticket from our prod JIRA queue, with tests please" (which I have actually been given as a test!)
I like to aim for a very simple task, like proposed above, with maybe one non-obvious part or edge case (which the spec mentions, so it's not a trick question). Just enough to demonstrate a) clean coding for the simple parts, and b) ability to solve something without using a completely naive solution.
You'd be surprised how many people stop looking at the perf numbers once all the tall tent poles are gone. They have no idea or tools for digging beyond that. I've worked on several projects where the customer still wanted the app to be 2-10 times faster (or worse, a competitor was 2-3 times faster) and people look at the charts and announce that they've done everything that can be done. Some hide in plain sight but others are just putting in work. For instance code duplication can hide a tall tent pole because it's chopped into five pieces.
I had a shared library running on a bunch of systems. One was VxWorks. Cross compiled. I knew nothing about VxWorks except that the code needed to run about 8x faster. I got about 2.5x out of my code by improving locality of reference problems that appeared to be intrinsic but just required experience to see. For the rest I filed a bug report to WindRiver and five to the cross compiler writer. At the end of this process I knew more about the VxWorks administration than anybody else in the group.
People give up, even when there's a clear business case for continuing.
I'm going to assume "couldn't accomplish the task" means couldn't get it entirely correct. Do you provide a testing baseline that determine the expected behaviour? With TDD being the norm and correct way to test your program behaviour, I'm absolutely not going to go through every edge case with a fine tooth comb to check your "correctly formatted number" calls the API as you expect for a homework assignment. Provide a testing baseline or clearly demarcate the various possible edge cases as would be discussed in real life.
What language was this in, out of curiousity? That timeframe seems very long for the task, but only because I tend to work with languages which have nice standard libraries.
Just wanted to say that I really like the idea of testing the "homework assignment" on your own team to hone it.
I had a similar experience a month ago. It was a similar "4 hour project" but it was open ended with "make it as complicated or as simple as you want". Being a single table backed CRUD app, you could only put in so many capabilities but it got me excited thinking about ways I could surprise the staff. In the end, I made it self-hosted (in a language that self-hosted is less common), included an installer, and made it support multiple connections from different devices with updates that pushed to the connected clients. The project took me about a day and a half (with two of those hours spent yelling at WIX), but the core parts took about two hours. I was genuinely surprised at how much fun I had putting it together. They also asked me to estimate my time spent -- since I had contract work I was doing at the time and couldn't dedicate a long stretch all at once to it -- I broke it down into its individual tasks and documented the whole thing. They had no problem with the fact that I delivered it a week after the interview.
I also second your "I can spare a half a day for a prospect" attitude. There's a flip-side benefit to that, as well. A candidate who feels over burdened with a simple assignment (provided it's tested as you mentioned and truly is simple) is probably not one I'd be interested in hiring, anyway. I'm not saying it's "one size fits all" -- I've had candidates walk in the door that I knew in advance were really good fits who were hired after a 30-minute screen. That, though, is a unicorn sighting.
What if the HW assignment became the norm. Could you spend half a day, or a whole day to 10 jobs? What if after you spend all that time you still don't get any offers, or even interviews? Job searches are already hard enough (for most people) without these huge time sinks becoming the norm.
That would be fair. But having that sort of expected result is impossible for many people. Of course, some people are good enough and sell themselves well enough that they'll never have problems passing such screens and getting offers. But for those that aren't, and worse don't know they aren't, will be able to reach that level of expected return without the process being designed not to exploit those situations.
My current company does something similar. I think back when I had to do it, it took 2-3 hours. It's a pretty basic problem using strategies and observers. The next time we're hiring I'll probably update it.
The kicker though, like someone else below mentioned, is that we would receive submissions that didn't even compile.
What I liked about the HW problem is that even something simple gives you and the candidate something to talk about. Why did you do this or that. Tabs or spaces ;) etc...
Requiring a take-home assignment, not just offering it as an alternative to an in-person interview phase, is a great way to systematically reduce the diversity of your hiring pool. It works well for some, but please don't require it of everyone.
I am confused by this, could you help me to understand why this is so?
In particular, it seems to me that take-home assignments are strictly better for candidate scheduling etc than in-person interviews, if time spent can be held equal. Maybe time spent is just not equal very often?
Yes, absolutely. I say this every time this homework assignment shit pops up. It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost should give everyone pause. It's plainly exploitative.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400 hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is to say, I should be the only one doing the assignment and there should be no wishy-washy rejections.
> It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost should give everyone pause. It's plainly exploitative.
There's nothing wrong with that, it's a competitive market, those who get hired are those willing to compete, not those who complain about unfairness about being asked to demonstrate their skills. Employers spend tons of time filtering candidates, even reviewing the submitted work takes time and effort, saying it's no cost to the employer whatsoever is not only untrue, it's naive.
> What is the employer putting up? Generally nothing.
Unless you've been in the shoes of a hiring manager, I'm unsure as to where this idea is coming from?
Having been a hiring manager, it is a horrible experience. Unless it's extremely naive, homework sets generally take time to come up with and formalize, and certainly take time to review the results for.
I suppose I can understand conflating the motivations of an interviewer with their employer, but it's not generally true to say that interviewers are just trying to exploit those they interview.
How many companies waive the coding requirement or postpone it towards the end, if you already have open sourced a bunch of code. It is also possible that the HR managers are just happy taking a route, where they can offload most work to the other party. The can instead act as better intermediaries, aligning the hiring goals of the company and the experience for the applicant. Whats wrong with asking them to create better value or understand the domain better so as to be able to create better value?
1) How many forks and stars do you have?
2) How often have you opensourced?
3) Have you blogged about them or created a documentation?
4) Why did they build each of those repos?
5) Commit logs... etc..
Very easy to develop such a framework.
One or two repos can be copied, but there is ample signature on a genuine open source contributors.
>homework sets generally take time to come up with and formalize, and certainly take time to review the results for
It's a one-off effort to come up with an assignment. So the cost per application can be extremely low. As far as evaluating them, the bad ones can usually be tossed out in minutes or less. Comparatively, the cost per applicant is basically nothing.
> So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing.
What?! Up until that point, typically, the employer had crafted the job posting, posted in on various channels, reviewed the responses, talked to promising candidates, and then give you the assignment. And a good assignment needs to be carefully crafted, a process that usually takes an order of magnitude time more than what it takes to prepare an answer.
An order of magnitude is not much though, that's only ten applicants? Most assignments waste the time of hundreds of applicants, so it sounds like its still a net win for the employer.
It feels to me like the root problem here isn't how many people are doing the homework problem or whatever. It's that this is increasing your time spent as a candidate relative to the in-person interview.
I think your last paragraph is very close, but I think the thing that matters is "no wishy-washy rejections". Not being the only person taking the test or how much time the company spends seem like ways to make that happen to me.
Maybe I'm just completely wrong here, but I think most of the objections to homework type tests are being caused by processes where the homework test was in addition to the in-person whiteboard interview, not instead of it.
How would you feel about a process that was: some preliminary screening -> nominal 4 hour homework thing -> in person meeting to confirm that you're not a complete asshole and negotiate salary?
Job seeking is a sales activity. You sell your labor on labor market. Every sales activity has a cost. If you do not want to bear this high cost of IT industry, you can work at McDonalds. Probably your sales cost would be much less but your profit also.
It is a sellers market. I get recruited every single day. Any company that disrespects me and wastes my time isn't going to be very successful at convincing me to work there.
Developers are the ones with the power in this market, and I have no problem throwing this power around.
That's a good way to get passed over for good jobs. As a potential candidate, you aren't in a position to making such demands and your unwillingness to demonstrate your skills on a simple assignment just means you won't be considered. Plenty of other fish in the sea who have no problem completing a simple assignment.
It depends how desperate the candidate is for a job. A coding assignment may have a hidden cost of filtering out many qualified candidates, if qualified candidates don't feel the need to apply to companies with these assignments. Anecdotally, these assignments seem like just another filter companies use, they don't replace any steps in the process, just one more hurdle to pass.
If you aren't willing to demonstrate your skills with a little coding assignment, I don't consider you a qualified candidate. I've used a coding test for years, always get exactly the good kind of people I want, it filters out the lazy and the liars, most programming candidates are liars and simply don't have the chops they claim.
>Plenty of other fish in the sea who have no problem completing a simple assignment.
And yet companies have an incredibly hard time hiring talent, even with so many other fish out there. Odd, that.
People need to start understanding their value as an employee. You create more value than you're worth; that's why companies need workers. You don't need to bend over for a potential employer. Plenty of other fish in the sea of employers, too.
It's not odd at all, those are old school companies playing by the old rules of looking for candidates locally. And demonstrating your skills isn't bending over, it's showing your value and it's how you get a job. Seriously, this notion that you should be hired for a high paying job without needing to audition for it is just absurd; I wouldn't hire anyone with your attitude.
It may depend on the industry, but in tech generally there's not plenty of other fish in the sea - there's a scarcity of qualified candidates but plenty of other potential employers who desperately want to hire.
That's really not true; there's only a scarcity if you insist on locality. There's no shortage of qualified remote workers when you open up your search the whole country or whole world. Employers crying about scarcity are employers who haven't figured out remote working is the future.
I get recruiting emails literally every day, and people call me on my cell phone, unprompted, once a week. So yes, I absolutely am in such a position to make such demands. If a company doesn't like it, then they can do me a favor by passing me over.
And I am honestly just an average SF engineer.
The current engineering market let's me choose whatever company I want.
So, I am not going to waste my valuable time with companies that disrespect me, when there are a hundred more that I could be talking to instead.
Great, but that's an artifact of living in the SF bubble which isn't anything like the rest of the planet. That is not the current engineering market, that's the current SF market. And asking you to demonstrate your skills isn't disrespecting you, you work in an industry that doesn't have a professional certification which means it's loaded with people trying to fake it for the big paycheck; due to that, it's perfectly reasonable for companies to ask for proof of skills. If that makes you feel disrespected, you aren't in touch which what it takes to run a business.
Don't care. I am the one with the power. And I have no problem using this power to my advantage.
The way that a business relationship works, is that the person with the power gets to make the decisions. And in the market that I am in, that person is me.
And I have no problem demonstrating my skills. I just demand that there is another person from the company, on the other end of the phone line or other end of the table, investing the same amount of time into the relationship as me.
If a company thinks that it is unreasonable for me to demand an equal time investment from them, well sucks to be that company, because as we have already established, they aren't the ones with the power, so their opinion doesn't really matter. They can go find someone else to interview.
Oh most happily will, even talking to you is a pain, I wouldn't work with you if you paid me. You're arrogant and entitled, you'd make an awful employee.
He is saying that the company needs to tell their people to fix the process and do what they are paid for. Not offload it entirely to the applicant.
Maybe just open separate channels for people who have an open source portfolio. I have about 30 repos in my OSS, but still have to take these assignments. Its such an utter waste of my time.
If you're applying for a job, you're asking for the company to pay you a salary and to believe you can provide labor worth that salary. You can find a few hours.
It does, if they're actually doing it in a way that's useful to them. At least one, probably two, engineers are going to evaluate what you send in, take notes, and then go over it with you. And they have to do that for every response.
So that's an absolute floor of an engineer-hour per candidate: Anything less than 30 minutes to read through it and take notes is useless, and figure another 30 minutes minimum for a call.
If they're not going over the responses in detail and then talking through them with the candidates, then they're just wasting their own time because they're not actually judging anything.
I agree its not a process of applying for college, its a job. Testing to see whether somehow is a recent BSCS helps to systemically impose all 3 of the most problematic isms, racism. sexism and ageism. You can get brilliant employees from bootcamps, my current coworker is a truly brilliant bootcamp guy. He's African American and decided to skip student loans and go direct for code bootcamp and for front end he's really good. But if you tested him over recent BSCS type of stuff you would be screening out a brilliant front end developer.
It never ceases to amaze me what situations people find a way to shoehorn -isms into.
It's a job. It has an application process predicated upon demonstration of some base level of aptitude. Fulfill it or fuck off and apply somewhere else.
I wouldn't hire a frontend-only guy instead of a qualified developer, for the same reason I wouldn't hire a guy who can only mount wall outlets instead of a qualified electrician. Everything he didn't care enough to learn is a risk.
If the homework assignment is at least somewhat related to the job (not CS esoterica, etc) then I can't really imagine a better way to measure the candidate's relevant programming skill, which is certainly not the only thing you're hiring for (if you're smart) but it's important and I don't really see how it reduces the "diversity" of the hiring pool in any undesirable way. Could you elaborate on your reasoning?
People have children or other life situations that make it harder and less desirable on average to give up weekends and evenings for a homework assignment. It optimizes for people more willing and able to use their off-hours for work, which is not an even distribution across the population or even necessarily a desirable trait in candidates for that matter.
Assuming the homework assignment is happening in the hiring phase after the initial screenings, I don't buy this. Getting to that point means it's time to show them why they should hire you, and it's going to take up your time whether you do it in person or "asynchronously". If they give you the opportunity to spend those couple of hours at home rather than at a whiteboard in their office, but you don't care enough to make the time, I don't know why they should care enough to hire you.
If the "homework assignments" are just a scheme to get work done for free, or a competent engineer's time is not set aside to review them, then that's a different story. But as a fair test of programming ability, administered in good faith, I don't see the problem, and such a test is important.
We really need to stop pushing this HW assignment nonsense. It's offloading the cost of hiring on to prospective candidates. The asymmetry of cost makes it plainly exploitative.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400+ hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is, I should be the only one doing the assignment and there should be no wishy-washy rejections.
To add on to this, a take-home assignment should be more than just another filter. The companies I interviewed for with take-home assignments didn't actually discuss the assignments at all in the in-person interview - I spent the time to prepare for and do the assignment, at least pretend it matters.
That just seems odd. How were they checking that you were the one who completed them?
All of the interview processes I've had recently have checked I did the exercise by either asking me to describe what I built with no aid, asking me to walk through the code with it in front of me, or asking me to conceptually expand on what I built on a whiteboard.
They gave me the standard whiteboard interviews on-site after that. This was for a data scientist position. I did not get offered the job (but I accepted a good job offer a couple of weeks later, so whatever).
What if a company offered to pay you a day's wages, or something reasonable still (if it's less than that)?
Whittle down the pool even further and the ones you think look good on paper, and after an interview (or a series of them), have them come in, on a paid basis.
It'd be a sign of good faith that they're both serious about finding a good candidate, and respectful of your time?
I know some companies do paid stuff like that. If only it were more commonplace.
That sounds like a fair process. Employers are in a position of power and can easily abuse it. When the employer also invests in me as a candidate it shows they're serious about me and also ethical. Sounds like a win-win.
I still wouldn't if I currently had a job. By the time I get home my brain is usually pretty fried and that 4 hours probably means giving up a couple of nights.
This reeks of entitlement. If you want the job, you'll find four hours. They don't owe you a job and they don't owe you anything for going through their hiring process that they have designed to find people to pay a salary.
I echo the grandparent's sentiments, myself, from my own personal experience as well.
So its entitlement to expect an employer to not exploit their position of power? Sure, I'll take that label.
>If you want the job, you'll find four hours.
But what happens when everyone starts having these 4 hour screens? Now its a 40 hour process to apply for 10 jobs in a week (and this is before any commitment on the part of the employer). It's not sustainable or fair. And yet, there's no incentive for employers not to do it. So we as job seekers just need to suck it up and be thankful for the opportunity, right?
And what's wrong with having a position of power? You say it like it's a dirty phrase.
Can you, personally, pay me a salary? If you can and I want to work with you, I absolutely think it's fair to ask four hours of my own time to demonstrate I'm worth consideration for it. If not, then why does your opinion matter?
You sound like a Libertarian. I'm assuming all the advances in worker's rights we've had over the last hundred years are also "entitlement"? If not, then I don't see what's different about expecting an employer to not exploit prospective applicants by offloading most of the cost to them.
I work at a small company in a remote part of an openly backward state when it comes to startups, and yet got two hundreds applicants to file a javascrip developer position.
After interviewing 10 (say two three hours spent on each between contacting, scheduling up etc) it was clear every unemployed with 'can use word' as skill boosted their resume and applied.
I'm sorry for the real developers looking for a job but a company can't dump 20 days work on chasing and weeding liers.
I'm not a fan of homeworks myself but this is what it has come to.
I'd be payng but that puts the company and the prospect in a very difficult position them towards tax laws and towards their current employer non competes, the company toward again tax laws and then there's the absurd of paying for contract work delivered that's 99% of times not up to spec.
Beside, the unwillingness to compromise smells itself and given the choice I'd never get a potential troublemaker in that only thinks time=money and screw everything else.
That little empathy for both sided of the problems sure is a red flag.
Edit: well unless it's a contracting job then it's fine.
This seems like something a 5 minute FizzBuzz-esque programming assignment as a pre-filter could easily fix. The goal seems to be a barrier that only serious applicants will even bother to attempt, and that's fair. But when that barrier then becomes a 4+ hour time sink for the serious applicants it becomes a problem. There are plenty of hackerrank style sites where someone can show a bare minimum of competency without costing either side much time.
yeah I think 4 hours is a bit too much, in the end you only need competency as a screening.
we used to give out this http://play.elevatorsaga.com/#challenge=3 and asking for solution to level 3 which can be done in a short time + is the first non trivial challenge.
of course solutions can be find in internet etc, but it's pretty easy to look them up if they smell fishy and it'd all come down crashing at the face-to-face anyway
it's also a very good exercise in real world problem solving and a source of endless technical discussion since there's no single good approach, especially with two elevators to schedule.
It's the power to get 30+ people to jump through hoops that only one person has a chance at being chosen. Even worse, a good 50-60% of those people have no chance out of the gate, they just don't know it. Encouraging people with little to no shot at the job to do significant work so make your job of filtering them out easier just reeks of exploitation. Come up with a filter that's minimal cost for those applying, fizzbuzz style problems seem suitable.
This phrasing may sound like entitlement, but try turning it around. Most of the people who will turn down a homework assignment are the ones who can afford to do so, because they already have a good job or know they can find one. Is that really the first filter you want on your candidate pipeline?
I have never seen a job posting I was sufficiently interested in to put in 4 hours with no investment from the company.
I have, and it's how I got the job I have now, which is by far the best job I've had.
Regardless, if you have a good job already, why apply to any position you're not particularly interested in and that you wouldn't spend four hours trying to get?
Because they contacted me and offered me to participate in the
recruitment process, and while I don't look for a job, I'm not against
checking where this can take us?
I think it is worth seeing what's out there even if you're happy. Maybe there's something better!
The standard process where the first two steps are some ordering of a 1-hour interview with someone technical and a chat with a recruiter is perfect for this. With minimal effort (maybe 90 minutes) you find out whether they're interested in you at all, and get to ask both a technical person and a recruiter about the job.
The big time investment typically comes later, and I can easily bail if I decide it isn't going to be a compelling offer. I've done this quite a bit.
I'm glad the homework process worked for you, but my point that it doesn't seem like a good idea for the company stands. If the first thing you do is filter based on how badly applicants need a job, you're losing a significant portion of the best people right off the bat.
I've never taken a job I've applied to myself. That is, they were the ones who approached me (some more directly than others[1])
Why would I spend any of my already very limited time if they didn't make sure the process is streamlined?
Also, I don't think it's uncommon to like what a company does and being interested without yet knowing if you're interested enough to put the time aside - most company sites and job descriptions are much too generic and vague, so I'd want to talk to some employees first (remember, interviews are two way conversations). So I might not know if I'm interested enough at the time of application, just that I might be.
[1] for my current job, the interview process was essentially them trying to persuade me to join them, and my previous job I was actively looking for a job and someone I know at the company said I should apply. My "application" consisted of telling him "ok set up an interview"
So now I'm doing a day and a half of unpaid work for them, taken out of evenings and time with the family.
Not hypothetical. I just described my last job hunt, 16 months ago.
The place I really wanted to go closed the req on me, and then opened it back up the day after I started at another place. My third choice (stars didn't align on the other 2 interviews, in part because of stuff like this).
I don't "want the job" per se, I'm interested, which is a bit different and we'll need the interview process to find out if I actually want the job.
Furthermore, I already have a job, just as most reasonable candidates do - so if you want to convince me that I should spend many hours on that, you'd better be sure that your offer has a really attractive first impression and seems likely to be better than the job I currently have, otherwise you'll be self-selecting your candidates only from the (generally less skilled) sub-population of workers for whom "If you want the job, you'll find four hours" is true because they actually are desperate for jobs, unlike most skilled people in tech industry.
It's a bit frustrating because different people react so strongly in both directions. I don't like live coding, but I'll be ecstatic with a pair programming session -- because I can evaluate them at the same time. Homework, I'm generally OK with, but I honestly hate it when you get a response like "We didn't like your code. Goodbye." On the other hand, I know some people who can't deal with the stress of writing code in an unfamiliar situation.
I wonder if the best thing to do is to simply offer the candidate a choice.
> It's offloading the cost of hiring on to prospective candidates.
Why is that a bad thing? The prospective candidates aren't entitled to anything, the burden should be on them to prove they have any capabilities for the job.
> What is the employer putting up? Generally nothing.
Totally incorrect, have you ever had to hire somebody? The initial phone screens alone to weed out the 99% of complete incompetents takes many hours/days. Even getting to the list of phone screens, involves many hours of sifting through resumes, where each resume is cleverly designed to hide the fact that the prospective candidate is incompetent.
I have not followed up with places because I was worried that the questions were representative.
Remember, the interview goes both ways. You can look dumb for your choice of questions just as much as the interviewee can look dumb for their choice of answers.
When I was hiring, I used a homework problem as well - something that we have actually built and released, and I encouraged candidates to look at our implementation (it was a single screen in a very UI-heavy mobile game). We did it in 8 hours, but almost all candidates took more.
Overall level was pretty low that after reviewing about 20 homework assignments, and I had wasted a lot of time on post-mortem interviews where I just had to politely decline with a short code review (after all, they spent all this time, so they deserve something in exchange). After all, I started asking FizzBuzz (yes, not changing the name or anything) on the first phone interview to save time. And yes, turns out, it's not a task that any candidate can do.
So, yes - homework assignments are great, but make sure that a candidate at least knows how to write a simple loop (and yes, I overlook off-by-one errors).
My advice to job seekers who are asked to complete a "homework assignment" by a potential employer:
Ask to see the company's employment agreements or contracts in writing first.
I was once given a 10 hour assignment, which indeed took that long to complete. Afterwards, the company expressed interest in bringing me on, and sent me a work agreement to sign, which included many unsavory terms such as being paid for completed work 90 days out.
Since their terms were shit, I didn't take the job, and lost 10+ hours of my life.
20,000 emails, 1000 phone calls, 100's of presentations to get your product off the ground. It never stops.
*Disclaimer: All numbers are estimates. Product can be something you work on internally for a big company to battle office politics or externally when doing your own thing.
Convincing people that what you do is righteous takes time.
> In the prior 17 years of professional experience, I have never needed to write a graph traversal algorithm.
I don't get this. I'm in my early 20s and have professionally implemented graph traversal algorithms at least five times. Graphs are extremely powerful data structures, and if you have them, then you can expect to traverse/search them.
It is also not a trivial thing to do efficiently though. I would much rather use a mature library to represent and traverse graphs than to try and implement a graph traversal algorithm myself. I do have some knowledge on the subject, which allows me to work with the available libraries, but some things are just too complex to implement from scratch in a timely fashion, I believe.
I agree that sometimes the best solution is to use existing library, but I can't agree that traversing graphs is "too complex to implement from scratch in a timely fashion".
Programming needs some technical skill anyway. And learning "how to use library X" often takes more time than learning "how to implement algorithm X". Libraries are often bloated, bugged, hard to debug or have terrible API. In these moments basic programming skill set is very useful.
The premise of open source ecosystems such as node is that reusable libs are tested by the whole community. Don't reinvent the wheel. I am not saying you shouldn't learn graphs/CS topics. But you should absolutely utilize the ecosystem.
I think it goes beyond subjective likes/dislikes. I personally like reinventing the wheel, you personally believe in ecosystem, these are just our beliefs.
But I think in the long run everything depends on exact project rather on our wishful thinking.
I had experiences when use of library was much better solution than own baked one, but I also had experiences where none of the libraries I had found were quite fitted for the project and many times I just had to throw library and write something from scratch in simple way.
> reusable libs are tested by the whole community.
reusable libs are often FOR the whole community and not for YOUR project. Ofc sometimes you need exactly what rest of community needs, but sometimes not.
Funny story(now, wasn't at the time but been a few years now). Made it to a final interview at one of the "elite" tech companies.
Was interviewing for a web development position, and the second whiteboard question was about designing a hash table. I of course was confused as I had no backend knowledge. Next one was about a graph traversal, and again I was confused.
So, because of that I was scared shitless that I needed to know that stuff and went out/crammed it.
Ended up interviewing other places, got an offer and didn't have to answer it.
However, because of that "elite" company's questions I went and learned more about graphs, queues, stacks etc.
So while I was pissed off at them for awhile for wasting my time, in the end without them I might not have spent the time learning and progressing through Comp Sci topics.
Unless you're designing to spec, how do you not need to understand graphs and hash tables for front end work? I'd imagine understanding hash tables would be useful in reasoning about caching, and graphs for dependency management if nothing else.
As a front end developer, if you're consuming APIs and building out interfaces, and passing data back to the APIs you don't have to know much about graphs or hash tables.
For most web developer/front-end positions, you can be perfectly fine without knowing much of the "advanced" topics of computer science. You're going to be designing to the product manager's specs, paired with a back-end programmer who is going to tell you the structure of the data that will be passed back down and then you build it from there.
As far as understanding hash tables or graphs, you can really get away with knowing them.
However, I agree with you after learning these topics. There's incredibly more efficient ways to build front end pages with the concepts of a hash table/lookup. Most recently I designed a page that basically uses the URL slugs as keys in a complex dictionary to grab content from a SQL database to render the pages.
In addition to that, it's also just wonderful to be part of the conversation when you go to design APIs or architecture so you can make sure to relay/point out concerns that may cause a lot of unneeded pressure on the front end.
Tl;dr You can be a web dev or front end without much knowledge of hash tables, graphs but I can't imagine you advancing your career very far without that knowledge.
Of course. Web applications are bigger and bigger, and quite often you have some hierarchical data structure in JSON (usually in form of tree, but tree is kind of graph anyway).
You have to deal with these kind of structures anyway. You have to be able to render such data as hierarchy of components, you have to be able to get some information from it, to transform it, to update it.
Tree traversal is almost as common in front-end world as using .map, .filter or .reduce, I think (and using `reduce` also can seem hard on beginning, but it's quite useful).
I like Triplebytes [1] focused approach to hiring engineers. I think they used to interview candidates to assess their skills rather than depend on whiteboards or resumes. If the candidate passes the interview, they would be connected to relevant companies, take up interviews with the companies and negotiate the offer(s). Now it looks like they changed the process slightly to introduce online coding tests to make it easier to screen more candidates. Edit : As pointed out below, the online coding test is the first step followed by one on one interviews.
I think Triplebyte's initial technical screen is kind of lame. It focuses on solving problems that aren't all that interesting to me, and they start a countdown timer before I've even had a chance to read the problem. It feels like I'm competing against a microwave cooking someone else's dinner.
I'm sorry you did not like our process! For what it's worth, we've tested a bunch of types of questions, and relatively easy questions answered quickly provide the highest accuracy / time ratio. We also cover a bunch of topics (to give everyone a chance to show their strength), so we don't expect everyone to do well on all the questions.
Yeah, I can understand that. I think you'll probably discover in the long run that you eliminate a whole bunch of people who might otherwise be good hires. It's kind of like only doing analysis on how your current customers like your product, while completely ignoring potential customers. And maybe that's for the best. It'd certainly better to build a recruiting tool that serves part of the market well vs. serving none of it well.
Here are a few things that kept personally me from completing the screening process (which I was checking out because I was a recruiter, and now I'm a developer):
- You don't (or didn't) support Elixir, which is what I've primarily been using for the last 12 months, so I spent a few minutes dusting off the "what's the syntax in Ruby" knowlegde
- I'm wasn't emotionally invested into applying, as I don't really know who you'll be connecting me with, and if I want to be challenged, I'll continue working on some personal code (which is what I did)
- Not having my glasses with me, and a bit of the ol' ADHD, made reading just a bit harder, so the countdown timer was basically reminding me the entire time that I couldn't read fast enough
He got 14 interviews from 54 companies. As a candidate, that's better than what you can expect from Triplebyte, which apparently only interviews 1/6th of applicants (2000 interviewed out of the 12000 who have applied).
If you don't make it to the interview phase with Triplebyte, then the process might as well be as opaque to you as any company. You aren't given much information about how well you tested and the reason you weren't selected to interview.
It would be great if there were a Triplebyte-like service for those of us in the lower 10,000, but until then it's back to applying to dozens of companies.
We suffer from adverse selection bias (as does any online job application). Many companies don't even hire from their online applications for this reason (instead using referrals). The truth is that almost all of the people we don't interview would not have passed (we have a small control group that we always interview, to measure our false negative rate). We do certainly make mistakes (our false negative rate is not 0). But I am proud of our process. It's more open than any other interview I know of. We routinely find awesome self-taught programmers who no other company would interview.
14 interviews from 54 companies is downright amazing. Better than I've ever seen. I've probably applied to hundreds of companies throughout my life/career and gained interviews at less than 20. Next time I'm looking for a job, 54 companies is going to be the minimum to apply to.
With a homework assignment that is not timed, you let the candidate do her best possible work in her ideal environment, and yes, while a work environment is sometimes chaotic, when she needs to get her work done, she is going to make her environment as perfect as possible.
I'd like to second this comment as someone who was recently laid off, did a bunch of interviews[0] and landed a job I'm enjoying very much. The company I picked had a "homework assignment". It wasn't a real project that had any useful purpose to them, which I've seen suggestions about doing[1], but a simple web based app.
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview a...
I remember somewhat recently there was a new job site for older engineers. Is there one for companies that don't do the bullshit Google style interview? Something like a noalgorithmdatastructurestechincalinterview.com? It would be great if not doing these kinds of interviews became a hiring advantage for companies and a site like that would help greatly.
I know that "homeworks" aren't very popular on HN or Reddit. However, when job-hunting I have infinitely greater respect for companies who use them, rather than sophomore C.S. whiteboard exercises.
As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world. I have written code against a set of requirements nearly every day of my professional career. Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago. Frankly, whenever I hear people favoring the latter over the former, I always assume that it's just because they're fresh out of school themselves.
Yeah, doing dozens of homework assignments for multiple companies could be a massive time sink. However, I've almost universally felt like I wasted my time when going through the interview process with companies who don't use them.
Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you. If I were closer to entry level, and therefore had to go through the process with a hundred companies to land an offer, maybe I'd have a different opinion.
Regardless, the original post here is some entrepreneur, writing a small novel about his lengthy process of figuring out which job title to seek at a real company ("CTO"? "VP"? "Architect"? "Principal"?) Of course it took him 200+ hours to get a clue.
If you're not willing to pay me on the order of $100/hr to do your homework, I'm not willing to play your recruitment game.
My time is valuable and if an employer isn't willing to give equal effort in the interview process, I'm not interested. Also I'm not particularly interested in a workplace – especially a startup – that insists on this in a recruitment process. It leads to only hiring employees who are willing to play the game which is much more likely to exclude people who want to live lives outside of their job and fosters a culture of obsessive overwork as well as not having the diversity of opinion/demeanor/background/etc. which is valuable in a healthy workplace.
If you want to bring me in and spend a day with me talking and working in your office, fine. I won't, however, spend 4-10 hours working on a problem for you while working at the job I already have and doing the rest of life.
Senior candidates are in the worst position for the typical algorithm/data structures interview. College is a distant memory and they need the weeks of study more than fresh graduates.
Exactly. As a 15+ year vet, I love coding homeworks because:
1) I've been coding forever, and have faced code reviews in multiple professional environments. So your "6-8 hours" will probably only take me less than two, and my code will probably look better than yours to a reviewer.
2) I get tired of having to re-read "Cracking the Code Interview", because almost none of that is frequently used outside of academia. And dealing with smug 5-year journeymen interviewers, who are accustomed to asking 1-year juniors to traverse binary search trees and name all of the design patterns that they've memorized from books.
No matter what process your company uses, there will always be people trash-talking it on the Internet. "I refuse to interview at those places", "I insist on being paid an hourly rate to deal with them", etc. I'm sure that 99% of this is chest-thumping bullshit. But regardless, if you're going to filter out part of the candidate pool no matter which path you take... I'd much rather filter those 25-year olds who feel too entitled to write some code, and legitimately believe that companies should just glance at their GitHub page and then hurl them a giant sack of money.
The thing is, the companies giving homework assignments still go through the regular onsite interview process. So it really is just an added burden of stress and time on the candidates. Arguably studying CS problems also helps job performance, though probably not as much as it could.
That's not always true. The best interview experiences I've had were at companies where you were given a homework assignment, and then invited on-site to talk about your solution for that assignment. That's not at all comparable to the standard Apple/Google/Microsoft/Facebook process where you're expected to come in and solve some graph theory or dynamic programming problems on a whiteboard for 4-6 hours.
I would much rather do a homework assignment and then come in and have a code review or design discussion about my solution than do the standard process. And, I would argue that it gives the company a much better look at my skills and aptitudes than the whiteboard interview. I have never once written a dynamic programming algorithm from scratch, in 11 years of programming. I have implemented graph theory algorithms only a handful of times. But create a relatively basic piece of functionality, and then justify my solution in code review? I do that literally every day, and sometimes more than once per day.
I understand that writing code for a homework assignment can be seen as giving the company I'm interviewing for "free labor". But, as others have pointed out, from the interviewee's perspective, there's not a huge distinction between spending 8 hours cramming Introduction to Algorithms and Cracking the Coding Interview versus spending 8 hours making a website using Django or Ruby on Rails (to name two interview tasks that I've actually been presented with). Frankly, I was much happier writing the websites, as it seemed much less pointless than memorizing how to write Floyd-Warshall all-pairs shortest path in Java on a whiteboard.
The difference is that you, as an applicant, want to be part of something someone else created and to use it to feed and clothe yourself.
Nobody is asking for free work. And I definitely would never hire someone that calls a four hour exercise that is part of an application process "an indication you don't value yourself highly." In fact, it would be a huge red flag and I'd throw that applicant'a resume into the trash can.
Ironically, one of the things outside interests can teach you about is something called overtraining, where you can actually get worse at doing something from doing it more or harder.
If you've heard cross-training in a fitness context, it's named that because it's intended to be an antidote for overtraining. Branching out makes you more resilient. Your ability to solve problems isn't helped by hyperfocussing on an extremely narrow context.
Any inventor can tell you that the really big stuff comes from fusing ideas from seemingly unrelated disciplines. And yet we bury our heads because we don't know any better, and we don't know better because we've buried our heads.
what's to prevent someone from outsourcing the "homework" to programmers on the internet ? I interviewed someone on skype 2 weeks ago and the applicant clearly had someone off-camera in the room feeding them answers. People like money, and they will go great and nefarious lengths to obtain it.
> As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world.
Yes, but what it is not is economical. There's a huge dropoff in the interview funnel at this step because it's largely not worth the effort for the interviewee. The choice is between spending 5-10 hours per take home test per company or spending 5-10 hours a week studying silly whiteboard exercises that apply to dozens of companies.
It's also just as uneconomical for the interviewer. I've completed about a dozen of these tests over the last year and only one company bothered to look at the test after I sent it in. Which makes sense; it takes time to look at a lot of code, figure out which of it is boilerplate and which of it was written fresh for the test. And even then, there's no guarantee somebody else didn't write it. And while you may not be interviewing with hundreds of companies, most companies easily get hundreds of applicants per position these days.
For whatever reason, it doesn't seem to be worth anybody's time to do these tests. This bums me out because I'm terrible at live coding tests. But, this is the reality of the situation, so I have to adapt.
The way I see it a lot of the best people would probably never bother going through a take home test. A company with a simpler, quicker interview process would have already snatched them up.
Marginal cost to get some sort of commitment/investment out of a company is lower, though. With 4-8 hour coding assignments, that's ~10 companies per two-week period to consider you for the on-site portion. With all the other process stuff, you'd wind up maybe being able to get 1 or 2 concurrent job offers.
All of the take-home interview questions I've done have been given after a phone screen has already been done. I would think that's the only way to do it - it's going to take some time for a company to look over what you submitted as well. I don't think I'd bother doing one without that.
FWIW, I've done a total of 1 for each of the last 2 job searches I've been on, and ended up getting an offer out of each one, so it seems perfectly worthwhile to me to do them.
I think there are two main problems with software development recruiting:
1) Companies who are not Google or Apple like to read books about Google and Apple, pretend that they are Google or Apple, and imitate recruiting processes that come from Google or Apple. Which doesn't serve them very well, because they are NOT Google or Apple.
2) Candidates who are not "the folks who can walk out of Apple's machine learning department on Friday and into Google's on Monday" like to pretend that they are, congregate on Internet forums with other candidates who also pretend that they are, and exchange interview advice geared toward people who are. Which doesn't serve them very well, because they are NOT.
> Companies who are not Google or Apple like to read books about Google and Apple, pretend that they are Google or Apple, and imitate recruiting processes that come from Google or Apple. Which doesn't serve them very well, because they are NOT Google or Apple.
A million times this. I got rejected from a company because the interviewer decided he didn't like that I used array concatenation to find the correct solution to a problem he gave me. This was for a job at a 100 person company. Companies writing line of business software need to stop acting like they are google.
> Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you.
Interesting. I've found the opposite. The farther I get in my career (and the older I get--not going to say the "A" word), the more difficult it is to get through the interview gauntlet. Companies looking to fill Junior "5th engineer from the left" roles tend to be fine with a warm body and a brain. Companies looking for super-senior-staff engineers, engineering managers or even more senior folks are extremely picky and probably reject almost everyone they meet. Even when I'm not actively looking for work, I try to answer every recruiter E-mail try to reach out to 1-2 companies a week just to keep irons hot and be able to act on the off chance that something spectacular pops up.
The older you get the MORE shoe rubber you need to go through pounding pavement.
> Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago.
Which company asked you to balance a red-black tree in an interview context? Name-and-shame them. Seriously. I've heard many whiteboard interview polemicists use this example, but have never seen any evidence for it actually happening.
In the programmingzen blog, Antonio Cangiano, a ruby programmer and author of several ruby books, there is a post about an interview for a job at IBM in which the question is to explain how the red-black tree algorithm works.
Random question: Did you just hand-cut the svg code for the graph on the left side of the page? I took a quick look through the source code and didn't see any evidence of a specific tool.
The reason I ask is that that kind of behavior, where you click on the node and it highlights the paths to that node but grays out everything else, is super useful for complicated architecture diagrams where you want to put a ton of stuff in one diagram, but still be able to see where edges go when you have a big spider web of connections. I've been looking for diagramming software that would create these clickable kinds of diagrams for my company, but haven't found anything that exactly fits the bill (Most tools I've found generate static diagrams, and some mind mapping software I've seen doesn't totally fit the bill, either).
He used D3.js. It produces standard HTML SVG elements which is why it looks handcrafted, but you can see the d3.min.js link at the bottom of the body. D3 is very powerful and can probably be used to do what you want but has a very steep learning curve imo.
For web things, I would recommend the combination of dagre (https://github.com/cpettitt/dagre), which will calculate node and edge positions, and d3.js (https://d3js.org/), which you can use to layout your nodes as an SVG.
This article does a decent job at capturing the grind that is interviewing in software engineering. The expectations during the interview process are high and only get higher the longer you've been in the industry. I wonder how much of this contributes to burnout?
I don't think the interview process contributes to burnout. That happens after you get the job. But, it does make recovering from a burnout a pain in the ass. You need to catch up with the latest trends and refresh on the basics like algos and DS. On top of all of that studying you need to come up with something to fill that 2-3 year gap in your resume.
I'm still breaking into a software job and had a homework problem for a company, with a week deadline. Since I worked full time, I spent the majority of my free time working on this problem.
I was happy with my solution, and disappointed when I was rejected.
Rejection is fine, but what upset me more was radio silence when asking for any feedback on my solution. As a young engineer it would have been amazing to get feedback from professionals.
334 comments
[ 0.20 ms ] story [ 357 ms ] threadThe thing to do is to remove proxies for competence and intermediary parties. Talk directly to the candidate, about the actual work (don't try to "see how the handle" odd questions).
However, this requires decentralization of decision making which reduces control.
If it gets me good people to defect (make my hiring easier), I win. The world won't copy me so once I hire people, I can still keep them.
I don't do this because I don't think I'll get good people.
Individual Germans may not have thought of themselves as particularly bigoted against Jews, but that didn't stop them from participating in regimes that marginalized them.
Most of the people fighting against slavery owned slaves. Jim Crow persisted well into the sixties and it's probable that many of the people against slavery would have been for Jim Crow.
It's very possible for the hiring market to be biased against liquidity even when everyone wishes it were easier to hire.
Note that Jim Crow/Davis Bacon/etc was a legal regime, a "must discriminate" set of laws to prevent people from acting in their own self interest. What such mechanism exists today preventing me from acting in my own interest?
I won't discount your theory of a bias, but I think it is a small influencer if it exists.
Well above avoiding mishires, loyalty concerns are paramount in every organization I've worked for. You'd think that they'd treat hiring as the quintessential bet-the-company decision, and operate accordingly, but this simply doesn't happen. Leadership creates policies and procedures designed to introduce opacity in the process with the ostensible goal of avoiding mishires, but the real goal of preserving power.
Absolutely agree and I'll add that you're shopping in a market for lemons. There are multiple adverse selection criteria against the pool of job seekers. (Excellent engineers are more likely to be retained/less likely to be fired. Excellent engineers are more likely to receive an offer when they do interview. Poor engineers are less likely to be retained, more likely to be fired, less likely to be hired.)
In steady state, the applicant pool is of lower overall average quality than the pool of employed engineers.
Our place has a 12-month contract before being made perm. Once you are perm you are not getting let go unless it's part of larger cutbacks (government). He was made perm before 12 months and I was astounded. They didn't even ask me what I thought before they did it.
Can you quantify? I read this all the time but nobody explains it.
Meanwhile, this talking point is used to justify an atrocious waste of interviewee time and ridiculous interview requirements.
2. opportunity cost of actually hiring someone good - by the time you've realized you've made a mistake, a good person is off the market.
3. you actually have to pay a bad hire for their time. you can't just not pay them. that's cold hard cash out the door.
4. everyone on the team starts wondering, how the hell did this guy make it through this process? are we being run by idiots?
5. if the hire is remote in another state, you have to register your business with their tax authorities, and deal with that whole payroll rigamarole.
6. also health insurance bullshit.
7. setting up accounts, changing passwords/keys when they get canned.
8. actually firing them is not fun unless you are a total sociopath, or they are actively causing damage by destroying value (which is extremely rare -- most people will just silently not do a fucking thing for weeks on end).
9. worrying about a lawsuit because ironically, people who cause damage like to threaten these things.
this is on top of your already full workload of doing actual, productive things for actual paying customers and good employees.
This only costs actual cash outlay, not opportunity cost of other team members (could be coding but are instead spinning up the new engineer), overhead from management salaries, cost to redo work of an other-than-productive employee, etc.
At Valley salaries hiring someone for a day costs you $100k++. In more moderately compensated locales, lower bound it at $40k or so.
Sorry, WHAT?
>>pportunity cost of other team members (could be coding but are instead spinning up the new engineer)
Hey I've got news for you. Developers don't just waltz in and start coding from day 1, they need to be brought up to speed on the systems. Your business-nosed attitude needs a reality check. Developers are human capital, not robots.
You're being unnecessarily defensive. Just because people are people does not change the fact that it costs companies money to onboard them. This whole subthread is "why is it expensive to hire people?" and one part of the answer is "because you have to pay them money while you train them, and pay their colleagues while they do it". It's not a reflection of any unrealistic expectations--it's the opposite.
> Sorry, WHAT?
Computer, desk, office space: $5k
Job ads: $500-3k per hire is not unrealistic
Time spent sifting through applications: Easily 5 hours per week. If the posting is up for a month, that's 20 hours. That translates to about $1.5k in fully loaded costs for an employee with a $100k salary.
Time spent conducting phone screens: Assume 50 phones screens for one position. 30 min per person (20 minute conversation, 10 minute notes/break). 25 hours. Approximately $1.8k for the same $100k employee.
Further interview rounds: Assume about 10 candidates per position. Assume at least 4 person-hours per candidate, plus at least 8 hours devising evaluations (programming tests or what-not). 48 hours = ~$3.6k. Add in $1k of travel costs per candidate = $10k.
Draw up an employment offer. Assume you've got a stock one, but the new hire would like to make an amendment. Run it by a lawyer. $500.
We're at about $25k. It's not $40k, but it's not free, either. It would be easy to spend more time on any of those steps. If you use a recruiter, it will be higher. If you calculate based on SV salaries and costs, it will be higher.
If that's the figure, then maybe bad hires really aren't that bad. Presumably the company won't have gotten zero or negative value from a bad hire, and even if it was $0, $25k isn't going to make or break any but the scrappiest bootstrapper.
I don't know how else to do it, especially when it is really rare for someone to have the skills we need out of the gate. That means testing (we have a testing process) and time talking with candidates. On the other side, streamlining things, we have made hiring missteps in the past, which had measurable and high cost to the business. For us at least, it isn't hot air. Hiring is a hard problem. We spent a lot of time trying to optimize it to find an ideal way that costs less time.
Consider this: every hour a consultant is interviewing they arent doing research or billable work. So we believe strongly enough in how we do things to put our money where our practices are. Maybe the economics for the Googles of the world are different, but for us it is a real decision to be made.
I don't think I've ever heard a manager even say the phrase 'turnover' until they have a few ex-employees that they really couldn't afford to have lost, and a bunch of disgruntled people still there. Once the horse has left it's too late to close the barn.
https://www.youtube.com/watch?v=hj7LRuusFqo
while homework problems can be better than in-person interviews, they can also be a lot worse. for one, because they are NOT time-bound. secondly, they are much more difficult to setup and evaluate: do you provide a spec? code to modify? third, asking people to do work for free in order to explore the potential relationship is a big ask. lastly, they are very time-consuming for the company.
in grad school i learned the hard way to always ask for in-class exams instead of take-homes :-). time and scope constraints can be favorable both for the company and the candidate.
> Enough with the graph traversal problems.
the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill. it is that they are a way to gain information about the candidate's problem-solving ability in a limited amount of time. unavoidably, these questions test for speed and familiarity along with problem-solving. that makes them a lot less than perfect, but i've generally found them valuable.
After the interview he said I was clearly weak on math because I didn't know the definition of a square root. Something I did know (10 or so college math courses in linear algebra, calc etc) and was describing, but was at a loss for words.
I find this point interesting, as to me, this would be an argument for, not against. Time constraints on homework assignments are a lot closer to the constraints you'd have programming for a job than "whiteboard coding" would be. I'd hazard a guess that doing a good job on an in-depth, open-ended problem given a few days to a week is a better indicator of ability to be a successful software engineer than being able to solve a logic puzzle on a whiteboard in fifteen minutes.
i agree that it seems easier to simulate working conditions with a homework assignment. (it does add a lot of work for the company to do homework problems well, but let's leave that aside)
the downside is that the candidate has a limited amount of time to talk to you. they often have a full-time job going on top of the job hunt. secondly, are you really game to spend 0.5-2 days of unpaid work just for the chance of being invited to interview further?
An additional perspective I didn't think of until now is whether this type of recruitment process would be more beneficial to the industry as a whole. An argument could be made that we'd all be better off as a whole if interviews tended to more accurately assess candidates and candidates were able to get a more realistic sense of the type of work they'd be doing for a company.
Um, this is a good thing, not a bad thing. Research shows that stress significantly reduces cognitive ability, and time pressure definitely adds to stress - sometimes substantially. What this means is that when you ask whiteboard coding questions during the interview, you're not measuring the candidate's on-the-job ability. You're simply measuring how well they perform under severe pressure. If that's fine with you, as in it's representative of your work environment where people have to churn out code around the clock, fine. Otherwise, homework questions are much better.
but also, keep in mind that the candidate has a limited amount of time, and often a full-time job on top of the job hunt. the homework questions are only good if the candidate has time and desire to them.
Also the stress of someone standing over your shoulder watching you work. At work, I enjoy hashing out problems with a coworkers at a whiteboard, but in an interview it's paralyzing to have someone watching me. As a student, I'd totally freeze up in the middle of a test whenever a teacher started walking down my aisle.
I've only experienced this once and it was stressful as could be, but was from a person I absolutely respected and still do. No-bs, either right or wrong and plenty to discuss in the followup conversation.
Did it from home, pulled the file at around 11pm right when I knew my best time to work was. Would happily do as part of an interview any time.
Asking to work somewhere without demonstrating that you can do the work is asking a lot. Unfortunately, job history alone is not enough to weed out people that cannot program (let alone well.)
> they are very time-consuming for the company.
Not if you implement a simple test spec or scoring system. We're engineers!
> the point of asking graph-traversals and similar algorithms questions is not that they are the relevant job skill.
So don't ask the generic questions. Ask someone to implement something that they would need to do on the job. Obviously don't ask them to implement a feature that takes 12 hours, but a basic screen that is a snapshot of 45 minutes of on-the-job experience isn't crazy.
We also do a take-home exercise first to see how you think without us staring at you. Day-to-day at Seneca Systems we aren't staring at you, so it feels more relevant.
Which other professions have this attitude? I've either worked or supported a pretty wide variety of other jobs, and I can't off the top of my head think of a single one that does.
Even if some examples are brought up (and I'm genuinely interested in what they'd be), it's far from the usual approach in most hiring situations.
I generally prefer an existing portfolio of code (e.g. github repos) to see the kind of code someone produces, but if you aren't able to show some fairly extensive examples of your work, homework is usually a bunch better assessment of your skills than what can be accomplished in a ~45 min interview.
When I wrote my comment, I was thinking of the medical profession, dentistry, auto & other mechanics, appliance and other device repair, construction ("there's a pile of lumber in the back, you have an hour to build a shed and then we'll consider hiring you"), plumbing, civil engineering ...
There are a lot of clowns out there, on both sides of the table. At this point I've had my quota of companies that sounded good in the interview process but ended up being like a bad second date. Full of chaos and not anything I needed in my life.
One place drilled me on testing, and I mistook that for caring about tests. Nope. It turned out that not only did they not have any test automation, but their code base was such a ball of mud that you couldn't have many of them if you wanted to. I had absolutely no interest in pushing for a renewal on that contract.
For example you can document one module and say "the rest of the docs are TODO due to time" - it shows you know how to document and you can move to another bit. My favourite submissions did 5% of many things to show they are aware of those areas, rather than super-polished trivial solutions of just the basic question.
Actually, my codebase has an honest to god graph traversal in it, on the front-end no less. Basic manipulation of graph relationships without much friction should be standard.
1. The "assignment" took me about 4 hours, which they expected and told me off the bat. 2. It involved mostly relevant code with 1-2 wrinkles that were not odd just not everyday sorts of things.
For the proceeding interview, I was able to sit with 3-4 engineers for 1-2 hours really just diving into details about my experience and their processes, etc.
So if I could offer advice it would be to find a "homework problem" your existing crew could build independently in 4 hours or so. As in you've actually had more than 1 team member attempt the request, and they did it in about 4 hours each.
This was actually an interview process I didn't mind. I have 15-ish years of experience doing development. I can spare half a day for a prospect no problem. It was actually kind of fun.
4 hours seems to be a good "sweet spot". You can ask something sufficiently technical and oriented to the daily work without being a bore. And when the interview comes around, you can make sure it's not some ego-bound contest, more of a conversation.
EDIT: I want to provide an anecdote around this. If your salary is $150k, it may cost you roughly $400 in gross pay to interview somewhere. Sure, you could take paid vacation, but vacation time is paid out in cash when you leave so it still costs you. Companies with unlimited vacation are a different story.
The crazy thing was that over half of the applicants that passed a resume screen and phone interview could not complete the homework assignment. Over half. Some couldn't get it at all (either gave up or had compile errors), some got it to run but it didn't work, a few got it to mostly work but their programs had serious bugs, and the last few actually got it right.
The assignment was to read a file containing a list of numbers (some formatted incorrectly, so there was some very simple parsing logic involved), call an API using each correctly formatted number as a parameter, and store what the API returned to a file.
I'm still shocked over half of developers couldn't accomplish that task. Makes me wonder what they were accomplishing in their existing roles.
There is no incentive to cheat. Why would you want a job you are not qualified to do.
From your experience, it sounds like these are good tests, because it's screening for qualified candidates, unlike the phone interview.
money.
What if you joined Google and was poached by Facebook after 1 year?
This does not contradict the fact that it still leaves/can leave a bad impression on impression if your were willing to be poached without a very good reason, since can be considered as a strong sign that your are not loyal to your employer and only hunting for money.
2. Poaching is rather uncommon in Germany. As I outlined in 1 being willing to be poached multiple times is a sign that you are not very loyal to your employer. Thus there is a strongly reduced incentive for employees to be poached.
Also for poaching to work, you have to offer lots of money - I'd be really surprised if there are lots of employers that are willing to start such an "arms race with money". Why not simply offer decent money in your regular job advertisements and get good people without any arms race?
Poaching might make sense if there you want to get "rockstars" that are not available on the "free job market", but can only be poached. But I'm also not aware of any industry sector in Germany where there exist employees that are treated as rockstars. :-(
Just imagine a random company in a tech hub. They host a meetup once... the days after the meetup, most of their employees have been contacted by recruiters and other companies who attended.
If they want anyone in particular and/or if any employee's salary is not on-par with their value, here they go! :D
I'm not commenting on whether that's a good or bad thing, because good or bad it's simply the way it is.
I am not so good with words sometimes... Haha...
Why do they do it? I have no idea.
I once asked a candidate to write a for loop in C#[0], using my laptop and Visual Studio and watched for a solid 5 minutes while he struggled -- And I don't mean jitters around syntax, I mean, failing to declare a variable properly and while he knew what a for loop was, he had no idea how to write one in C#. His resume touted a series of projects that we estimated put him a little above a Junior developer and the only language he referenced was C#. I don't normally do these kinds of "on the spot tests" because I, personally, hate them, but I was cheering when I read Coding Horror's "Fizz Buzz" article[1] on the subject.
This was the most foul example and my only experience is with developer and ops/security positions, but if you're getting half of developers who are in the same time zone as the title[2], you're doing well. Does this happen in other parts of the business world? Some of the folks we get in would be like a Walmart cashier applying to run a finance department because "they've handled money before". And what if they got the job, do they think they'd come close to being successful? I'm all for the "fake it until you make it" and I'm very willing to overlook weaknesses around experience/education if the candidate is really passionate and I can get enough of a comfort level that they'll be a quick study and love the job, but you have to be realistic about your deficiencies.
A lot of the problem, I've found (being on both sides of this in the past) is with recruiters, as well. They tend to operate on a "throw darts at the wall, blindfolded" approach, with the assumption that if they send out a million candidates for a million jobs without regard for qualifications, that they'll get lucky, one will land somewhere and they'll get a commission from sheer volume on bad odds. This serves nobody well -- it causes companies to put up multi-step gates that involve people who aren't close enough to the field to discern a good candidate from a bad one via resume performing a filter operation that disproportionately disqualifies the really passionate but "not quite a perfect fit on the qualifications side" and it causes job seekers to take a bunch of interviews for positions they won't like/aren't qualified for and won't land -- causing them to have a miserable experience interviewing (and probably resulting in some dropping out of the job market entirely).
[0] The exercise was, specifically, write a for loop with an int value that is incremented by one, adds that int value to another int variable, and multiplies the result against the counter, assigning the value to a third variable. Print the result to the console. This was before "Fizz Buzz" became all the rage, I did my best to give a blindingly simple problem to just validate that the person could do something in the language.
[1] https://blog.codinghorror.com/why-cant-programmers-program/
[2] Never mind the details "Must have 4 years of experience in Angular", nonsense -- a topic for another post, but the way we write job requirements is about stupid. I mean, I've seen experience requirements that exceed the length of time the technology was in existence, and I'd rather see a candidate demonstrate an ability to adapt and learn new things than be overly concerned with whether or not they've used a framework that is really similar to a bunch of other frameworks.
If candidates who can't code are actually getting on site, your phone screening is inadequate and vulnerable to "smooth talkers".
Hiring at the company worked like this, provided the resume wasn't passed directly from a member of staff close to that team:
(1) Applicant applied through web/recruiter/e-mail. (2) Resume was reviewed by a generic hiring team who covered every kind of job the company had and none of whom had even the most basic understanding of what a developer's qualifications should be[0]. (3) This same team does a myriad of phone screens to further whittle the pile. (4) Team's manager reviews remainder and often didn't perform a second screen since one was already done[1]. (5) Candidate arrives.
[0] These same people provided many of the "templates" used for the job descriptions, so you can imagine these were similarly bad. This practice resulted in the company, for years, refusing all resumes where the candidate did not complete a 4-year degree program. I pushed for an end to that for dev jobs after two candidates I hired in who had limited/no college experience turned out to be excellent employees.
[1] And in some cases, the manager knew nothing about software development and was put there from some part of the business, so s/he'd be about as effective at a phone screen as the hiring team.
Bad employees are constantly in need of placement: people get fired very frequently.
If you're hearing about a particular job or candidate, the odds are they are bad. Most jobs are filled internally or by referral. IIRC, it's only about 20% of jobs that are filled by recruiters and/or cold job applications.
As for bad jobs, even bad jobs need someone who is functionally capable of doing them. Sure, you have to be vigilant about filling those positions, but it's rare to encounter a "bad job" in this field that would be fine to have a terrible candidate in.
Regarding jobs being from internal candidates: It's interesting, because I hear that statistic rather frequently and often wonder where it originates or whether or not it has the meaning that is often attributed to it. Again, pulling from my anecdotal experience, though my resume was passed in from an internal employee for the two jobs I was most interested in[1], in one case my resume was handed to a sales person who reached out to me with a question about the company I had just been laid off from -- I had never talked to him or met him before and he lived about 100 miles away and in the other case, my sister handed my resume off to a friend of hers who worked there[2]. This friend and I met for the first time on my second day of work there. I don't even know if the guy is a software developer or not -- and that's the crux of the issue. If the recommendation is coming from someone doing the same/similar work on the same/similar team, that candidate is probably a good fit. If it's from someone doing wildly different work or who has no idea what a good candidate would look like, it's not much better than the blind solicitation from a recruiter. This is further complicated by the fact that many/most companies grant a referral bonus, incentivizing staff to solicit and (possibly) lowering the quality of candidates coming through that channel[3]. I'm really curious about others' thoughts on this, particularly those who've hired for their own companies.
The last issue with the 20% rule is that in companies that are dedicated to filling a diversity standard of some kind, it's hard to get there from internal employees unless you already have a very diverse staff. Reality being non-ideal, you tend to hang out with folks who are similar to you[4].
[0] There are plenty of reasons people get laid off/lose a job that have nothing to do with the competency of the employee. Companies go completely out of business, shedding all jobs or merge with companies where the whole department gets eliminated. Less "great" excuses are role elimination, though, this happened directly to me in January but I was offered a position for every job I interviewed for, so given appropriate context it can be fine as well.
[1] I interviewed at four places, all four offered me a position, two of the places were the highest on my list and I took one of my top choices. The two that were high on my list technically had my resume passed in from someone who already worked there.
[2] That it was my sister certainly helped -- she's been the CFO of a very well known global company and is currently the CEO of a mid-sized company -- her recommendation is trustworthy and people who know her know that she wouldn't embellish anything just because of family relation.
[3] I'm on the fence about this, though, personally. On the one hand, I'm less likely to recommend anything but the absolute best candidate for a role at company I work for if there's not a financial incentive -- it's nearly all downside otherwise. If the candidate works out, it's unlikely anyone is going to remember you sent that resume in. If the candidate is a mess, you get a ding on your record for recommending a fool and a subconscious ding to your own reputation that you associate yourself with fools (unfair as that is, I've seen it). If there's a financial incentive, there's always the "...
Not true. Firing is a rare event.
If anything, bad employees will end up in a group that is "relaxed" toward productivity and stays there for a while.
Shady body shops come to mind.
I've witnessed exactly 2 people fired in 7 years.
And anecdotally, the bigger the company the rarer the firings.
Where were you working at that people got fired all the time?
But reading a file, doing some parsing adjustments, and calling an API sounds like a small amount of python code... so I guess not so bad :\
Your one liner is neat and all, but now you have to do 2-3 hours of initial work + w/ever time it takes for adjustments in 2 hours scheduled for an interview.
You could've thought out this toy architecture up front with initial simple implementations or at least discuss what they expect you to provide, but instead you wrote a shell script for some nefarious reason, even though you saw obvious disconnect between estimated time and time you actually spent.
When I was a junior developer, I was just trying to get code to work. When I learned design patterns, and had some bad experiences, then I started architect "robust" architectures for everything. After perhaps 6 or 7 years of doing this, I started to question my insanity. Sure there were several instances where I was like "Damn, really glad I built it like that" but there were an equal amount where it just ended up turning a simple problem into a complex problem. There's no good excuse for that.
So now, I go high school simple first, and college hard later. If it means I have to do some rework, fine. I'm okay with a little rework, because in the end it aggregates to a simpler platform that took less time to build.
Anything beyond 10-20 lines for the solution to this problem is pointless levels of abstraction and would be a huge red flag.
We have an in office test--the dev sits down at a laptop with internet access, gets clear instructions and scaffolding code. We know that the tasks take n minutes because we see the time that candidates take.
Never had a problem with that. It's well known that C++ can't do anything on it's own.
I'm perfectly competent at writing complex code. In fact I'm usually the guy who fixes bugs or perf issues that nobody else believes exist.
A problem you came up with on your own is going to seem simpler to you than to everyone else. Why? Because you thought you picked it at random but your subconscious picked the one it thought was good. In other words, you picked the one you were already primed to answer, and it seemed simple.
So either people are going to work at it longer, the timid will give up (there's nothing wrong with timid coworkers as long as they have someone to listen to them), and the brash will get it done in the time allotment by cutting corners.
Are those the people you were trying to hire? Because that's who you're gonna get.
I absolutely would not hire someone who failed that test. Your defence of it seems bizarre to me.
I think these tests are a great tool, but like anything we all just need to be aware and careful of the biases introduced. Also, we need to be mindful that really good developers who are employed, have families etc, may not have an over abundance of time to spend on these, so if 10 companies each send a challenge that takes 4 hours, and it's unclear if the assessment will lead to further interviews, some better candidates may give up on the process or be selective about the ones they complete.
I think this creates an additional bias, that the people who complete your assessments may be the ones with the most time to invest in the process (ie the currently unemployed).
The catch? There is no internet in Cuba, so they had to do it using a book, and asking questions to a programmer who was helping them there.
It's also better to aim small (a la FizzBuzz) and walk up the complexity slowly, as opposed to starting with "Solve this ticket from our prod JIRA queue, with tests please" (which I have actually been given as a test!)
I like to aim for a very simple task, like proposed above, with maybe one non-obvious part or edge case (which the spec mentions, so it's not a trick question). Just enough to demonstrate a) clean coding for the simple parts, and b) ability to solve something without using a completely naive solution.
Uhuh.
I had a shared library running on a bunch of systems. One was VxWorks. Cross compiled. I knew nothing about VxWorks except that the code needed to run about 8x faster. I got about 2.5x out of my code by improving locality of reference problems that appeared to be intrinsic but just required experience to see. For the rest I filed a bug report to WindRiver and five to the cross compiler writer. At the end of this process I knew more about the VxWorks administration than anybody else in the group.
People give up, even when there's a clear business case for continuing.
I'm going to have to assess come applicants soon and have been trying to come up with something appropriate, which is why I'm interested.
I had a similar experience a month ago. It was a similar "4 hour project" but it was open ended with "make it as complicated or as simple as you want". Being a single table backed CRUD app, you could only put in so many capabilities but it got me excited thinking about ways I could surprise the staff. In the end, I made it self-hosted (in a language that self-hosted is less common), included an installer, and made it support multiple connections from different devices with updates that pushed to the connected clients. The project took me about a day and a half (with two of those hours spent yelling at WIX), but the core parts took about two hours. I was genuinely surprised at how much fun I had putting it together. They also asked me to estimate my time spent -- since I had contract work I was doing at the time and couldn't dedicate a long stretch all at once to it -- I broke it down into its individual tasks and documented the whole thing. They had no problem with the fact that I delivered it a week after the interview.
I also second your "I can spare a half a day for a prospect" attitude. There's a flip-side benefit to that, as well. A candidate who feels over burdened with a simple assignment (provided it's tested as you mentioned and truly is simple) is probably not one I'd be interested in hiring, anyway. I'm not saying it's "one size fits all" -- I've had candidates walk in the door that I knew in advance were really good fits who were hired after a 30-minute screen. That, though, is a unicorn sighting.
If 10 good assignments meant at least 3 offers, it'd be a different story, right?
The kicker though, like someone else below mentioned, is that we would receive submissions that didn't even compile.
What I liked about the HW problem is that even something simple gives you and the candidate something to talk about. Why did you do this or that. Tabs or spaces ;) etc...
In particular, it seems to me that take-home assignments are strictly better for candidate scheduling etc than in-person interviews, if time spent can be held equal. Maybe time spent is just not equal very often?
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400 hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is to say, I should be the only one doing the assignment and there should be no wishy-washy rejections.
There's nothing wrong with that, it's a competitive market, those who get hired are those willing to compete, not those who complain about unfairness about being asked to demonstrate their skills. Employers spend tons of time filtering candidates, even reviewing the submitted work takes time and effort, saying it's no cost to the employer whatsoever is not only untrue, it's naive.
Unless you've been in the shoes of a hiring manager, I'm unsure as to where this idea is coming from?
Having been a hiring manager, it is a horrible experience. Unless it's extremely naive, homework sets generally take time to come up with and formalize, and certainly take time to review the results for.
I suppose I can understand conflating the motivations of an interviewer with their employer, but it's not generally true to say that interviewers are just trying to exploit those they interview.
How do I know you wrote it and didn't merely paste your name over some project I've never heard of?
1) How many forks and stars do you have? 2) How often have you opensourced? 3) Have you blogged about them or created a documentation? 4) Why did they build each of those repos? 5) Commit logs... etc..
Very easy to develop such a framework.
One or two repos can be copied, but there is ample signature on a genuine open source contributors.
It's a one-off effort to come up with an assignment. So the cost per application can be extremely low. As far as evaluating them, the bad ones can usually be tossed out in minutes or less. Comparatively, the cost per applicant is basically nothing.
What?! Up until that point, typically, the employer had crafted the job posting, posted in on various channels, reviewed the responses, talked to promising candidates, and then give you the assignment. And a good assignment needs to be carefully crafted, a process that usually takes an order of magnitude time more than what it takes to prepare an answer.
I think your last paragraph is very close, but I think the thing that matters is "no wishy-washy rejections". Not being the only person taking the test or how much time the company spends seem like ways to make that happen to me.
Maybe I'm just completely wrong here, but I think most of the objections to homework type tests are being caused by processes where the homework test was in addition to the in-person whiteboard interview, not instead of it.
How would you feel about a process that was: some preliminary screening -> nominal 4 hour homework thing -> in person meeting to confirm that you're not a complete asshole and negotiate salary?
It is a sellers market. I get recruited every single day. Any company that disrespects me and wastes my time isn't going to be very successful at convincing me to work there.
Developers are the ones with the power in this market, and I have no problem throwing this power around.
And yet companies have an incredibly hard time hiring talent, even with so many other fish out there. Odd, that.
People need to start understanding their value as an employee. You create more value than you're worth; that's why companies need workers. You don't need to bend over for a potential employer. Plenty of other fish in the sea of employers, too.
And I am honestly just an average SF engineer.
The current engineering market let's me choose whatever company I want.
So, I am not going to waste my valuable time with companies that disrespect me, when there are a hundred more that I could be talking to instead.
The way that a business relationship works, is that the person with the power gets to make the decisions. And in the market that I am in, that person is me.
And I have no problem demonstrating my skills. I just demand that there is another person from the company, on the other end of the phone line or other end of the table, investing the same amount of time into the relationship as me.
If a company thinks that it is unreasonable for me to demand an equal time investment from them, well sucks to be that company, because as we have already established, they aren't the ones with the power, so their opinion doesn't really matter. They can go find someone else to interview.
Oh most happily will, even talking to you is a pain, I wouldn't work with you if you paid me. You're arrogant and entitled, you'd make an awful employee.
Maybe just open separate channels for people who have an open source portfolio. I have about 30 repos in my OSS, but still have to take these assignments. Its such an utter waste of my time.
That is a huge red flag, and means that they will disrespect me at work as well.
So that's an absolute floor of an engineer-hour per candidate: Anything less than 30 minutes to read through it and take notes is useless, and figure another 30 minutes minimum for a call.
If they're not going over the responses in detail and then talking through them with the candidates, then they're just wasting their own time because they're not actually judging anything.
It's a job. It has an application process predicated upon demonstration of some base level of aptitude. Fulfill it or fuck off and apply somewhere else.
If the "homework assignments" are just a scheme to get work done for free, or a competent engineer's time is not set aside to review them, then that's a different story. But as a fair test of programming ability, administered in good faith, I don't see the problem, and such a test is important.
So you block off 4-8 hours of your life for this job you're really excited about. What is the employer putting up? Generally nothing. They get a sea of responses and they get to pick a few of the best ones. The 100 responses they got cost these candidates 400+ hours of time for no cost to the employer whatsoever.
Take home assignments are fine, but the employer needs to put up something as well. Either pay for the time I spend on it, or have the assignment late in the interviewing process and have employment contingent on passing some known bar ahead of time. That is, I should be the only one doing the assignment and there should be no wishy-washy rejections.
All of the interview processes I've had recently have checked I did the exercise by either asking me to describe what I built with no aid, asking me to walk through the code with it in front of me, or asking me to conceptually expand on what I built on a whiteboard.
Whittle down the pool even further and the ones you think look good on paper, and after an interview (or a series of them), have them come in, on a paid basis.
It'd be a sign of good faith that they're both serious about finding a good candidate, and respectful of your time?
I know some companies do paid stuff like that. If only it were more commonplace.
Note that 5-minute-long task is a good pre-filter on its own.
I did an assignment like that for a company once, I'm still waiting for the money for more than one year...
I echo the grandparent's sentiments, myself, from my own personal experience as well.
>If you want the job, you'll find four hours.
But what happens when everyone starts having these 4 hour screens? Now its a 40 hour process to apply for 10 jobs in a week (and this is before any commitment on the part of the employer). It's not sustainable or fair. And yet, there's no incentive for employers not to do it. So we as job seekers just need to suck it up and be thankful for the opportunity, right?
Can you, personally, pay me a salary? If you can and I want to work with you, I absolutely think it's fair to ask four hours of my own time to demonstrate I'm worth consideration for it. If not, then why does your opinion matter?
After interviewing 10 (say two three hours spent on each between contacting, scheduling up etc) it was clear every unemployed with 'can use word' as skill boosted their resume and applied.
I'm sorry for the real developers looking for a job but a company can't dump 20 days work on chasing and weeding liers.
I'm not a fan of homeworks myself but this is what it has come to.
I'd be payng but that puts the company and the prospect in a very difficult position them towards tax laws and towards their current employer non competes, the company toward again tax laws and then there's the absurd of paying for contract work delivered that's 99% of times not up to spec.
Beside, the unwillingness to compromise smells itself and given the choice I'd never get a potential troublemaker in that only thinks time=money and screw everything else.
That little empathy for both sided of the problems sure is a red flag.
Edit: well unless it's a contracting job then it's fine.
we used to give out this http://play.elevatorsaga.com/#challenge=3 and asking for solution to level 3 which can be done in a short time + is the first non trivial challenge.
of course solutions can be find in internet etc, but it's pretty easy to look them up if they smell fishy and it'd all come down crashing at the face-to-face anyway
it's also a very good exercise in real world problem solving and a source of endless technical discussion since there's no single good approach, especially with two elevators to schedule.
Most of the places I've been at were desperate for 'good' people and they couldn't seem to find them. It's still a seller's market out there.
If this were true people would be poached a lot more frequently than what we can observe now.
I have never seen a job posting I was sufficiently interested in to put in 4 hours with no investment from the company.
Regardless, if you have a good job already, why apply to any position you're not particularly interested in and that you wouldn't spend four hours trying to get?
The standard process where the first two steps are some ordering of a 1-hour interview with someone technical and a chat with a recruiter is perfect for this. With minimal effort (maybe 90 minutes) you find out whether they're interested in you at all, and get to ask both a technical person and a recruiter about the job.
The big time investment typically comes later, and I can easily bail if I decide it isn't going to be a compelling offer. I've done this quite a bit.
I'm glad the homework process worked for you, but my point that it doesn't seem like a good idea for the company stands. If the first thing you do is filter based on how badly applicants need a job, you're losing a significant portion of the best people right off the bat.
Why would I spend any of my already very limited time if they didn't make sure the process is streamlined?
Also, I don't think it's uncommon to like what a company does and being interested without yet knowing if you're interested enough to put the time aside - most company sites and job descriptions are much too generic and vague, so I'd want to talk to some employees first (remember, interviews are two way conversations). So I might not know if I'm interested enough at the time of application, just that I might be.
[1] for my current job, the interview process was essentially them trying to persuade me to join them, and my previous job I was actively looking for a job and someone I know at the company said I should apply. My "application" consisted of telling him "ok set up an interview"
So now I'm doing a day and a half of unpaid work for them, taken out of evenings and time with the family.
Not hypothetical. I just described my last job hunt, 16 months ago.
The place I really wanted to go closed the req on me, and then opened it back up the day after I started at another place. My third choice (stars didn't align on the other 2 interviews, in part because of stuff like this).
Furthermore, I already have a job, just as most reasonable candidates do - so if you want to convince me that I should spend many hours on that, you'd better be sure that your offer has a really attractive first impression and seems likely to be better than the job I currently have, otherwise you'll be self-selecting your candidates only from the (generally less skilled) sub-population of workers for whom "If you want the job, you'll find four hours" is true because they actually are desperate for jobs, unlike most skilled people in tech industry.
It goes both ways.
I wonder if the best thing to do is to simply offer the candidate a choice.
Why is that a bad thing? The prospective candidates aren't entitled to anything, the burden should be on them to prove they have any capabilities for the job.
> What is the employer putting up? Generally nothing.
Totally incorrect, have you ever had to hire somebody? The initial phone screens alone to weed out the 99% of complete incompetents takes many hours/days. Even getting to the list of phone screens, involves many hours of sifting through resumes, where each resume is cleverly designed to hide the fact that the prospective candidate is incompetent.
Remember, the interview goes both ways. You can look dumb for your choice of questions just as much as the interviewee can look dumb for their choice of answers.
Overall level was pretty low that after reviewing about 20 homework assignments, and I had wasted a lot of time on post-mortem interviews where I just had to politely decline with a short code review (after all, they spent all this time, so they deserve something in exchange). After all, I started asking FizzBuzz (yes, not changing the name or anything) on the first phone interview to save time. And yes, turns out, it's not a task that any candidate can do.
So, yes - homework assignments are great, but make sure that a candidate at least knows how to write a simple loop (and yes, I overlook off-by-one errors).
Ask to see the company's employment agreements or contracts in writing first.
I was once given a 10 hour assignment, which indeed took that long to complete. Afterwards, the company expressed interest in bringing me on, and sent me a work agreement to sign, which included many unsavory terms such as being paid for completed work 90 days out.
Since their terms were shit, I didn't take the job, and lost 10+ hours of my life.
- The homework assignment wasn't open and public: it was an option after doing a brief (30 minute) phone interview.
- The other option was basically coming in for onsite technical interviews.
*Disclaimer: All numbers are estimates. Product can be something you work on internally for a big company to battle office politics or externally when doing your own thing.
Convincing people that what you do is righteous takes time.
I don't get this. I'm in my early 20s and have professionally implemented graph traversal algorithms at least five times. Graphs are extremely powerful data structures, and if you have them, then you can expect to traverse/search them.
This very statement reminds me of famous "why can't programmers program" article: https://blog.codinghorror.com/why-cant-programmers-program/
Programming needs some technical skill anyway. And learning "how to use library X" often takes more time than learning "how to implement algorithm X". Libraries are often bloated, bugged, hard to debug or have terrible API. In these moments basic programming skill set is very useful.
But I think in the long run everything depends on exact project rather on our wishful thinking.
I had experiences when use of library was much better solution than own baked one, but I also had experiences where none of the libraries I had found were quite fitted for the project and many times I just had to throw library and write something from scratch in simple way.
> reusable libs are tested by the whole community.
reusable libs are often FOR the whole community and not for YOUR project. Ofc sometimes you need exactly what rest of community needs, but sometimes not.
Funny story(now, wasn't at the time but been a few years now). Made it to a final interview at one of the "elite" tech companies.
Was interviewing for a web development position, and the second whiteboard question was about designing a hash table. I of course was confused as I had no backend knowledge. Next one was about a graph traversal, and again I was confused.
So, because of that I was scared shitless that I needed to know that stuff and went out/crammed it.
Ended up interviewing other places, got an offer and didn't have to answer it.
However, because of that "elite" company's questions I went and learned more about graphs, queues, stacks etc.
So while I was pissed off at them for awhile for wasting my time, in the end without them I might not have spent the time learning and progressing through Comp Sci topics.
As a front end developer, if you're consuming APIs and building out interfaces, and passing data back to the APIs you don't have to know much about graphs or hash tables.
For most web developer/front-end positions, you can be perfectly fine without knowing much of the "advanced" topics of computer science. You're going to be designing to the product manager's specs, paired with a back-end programmer who is going to tell you the structure of the data that will be passed back down and then you build it from there.
As far as understanding hash tables or graphs, you can really get away with knowing them.
However, I agree with you after learning these topics. There's incredibly more efficient ways to build front end pages with the concepts of a hash table/lookup. Most recently I designed a page that basically uses the URL slugs as keys in a complex dictionary to grab content from a SQL database to render the pages.
In addition to that, it's also just wonderful to be part of the conversation when you go to design APIs or architecture so you can make sure to relay/point out concerns that may cause a lot of unneeded pressure on the front end.
Tl;dr You can be a web dev or front end without much knowledge of hash tables, graphs but I can't imagine you advancing your career very far without that knowledge.
You have to deal with these kind of structures anyway. You have to be able to render such data as hierarchy of components, you have to be able to get some information from it, to transform it, to update it.
Tree traversal is almost as common in front-end world as using .map, .filter or .reduce, I think (and using `reduce` also can seem hard on beginning, but it's quite useful).
I like Triplebytes [1] focused approach to hiring engineers. I think they used to interview candidates to assess their skills rather than depend on whiteboards or resumes. If the candidate passes the interview, they would be connected to relevant companies, take up interviews with the companies and negotiate the offer(s). Now it looks like they changed the process slightly to introduce online coding tests to make it easier to screen more candidates. Edit : As pointed out below, the online coding test is the first step followed by one on one interviews.
[1] https://triplebyte.com/
Here are a few things that kept personally me from completing the screening process (which I was checking out because I was a recruiter, and now I'm a developer):
- You don't (or didn't) support Elixir, which is what I've primarily been using for the last 12 months, so I spent a few minutes dusting off the "what's the syntax in Ruby" knowlegde
- I'm wasn't emotionally invested into applying, as I don't really know who you'll be connecting me with, and if I want to be challenged, I'll continue working on some personal code (which is what I did)
- Not having my glasses with me, and a bit of the ol' ADHD, made reading just a bit harder, so the countdown timer was basically reminding me the entire time that I couldn't read fast enough
http://blog.triplebyte.com/12-000-engineers-evaluated
If you don't make it to the interview phase with Triplebyte, then the process might as well be as opaque to you as any company. You aren't given much information about how well you tested and the reason you weren't selected to interview.
It would be great if there were a Triplebyte-like service for those of us in the lower 10,000, but until then it's back to applying to dozens of companies.
> Still getting programming homework interviews
Kill me.
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview a...
https://news.ycombinator.com/item?id=13194074
http://tryoldster.com/
As imperfect as ALL technical recruiting styles may be, at least a short coding assignment (i.e. < 4 hrs) is halfway-related to the real world. I have written code against a set of requirements nearly every day of my professional career. Outside of the interview context, I haven't balanced a red-black tree since I was in undergrad over 15 years ago. Frankly, whenever I hear people favoring the latter over the former, I always assume that it's just because they're fresh out of school themselves.
Yeah, doing dozens of homework assignments for multiple companies could be a massive time sink. However, I've almost universally felt like I wasted my time when going through the interview process with companies who don't use them.
Perhaps my perspective is a bit colored, as I'm far enough along in my career that I can be choosy about which companies I invest time in. I only talk with a dozen or fewer companies in each job search, and so I won't do a 4-hour homework assignment unless I'm already interested in working for you. If I were closer to entry level, and therefore had to go through the process with a hundred companies to land an offer, maybe I'd have a different opinion.
Regardless, the original post here is some entrepreneur, writing a small novel about his lengthy process of figuring out which job title to seek at a real company ("CTO"? "VP"? "Architect"? "Principal"?) Of course it took him 200+ hours to get a clue.
My time is valuable and if an employer isn't willing to give equal effort in the interview process, I'm not interested. Also I'm not particularly interested in a workplace – especially a startup – that insists on this in a recruitment process. It leads to only hiring employees who are willing to play the game which is much more likely to exclude people who want to live lives outside of their job and fosters a culture of obsessive overwork as well as not having the diversity of opinion/demeanor/background/etc. which is valuable in a healthy workplace.
If you want to bring me in and spend a day with me talking and working in your office, fine. I won't, however, spend 4-10 hours working on a problem for you while working at the job I already have and doing the rest of life.
1) I've been coding forever, and have faced code reviews in multiple professional environments. So your "6-8 hours" will probably only take me less than two, and my code will probably look better than yours to a reviewer.
2) I get tired of having to re-read "Cracking the Code Interview", because almost none of that is frequently used outside of academia. And dealing with smug 5-year journeymen interviewers, who are accustomed to asking 1-year juniors to traverse binary search trees and name all of the design patterns that they've memorized from books.
No matter what process your company uses, there will always be people trash-talking it on the Internet. "I refuse to interview at those places", "I insist on being paid an hourly rate to deal with them", etc. I'm sure that 99% of this is chest-thumping bullshit. But regardless, if you're going to filter out part of the candidate pool no matter which path you take... I'd much rather filter those 25-year olds who feel too entitled to write some code, and legitimately believe that companies should just glance at their GitHub page and then hurl them a giant sack of money.
I would much rather do a homework assignment and then come in and have a code review or design discussion about my solution than do the standard process. And, I would argue that it gives the company a much better look at my skills and aptitudes than the whiteboard interview. I have never once written a dynamic programming algorithm from scratch, in 11 years of programming. I have implemented graph theory algorithms only a handful of times. But create a relatively basic piece of functionality, and then justify my solution in code review? I do that literally every day, and sometimes more than once per day.
I understand that writing code for a homework assignment can be seen as giving the company I'm interviewing for "free labor". But, as others have pointed out, from the interviewee's perspective, there's not a huge distinction between spending 8 hours cramming Introduction to Algorithms and Cracking the Coding Interview versus spending 8 hours making a website using Django or Ruby on Rails (to name two interview tasks that I've actually been presented with). Frankly, I was much happier writing the websites, as it seemed much less pointless than memorizing how to write Floyd-Warshall all-pairs shortest path in Java on a whiteboard.
A willingness to give away your time for free is an indication you don't value yourself that highly.
Nobody is asking for free work. And I definitely would never hire someone that calls a four hour exercise that is part of an application process "an indication you don't value yourself highly." In fact, it would be a huge red flag and I'd throw that applicant'a resume into the trash can.
If you can't see the symmetry in the situation it says something about your values, but it is not a compelling argument.
Wanting the employer to spend "equal effort" is silly. Their loss is not necessarily your gain.
Ironically, one of the things outside interests can teach you about is something called overtraining, where you can actually get worse at doing something from doing it more or harder.
If you've heard cross-training in a fitness context, it's named that because it's intended to be an antidote for overtraining. Branching out makes you more resilient. Your ability to solve problems isn't helped by hyperfocussing on an extremely narrow context.
Any inventor can tell you that the really big stuff comes from fusing ideas from seemingly unrelated disciplines. And yet we bury our heads because we don't know any better, and we don't know better because we've buried our heads.
Yes, but what it is not is economical. There's a huge dropoff in the interview funnel at this step because it's largely not worth the effort for the interviewee. The choice is between spending 5-10 hours per take home test per company or spending 5-10 hours a week studying silly whiteboard exercises that apply to dozens of companies.
It's also just as uneconomical for the interviewer. I've completed about a dozen of these tests over the last year and only one company bothered to look at the test after I sent it in. Which makes sense; it takes time to look at a lot of code, figure out which of it is boilerplate and which of it was written fresh for the test. And even then, there's no guarantee somebody else didn't write it. And while you may not be interviewing with hundreds of companies, most companies easily get hundreds of applicants per position these days.
For whatever reason, it doesn't seem to be worth anybody's time to do these tests. This bums me out because I'm terrible at live coding tests. But, this is the reality of the situation, so I have to adapt.
FWIW, I've done a total of 1 for each of the last 2 job searches I've been on, and ended up getting an offer out of each one, so it seems perfectly worthwhile to me to do them.
All the take home tests have been given before I had any phone screen with any engineer. The point is to save both your time.
It's worthless if you have the test after already wasting an afternoon on the phone.
1) Companies who are not Google or Apple like to read books about Google and Apple, pretend that they are Google or Apple, and imitate recruiting processes that come from Google or Apple. Which doesn't serve them very well, because they are NOT Google or Apple.
2) Candidates who are not "the folks who can walk out of Apple's machine learning department on Friday and into Google's on Monday" like to pretend that they are, congregate on Internet forums with other candidates who also pretend that they are, and exchange interview advice geared toward people who are. Which doesn't serve them very well, because they are NOT.
A million times this. I got rejected from a company because the interviewer decided he didn't like that I used array concatenation to find the correct solution to a problem he gave me. This was for a job at a 100 person company. Companies writing line of business software need to stop acting like they are google.
Interesting. I've found the opposite. The farther I get in my career (and the older I get--not going to say the "A" word), the more difficult it is to get through the interview gauntlet. Companies looking to fill Junior "5th engineer from the left" roles tend to be fine with a warm body and a brain. Companies looking for super-senior-staff engineers, engineering managers or even more senior folks are extremely picky and probably reject almost everyone they meet. Even when I'm not actively looking for work, I try to answer every recruiter E-mail try to reach out to 1-2 companies a week just to keep irons hot and be able to act on the off chance that something spectacular pops up.
The older you get the MORE shoe rubber you need to go through pounding pavement.
Which company asked you to balance a red-black tree in an interview context? Name-and-shame them. Seriously. I've heard many whiteboard interview polemicists use this example, but have never seen any evidence for it actually happening.
Hey, I googled for it, here it is: http://programmingzen.com/things-ive-learned-from-hiring-int...
> the question is to explain how the red-black tree algorithm works.
From your link:
> Sample questions included: What’s a Red-Black Tree and what would you use it for?
I don't see how those are equivalent.
I also don't see how the latter two questions are equivalent to asking an interviewee to "balance a red-black tree".
The reason I ask is that that kind of behavior, where you click on the node and it highlights the paths to that node but grays out everything else, is super useful for complicated architecture diagrams where you want to put a ton of stuff in one diagram, but still be able to see where edges go when you have a big spider web of connections. I've been looking for diagramming software that would create these clickable kinds of diagrams for my company, but haven't found anything that exactly fits the bill (Most tools I've found generate static diagrams, and some mind mapping software I've seen doesn't totally fit the bill, either).
I did some work on something a little bit similar for an EU research project, which you are welcome to take a look at. It's probably not directly applicable, but it may give you some thoughts about what does and doesn't work as a UI: https://github.com/cse-bristol/process-model/ https://tools.smartsteep.eu/process-model/?name=actionplanni...
I was happy with my solution, and disappointed when I was rejected.
Rejection is fine, but what upset me more was radio silence when asking for any feedback on my solution. As a young engineer it would have been amazing to get feedback from professionals.