Might make it more expensive to hire, favoring insiders who are employed at the expense of outsiders. Also makes it harder to switch jobs if you’re unhappy/deserve better.
Some places, yes. But surprisingly many places do a really bad job at encouraging internal recruitment, even when it would often let them identify talent they are unaware of and let them shift people into positions where they might be a better fit or be stay for longer. It's especially bad as it can often help shift more of their external hiring to lower risk entry level hires where a bad hiring choice has a lot less impact.
I don't necessarily buy fully into all of these (I am in favor of homeworks, but they should be paid and later staged [this usually means more phone screens. can't win em all]). But since you asked:
1) because meaningfully large swaths of applicants are too stupid to fizzbuzz
2) because passing fizzbuzz is not the same as writing software
3) because teams are comprised of many people in many roles and just because team/person X likes you, doesn't mean you'll be a good aggregate fit.
1. If applicants can't pass fizzbuzz then you shouldn't hire them and you can end the interview process right there.
2. Asking them to write fizzbuzz is exactly the same as asking them to write a method that does anything else, unless if they're aware of it and already knew how to do it, but of course you're asking them other questions as well?
3. Agreed, but it's impossible to match a new candidate with everyone in the company. If they're going to be working closely with person X then have that person sit in the interview for 15-30 minutes and see if they can work together on a problem.
1) I don't get fizzbuzz in first coding rounds anymore. It's usually graphs
2) Ok. I write software. Your whole team can go take a look at my software. Done coding?
3) Ok, so the last on-site round should be meet and greet then? Because I already submitted my code before. You guys all should have looked at the code and agreed to bring me on-site and just meet me. Why ask me to traverse trees again?
Hiring manager: Hey recruitment team, I've read articles saying <interview type A> doesn't work very well, we should try <interview type B>. <Interview type B> solves <concern X>
Recruitment team: Sure thing, we'll start with a parallel run so we can see how <interview type A> results compare with <interview type B>
Hiring manager: We've run both for a few months now, should we stop doing <interview type A> ?
Recruitment team: Some hiring managers still like it, as it addresses <concern Z>. We don't see it as our job to second-guess them. You can spend your time and political capital convincing every other hiring manager, if you like?
Hiring manager: I'm awfully busy with things that are a better use of my time.
It provides a diversity of opinion. You might be surprised at how much disagreement there is about most candidates.
Its probably one of the few parts of the process that actually makes sense, but the fact that people rarely agree belies the deeper problems with tech interviews.
The selection criteria is way too subjective and the bar at so many companies set artificially high.
In my experience your best bet is still to know someone on the inside.
Though even that doesn't always work. A former intern of mine was rejected for a full time job at the company because he was "shy". Mind you this was someone who built something still running in production today.
I do 3 or more interviews a week and the process seems like complete nonsense. (but its my job so I do what Im asked)
Did they do something to correct the intern not being hired or they just wanted extroverts/loud people?
I've erred on the wrong side going the other way, recall one interview on a team where I thought something was off with the candidate because they were so abrasive and then they got into a shouting match with one of my teammates (we thought they were fighting in the conference room), but when it came down to it I didn't want to be the only no and reject them. That was my fault because they turned out to really be a sociopath after a couple of months.
Think of these interviews as a means by which interviewers assure themselves they're working on deeply intellectual CS problems and working with people who are incredibly intellectually gifted mathematicians practing computer science (like themselves), or that they're of the same caliber as what they believe of developers at FAANG, and it begins to make more sense.
As part of an interview I did recently I was asked to build a web app that mimicked a part of their service and deploy it somewhere for them to view. I probably spent about 15-20 hours on it in total and I believe I did more than what the brief asked for only to be told I did not get the job with no explanation, even after I asked if they had any feedback.
I thing is I don't think anybody even looked at what I made because to view everything in the app required the user to login (part of their brief) but no new users showed up on the site at any point. Pretty frustrating experience overall.
Focusing on the “what to do instead”... There's an interesting suggestion at the end:
> Holiday’s company [CallRail, Atlanta, GA] now uses a structured “live-coding interview” format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.
I wish there were more details, like whose hands are on the keyboard, or whether the interviewer has a rubric to structure their evaluation.
I'm waiting for recruiting doctors like that. Oh, you are a surgeon with 17 years of experience? Cool. How about you make a surgery at home, record it, and send us a video?
More pathetic are only companies which have only one question at the beginning (without any further contact) "how much do you want to earn" and after that there is no negotiation. Sometimes I feel like I'm drowning in an endless sea of candidates. Oh, wait, no, the same companies publish the same job ads over and over for years. So maybe not.
Would you want to work in a world where all programmers have to be trained, licensed and regulated with the same rigor as surgeons?
Surely the fact that programming is one of the few 'white collar' jobs where someone with no education and no relevant work experience can get hired based purely on the fact that they've taught themselves the necessary skills and can do the job is a net good thing.
So you either use no commercial software at all, or you're happy paying a hundred times what you paid for it, and doing so in return for the privilege of running 1980s-era applications in 2018.
The ability to build a top tier career from self-teaching is remarkable. It would be such a tragedy if this was stripped and replaced with a signalling based equilibrium run and/or structured by the government :(
Very true. These people literally don't know what they're talking about, or what they're asking for. They've thought their plan through about five seconds into the future.
Either that, or they're blatant featherbedders who aren't bothering to disclose their interests.
Yes exactly that. The problem is that anyone can stitch a leg back on. We need people who can stitch the leg back on and the patient not die 3 months later or the leg fall off again.
If you look at other engineering disciplines there are formal structured approaches to education and testing leading to chartered engineers etc. We could really do with the same.
At the moment we have certifications which merely prove that the individual can pass an exam. I can routinely get near 100% on these sorts of test without knowing the subject or having any experience.
I want to live in a world when we have normal recruitment process for programmers. I have about 17 years of experience. And really my university grades are not important (yes, some companies want to see that). In all places where I worked I knew databases much more than all other people together, I organized training for them etc. Despite my database-oriented CV, I'm always asked basic database questions usually asked by someone who doesn't understand them, and then doesn't understand my answers. I also got some stupid IQ tests with multiple good answers (however exactly one was in the key).
I can talk about bad recruitment processes for hours. I just want to meet competent recruiters, and managers - just like they want to find a competent candidate.
I've only been asked for this when I applied to a "digital design agency" (i.e. web development) during a moment of madness.
They asked me to write a GAE based web app for something or the other. I told them (politely!) that I'd do it, if they provided me with their diversity policy, specifically with an explanation of how this sort of exercise didn't discriminate against anyone (as noted in the article.) I got a "sure, we can do that" and then never heard from them again. I didn't consider that a loss.
I don't get it. The whole industry is complaining about not being able to hire developers, yet there seems to be some kind of competition who can come up with the most awkward hiring process.
Requiring homework is a great way to deter the kind of people companies should be most interested in hiring: those who are not actively looking for a new job.
I find this trend especially surprising considering that the U.S. has at-will employment where you can fire anyone anytime (at least that is my understanding, I'm from Germany). Why not get people at a desk quickly and evaluate them on the job?
Just that firing someone is legal does not mean it is easy. Especially if you have been working closely with the said person for months. And then there are people who are very nice human beings but their performance just isn't good enough. Firing them is hard and pretty bad for team morale.
I'd much rather do that extra work during the hiring process than have to deal with a bad hire later.
Money where your mouth is; are you willing to pay for it? Would you pay the candidate a reasonable sum, certainly over a hundred Euros a day (after tax etc.) for their pre-interview homework? Given that the cost of a bad hire is orders of magnitude more than that, this seems like a really good deal for the company.
Absolutely. As you say, spending a few hundreds bucks is way cheaper than wasting almost half a day of the candidate and our team when we bring someone in and it ends in a no. For remote candidates it makes even more sense since we end up spending thousands of dollars in travel costs when we fly them in for onsite interviews.
are you willing to pay for it? Would you pay the candidate a reasonable sum
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
No matter how you slice it, these multi-hour homeworks are a stupid idea. And the real reason for them is to discourage candidates to justify getting indentured labour instead. Or to discriminate against older workers with more existing time commitments.
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
It seems wrong to reject a good idea because other bad idea will interfere with it. Maybe I'm fortunate; I take extra irregular work in the sum of about 30 days a year, and at the end of the year HMRC and I settle up very simply and easily. I couldn't imagine taking a contract that said I couldn't do any other work; it's by no means uncommon in the UK for people to have extra part-time work that fits around their main job. I think we shouldn't reject a good idea - paying people if you want them to do multi-day interviews - just because other bad ideas (tax systems that can't handle earning extra cash, employers who view their staff as property) will make it awkward for some people.
couldn't imagine taking a contract that said I couldn't do any other work
It’s usually about avoiding conflicts of interest - you can generally get the box ticked easily enough to do something unrelated to your job with them. But “doing an audition at a competitor” is unlikely to be signed off!
But even so - these homework assignments are stupid, and discriminatory on factors unrelated to job performance, and no other industry does it. In fact they yield worse candidates because quality candidates don’t tolerate this kind of nonsense.
And how would this work - you take vacation to do your trial period? You quit to do a trial period knowing it’s still an interview not even an offer?
We want people who are desperate, people who will give up their precious free time to do our bidding, people who don't know when they're being asked to do something plainly ridiculous. We want people who will give up their vacation to try to please us. These are the people who will embrace poor working conditions and tie themselves to us through Stockholm syndrome.
The homework is, in part, a personality test to see how much the company will be able to take advantage of an applicant. This may or may not not be the intent, but it is a result.
Firing someone is always hard, no doubt about it. But I disagree that it's bad for team morale to fire a nice underperformer. In fact, I've seen the opposite happen, where having to drag along a team member was a reason for people leaving.
IMHO, hiring based on homework is no guarantee that you won't end up with a bad hire anyway. It is just another imperfect data point to base your decision on, but one that comes at the cost of massively reducing your number of candidates.
Lately most of the companies I had an interview with wanted me to be a contractor (most of the time with all the benefits of being a contractor removed, from a law standpoint this is possible), this is a common practice around here, not just out of caution, but for tax evasion as well.
There's not much I can do, it's so common a practice nowadays (literally all of my team members are in a similar construction and we are working on fixed projects from nine to six) that with my next workplace I will do the same.
Honestly, keeping someone around who is under par is bad for morale because your other teammates have to make up for the difference. It is hard though, but it's business and if you aren't up for it, you probably shouldn't be in that position.
All of the complaining is just a tantrum fit of an emotionally immature industry who, just like a spoiled child has had it very easy so far.
"Why aren't there 100x more people than I actually need (because we only hire top 1%, right?) with 100 years of working experience with this technology that I just dreamt last night that want to work for free?"
Developers have a similar attitude, despite currently being one of the most privileged segments of the workforce save perhaps investment bankers. We have a straightforward professional path that does not require massive debt and burning half of our youth doing academic busywork and entry level drudgery - like it happens in almost any other professional field with similar compensation.
The industry requirement to prove that you can produce software is not unreasonable, coding is a craft and selecting developers is like selecting painters: ability to bullshit your way through an interview is orthogonal to the skill.
I might assert that what developers have is not a privilege. A privilege is something granted out of generosity or grace by another party, perhaps even undue (and hence the grace).
What developers have is some "game", which makes them a "player". As soon as they lose their game, they cease to be players. Their relation to the game is exactly their value and nothing less. Tech employers are higher level players who know that the economy takes no excuses, and they will trim the fat, so to speak, of anyone who cannot contribute to their gamesmanship.
Generally, in most industries, seniority is respected in and of itself. But in software, you see disgraced older people who didn't play the game right, and you wonder if they are to decline away out of sight. That suggests this is not grace granted by another party. Does that look more like a privilege granted (by whom? charitable businesses?), or a loser in a cold and harsh game?
I see more question marks over the futures of software people than I see in other industries where other peers have gone. In other industries, I feel there has been more of a multi-factored consideration for the generational passing of the torch. In software, I feel everyone is always in a sink or swim test, and I think older people sink a little more.
Very well thought out rebuttal. I live in the convergence between telecom/network engineering and code, and I definitely see the disparity that you describe.
"Privilege" is not generally understood to be granted out of generosity or grace. The dictionary definition is "a right, immunity, or benefit enjoyed only by a person beyond the advantages of most" (http://www.dictionary.com/browse/privilege) which sounds like it applies to software developers when compared to the rest of the workforce. One can enjoy a privilege even if it wasn't intentionally granted by another party. And I certainly feel privileged compared to most of my friends and family members who do not work in the software world.
I agree we have a problem with age discrimination, but I can't conclude if our problem is any better or worse than the rest of the work force.
I thought of undue grace or generosity because typically the term is used to criticize the unmeritorious for undue position -- is it not?
What does it mean if I say you are privileged? When one looks further at the definitions listed for privilege, one sees examples discussing kings and royalty for birthrights. If we're discussing whether people are unworthy, then we are on the same page.
To me, "undue grace" and "generosity" imply some actor who imparts such grace or generosity. But that's not required for privilege. Particularly when people talk about enjoying a privilege in the social-justice way (which I believe to the kind of criticism you're talking about), they're often using it to mean a lack of discrimination as opposed to granting an elevated status. For example, if someone has the expectation that they can walk through a store without getting harassed, we may say they enjoy a privilege, even though most people would agree that should be the norm - because there are people who do expect to get harassed. The reason I think it's important to remove an actor who grants privilege is that its causes are often structural and implicit, as opposed to intentional and explicit. And it's less a criticism than it is a way of pointing out that some people's default expectations for how the world works and will react to them is quite different from others.
Put another way, privilege is often used when some people tend to experience a fairer world than others. But expecting a fair world is how it should be! That's not "undue grace", as everyone should experience a fair world.
(Just so we're clear: this is a full-on semantic discussion, and I'm okay with that.)
>A privilege is something granted out of generosity or grace by another party, perhaps even undue (and hence the grace).
That's quite a narrow definition. A privilege is any kind of advantage enjoyed by virtue of belonging to a group as opposed to personal merit. It's not fundamentally imoral and may well be temporary. Certainly no one has to grant someone the privilege of beauty or intelligence.
In relation to other professions , programmers have the strong privilege of being highly in demand in the labour market. This is clearly a temporary situation, and they should absolutely make the most of their "game" towards the tech employers.
What I was underlining is that the tech job market is ballanced, and certainly more favorable towards labour than almost any other.
If you believe that merit is the focus of discussion, then we're actually on the same page and no further definition work is needed.
I saw focus on meritoriousness, undue worth, or whatever is evoked under the sense of "birthright". I'm arguing that it's gamesmanship at play, and not metaphorical birthright, and hence I speak of some of the losers of the game, as well as the higher level players who interface with the economy directly.
A thousand times this. Although I still write code, and I would be comfortable being a tech lead forever, after 20 years I decided to focus my career on management for my own job security. It was either that or specialize in some hot field, but that's not where my heart is. So as a generalist I would be perpetually competing with the ever widening cohort of "senior" devs who have the SV-standard 5-years experience that represents the capping out of their signalable value.
The picture gets worse when you look at typical interview panels at hot SV companies: median-age 26, went to Stanford/MIT, interned at AmaGooFace, base their hiring process on established "rigorous" stack ranking of candidates based on their performance on fixed algo/whiteboard questions. While I can generally do pretty well at these, variance is high and it does nothing to differentiate me from smart but inexperienced people.
I think it's ridiculous that SV treats programmers like sports players with a limited shelf life. Code bases would probably be a lot better, and workplaces more pleasant to work in if experience were more valued. However I hold no illusions about the way VCs work: they require a steady influx of impressionable young blood to exploit with visions "changing the world", so it makes sense that older jaded devs don't fit into that picture. I do think there is an arbitrage opportunity for older talent though.
Spoken like a programmer with years of experience! I feel this all too acutely. My only game these days is to find small companies who are desperate for engineering experience and leadership because their in-house/contract devs aren't getting the job done.
If I went to google or FB or whatever, it would likely be a nightmare of constant sink or swim projects and unrealistic expectations.
Programmer is one of the only jobs where you have to actually show work related talent in an interview. Some jobs have strong credentials, most jobs it's bullshitting and resume.
investment bankers. We have a straightforward professional path that does not require massive debt and burning half of our youth doing academic busywork and entry level drudgery
You’ve never worked in banking I see. There’s a 3-5 year grind of doing nothing but updating Excel sheets 80-100 hours/week at entry level. Pay is decent for the 1-in-10 that make it through this stage (i.e to Associate 3 and up) but the hours are still long and the work dull.
And there’s nothing at all straightforward about guessing which language or “framework” will be fashionable next...
When I was younger, I felt a written test should be the only factor in hiring.
If you can pass a very hard test in your speciality(almost any field); you get the job. It's bypasses all the nepotism, and subpar people with seemingly spectacular resumes. It's a fair way of hiring. Plus--there's a lot of diamonds in the mix, who didn't go to fancy schools, but are hungry, and get any job done.
I've given up on that because I have never seen just a test used extensively anywhere, except maybe the armed forces. Then again, George Bush jumped past a few hundred pilots because of his connections.
I'm now down to hire a local applicant for a few weeks. (A local applicant!)
See how they work out. And no--you don't want to drag a guy/woman across the country for a temp job, but hiring someone near the job site seems fair?
If the local does well in a few weeks--they get the job.
I've seen so many companies/entities feel they to hire nationally, and that new exotic employee is just an average dude with a fancy resume. The company stooges are happy because no one got their ego bruised, but the company just hired a foxey resume.
Now I'm thinking about it--there's probally a reason progressive companies make people jump through hoops. I imagine the industry hired too many subpar employees by just looking at the carefully crafted resume.
So, if I was hiring for a large company, I would make them jump through hoops too? Or, try to talk the owner into hiring just on test performance?
After all my rambling, guess spending a weekend, at the library, reading the Cracking the Coding Interview, by Iforgothername is not bad advice?
Also from Germany. Last time someone told me "we just can't get enought decent devs" was a company with the worst hiring process I have ever seen.
They wanted me to build an app, with backend and frontend, and design (the design had to be "beautiful"), test it, all in under a weekend while I still was working fulltime at some other job, and I told them, they knew.
It was a Deutsch company. The name is GuteFre.The worst take home test ever, of course I didn't do it.
Lol as an American living in Germany I totally empathize with you, it made me smile.
Germany also says it suffers from a lack of developers, but in my opinion what they don't say is that they need more developers to treat poorly and with a low salary.
That's the same in America. You see job posting like:
Developer position, independent contractor, 5+ years experience Erlang, COBOL, Salesforce, Active Directory, IBM Mainframe, and Java required. Starting salary: $40,000.
And they're whining that there is a great developer shortage in America but they did get some offers from overseas that claim to have those skills, if they could only get a H1B...
I don't know why you're being downvoted. This is exactly how employers abuse the H1B system.
Give an employer absolute power over an immigrant worker with no connections, support network, or other job options and their residency status on the line--what could possibly go wrong?
To be fair, such an employer usually can't get a H1B these days. Which is probably for the best because those people were probably not completely truthful when they described their tech experience.
There is also an filtering effect on job sites where good jobs get snapped up right away and disappear from the site, while delusional prospects concentrate over time, causing people to think that those sites have nothing but garbage offers.
I wonder how much of the shortage is due to there only being a relatively small portion of jobs that developers actually want.
Every time I've worked at a small company that had obvious perks (using a cool language, building a cool product), we had a bunch of applicants to pick from. We could be picky.
But the few times I worked at a company on soul-sucking work like internal legacy tools in VBScript for a non-tech company, they were the ones complaining about the great dev shortage.
They paid well but you also had to jump through hoops to see what it was which is too typical in our work culture. So if the job description doesn't compel someone to apply, GG.
I never see a shortage of people applying. I usually see a great shortage of people I want to hire.
Mid-level position, asking for 5 years of related experience? 85% of the resumes have no related experience. Fresh out of school, that is. The virtual pile gets much shorter when minimal criteria are applied.
Are there companies where people don't periodically move on? The culture right now is to hop between companies. It's not necessarily a bad thing. People get tired of their work and need novelty.
Yes, not crappy ones that pay well. I worked at a company where everyone automatically got a week and a half off at Christmas. This was above and beyond our normal PTO. I would have worked there forever, but we got acquired by a soul sucker. That was the first perk to go.
When you say you require 5 years of related experience as a minimum bar, most applicants read we don't train or invest in our employees. It's not surprising you don't get the best candidates applying.
If I want a junior person, I place an ad for a junior person. We do quite well with those.
If I want somebody with 5 years of experience, it's because I need someone who has already got familiarity with the tools and can apply good judgement.
I have had both positions open simultaneously on occasion, and discovered the same people applying for both. Some of them were even qualified for the junior position.
I think our contention centers mainly around this word. When I see the word related I envision most standard job offers that have a laundry list of 10+ technologies, not all of them particularly related.
Software engineering is such a wide field with such a huge array of available technology options that if you're limiting to 5+ years in a specific stack you're already massively narrowing down your field to a small percentage of the available workforce. If you aren't offering significant advantages to offset that huge initial filter you're not going to get many candidates you find acceptable.
At the company I work at we hire for "general software engineering ability". You can pick whatever language or tool you want to get through the interview, we don't care. Most strong candidates will ramp up on whatever specific stack way more quickly than you expect.
If I ask for five years of related experience, I mean that if my list includes an object-oriented language with a well-known framework, I expect to see someone who has worked with an object-oriented language with a well-known framework and has perhaps dabbled in the particular one I mentioned.
Instead I get many resumes from people who have never used an object-oriented language to contribute to any software project that wasn't assigned by their professor. They haven't got five years of experience, period.
When I was fresh out of University I had to send at least 2 resumes per week to get my unemployment money. Sometimes there were no positions open that matches my criteria, so I just send it to what I could find.
same here, looking for a local web developer, getting people that can't spell the difference between the various css positioning. 8 in 10 couldn't pass a javascript version of foobar test and 3 in ten even got some basic stuff from the assignment wrong (like, printing 0 to 99 instead of 1 to 100)
That's probably because the rest of the job posting doesn't sound that difficult.
And you may even say you want "rock stars", completely forgetting that actual rock stars don't need 5 years of experience to do an exemplary job with a new tech stack.
Most job postings are complete BS, cut through that and the quality of candidates will likely improve.
Yes! If the best thing an employer can come up with to say about their compensation and benefits is that they are “competitive“ this is a huge red flag for any candidate who is (or believes they are) above average.
Especially irritating considering I've dedicated tens of thousands of dollars and six years of my life completing a bachelor's and a master's degree in computer science where people whose entire job was to give me tests and evaluate me on them - maybe you'd like to take their word for it?
I went to a large state school for undergrad and tutored people regularly. By senior year, even the bottom performing students wouldn't have had a problem with fizzbuzz or similar screening problems. We regularly had coding problems much harder than that on tests.
Unless there were other factors involved like maybe the pressure caused by the weirdly adversarial hazing process that the tech interview has become.
Yeah, I'd really like to know what colleges these people are coming out of - if they're real colleges and not just paper diploma mills, I'd have to question their accreditation. I didn't go to MIT - I went to a tier-3 (or 4...) school and you couldn't have graduated, with any GPA no matter how low, without having produced dozens of working programming assignments, many much more difficult than any take-home work assignment.
I personally went to college with someone who could not program at all when he graduated. He made a lot of friends and had a lot of help the entire way through.
It's been 7 years and he has never worked as a developer.
>I personally went to college with someone who could not program at all when he graduated.
If you mean can't program fizzbuzz or simliar, how did he make it through tests? Did he cheat? In most classes I had, tests were around 60% of your grade.
I got a TDD fizzbuzz question once. Later learned that the interviewer told people it was the best interview he’d ever done.
But halfway through I got stuck. Something wasn’t clicking. This is stupid. I could do this in my sleep what’s going on? How much time do I have left?
So I did the only thing I could do. I wrote more tests, figured it out and got the job. Became a lead.
Conversely I’ve worked with many people who can handle trivial problem after trivial problem all day long without breaking a sweat. Give them data that isn’t arranged the way they need it though, and they crumble. Convoluted code full of redundant decisions or data rearranged in an inconsistent (or even data loss) fashion. Always had to be rewritten because we had five more features to add in the same place and you can’t build on sand.
That's a bunch of hooey used to justify crappy interviewing practices. Now there are people who say that have those degrees that don't, but that's verified by a simple phone call to the school.
This is true, although I've no idea how prevalent it is, outside of my own sphere. Over the years I have had several devs in my offshore team in India who had not just a Bsc, but an Msc in computing - and they are literally incapable of performing even the most basic of tasks.
And yet, every last freakin employer is requiring a BS in computer science. Combine that with ever longer programming assignments and stagnant wages and These are all signs of a vast talent surplus.
When the whole industry complains about not being able to hire developers, it's not due to lack of engineering supply. It's due to lack of hiring ability. Most engineers, quite simply are terrible at interviewing, at least until someone gives some actual training and mentoring, but even that is quite rare.
> Requiring homework is a great way to deter the kind of people companies should be most interested in hiring: those who are not actively looking for a new job.
Yeah. I was pretty interested in a place recently, my initial chat with them seemed great and everything, but I lost interest when they gave me the homework test. It's not that I resented the homework -- hell, I've even worked in a place that had a take-home test which typically resulted in a good signal for a candidate's ability. I just didn't feel like doing more programming shit after doing it all day at work.
I think to make this idea work for candidates that aren't actively looking, a company needs to make it clear what they're offering beyond "we're hiring". Employers have been so in control for decades now that they think they should just be able to hang that phrase out there and get what they want.
(And nobody cares about foosball and free soda as perks anymore, we all know that shit doesn't cost anything.)
Very well summed up. Can confirm from the PoV of Berlin, Germany.
I've been on an interviewing spree in January and February, totaling around 20 companies. There were maybe 3 or 4 with realistic and respectful hiring process.
In the end I was left with just one company I actually liked.
Out of interest, what do you define as a 'realistic' hiring process?
I end up having to hire new devs for my offshore team in India every 4-6 months or so, and I just haven't found a good way to ensure candidates are any good.
I actually gave a link to my Stripe account and told the company I was applying to to pay me hourly as a contractor for doing this work. In the end, they took it offensively and denied me.
What you want from a hiring process is to be as objective as possible. For fresh engineers, the industry has settled on basic coding right from fizz-buzz to reverse a linked list and beyond.
For more senior roles, majority still rely on pedigree (previous employer, resume etc), and those who try to be more objective hand out a take-home which ends up frustrating both sides. Candidates have to spend a lot of time and effort and the hiring manager/devs have to setup/run/review.
This is where the industry needs to make the process easy, and operationally objective and efficient for the initial screen: An hour or 2 spent by candidate on a take home problem, which can be reviewed in under 5-10 minutes by the hiring manager. (This disparity in time is because the ratio of interviewed to screened is roughly that.)
(Disclosure: I work at hackerrank, and build the take-home equivalent of a coding test.)
An interesting thing happened to me when I applied for an internship with your employer. I got a call, saying that I would have an interview few days days later even gave me the exact time. Come the day of interview, I got no call. And no response to follow up emails/calls. So, when somebody from hackerrank comments on the hiring process, it is a bit vexing.
I don't want to name names but one YC company I interviewed at left me with the distinct impression that they talked a big game, just to get me to work on one of their (apparently) "strictly-hiring-related toy projects" as homework. Homework, "sure", except it: 1) integrated with a real system, 2) mirrored something they actually did, and 3) they would only agree to proceed if I signed an IP/work contract and accepted their (sorry, too small) payment of USD450 for the (not insignificant) project.
I pushed back at the contract, and asked them to ask their lawyer if that was really necessary. When they insisted on a contract for the homework project I pulled out of the process. It just felt so wrong.
The worst part, is not that they would seek some way to freelance a small part of their system, nor that they would actually use ideas from hiring interviews in their product -- but that (it seems) they would so blatantly lie to someone they professed to want to hire that this was homework, when in fact it was not. Even worse was the feeling of disrespect that they wanted to freelance part of their system for USD450, trying to abuse the hiring process/ talking a big game about offers, to unfairly lowball the price. Like they are a well funded, established, largish YC alum, at least they can afford to pay market. Anyway, everytime I see "XXXX is hiring a YYYY in ZZZZ" on here from them, I'm reminded. Feels good to vent.
Just sad. Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Anyway, I only added the above anecdote after writing the following comment.
This is why Triplebyte is so good. You do a single technical interview ( after a quick online quiz ), then get fast tracked to final interviews at various YC / silicon valley startups.
Sure, maybe not everyone wants to work for a smallish startup. But some of their clients are 200+ people. So...it's not all the same.
I have noticed a trend of not naming companies here. Is it against the rules? I fell that we should name and shame companies that are disrespectful of peoples time.
If not here, where? There's no other forum where it will be easier to hear both sides of the story, naming a name on HN is probably the best place to name names because it's pretty likely to actually get hashed out in the comments.
There perhaps now is another forum because I made it ( actually reused some old code ), after reading these comments. But it's not a forum. More like a very simple database with elastic search. I'm replying you because you seemed interested in this. Hope this is useful somehow, the HN post is here: https://news.ycombinator.com/item?id=16895043
It’s a chilling effect caused by overzealous moderation.
When Coinbase first launched, someone was having issues getting funds out of there. That person made a thread on HN, and the response has always been “HN is not a support forum for YC companies” by the moderation team.
I've had multiple comments detached (as you can see by my comment history) for calling out companies that suck to apply for in the monthly hiring threads. Either the people who posted must have complained so they removed it, or HN doesn't want people to know some companies suck to deal with when it comes to hiring process.
Would Reddit be more open to something like this as in could you create a subreddit and remind HN readers to check?
I see a few people seem interested in the idea of naming such companies. I have wasted a few weekends worth of my time (which at freelance rates is a decent amount of money) and have been ignored when I asked for feedback (sometimes just ignored completely). Its disrespectful and companies should be called out for it. I could name at least one one company that regularly posts in the hiring threads here.
I'm still undecided about the best option for these situations, but I made this ( simple ) site as an experimental database of people's hiring experiences, and posted to HN. I'm letting you know because you cared about this: https://news.ycombinator.com/item?id=16895043
I made a place that is not moderated by HN, where you can share your experiences of applying. It's called hirepwned.xyz and that's the domain name. Just wanted to let you know because you were passionate about this.
I don't want to pass judgement on your coding exercise, and it certainly seems fishy with the contract, however...
I would point out that on the hiring side, evaluating a submitted project in a way that can differentiate between good and great candidates, and that the reviewers can understand well enough to get the signal they need, is very difficult.
Because of this, I know several times we have asked candidates to build a simplified version of something that we have already built internally, where we understand the problem space deeply. We only ever do something we have already built, as that's the whole point, and we typically simplify it, although that might not be obvious from the outside. We also cap the time at 1-3 hours depending on the task as we care about evaluating the candidate, not getting a complete/usable/production ready solution, and so that it's not dependent on how much time a particular candidate can spend on the problem, which would introduce biases.
This isn't perfect, and we try not to do this, and to use problems that are obviously toy problems, but sometimes it's necessary.
That is fine, just be upfront about it. Still require a contract for it is reeking of trying to get work for "free". If you have it running in production, and use it because you understand it thoroughly, it shouldn't matter what the candidate comes up with. Any new insight he might bring you can always incorporate with your own code. In any case none of the code is expected to be used in production, and if it turns out you want to, it's probably the candidate you want to hire anyway.
If you want to use the candidates code, then you have to have a signed contract. Nothing prevents you from looking at the code, and write your own version of it.
Just a minor bit of friendly feedback on your comment, I hope you don't mind...
You said "Any new insight he might bring". To refer to a hypothetical developer as "he" contributes to the stereotype of developers being male, and that can make those who don't conform to that feel less welcome in the community.
I'm sure this wasn't a conscious decision, and I know it's something I slip up with frequently, but I'm trying not to do it, and I think writing that is more inclusive is more persuasive to more people and generally better.
this, just have them do a small subset of something you already do. It tells you so much. I also tell the candidate that if they complete it and it works they are guaranteed to be hired at some level. At the end of the day if you can't show me your work on a open source project, or demo me something you made, there is no way for me to know you can actually build something until you do! Especially true for junior level positions!
> Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Why not? create a throwaway if its a matter of coming back to you. Otherwise we really need to name and shame these companies if we have any hope of getting the practices to improve. Sunlight is a great disinfectant!
I don't understand this development. In Germany, we've basically got full employment in the IT sector. Why would anyone waste their time with this? Just apply for the Job next door.
As an employer, hiring someone new is always a leap of faith. Might work out, might not. If it doesn't, dont feel bad to fire people again.
"In Germany, we've basically got full employment in the IT sector."
I can only confirm this. I used to be a freelancer and I got almost daily offers per email and phone. In order to stop I ask them a very hi price per hour ( 90-100 euros ) and to my very surprise yesterday the guy said, "that's fine, when do we start?". I backed up a bit and after some discussion he told me that they really could not find anyone. There is really no IT people in Germany without job, so the only way to get work do is to turn to freelancers and offer a good rate.
Contrary to what you seem to expect, that price is a steal. A more realistic rate for contractors doing basic IT work (like configuring DB backups) is 400-800 euros per hour.
Please tell me where I can find these jobs that will pay six figures a year for two hours a week of basic IT work. This is more than an order of magnitude higher than anyone I know makes.
You have a bit of the opposite problem in America. The market is flooded with applicants. The real issue is that many of them are woefully underqualified (see the fizzbuzz debate). It's a constant struggle to find a system that filters these people out without insulting the people you are trying to hire in the first place.
In Denmark we put a job offer out on the various job banks, receive the applicants, go through them in the hiring team, starring them 1-5 and we call the best 5 candidates in for a 30-60 minute interview.
Mainly it’s just a talk to see what personality is the best fit. Sometimes we do a personality test, but only for people who will do any form of project management. We’ve never done a technical test or had anyone program for us, I know some companies do it for juniors that are fresh out of school, but otherwise it’s faily uncommon.
That’s it really.
I’ve hired around 20 developers like this over the years, none have failed me. One junior wasn’t really worth his salt at first, but he learned quickly.
When I read about these American hiring practices I wonder why anyone would want to work there. I wonder if it’s because a lot of American developers are autodidact? I mean, I’ve never interviewed someone who wasn’t educated in some form of CS, academy level and up.
3 days of work is certainly an excessive commitment. But limiting the verbal interview to one hour then leaving the candidate with a a computer and a well defined problem that takes half a day to solve it's not unreasonable, in my opinion.
They have already committed that day to this interview, there is no abuse - show us you can solve problems and not give up after a couple of Google searches, like a good half of programmer-aspirants do.
Latest trend? Haven't we all gone through this before? Back in 2012 I did a "homework assignment" too, just to get an interview with a game studio. The task? Build a SOAP client and server that do some silly little thing, I don't even remember what. Basically, just something to prove competency. It took about an hour, mostly time I spent reading the manual for the programming language they requested. Trivial shit... WTF are people being assigned that takes multiple days? Ridiculous...
It makes sense to me, as long as it's somewhat related to the job context, so you can have a first impression.
If it's directly contributing to them, of course they must pay.
In any case this kind of work is a good practice, and is reusable, so applicants are losing anything
In the UK I see a lot of tests that say "don't spend more than 2 hours on this", and that I'd generally expect to take 3-5 hours tops. Americans seem to have a strange employee / employer dynamic
Testing coding abilities should only be done in the context of finding out if I can trust the resume (which should be relatively minimal), or if the candidate is looking for their first job (again, minimal as they won't have experience).
The rest should be possible to analyse TALKING to the candidate or doing some join design session...
Good luck making people spending more than 4 hours in a selection process if you're not Google (and even so)... Seems like a great way of massively discarding anyone that's busy...
For my previous job I spent around a week to build a simple microservice from scratch, with login, jwt, email notification etc. - it's still not a full app of course, but it was very very time consuming (it was a bit out of my field), considering that I did not spend that time with family, hobby et cetera.
I know noone forces me to do it, but back then I really had high hopes for that job. Nowdays the maximum I'm willing to spend with an entry app is two to four days but deep in my heart I still find it a bit too much, but I still prefer this over live coding sessions in front of a bunch of random devs.
These are unreasonable asks for just an interview. A read of their CV and a 30-minute phone screen, maybe two with different people, is all that is needed to determine if a candidate is worth bringing onsite, unless the interviewer and their company is utterly incompetent.
I share your opinion Gaius, but the recruiters seem to think otherwise. In the last year I've been to seven interviews (by all standards I am considered a senior dev with an appealing pedigree) and this is not something unexpected at all.
The worst is that when they ask you to take three or five days off and work with the future team to see how you fit in (not for free though). I'm not too enthusiastic about it, but I already did it twice.
Can I ask which geography you are in? I am in England and that would be excessive here. I think the most I have experienced is 3 phone screens and two full but non-contiguous day’s of on-sites.
Hungary, but I had two non-hungarian interview processes as well, one for Berlin and another for a multinational company (for the Hungarian office though, but six interviews out of seven was held in English), same experience.
The latter one had a fairly complex entry task which took me three full days and I had six interviews after that - during the seventh I gave up and told them we are not looking for each other here.
It's not new - I did my first assignment around 18 years ago - and it's not necessarily a bad thing if done right.
One problem is that assignments take too long and have over-optimistic estimates, which apart from taking up time can lead to feelings of self-doubt. Feedback on the assignments is lacking to no-existent, and this is an unforgivable sin in my book. Also, anything that looks more like a contribution to their own software than an academic test is a red flag.
The other options noted in the article are also pretty bad. Whiteboards and fibonacci numbers, forget it. "Pair programming" in an interview is deceptive, nothing like normal pairing; it's a test (fair enough, but don't call it pairing). It's also problematic for me because I have a problem with my eyesight and any "accommodation" requested up front tends to be inadequate, leaving me at a disadvantage.
For people with busy lives or family commitments, a short "interview prep homework" of 30-90 minutes would help the person to prepare, in their own time, in a relatively less pressured environment. The interviewer could then expand on this during the interview.
I've done assignments like this where it proved a useful opportunity to brush up on a particular language (if cross-training or coming back from management), sometimes
even enjoyed it. Also, it means recruiting is less of a numbers game - you might interview with a handful of companies instead of a dozen, choosing the ones you really care about.
IMHO what we're seeing is a terrible implementation of a reasonable idea.
As a potential employee, I am just not going to do this. Nor am I going to complete puzzles (and I LIKE puzzles) or dance around in front of a whiteboard.
What I would be willing to do is work as a well paid contractor for 1-3 months to make sure that we are a good fit for each other. I'm not sure how anything other than actually working together could suffice.
I am fortunate enough not be concerned about immediate benefits like medical due to my wife's employment--I recognize that this might not be a good option for everyone.
Despite some of the comments here, I'm actually a fan of the 'homework' format. I'm not always great at hiring, but I know how to gauge the 'homework' interviews, and I've usually been right about the interviewee's abilities afterwards, WAY moreso than whiteboarding or riddles or any of the other alternatives.
That being said, I would add some constraints:
1. The 'task' shouldn't take more than 30 min to understand, and no more than 1-2 hours to do. If it takes more than that, you should make some obfuscated dummy-code endpoints that do everything you're not directly evaluating on.
2. You should let the interviewee know that it should only take 2 hours as well. If it takes more, then there was either a misunderstanding in the task, or the interviewee doesn't have the right knowledge/experience/qualifications etc.
3. You should do your homework as well, meaning you've thoroughly reviewed the code submitted and have detailed questions ready to go for the interviewee. This should also be the 'control' to make sure the interviewee really did write the code (you'll know really quickly if they didn't)
4. No other coding tasks in the interview process. You can have them review code or suggest approaches to solve problems (things that people do on the job with others)
5. The interview should be shorter than if they were asked to do whiteboarding or live coding or anything like that.
Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour coding assignment followed by 1 hour of well-crafted questions about the assignment (plus the usual interview stuff) should be enough to know whether or not you want to give the employee a chance for 3 months.
(I realize most employers don't use the evaluation period for evaluative tasks, they just have the person start and hope for the best, but if done right, it's actually a pretty good system)
Well when done this way with all the constraints you specified, then yes the homework interview works very well. Unfortunately, the vast majority of hiring managers do not come anywhere close to any of the points you mentioned.
Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it.
I ended up withdrawing from the process.
If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.
Senior roles should have more demanding technical interviews than junior roles. The company had a bullshit-preventing immune system. It's not a culture fit for you. Win-win.
When hiring for a senior role, you should also consider that the person, who is already in a similar role, would not have the time to spend on your humongous homework app. When literally the only free time I have for myself and my family is a few hours everyday and then the weekends, I am definitely not going to spend the majority of that on an interview, not would I expect my candidates to do so if I were the one doing the hiring.
So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
There's a trade-off for companies. This kind of hiring process will filter out 99% of low-quality candidates, while also filtering out <99% of quality candidates. So it increases their odds of a not selecting a poor choice.
This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.
I'm going to disagree with you on a number of reasons.
1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill.
2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference?
3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes).
4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc.
5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure.
6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.
Regarding #2, I believe your observations and concerns are correct and valid. However, in these situations where the candidate spills his technical heart out, you really start to see the difference between good candidates and bad candidates. Good candidates just have a deep sense of the problem space, can enumerate issues, document such things, etc.
I find code review-style discussion of a simple coding challenge to be great for assessing both culture fit and skill.
What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades.
The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever.
Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time?
You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?
This actually sounds really interesting, putting some code up for one interview or take home assignment, and then asking them to walk through a code review. You could also throw refactoring into the mix, ask them to actually refactor it if needed.
It also got me thinking: what if you just had some generic code, which was low quality, or a prototype front end app, and then asked them to refactor or code review it? That could spark some interesting discussion and be less stressful, but may be easily bsed I think.
I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into.
Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!).
This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours.
I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style.
Afterward, I am looking for how they handle constructive criticism.
This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session.
I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad.
Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.
I think assignments are a really good way to test people, but they really do have to be short. The second-last job I got took around 4 hours which I think is fine. Helps if the task is a bit fun too.
Also it shouldn't be the first stage of the interview process. That's just taking a piss. For this job it was after a phone interview, and with a scheduled face-to-face interview (where we discussed the code I'd written).
Getting a (new) job in IT is an exercise in futility if you don't know someone inside, so essentially there's not an interview but an invitation.
It's lower-status, more degrading and has worser odds than cold-call telemarketers trying to sell you some new credit card.
By the way, when a telemarketer tricks you into answering his call, you don't subject him to a 3-days intensive rectal exam ("homework") just for the remote possibility of getting a new credit card, which at the end you still don't buy anyways on some made-up pretense.
So developers agree to way worse level of abuse and degradation than the lowliest sales guy.
Also there's one and only one defense with respect to abuse either at interviews or after you get the job: having a large stash of money, which allow you to say NO to abuse.
At least 5 years runway, entirely on your own "payroll". If you're actually competent, that's more than enough time to pick any skill (possibly starting your own business rather than look for a job using it), if you can do 12 hours per day by your own choosing on whatever you see fit.
Don't have those 5 years but only 1-2 or, god forbid, have nothing more than a few months on top of a large amount of credit you have to pay back.... and you're in the 99% percent of desperate and destitute crowd who will submit to anything in exchange for a bowl of soup (guarantee they never break out of slavery).
Problem with the solution is that it's a vicious circle: if you don't already have those 5 years of savings, chances are you're never going to save them anyways.
1,025 comments
[ 4.4 ms ] story [ 485 ms ] thread2. A homework
3. An onsite with 5 rounds
Seriously, what does that additional tree traversal round prove?
1) because meaningfully large swaths of applicants are too stupid to fizzbuzz
2) because passing fizzbuzz is not the same as writing software
3) because teams are comprised of many people in many roles and just because team/person X likes you, doesn't mean you'll be a good aggregate fit.
2. Asking them to write fizzbuzz is exactly the same as asking them to write a method that does anything else, unless if they're aware of it and already knew how to do it, but of course you're asking them other questions as well?
3. Agreed, but it's impossible to match a new candidate with everyone in the company. If they're going to be working closely with person X then have that person sit in the interview for 15-30 minutes and see if they can work together on a problem.
1) I don't get fizzbuzz in first coding rounds anymore. It's usually graphs
2) Ok. I write software. Your whole team can go take a look at my software. Done coding?
3) Ok, so the last on-site round should be meet and greet then? Because I already submitted my code before. You guys all should have looked at the code and agreed to bring me on-site and just meet me. Why ask me to traverse trees again?
Hiring manager: Hey recruitment team, I've read articles saying <interview type A> doesn't work very well, we should try <interview type B>. <Interview type B> solves <concern X>
Recruitment team: Sure thing, we'll start with a parallel run so we can see how <interview type A> results compare with <interview type B>
Hiring manager: We've run both for a few months now, should we stop doing <interview type A> ?
Recruitment team: Some hiring managers still like it, as it addresses <concern Z>. We don't see it as our job to second-guess them. You can spend your time and political capital convincing every other hiring manager, if you like?
Hiring manager: I'm awfully busy with things that are a better use of my time.
Its probably one of the few parts of the process that actually makes sense, but the fact that people rarely agree belies the deeper problems with tech interviews.
The selection criteria is way too subjective and the bar at so many companies set artificially high.
In my experience your best bet is still to know someone on the inside.
Though even that doesn't always work. A former intern of mine was rejected for a full time job at the company because he was "shy". Mind you this was someone who built something still running in production today.
I do 3 or more interviews a week and the process seems like complete nonsense. (but its my job so I do what Im asked)
I've erred on the wrong side going the other way, recall one interview on a team where I thought something was off with the candidate because they were so abrasive and then they got into a shouting match with one of my teammates (we thought they were fighting in the conference room), but when it came down to it I didn't want to be the only no and reject them. That was my fault because they turned out to really be a sociopath after a couple of months.
I thing is I don't think anybody even looked at what I made because to view everything in the app required the user to login (part of their brief) but no new users showed up on the site at any point. Pretty frustrating experience overall.
> Holiday’s company [CallRail, Atlanta, GA] now uses a structured “live-coding interview” format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.
I wish there were more details, like whose hands are on the keyboard, or whether the interviewer has a rubric to structure their evaluation.
More pathetic are only companies which have only one question at the beginning (without any further contact) "how much do you want to earn" and after that there is no negotiation. Sometimes I feel like I'm drowning in an endless sea of candidates. Oh, wait, no, the same companies publish the same job ads over and over for years. So maybe not.
Would you want to work in a world where all programmers have to be trained, licensed and regulated with the same rigor as surgeons?
Surely the fact that programming is one of the few 'white collar' jobs where someone with no education and no relevant work experience can get hired based purely on the fact that they've taught themselves the necessary skills and can do the job is a net good thing.
Uh, yes...
There are too many terrible developers out there.
Either that, or they're blatant featherbedders who aren't bothering to disclose their interests.
In neither case do they belong on this site.
If you look at other engineering disciplines there are formal structured approaches to education and testing leading to chartered engineers etc. We could really do with the same.
At the moment we have certifications which merely prove that the individual can pass an exam. I can routinely get near 100% on these sorts of test without knowing the subject or having any experience.
I can talk about bad recruitment processes for hours. I just want to meet competent recruiters, and managers - just like they want to find a competent candidate.
Yes. Please, God, yes.
They asked me to write a GAE based web app for something or the other. I told them (politely!) that I'd do it, if they provided me with their diversity policy, specifically with an explanation of how this sort of exercise didn't discriminate against anyone (as noted in the article.) I got a "sure, we can do that" and then never heard from them again. I didn't consider that a loss.
Yeah you dodged a bullet there. Digital design agencies are almost always sweatshops.
Requiring homework is a great way to deter the kind of people companies should be most interested in hiring: those who are not actively looking for a new job.
I find this trend especially surprising considering that the U.S. has at-will employment where you can fire anyone anytime (at least that is my understanding, I'm from Germany). Why not get people at a desk quickly and evaluate them on the job?
I'd much rather do that extra work during the hiring process than have to deal with a bad hire later.
This would be additional income complicating tax, and many companies restrict employees from taking on outside paid work without approval.
No matter how you slice it, these multi-hour homeworks are a stupid idea. And the real reason for them is to discourage candidates to justify getting indentured labour instead. Or to discriminate against older workers with more existing time commitments.
It seems wrong to reject a good idea because other bad idea will interfere with it. Maybe I'm fortunate; I take extra irregular work in the sum of about 30 days a year, and at the end of the year HMRC and I settle up very simply and easily. I couldn't imagine taking a contract that said I couldn't do any other work; it's by no means uncommon in the UK for people to have extra part-time work that fits around their main job. I think we shouldn't reject a good idea - paying people if you want them to do multi-day interviews - just because other bad ideas (tax systems that can't handle earning extra cash, employers who view their staff as property) will make it awkward for some people.
It’s usually about avoiding conflicts of interest - you can generally get the box ticked easily enough to do something unrelated to your job with them. But “doing an audition at a competitor” is unlikely to be signed off!
But even so - these homework assignments are stupid, and discriminatory on factors unrelated to job performance, and no other industry does it. In fact they yield worse candidates because quality candidates don’t tolerate this kind of nonsense.
And how would this work - you take vacation to do your trial period? You quit to do a trial period knowing it’s still an interview not even an offer?
We want people who are desperate, people who will give up their precious free time to do our bidding, people who don't know when they're being asked to do something plainly ridiculous. We want people who will give up their vacation to try to please us. These are the people who will embrace poor working conditions and tie themselves to us through Stockholm syndrome.
IMHO, hiring based on homework is no guarantee that you won't end up with a bad hire anyway. It is just another imperfect data point to base your decision on, but one that comes at the cost of massively reducing your number of candidates.
There's not much I can do, it's so common a practice nowadays (literally all of my team members are in a similar construction and we are working on fixed projects from nine to six) that with my next workplace I will do the same.
"Why aren't there 100x more people than I actually need (because we only hire top 1%, right?) with 100 years of working experience with this technology that I just dreamt last night that want to work for free?"
The industry requirement to prove that you can produce software is not unreasonable, coding is a craft and selecting developers is like selecting painters: ability to bullshit your way through an interview is orthogonal to the skill.
What developers have is some "game", which makes them a "player". As soon as they lose their game, they cease to be players. Their relation to the game is exactly their value and nothing less. Tech employers are higher level players who know that the economy takes no excuses, and they will trim the fat, so to speak, of anyone who cannot contribute to their gamesmanship.
Generally, in most industries, seniority is respected in and of itself. But in software, you see disgraced older people who didn't play the game right, and you wonder if they are to decline away out of sight. That suggests this is not grace granted by another party. Does that look more like a privilege granted (by whom? charitable businesses?), or a loser in a cold and harsh game?
I see more question marks over the futures of software people than I see in other industries where other peers have gone. In other industries, I feel there has been more of a multi-factored consideration for the generational passing of the torch. In software, I feel everyone is always in a sink or swim test, and I think older people sink a little more.
I agree we have a problem with age discrimination, but I can't conclude if our problem is any better or worse than the rest of the work force.
What does it mean if I say you are privileged? When one looks further at the definitions listed for privilege, one sees examples discussing kings and royalty for birthrights. If we're discussing whether people are unworthy, then we are on the same page.
Put another way, privilege is often used when some people tend to experience a fairer world than others. But expecting a fair world is how it should be! That's not "undue grace", as everyone should experience a fair world.
(Just so we're clear: this is a full-on semantic discussion, and I'm okay with that.)
That's quite a narrow definition. A privilege is any kind of advantage enjoyed by virtue of belonging to a group as opposed to personal merit. It's not fundamentally imoral and may well be temporary. Certainly no one has to grant someone the privilege of beauty or intelligence.
In relation to other professions , programmers have the strong privilege of being highly in demand in the labour market. This is clearly a temporary situation, and they should absolutely make the most of their "game" towards the tech employers.
What I was underlining is that the tech job market is ballanced, and certainly more favorable towards labour than almost any other.
I saw focus on meritoriousness, undue worth, or whatever is evoked under the sense of "birthright". I'm arguing that it's gamesmanship at play, and not metaphorical birthright, and hence I speak of some of the losers of the game, as well as the higher level players who interface with the economy directly.
The picture gets worse when you look at typical interview panels at hot SV companies: median-age 26, went to Stanford/MIT, interned at AmaGooFace, base their hiring process on established "rigorous" stack ranking of candidates based on their performance on fixed algo/whiteboard questions. While I can generally do pretty well at these, variance is high and it does nothing to differentiate me from smart but inexperienced people.
I think it's ridiculous that SV treats programmers like sports players with a limited shelf life. Code bases would probably be a lot better, and workplaces more pleasant to work in if experience were more valued. However I hold no illusions about the way VCs work: they require a steady influx of impressionable young blood to exploit with visions "changing the world", so it makes sense that older jaded devs don't fit into that picture. I do think there is an arbitrage opportunity for older talent though.
If I went to google or FB or whatever, it would likely be a nightmare of constant sink or swim projects and unrealistic expectations.
You’ve never worked in banking I see. There’s a 3-5 year grind of doing nothing but updating Excel sheets 80-100 hours/week at entry level. Pay is decent for the 1-in-10 that make it through this stage (i.e to Associate 3 and up) but the hours are still long and the work dull.
And there’s nothing at all straightforward about guessing which language or “framework” will be fashionable next...
When I was younger, I felt a written test should be the only factor in hiring.
If you can pass a very hard test in your speciality(almost any field); you get the job. It's bypasses all the nepotism, and subpar people with seemingly spectacular resumes. It's a fair way of hiring. Plus--there's a lot of diamonds in the mix, who didn't go to fancy schools, but are hungry, and get any job done.
I've given up on that because I have never seen just a test used extensively anywhere, except maybe the armed forces. Then again, George Bush jumped past a few hundred pilots because of his connections.
I'm now down to hire a local applicant for a few weeks. (A local applicant!)
See how they work out. And no--you don't want to drag a guy/woman across the country for a temp job, but hiring someone near the job site seems fair?
If the local does well in a few weeks--they get the job.
I've seen so many companies/entities feel they to hire nationally, and that new exotic employee is just an average dude with a fancy resume. The company stooges are happy because no one got their ego bruised, but the company just hired a foxey resume.
Now I'm thinking about it--there's probally a reason progressive companies make people jump through hoops. I imagine the industry hired too many subpar employees by just looking at the carefully crafted resume.
So, if I was hiring for a large company, I would make them jump through hoops too? Or, try to talk the owner into hiring just on test performance?
After all my rambling, guess spending a weekend, at the library, reading the Cracking the Coding Interview, by Iforgothername is not bad advice?
They wanted me to build an app, with backend and frontend, and design (the design had to be "beautiful"), test it, all in under a weekend while I still was working fulltime at some other job, and I told them, they knew.
It was a Deutsch company. The name is GuteFre.The worst take home test ever, of course I didn't do it.
Germany also says it suffers from a lack of developers, but in my opinion what they don't say is that they need more developers to treat poorly and with a low salary.
Developer position, independent contractor, 5+ years experience Erlang, COBOL, Salesforce, Active Directory, IBM Mainframe, and Java required. Starting salary: $40,000.
And they're whining that there is a great developer shortage in America but they did get some offers from overseas that claim to have those skills, if they could only get a H1B...
Give an employer absolute power over an immigrant worker with no connections, support network, or other job options and their residency status on the line--what could possibly go wrong?
There is also an filtering effect on job sites where good jobs get snapped up right away and disappear from the site, while delusional prospects concentrate over time, causing people to think that those sites have nothing but garbage offers.
Where are all the good employees? They have jobs.
Where are all the good jobs? They have employees.
Every time I've worked at a small company that had obvious perks (using a cool language, building a cool product), we had a bunch of applicants to pick from. We could be picky.
But the few times I worked at a company on soul-sucking work like internal legacy tools in VBScript for a non-tech company, they were the ones complaining about the great dev shortage.
They paid well but you also had to jump through hoops to see what it was which is too typical in our work culture. So if the job description doesn't compel someone to apply, GG.
Mid-level position, asking for 5 years of related experience? 85% of the resumes have no related experience. Fresh out of school, that is. The virtual pile gets much shorter when minimal criteria are applied.
If I want somebody with 5 years of experience, it's because I need someone who has already got familiarity with the tools and can apply good judgement.
I have had both positions open simultaneously on occasion, and discovered the same people applying for both. Some of them were even qualified for the junior position.
I think our contention centers mainly around this word. When I see the word related I envision most standard job offers that have a laundry list of 10+ technologies, not all of them particularly related.
Software engineering is such a wide field with such a huge array of available technology options that if you're limiting to 5+ years in a specific stack you're already massively narrowing down your field to a small percentage of the available workforce. If you aren't offering significant advantages to offset that huge initial filter you're not going to get many candidates you find acceptable.
At the company I work at we hire for "general software engineering ability". You can pick whatever language or tool you want to get through the interview, we don't care. Most strong candidates will ramp up on whatever specific stack way more quickly than you expect.
If I ask for five years of related experience, I mean that if my list includes an object-oriented language with a well-known framework, I expect to see someone who has worked with an object-oriented language with a well-known framework and has perhaps dabbled in the particular one I mentioned.
Instead I get many resumes from people who have never used an object-oriented language to contribute to any software project that wasn't assigned by their professor. They haven't got five years of experience, period.
Does that help?
The worst I could get was a no :)
And you may even say you want "rock stars", completely forgetting that actual rock stars don't need 5 years of experience to do an exemplary job with a new tech stack.
Most job postings are complete BS, cut through that and the quality of candidates will likely improve.
Because then the other developers would not be able to haze the candidates. Seriously, there's way to much ego involved in development these days.
I went to a large state school for undergrad and tutored people regularly. By senior year, even the bottom performing students wouldn't have had a problem with fizzbuzz or similar screening problems. We regularly had coding problems much harder than that on tests.
Unless there were other factors involved like maybe the pressure caused by the weirdly adversarial hazing process that the tech interview has become.
It's been 7 years and he has never worked as a developer.
If you mean can't program fizzbuzz or simliar, how did he make it through tests? Did he cheat? In most classes I had, tests were around 60% of your grade.
But halfway through I got stuck. Something wasn’t clicking. This is stupid. I could do this in my sleep what’s going on? How much time do I have left?
So I did the only thing I could do. I wrote more tests, figured it out and got the job. Became a lead.
Conversely I’ve worked with many people who can handle trivial problem after trivial problem all day long without breaking a sweat. Give them data that isn’t arranged the way they need it though, and they crumble. Convoluted code full of redundant decisions or data rearranged in an inconsistent (or even data loss) fashion. Always had to be rewritten because we had five more features to add in the same place and you can’t build on sand.
Until we can get useful credentialing in place, my vote is still with white-boarding, with a preliminary remote coding interview over online tools.
Yeah. I was pretty interested in a place recently, my initial chat with them seemed great and everything, but I lost interest when they gave me the homework test. It's not that I resented the homework -- hell, I've even worked in a place that had a take-home test which typically resulted in a good signal for a candidate's ability. I just didn't feel like doing more programming shit after doing it all day at work.
I think to make this idea work for candidates that aren't actively looking, a company needs to make it clear what they're offering beyond "we're hiring". Employers have been so in control for decades now that they think they should just be able to hang that phrase out there and get what they want.
(And nobody cares about foosball and free soda as perks anymore, we all know that shit doesn't cost anything.)
I've been on an interviewing spree in January and February, totaling around 20 companies. There were maybe 3 or 4 with realistic and respectful hiring process.
In the end I was left with just one company I actually liked.
I end up having to hire new devs for my offshore team in India every 4-6 months or so, and I just haven't found a good way to ensure candidates are any good.
For more senior roles, majority still rely on pedigree (previous employer, resume etc), and those who try to be more objective hand out a take-home which ends up frustrating both sides. Candidates have to spend a lot of time and effort and the hiring manager/devs have to setup/run/review.
This is where the industry needs to make the process easy, and operationally objective and efficient for the initial screen: An hour or 2 spent by candidate on a take home problem, which can be reviewed in under 5-10 minutes by the hiring manager. (This disparity in time is because the ratio of interviewed to screened is roughly that.)
(Disclosure: I work at hackerrank, and build the take-home equivalent of a coding test.)
I pushed back at the contract, and asked them to ask their lawyer if that was really necessary. When they insisted on a contract for the homework project I pulled out of the process. It just felt so wrong.
The worst part, is not that they would seek some way to freelance a small part of their system, nor that they would actually use ideas from hiring interviews in their product -- but that (it seems) they would so blatantly lie to someone they professed to want to hire that this was homework, when in fact it was not. Even worse was the feeling of disrespect that they wanted to freelance part of their system for USD450, trying to abuse the hiring process/ talking a big game about offers, to unfairly lowball the price. Like they are a well funded, established, largish YC alum, at least they can afford to pay market. Anyway, everytime I see "XXXX is hiring a YYYY in ZZZZ" on here from them, I'm reminded. Feels good to vent.
Just sad. Anyway, I'm not going to name names because this is unwise to do that in a public forum.
Anyway, I only added the above anecdote after writing the following comment.
This is why Triplebyte is so good. You do a single technical interview ( after a quick online quiz ), then get fast tracked to final interviews at various YC / silicon valley startups.
Sure, maybe not everyone wants to work for a smallish startup. But some of their clients are 200+ people. So...it's not all the same.
When Coinbase first launched, someone was having issues getting funds out of there. That person made a thread on HN, and the response has always been “HN is not a support forum for YC companies” by the moderation team.
I see a few people seem interested in the idea of naming such companies. I have wasted a few weekends worth of my time (which at freelance rates is a decent amount of money) and have been ignored when I asked for feedback (sometimes just ignored completely). Its disrespectful and companies should be called out for it. I could name at least one one company that regularly posts in the hiring threads here.
The HN post is here: https://news.ycombinator.com/item?id=16895043
Pretty simple (beanstalk / elasticsearch, no sessions): https://news.ycombinator.com/item?id=16895043
Hope you find this database useful.
I would point out that on the hiring side, evaluating a submitted project in a way that can differentiate between good and great candidates, and that the reviewers can understand well enough to get the signal they need, is very difficult.
Because of this, I know several times we have asked candidates to build a simplified version of something that we have already built internally, where we understand the problem space deeply. We only ever do something we have already built, as that's the whole point, and we typically simplify it, although that might not be obvious from the outside. We also cap the time at 1-3 hours depending on the task as we care about evaluating the candidate, not getting a complete/usable/production ready solution, and so that it's not dependent on how much time a particular candidate can spend on the problem, which would introduce biases.
This isn't perfect, and we try not to do this, and to use problems that are obviously toy problems, but sometimes it's necessary.
Only if the contract is signed.
Just a minor bit of friendly feedback on your comment, I hope you don't mind...
You said "Any new insight he might bring". To refer to a hypothetical developer as "he" contributes to the stereotype of developers being male, and that can make those who don't conform to that feel less welcome in the community.
I'm sure this wasn't a conscious decision, and I know it's something I slip up with frequently, but I'm trying not to do it, and I think writing that is more inclusive is more persuasive to more people and generally better.
Why not? create a throwaway if its a matter of coming back to you. Otherwise we really need to name and shame these companies if we have any hope of getting the practices to improve. Sunlight is a great disinfectant!
Ok, I created it: https://news.ycombinator.com/item?id=16895043
I'm replying you because you were interested in this ( or seemed to be ).
As an employer, hiring someone new is always a leap of faith. Might work out, might not. If it doesn't, dont feel bad to fire people again.
I can only confirm this. I used to be a freelancer and I got almost daily offers per email and phone. In order to stop I ask them a very hi price per hour ( 90-100 euros ) and to my very surprise yesterday the guy said, "that's fine, when do we start?". I backed up a bit and after some discussion he told me that they really could not find anyone. There is really no IT people in Germany without job, so the only way to get work do is to turn to freelancers and offer a good rate.
Contrary to what you seem to expect, that price is a steal. A more realistic rate for contractors doing basic IT work (like configuring DB backups) is 400-800 euros per hour.
Mainly it’s just a talk to see what personality is the best fit. Sometimes we do a personality test, but only for people who will do any form of project management. We’ve never done a technical test or had anyone program for us, I know some companies do it for juniors that are fresh out of school, but otherwise it’s faily uncommon.
That’s it really.
I’ve hired around 20 developers like this over the years, none have failed me. One junior wasn’t really worth his salt at first, but he learned quickly.
When I read about these American hiring practices I wonder why anyone would want to work there. I wonder if it’s because a lot of American developers are autodidact? I mean, I’ve never interviewed someone who wasn’t educated in some form of CS, academy level and up.
They have already committed that day to this interview, there is no abuse - show us you can solve problems and not give up after a couple of Google searches, like a good half of programmer-aspirants do.
Testing coding abilities should only be done in the context of finding out if I can trust the resume (which should be relatively minimal), or if the candidate is looking for their first job (again, minimal as they won't have experience).
The rest should be possible to analyse TALKING to the candidate or doing some join design session...
Good luck making people spending more than 4 hours in a selection process if you're not Google (and even so)... Seems like a great way of massively discarding anyone that's busy...
I know noone forces me to do it, but back then I really had high hopes for that job. Nowdays the maximum I'm willing to spend with an entry app is two to four days but deep in my heart I still find it a bit too much, but I still prefer this over live coding sessions in front of a bunch of random devs.
The worst is that when they ask you to take three or five days off and work with the future team to see how you fit in (not for free though). I'm not too enthusiastic about it, but I already did it twice.
The latter one had a fairly complex entry task which took me three full days and I had six interviews after that - during the seventh I gave up and told them we are not looking for each other here.
One problem is that assignments take too long and have over-optimistic estimates, which apart from taking up time can lead to feelings of self-doubt. Feedback on the assignments is lacking to no-existent, and this is an unforgivable sin in my book. Also, anything that looks more like a contribution to their own software than an academic test is a red flag.
The other options noted in the article are also pretty bad. Whiteboards and fibonacci numbers, forget it. "Pair programming" in an interview is deceptive, nothing like normal pairing; it's a test (fair enough, but don't call it pairing). It's also problematic for me because I have a problem with my eyesight and any "accommodation" requested up front tends to be inadequate, leaving me at a disadvantage.
For people with busy lives or family commitments, a short "interview prep homework" of 30-90 minutes would help the person to prepare, in their own time, in a relatively less pressured environment. The interviewer could then expand on this during the interview.
I've done assignments like this where it proved a useful opportunity to brush up on a particular language (if cross-training or coming back from management), sometimes even enjoyed it. Also, it means recruiting is less of a numbers game - you might interview with a handful of companies instead of a dozen, choosing the ones you really care about.
IMHO what we're seeing is a terrible implementation of a reasonable idea.
That being said, I would add some constraints:
1. The 'task' shouldn't take more than 30 min to understand, and no more than 1-2 hours to do. If it takes more than that, you should make some obfuscated dummy-code endpoints that do everything you're not directly evaluating on.
2. You should let the interviewee know that it should only take 2 hours as well. If it takes more, then there was either a misunderstanding in the task, or the interviewee doesn't have the right knowledge/experience/qualifications etc.
3. You should do your homework as well, meaning you've thoroughly reviewed the code submitted and have detailed questions ready to go for the interviewee. This should also be the 'control' to make sure the interviewee really did write the code (you'll know really quickly if they didn't)
4. No other coding tasks in the interview process. You can have them review code or suggest approaches to solve problems (things that people do on the job with others)
5. The interview should be shorter than if they were asked to do whiteboarding or live coding or anything like that.
Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour coding assignment followed by 1 hour of well-crafted questions about the assignment (plus the usual interview stuff) should be enough to know whether or not you want to give the employee a chance for 3 months.
(I realize most employers don't use the evaluation period for evaluative tasks, they just have the person start and hope for the best, but if done right, it's actually a pretty good system)
Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it.
I ended up withdrawing from the process.
If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.
So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.
1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill.
2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference?
3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes).
4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc.
5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure.
6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.
What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades.
The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever.
Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time?
You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?
It also got me thinking: what if you just had some generic code, which was low quality, or a prototype front end app, and then asked them to refactor or code review it? That could spark some interesting discussion and be less stressful, but may be easily bsed I think.
No! Studies show that above an sufficient baseline level, having a baseline of soft skills matters more than marginal differences in tech skills.
I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into.
Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!).
This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours.
I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style.
Afterward, I am looking for how they handle constructive criticism.
This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session.
I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad.
Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.
Also it shouldn't be the first stage of the interview process. That's just taking a piss. For this job it was after a phone interview, and with a scheduled face-to-face interview (where we discussed the code I'd written).
It's lower-status, more degrading and has worser odds than cold-call telemarketers trying to sell you some new credit card.
By the way, when a telemarketer tricks you into answering his call, you don't subject him to a 3-days intensive rectal exam ("homework") just for the remote possibility of getting a new credit card, which at the end you still don't buy anyways on some made-up pretense.
So developers agree to way worse level of abuse and degradation than the lowliest sales guy.
At least 5 years runway, entirely on your own "payroll". If you're actually competent, that's more than enough time to pick any skill (possibly starting your own business rather than look for a job using it), if you can do 12 hours per day by your own choosing on whatever you see fit.
Don't have those 5 years but only 1-2 or, god forbid, have nothing more than a few months on top of a large amount of credit you have to pay back.... and you're in the 99% percent of desperate and destitute crowd who will submit to anything in exchange for a bowl of soup (guarantee they never break out of slavery).
Problem with the solution is that it's a vicious circle: if you don't already have those 5 years of savings, chances are you're never going to save them anyways.