Great, so now I have to maintain an updated LinkedIn profile, contribute to various open source projects on GitHub, sustain an active StackOverflow profile, and complete coding challenges on HackerRankX.
I hope the employer who hires me doesn't mind if I do all that on the clock!
Don't forget you'll also need to know Ruby, PHP, Python, Node.js, all .net languages, Javascript, HTML5, CSS3, Nginx, Linux, Nosql, MySQL, Photoshop, Art and Design, among other skills.
Can you prove that your support vector machine works properly, fed with data from Arduino sensors, and can you scale that onto a cloud network while integrating with Active Directory so that users can view their results in a single-page app?
In my experience, the interview process for jack of all trades job descriptions plays out the same way every time.
1.) The candidate shows up with a effectively all the required skills. Predictably, logically, the candidate has a roughly average level of skill across the range on average.
2.) The interviewers decide the candidate won't do because he/she is 'only' average in some area that should have been one entry on a short list of required skills in the first place.
Either hire for the skills you actually need and be specific about it or hire the smartest, hungriest people you can find and be honest about what they'll be expected to learn.
3.) Applicant is hired, and paid criminally below-market with above-market stress, because the types of people who explicitly hire "jack of all trades" are generally trying to save money.
While this may be uncomfortable for you, please understand that the absolute majority of people in the world is much worse off than you.
And while contributing simultaneously to github, stackoverflow and others community projects could be cumbersome for you, positive externalities are immense. The whole ecosystem benefits from this.
Even if this burden is too heavy, there are always options of 4-day working week, freelance work, project-based work, etc. etc. Programmers are lucky to enjoy most liberty from their profession.
So come on, don't complain, you DON'T HAVE to do anything, unless you want to be the very best.
Yes, at the end of the day, software developers have it pretty good in the current job market.
Sometimes it gets a little ridiculous on the requirements/wants on job postings. Some companies just ask for the world; it can lead to impostor syndrome and it can be stressful.
This is not about projects and work that help other people, but doing this for future employers an a career, not out of altruism for others. For example you didn't bring up a linked in profile in your list of things.
Arguably there are better things that programmers can do with their talent, not just for themselves but for the world, than updating social media websites and building a personal online image.
I consider the activity of updating stackoverflow and github with new knowledge and code to be incredibly good for the world. The need to update Linkedin and other resume-like social media website is just an inevitable consequence to the (prisoner dilemma)-like evolution of the credentials game.
I don't just question the ever-growing list of must-have profiles and time required to maintain them, but their value in general.
It feels like a sort of arms race to engineer and automate hiring.
It's generally agreed that actually sitting down and talking with a person is strongest source of signal [1] in the hiring model, but rather than focusing on that the model is continually stuffed with a growing set of noisy variables (profiles).
At what point does this filtering start identifying the best profile builders as opposed to the best or brightest employees? Does hiring become a full-on game of prep services and checklists of activities along the lines of college admissions?
I think the critical role for tech in recruiting is to help work out who to meet, not to replace the conversation. It's definitely an arms race to turn the 1000 resumes into 10 people to be interviewed, but I don't even know where to begin taking it past that realistically.
The first automation software for the full hiring process is a going to be a very well marketed solution, not necessarily an effective one.
Ideally, I would see less of a need for specialist recruiters as tech has made connecting with appropriate hires simple enough for someone from the team the hire will work with to conduct interviews.
There's already an element of SEO-style optimisation in resumes as it is. I have a few outside-the-box approaches on how to get around this and I'll be doing a Show HN once I have something together :)
It's OK. Maintaining a HackerRankX is just a subset of your ability to Google.
Anyway, I'd submit that's the real threat here... this sort of thing is easy to game, and you are talking about the group of people most capable of gaming even the smartest AI algorithm... The top end of the "leaderboards" is going to be a list of cheaters pretty quickly.
The family of Lisps isn't functional by definition. There are Lisps which are as non-functional as one could imagine.
Anyway, where the hell are you seeing the lambda calculus in JavaScript (ignoring that "it's based on lambda calculus" is probably as useful as "computers usually work with 0s and 1s")?
I think I prefer Gild's approach (I've only superficially skimmed their site, no actual experience of their product). They're analysing existing data points to predict the right applicants - StackOverflow, Github etc.
Seems more elegant to use what exists than to ask recruiters to come up with a custom challenge. If they can't make realistic asks at the moment it's unlikely they can create realistic tests on HackerRankX either.
I'm still waiting to see the first job that requires 5 years + experience in Swift. Shouldn't take more than a few months.
I feel like coding challenges like these are contrived and really prove nothing. Cool, you can create a "palindrome index." Can you effectively lazy load assets in an app on demand? Can you architect your CSS in a way that only the relevant styles show up on a page? Do you know why a standard for loop is faster in most cases than a for in loop? Tests like what this site is trying to do don't really prove anything other than you're good at solving puzzles with code whereas as an employer I'm more interested in the quality of your applicable code and architecting thought processes.
I'm not saying these tests are totally useless. Obviously it demonstrates a dev's ability to code within one or more languages, but they're hardly enough to make a hire off of. When it comes to building actual product, these tests always fall short.
If your core business is built around a specific CS problem [e.g. Uber would be the traveling salesman, Google is searching with NLP], picking candidates that demonstrate they can solve that sort of problem effectively [not perfectly, just 'good enough'] is pretty relevant. [e.g. Code Combat's Greed tournament would be a good recruiting platform for finding developers for Uber that worked on the backend since the traveling salesman problem seems really relevant to them]
However, I think you are right in that most of them are useless because they don't focus on that sort of detail.
Would it be though? Amazon spends a lot of time on the traveling salesman problem too, but only the extreme high end architect level people, on a very tiny number of teams.
99.9% of engineers at Amazon are writing, well, downright normal code.
It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce.
It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me. It doesn't to other people.
Think of it as a more relevant FizzBuzz.
> It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce.
> It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
Why do we want a naive solution to TSP? The naive solution to TSP has no applicability to anything - it's purely a thought exercise in the same vein as "why are manhole covers round" and "how many jelly beans are in this jar".
Remember that 10 years ago we thought those questions were "predictors" of programmer ability. Turns out that was full of crap.
So your notion of testing someone's suitability for a job is to ask them to solve a problem unlike any they will encounter in the job, expecting them to generate a solution that would be wildly insufficient even if the problem was relevant to their job?
In the mean time I rarely see companies testing for abilities that are used on a regular basis in these jobs: the ability to architect, knowledge and familiarity with best practices and design patterns, writing testable code, etc etc. All of these skills are far from universal, but yet we spend no time ensuring they're there. No, we blow the valuable 45-60 minutes we have with candidates twiddling around with TSP.
We ask candidates simplified things from what goes in IRL, because we don't have all day. That can't be helped - the least we can do is make sure what we ask is actually a simplified version of what the candidate will be responsible for, instead of simplified versions of things the candidate will never touch, ever.
> "Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me."
It doesn't to me. More accurately, it doesn't seem relevant to the majority of coding jobs out there.
In reality the number of jobs that ever involve implementing real algorithms is really low. Even in research-heavy companies like Google it's limited to a small subset of employees in a small subset of teams.
The vast majority of everyone will not write a substantial algorithm from scratch in their day job. Ever. So why are we testing for this ability?
It really isn't. If anything it'd be Djikstra's or other optimized pathfinding algos. Even then, Uber doesn't do its own navigation, it's handled by Google/Apple/etc.
Uber, like most tech companies, is an integrator. Their need to write complex algorithms from scratch is practically nil. Looking for algo synthesis ability in a candidate is a waste.
Uber isn't built around traveling salesman. When you put in your location and ask Uber for a ride it effectively broadcasts your intent to nearby drivers (with some filtering logic put in place based on your rating and the driver's rating). AFIAK the app doesn't even take into account 1-way streets when computing anything.
The idea has always been to judge ability and potential rather than domain-specific knowledge.
The problem is that, as with most tests, these can be gamed. Every programmer under the sun now knows about FizzBuzz. So now there's an entire art dedicated to the craft of inventing new programming interview questions. Inevitably, the questions which were supposed to be the antithesis to trivia start to become their own trivia.
Really, though, it's just a hard problem. There is no easy way to really accurately assess a programmer you've never met or worked with. Your specific questions could easily turn down an incredibly talented programmer who just hasn't done much front-end work before but could pick it up in a few afternoons.
Sure, anything popular can be gamed. And countermeasures will in-turn be applied (checking for plagiarism, etc).
The point to note is that there is (usually) a job on the line, and a programming challenge is not the only part of the screening process -- it will usually be followed by a live coding session/interview. So the question is would someone risk gaming the process, when the chance of getting caught is high. We think not.
...but when a programmer who has gamed the system gets an interview and then is discovered to be a fraud, doesn't that defeat the purpose of using this service in the first place (to save thousands of interview hours, per the article)?
Even with all these metrics I still have to write code even for the phone screen and then have a 6-8 round coding interview where I have to algorithms that I will not use in my job. I don't see that going away.
Also I don't see bigger companies adopting these anyway. They will have their own way of hiring and is usually more relaxed. All these hurdles just make an experienced candidate cringe more applying to a startup.
For what it's worth, I kinda like the idea. I don't think coding interviews are going to go away anytime soon and I'd much rather use HackRank's fancy interface from the comfort of my home than stand awkwardly at a white board while one or two people stare judgingly at the back of my head.
Despite the somewhat sensationalist headline, the article itself actually seems to be pretty realistic (aside from the "and in a year from now, HackerRank will be like that" comment).
To me, it seems like another tool people who are not good at networking can use to distinguish themselves.
It seems odd that they don't combine the two. You could have a paid version for companies that lets them create their own coding challenges and view candidates who have completed challenges in problem domains and languages they are interested in.
We're figuring out exactly how this could work -- but given the disparate audience of the two applications, they are separated for now. Like Github normal, and Github Enterprise.
> "[...] an obscure theory, like graph theory [...]", [founder] says
Even nowadays, no more or less respectable university will let you graduate with a CS degree without learning at least the basic ropes of graph theory (more than rightfully so). If this is representative of their priorities, I am left wondering if their ranking will actually be of interest to any organisation looking for competent programmers (as opposed to bedroom-trained PHP/Python hackers).
I was recently interviewing for a new job. One company asked me to solve a coding challenge. I had one free evening to do it and managed to get it mostly done but didn't feel comfortable submitting it as there was still a few issues and I didn't have time to clean it up like I wanted to.
The other company asked for some code samples of work I had already done and then I walked them through it to show that I understood it and what design decisions and trade-offs I had made and why.
The company that asked for code samples ended up making me an offer and due to lack of time and the fact that another company was already interested in hiring me, I never even continued the conversation with the coding challenge company.
It seems that if a coding challenge is trivial then it doesn't really prove much. There's probably an answer for something close to it online and even if the candidate did solve it on their own it's still too trivial.
If a coding challenge is complex enough to actually test a candidates range of knowledge then it's often not worth it to complete the challenge for a possible job opportunity.
I was into HackerRank for a while (in a previous life I did a lot of TopCoder/CodeJam/ICPC/USACO/etc) but it just wasn't the same. Programming competitions have a long history and high standards, and there were just too many repeated failures on both the side of execution and problem selection. An objective function that immediately allows a trivial solution for record points. Lack of compiling with optimizations. Changing what libraries are allowed partway through (more accurately, some server instances allowed the library, but they all went away by the end). Too basic algorithms. That said, I did have a lot of fun, especially having free reign to use ML and NLP and math libraries. But even that just made me realize that ML competitions tend by nature to be less about the "Aha" moment then they are about getting all your ducks in a row and doing the appropriate due diligence, and I might as well spend that time consulting for hedge funds.
Or companies could just ask for code samples. A resume is just a summary of experience, not sure how programming challenges would show the person can maintain a working relationship in a company.
58 comments
[ 10.0 ms ] story [ 112 ms ] threadI hope the employer who hires me doesn't mind if I do all that on the clock!
Full-stack indeed. :(
1.) The candidate shows up with a effectively all the required skills. Predictably, logically, the candidate has a roughly average level of skill across the range on average.
2.) The interviewers decide the candidate won't do because he/she is 'only' average in some area that should have been one entry on a short list of required skills in the first place.
Either hire for the skills you actually need and be specific about it or hire the smartest, hungriest people you can find and be honest about what they'll be expected to learn.
3.) Applicant is hired, and paid criminally below-market with above-market stress, because the types of people who explicitly hire "jack of all trades" are generally trying to save money.
And while contributing simultaneously to github, stackoverflow and others community projects could be cumbersome for you, positive externalities are immense. The whole ecosystem benefits from this.
Even if this burden is too heavy, there are always options of 4-day working week, freelance work, project-based work, etc. etc. Programmers are lucky to enjoy most liberty from their profession.
So come on, don't complain, you DON'T HAVE to do anything, unless you want to be the very best.
Sometimes it gets a little ridiculous on the requirements/wants on job postings. Some companies just ask for the world; it can lead to impostor syndrome and it can be stressful.
Software developers here are competing for the best positions on the market, so why the surprise about the requirements?
The opportunities are plentiful for experienced programmer who doesn't play the credentials game and agrees to earn a little less.
I don't just question the ever-growing list of must-have profiles and time required to maintain them, but their value in general.
It feels like a sort of arms race to engineer and automate hiring.
It's generally agreed that actually sitting down and talking with a person is strongest source of signal [1] in the hiring model, but rather than focusing on that the model is continually stuffed with a growing set of noisy variables (profiles).
At what point does this filtering start identifying the best profile builders as opposed to the best or brightest employees? Does hiring become a full-on game of prep services and checklists of activities along the lines of college admissions?
1: http://www.nytimes.com/2013/06/20/business/in-head-hunting-b...
The first automation software for the full hiring process is a going to be a very well marketed solution, not necessarily an effective one.
Ideally, I would see less of a need for specialist recruiters as tech has made connecting with appropriate hires simple enough for someone from the team the hire will work with to conduct interviews.
There's already an element of SEO-style optimisation in resumes as it is. I have a few outside-the-box approaches on how to get around this and I'll be doing a Show HN once I have something together :)
Thankfully HackerRankX seems about as likely to catch on as Klout.
Anyway, I'd submit that's the real threat here... this sort of thing is easy to game, and you are talking about the group of people most capable of gaming even the smartest AI algorithm... The top end of the "leaderboards" is going to be a list of cheaters pretty quickly.
Scala, Clojure, Erlang, Haskell, Ocaml, Lisp, F#.
not the most popular languages.
Anyway, where the hell are you seeing the lambda calculus in JavaScript (ignoring that "it's based on lambda calculus" is probably as useful as "computers usually work with 0s and 1s")?
Aside from Dylan, J or Scheme, I can't think of anything else to add.
You might be able to make an argument to add is ES6 with a pile of additions to Function.prototype, but FP in JS isn't as natural as other languages.
Seems more elegant to use what exists than to ask recruiters to come up with a custom challenge. If they can't make realistic asks at the moment it's unlikely they can create realistic tests on HackerRankX either.
I'm still waiting to see the first job that requires 5 years + experience in Swift. Shouldn't take more than a few months.
I'm not saying these tests are totally useless. Obviously it demonstrates a dev's ability to code within one or more languages, but they're hardly enough to make a hire off of. When it comes to building actual product, these tests always fall short.
However, I think you are right in that most of them are useless because they don't focus on that sort of detail.
EDIT: Okay, I had a bad example. Sorry.
Many of the questions I've been asked in interviews are more or less, "Someone had to do X at some point as part of our product, how would you do it?"
The goal is to figure out if you are competent at solving business problems logically that are relevant to the business in some way.
99.9% of engineers at Amazon are writing, well, downright normal code.
It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce.
It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
Think of it as a more relevant FizzBuzz.
> It seems silly to test someone on something they will never, ever run across in the course of their duties, and which you reserve for only a tiny, select portion of your workforce. > It'd be like requiring all the waiters in a restaurant to also be qualified chefs. Yeah, cooking is the core of your business and highly relevant to the organization as a whole - but not to this position.
Is a naive solution to TSP really that hard?
Why do we want a naive solution to TSP? The naive solution to TSP has no applicability to anything - it's purely a thought exercise in the same vein as "why are manhole covers round" and "how many jelly beans are in this jar".
Remember that 10 years ago we thought those questions were "predictors" of programmer ability. Turns out that was full of crap.
So your notion of testing someone's suitability for a job is to ask them to solve a problem unlike any they will encounter in the job, expecting them to generate a solution that would be wildly insufficient even if the problem was relevant to their job?
In the mean time I rarely see companies testing for abilities that are used on a regular basis in these jobs: the ability to architect, knowledge and familiarity with best practices and design patterns, writing testable code, etc etc. All of these skills are far from universal, but yet we spend no time ensuring they're there. No, we blow the valuable 45-60 minutes we have with candidates twiddling around with TSP.
We ask candidates simplified things from what goes in IRL, because we don't have all day. That can't be helped - the least we can do is make sure what we ask is actually a simplified version of what the candidate will be responsible for, instead of simplified versions of things the candidate will never touch, ever.
> "Proving people know how to implement an algorithm [even one that is a pretty bad solution to TSP] seems relevant to me."
It doesn't to me. More accurately, it doesn't seem relevant to the majority of coding jobs out there.
In reality the number of jobs that ever involve implementing real algorithms is really low. Even in research-heavy companies like Google it's limited to a small subset of employees in a small subset of teams.
The vast majority of everyone will not write a substantial algorithm from scratch in their day job. Ever. So why are we testing for this ability?
Uber, like most tech companies, is an integrator. Their need to write complex algorithms from scratch is practically nil. Looking for algo synthesis ability in a candidate is a waste.
The problem is that, as with most tests, these can be gamed. Every programmer under the sun now knows about FizzBuzz. So now there's an entire art dedicated to the craft of inventing new programming interview questions. Inevitably, the questions which were supposed to be the antithesis to trivia start to become their own trivia.
Really, though, it's just a hard problem. There is no easy way to really accurately assess a programmer you've never met or worked with. Your specific questions could easily turn down an incredibly talented programmer who just hasn't done much front-end work before but could pick it up in a few afternoons.
There are tasks/competencies that can only be tested via manual intervention. But things that can be automated are in our roadmap.
They're also easy to fake. If this became widely adopted, you could hire someone on Fiverr to boost your score.
The point to note is that there is (usually) a job on the line, and a programming challenge is not the only part of the screening process -- it will usually be followed by a live coding session/interview. So the question is would someone risk gaming the process, when the chance of getting caught is high. We think not.
Also I don't see bigger companies adopting these anyway. They will have their own way of hiring and is usually more relaxed. All these hurdles just make an experienced candidate cringe more applying to a startup.
Despite the somewhat sensationalist headline, the article itself actually seems to be pretty realistic (aside from the "and in a year from now, HackerRank will be like that" comment).
To me, it seems like another tool people who are not good at networking can use to distinguish themselves.
Even nowadays, no more or less respectable university will let you graduate with a CS degree without learning at least the basic ropes of graph theory (more than rightfully so). If this is representative of their priorities, I am left wondering if their ranking will actually be of interest to any organisation looking for competent programmers (as opposed to bedroom-trained PHP/Python hackers).
The other company asked for some code samples of work I had already done and then I walked them through it to show that I understood it and what design decisions and trade-offs I had made and why.
The company that asked for code samples ended up making me an offer and due to lack of time and the fact that another company was already interested in hiring me, I never even continued the conversation with the coding challenge company.
It seems that if a coding challenge is trivial then it doesn't really prove much. There's probably an answer for something close to it online and even if the candidate did solve it on their own it's still too trivial.
If a coding challenge is complex enough to actually test a candidates range of knowledge then it's often not worth it to complete the challenge for a possible job opportunity.