I wonder how much of this is due to novelty factor. I doubt that one could reach such good numbers, if asking for solving puzzles becomes a common practice.
There is something to be said for the novelty probably getting more interest, but I think that giving puzzles around the real work the hire will perform tells them as much about your company and the problems they can help you solve, as it tells you about the candidates ability to do so. A win/win as they say.
I already ask about 10 technical questions in each of the interviews I conduct, and having a problem online for each candidate to hack on would be really useful.
Now that I think about it, I may even consider using something like this to drive internal improvement projects. Some people just love the challenge, and the 'a' people always step up, you just have to ask.
Yes, this is a bit annoying. First he relates a not entirely interesting story about his HR problem, then says he solved it by including an interesting puzzle. OK, cool story bro, NOW WHERE'S MY PUZZLE?!?!? I'm not looking for a job, or looking to hire, but like most Hacker News readers I like puzzles. Now I'm angry and have to find a puzzle somewhere else.
But I'd suggest that except in special cases, posts with titles like mine are probably not worth reading if you are uninterested in hiring or finding a job.
Well, you never can tell. When someone says they've found a novel test for weeding out bad applicants, that sounds interesting. When the test is described with a handwave as an interesting puzzle that the best applicants leap at the chance of solving before reading the rest of the interview.... and then you don't even give us the puzzle, but rather launch back into the HR aspect... well, that just seems cruel. It's like reading a murder mystery where the detective details how he figured out who the murderer is, and then never tells anyone. From my authoritative position as a random anonymous internet jerk, I assert that if you allude to a puzzle, you ought to at least include it.
I am apparently puzzle obsessed today. Slow day at work.
Providing you're testing the actual skills that you want, this works well.
But if you're throwing out maths puzzles when you actually want people to write Java business logic then I can see you ending up with the wrong people.
The math puzzle we used required only prime numbers and modular arithmetic. Any competent programmer could wikipedia it and solve in 20-30 minutes (and many did).
It seems like this criticism should apply to FizzBuzz too, and that's why I'm inclined to dismiss it. (Specifically: how often does a Java business logic programmer invoke the modulus operator?) But I think I would agree with a more general point of avoiding difficult Project Euler style problems in favor of simpler ones that nevertheless need a written program to solve (or hardcore paper and pencil math skills).
FizzBuzz isn't about creating a correct compilable piece of software in a particular language.
FizzBuzz is useful for finding the people who cannot even begin the problem; the people who couldn't craft even a vague approach in pseudocode. Some people can't sketch out a few loops and modulus tests.
You know that the person who isn't sure of the precise syntax of a modulus operator can pick it up. You also know that the person who is baffled by FizzBuzz probably will need too much hand-holding.
The only jobs that caught my eye were the ones that offered some type of challenge to apply (Apply Via API, Etc.) If I'm going to end up talking to someone from Human Resources that's going to ask me the "Man Hole Cover" or "Lightbulb Question" then forget about it, I'm not interested in working in more stuffy corporate environments.
-(void)fizzbuzz
{
int i = 1;int t = 3;
int f = 5;int h = 101;
int z = 0;
// here we do a for loop using i and incrementing it until it's equal to z
for (i = i; i < h; i++)
{
/* we check if both i/t is equal to z and if i/f is equal to z
if both are true we log fizzbuzz in the console */
if (i % t == z && i % f == z)
{
NSLog(@"FizzBuzz");
}
//check if i/t = z and log
else if (i % t == z)
{
NSLog(@"Fizz");
}
//check if i/f = z and log
else if (i % f == z)
{
NSLog(@"Buzz");
}
// if none of them are true we just act like we're learning to count
else
{
NSLog(@"%d", i);
}
}
}
Do you want a critique of that? I'm mainly interested because I'd be interested in a critique of my critique... We use Fizzbuzz as our lowest bar question and I'm curious to know what people consider significant points and what are not.
Does it work is the most important thing by a mile but if you've asked them to wrie production ready code then you've got to ask "how would I feel if one of my developers checked this into version control and how would it fair in a code review?"
Maintainability, readability, efficiency and most significantly the candidates ability to explain why they took the approach they did.
I wouldn't hold any approach against someone so long as they could talk intelligently about the trade offs and why they'd done what they'd done.
Even really simple problems are a chance to see whether someone thinks about things.
Shouldn't it say until it ceases to be less than h?
Your comment supposes that the loop will increment without interference. While you'd hope that was the case, it's not necessarily true.
Obviously with the loop as coded your comment is correct (that'll be the first condition that terminates the loop) but it could raise a question about whether comments should reflect the routine as written or just the section they're immediately referring to.
Or indeed whether comments for something so self evident are even useful or necessary... Something I'd definitely ask in an interview if I was presented with this code, not looking for a specific answer, just looking for some indication that the candidate had (or could come up with) some vaguely informed thoughts on the subject.
I'm in the readability camp, I'm one to comment my code even if it's something simple and I also like to use verbose variable names. I always keep in the back of my mind that if someone had to pick up my code without knowing how anything was implemented they might appreciate my efforts (even if it's just a personal project that I don't plan on anyone seeing). You can't please everyone I'm sure for as many people that like it compact there are people that like the whitespace.
Pertinent passage:
FizzBuzz was presented as the lowest level of comprehension required to illustrate adequacy. There's no glory to be had in writing code that establishes a minimum level of competency. Even if you can write it in five different languages or in under 50 bytes of code.
Yup, this worked really well for us too - we asked candidates to submit a 20-line "binary chop" search over a given data file (well, that was the solution, we described the problem and restrictions such that that was the only right answer). All the interviewees were good when we did that, and it also seems like a good test for recruitment agencies (i.e. which ones understand what we're trying to do).
We did get one answer that was both incorrect, too slow, and massively over-engineered. I got emailed a multi-megabyte Rails project, boilerplate, tests for correctness and all. Bit It Didn't Solve The 20-line Problem. He was the guy we wrote the test to avoid!
I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them. People who've taken Haskell or even Scala into prime-time production are extremely rare. People who would be excited to try, and who could be trained up into that role are a lot more common.
I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)
"I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them."
A good way to do this is a humans.txt, whose purpose is not just to credit the developers but also the technologies in use. It's funny that a lot of companies are crazy about hiring right now, but haven't taken 30 minutes to add this simple file to their web server.
I'd go into management before I'd work in Java or C++ full-time.
I suspect that statement is more absolute than you intend it to be. Would you not work on a new language if the compiler for that language generated C++, which meant that the language's runtime was implemented in C++? Which, of course, implies that you will have to write a decent amount of C++ code.
If you would work on such a project, then I suspect you mean "If the problem and solution itself are not novel, then I have no interest in using Java and C++."
Fair point. Actually, you've nailed it. C++, for example, isn't some horrible language without purpose. It has a niche that it fills well (although I'd prefer C) but most C++ jobs aren't in that niche.
My experience is that language adjustment is up to a 0.5-point drop (the scale is here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) in an engineer's ability. This is on a 3.0 scale where most engineers are between 1.0 and 2.0, so that's a huge change. Compensation tends to be linear in proficiency but exponential in value-add. 0.5 points is about $50,000 in market pay but a 2-4x (depending on the type of company) difference in productivity.
Sometimes it is worth the drop, such as when one's learning a dramatically new and more effective approach, or solving a fresh class of problem (e.g. real-time software, which is done in C). That isn't the case with enterprise, "pattern"-afflicted Java. Instead of solving the problem appropriately and well because I know how to do it, I have solve problems inappropriately and badly because I don't know the crappy idioms that people invented because they don't have appropriate tools.
On that scale above, I put myself around a 1.7. If I set foot in a typical C++ or Java shop, I'd drop to 1.2 or 1.3. Which means I'd risk falling to the level of a junior programmer and being typecast that way. Which means I wouldn't get the benefit of the doubt if I say, "This project wouldn't be complete shit if we'd done it in an FP language" (in some nicer words, of course). Your first 6 months determine whether you have technical credibility or not. If you don't, you can't blame the tools, even if bad tools are (as they often are) the cause of the problem. The unenlightened tend to think of all tools as equally good (all languages are Turing complete). No, not so. A 3x difference in CPU performance may not matter for most problems, but the 3x difference in human performance that hinges on whether one uses the wrong tools certainly does.
The drop to 1.2 is something I'd tolerate if moving to something genuinely interesting, knowing it'd only take a year to pop back up to the same or even higher level. An example might be high-performance C++ in weather forecasting, bioinformatics, or artificial intelligence. I'd gladly become a junior again for a year for that kind of stuff. On the other hand, most enterprise Java/C++ bullshit isn't really worth learning, not for a creature that has 60 years left to live instead of 600,000 (#ShitYouCantSayAtWork).
So yes, my language bigotry is more directed at the typical C++ or Java job than at the language itself.
I don't think I'd take an enterprise job. However, When someone mentions C++ my assumption is we're talking about fairly low level programming. Java though, I tend to think of as being strongly enterprise-y.
I assume my reaction to C++ is that way because my C++ experience is entirely in embedded software / consumer electronics (Strongly C-driven but way more C++ than you might imagine).
I'd suggest that you're being a bit ideologically biased against Java. Ain't no easier language to do concurrent programming, aside from Clojure (jvm), Scala (jvm) and I hear Erlang (don't know much about it). Every serious project in Clojure or Scala that I've seen ignores their concurrency stuff and goes straight to Java.
It turns out that if you implement a bunch of Callables and ExecutorServices with LinkedBlockingQueues or SynchronousQueues.. it's basically functional programming. Don't get too hung up on the syntax.
You're talking about at least 3 distinct models: direct thread management, with improved locking routines (java.util.concurrent, see recent Prismatic thread on HN), actors/green threads (Akka and erlang BEAM), and STM/guarded read/writes (clojure, and you should read about Microsoft's attempt to graft STM onto C#, Joe Duffy's blog)
>Actors and STM are great thought experiments but real-world production stuff tends to always circumvent them for the java.util.concurrent stuff.
Its a mixed picture, if you want to be honest. There are places like bofa where I've pushed into production a cvar processing module on 500,000 loans which uses only actors. No low level tinkering with util.concurrent, no Threadpool setting, no Executors, no synchronized anywhere in my code. In the past at GS, I wrote concurrent code where I explicitly specified the number of threads & we had a ThreadPool manager & internally it used the util.concurrent classes. At LinkedIn, I believe they started with actors & switched parts of it to Doug Lea's framework ( which was the sort of the foundation of util.concurrent). I have seen financial shops where they do half & half - the quants seem to like actors but the swing programmers like to have control on threadcounts, & they tend to do things like naming threads & creating parent-child relationships so swing can paint reliably & they can debug threads by name when it doesn't. I did the same in the past when most of my work was on the swing side, creating custom blotters to show trade tickets & the like. But now I primarily do quant model building work with actors & no more util.concurrent stuff. If you want to construct a lattice model to price structured options on a 100 different strikes per month for 5 years in advance, you can find no better friend than an actor. Actors are certainly not a thought experiment, very much a reality.
Great post, yeah I overstated a little bit and actors are certainly valid for certain cases. I was just reacting to what I saw as an overstatement on the other side.
You've made this point twice now ("Every serious project in Clojure or Scala that I've seen ignores their concurrency stuff and goes straight to Java") and I'm curious to hear some examples as well as your understanding of why this would be.
For examples, the most recent ones I've seen are linkedin's kafka (scala), and bradford cross's recent post regarding clojure at prismatic (and other posts I've seen regarding both).
For why, I'd assume that they found the abstractions were getting in their way. java.util.concurrent is sort of the minimum abstraction (IMO) between nicer constructs and OS-level threads.
My problem with Java is the language. Scala and Clojure are great.
The platform is excellent technology. The language is mind-numbingly boring.
When Scala and Clojure projects go "straight to Java", it's generally for specialized purposes (where those languages don't have native maturity) or in already-discovered bottlenecks. I think some degree of that's inevitable. Like database tuning, it's just something you have to do when you get to a certain point where (a) your problem is well-defined, and (b) shaving 20% off latency is worth doing. I consider these good problems to have. If I have to rewrite a class in Java for performance reasons, fine. I'd never start a multi-thousand-line project in Java, though.
Also, I really don't think anyone mature minds writing code in underpowered or difficult languages. C++, I find, is actually a lot of fun to write. It's reading other peoples' code, in large amounts, that becomes intolerable in these languages. If I have to spend a day to answer a simple question like "how do I initialize this object?", that's bad.
Interesting fact: Closures were cut from the 1.0 release for time. Gosling wanted them in, but Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes", so they were far from essential for that domain. The folk history (which I once believed) of Java being deliberately underpowered so that teams of hundreds of mediocre programmers could be stapled together in it is false. It was, however, co-opted by the "let's commoditize developers" crowd in the late '90s, and we've had to deal with FactoryVisitorDelegates ever since.
> Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes
from my time at sun, i remember speeches by Gosling that included microwaves and refrigerators ( i vaguely recall an example of the fridge telling you via java applet that you had run out of milk ). the hotjava thing ( web browser written entirely in java http://en.wikipedia.org/wiki/HotJava) created some anticipation around Word & Photoshop & Powerpoint written in java, which were brought up when Sun hosted a gigantic industry conference in Vegas. The Java Chip ( picojava http://www.jopdesign.com/perf.jsp ) was supposed to revolutionize the industry by running java bytecode natively. Honestly I think the dumbing down began once EJB's crept in. There was some nasty visual shit called the Beantool which automatically inserted getters & setters by mandating every EJB to be a Bean, and from then on things got downhill really fast.
If reading code is your problem rather than writing code, I find that Java excels. Most of the complaints are about writing code in Java.
You know how awesome that type inference is in Scala, and how stupid and annoying it is in Java to type stuff like Map<K,V> myMap = new HashMap<K,V>? I totally agree with how annoying that is. Here's a related example in scala.
def myThing = something.getSomeThing()
What is the type of myThing?
Java has weaknesses for sure but reading code is actually one of its strengths, precisely because of the boilerplate. Note that I'm not defending EJB or any other atrocities here, just the core language.
Lastly, I agree that syntactically-nicer closures would be better for Java. But it already has closures,
final Something variableImClosingAround;
Runnable myClosure = new Runnable() { public void run() { variableImClosingAround.doSomething(); }};
Yes, the sad all stuff which has been used to argue over a decade now. Do you think people are stupid? I don't think there is anyone left who didn't hear these straw men. People moved on, maybe you should too?
i actually find c++ more pleasant to read than to write. when writing c++ i am constantly annoyed at the tedium involved in things that would be a few lines of ruby/clojure/ocaml (my top 3 hobbyist languages), but once the tedium activation barrier is crossed and the code is written, it usually ends up pretty clear and straightforward to read and maintain. this is especially true in google, where coding standards are taken seriously and refactoring is a culturally valued operation (that was actually one of the main draws the job had for me).
scala> (1 to 10).par.foreach(x=>print(x*x+","))
4,9,1,49,36,16,81,25,64,100,
Let me know how you'd parallelize the above in Java, how many lines of code it'll take, what syntactic constructs you'd end up using, what libraries you'd import, and finally your my mental model would be - in terms of locks and semaphores and so forth. The thing with scala is that concurrency - both data parallelism ( above example ) and task parallelism ( using actors in scala, a slightly longer example) are baked right into the language. A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency.
Ideological bias is never justified. On to the justified bias:
For that problem, I'd just do it serially, because it's only 10 numbers.
Oh, it was an example for really big sets of numbers? Like really really big? Then I'd skip the .par.foreach and deal with java.util.concurrent directly, because it's worth my while if I have a serious problem worth optimizing.
"A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency."
A newbie in concurrency needs to understand how many threads are in their threadpool, or else they should just do it serially. I'm not talking rocket science here, it's just a single integer argument.
Tried that. Now I'm back coding Java, Groovy, JS and looking into Scala.
The perfect management role for me is one where I can shape product direction, mentor some younger guys, and still code. The problem is that in all the companies I've seen, management ends up being I sit in meetings all day and get completely cut off from coding or even having deeper tech discussions. I just don't feel like I'm still creating at that point.
I think it's the 2 surfers outpaddling the shark scenario: you don't have to be great at hiring, just slightly better than other companies in your demographic(geographic and "itch yoru scratching". You do have to be great at dev retention.
I currently write Scala at work. As far as I can tell, it's a lot like C or C++, except that you have to use while loops instead of for loops and you have to perform unholy incantations to fake arrays of structs. If you don't do this, you can either try to allocate a few hundred million tuples and OutOfMemoryError, or you can allocate one array of a primitive for each field of the struct and give up on the idea of having decent cache locality. It's less capable of TCO than gcc, which is pretty hilarious for a functional programming language. Generic programming in Scala is lots of fun until you try to use an Ordering[Double] or an ArrayBuffer[Double] in a critical path.
This is true. The only things that are trivial to fix are the absence of for loops and the use of ArrayBuffer[AnyRef] to implement ArrayBuffer[Double]. It might also be possible to make TCO of mutually recursive final methods work, but it's probably more trouble than it's worth.
Regarding ArrayBuffer: It is not as easy as it looks like, because of the conflicting nature of Arrays vs. Generics (yet another JVM problem).
Amusingly I tried using Manifests to do exactly that last week. I hit the wall because the Manifests seemed to spread arbitrarily. Will try with TypeTags/macros when they become available.
Matching problem. Hard to find jobs in good languages, because they're rare. Hard to hire in good languages, because the people are rare. Fuck Java/C++.
I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them. People who've taken Haskell or even Scala into prime-time production are extremely rare. People who would be excited to try, and who could be trained up into that role are a lot more common.
I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)
This reminds me of a stupid NH job posting where they said they want, and then there were two hashes:
Googling for those hashes revealed in the first snippet first standing for "good", second for "developers".
The message was that anybody who has the common sense to google an unknown string is qualified duh?
The first puzzle job ad is nice, but it quickly wears off, and would start to bring bored "oh, not again" feelings. And many people just aren't good at making puzzles.
If you are looking for a good candidate, and ready to spend the time doing it, why not look around the developers who have done something in the field you are active on.
For example, if you are looking for a Front-End guy, why not look around web magazines, Web 2.0 products and services, Github, Dribbble and pick the guys you like and contact them.
Good puzzle, as opposed to a bad one, is an excellent attention getter for good programmers. They can judge you and your intellect by your puzzle and be attracted to work with you. Ie. your input makes the recruitment a two-way street.
The HUGE peer-reviewed professional literature on industrial and organizational psychology has many articles devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but still below the .50 level, standing alone). One is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Another is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Each of these kinds of tests has about the same validity in screening applicants for jobs. Neither is perfect (both operate at about .4x level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Companies outside the United States are regulated by different laws.
Schmidt, F.L. & Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings,” Psychological Bulletin,
Thanks. I had read the conclusion I mentioned in monographs about psychology I was reading for other research interests, and I'm glad you cited a key review article. I found the article's full text
The author's point, however, is wider than what you discussed. Filtering is one thing, which it sounds like the literature you mentioned covers. But filtering makes the assumption that the candidates have already decided to apply. The author of this piece found that he got significantly more applications when he added a test. So it's not just filtering, but also advertising.
I applied at Company A that had an test similar to an IQ test, and they returned back that I would be better suited to an entirely different job than what I had applied for. It also took them 3 full weeks to get this information back to me, then another 3 months to determine that they wanted to interview me for the original position I had applied for. At that point, I was already working at Company B making 10k more than Company A said I would be eligible to earn after working at their company for 3 years.
Yeah, I'd pretty much say that the IQ test was pointless in this case and luckily that was the only company I had run into using it.
EDIT: Forgot to mention that the test also took 3 hours to go through and had no limit on the questions that you would get. There was no possible way you could finish every question on the test.
How specific to the job does the validation of IQ test to job have to be to pass the legal test?
Could someone do an (awesome) academic study which shows that all programmers benefit from g, or at least that all developers in x language do? Or add some specific problem where g is hugely useful (algorithm design?) and then make that problem a requirement for many programming positions?
There would be a huge benefit to technology hiring if we could start using a cheap, standard test result vs. relying on proxies like school admittance (where they can screen for IQ), or expensive work-sample tests (which burn more of the applicant's time, selecting against busy/good candidates).
I'd rather advantage poor, possibly non-US, high-IQ, busy applicants over either graduates of elite schools (correlated with rich) or people willing to spend the time to take a 4-40 hour unpaid assignment.
"we hadn’t really planned for so many good applicants so we were very slow to respond to everyone"
So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.
The key takeaway for me is not how people filter their applicants on real criteria, it's what they do when they have a large amount of seemingly equally skilled applicants. The original article demonstrated the 'unlucky' solution, I'm not sure what this article demonstrates since they still got too many applicants and they're spending time responding to all of them.
Having a lot of highly skilled applicants is a much better than having a lot of unskilled applicants. In the former situation, you pretty much can't go wrong...even if you use completely arbitrary filters, it doesn't really matter.
This article demonstrates a way to get a lot of highly skilled applicants-that's extremely useful.
So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.
The original article talks about finding 4-5 people worth talking to out of 100 people, 50 of whom can't fizzbuz.
We were in the situation of 20 interesting people out of 40 candidates, all of whom could fizzbuz.
If it were truly unmanageable, we'd probably have used weak Bayesian filters. E.g., degree, rockstar ninja, etc, all the things Raganwald originally advocated against using. I gather this is what Google and Facebook do.
I call BS on this one. I'd even say that this will filter out the top level of folks (†).
If you give me a synthetic puzzle to solve I wouldn't bother applying to your organization. It's much better to craft a problem to the specific domain that your company is dealing with. Then you can test for both domain expertise and problem solving ability in one fell swoop with the extra bonus of it not seeming pointless.
† The important addendum on this is that you're not going to have the top level of folks sending out blind applications unless that person is in the process of making a lateral jump into another sub-field. And even then probably not. But if you give a top applicant a synthetic puzzle after your recruiting contacts them, expect a large portion of them to thumb their nose at you. Similar note on top folks, though this is mostly pedantry -- the top OSS projects aren't on GitHub and similar. The largest and most prestigious of the OSS projects host their repos themselves.
"the top OSS projects aren't on GitHub and similar"
True, however if you're a contributor to a top OSS project, you'd tell me that when you apply, and send me an example of one of your contributions that you're particularly proud of (at least I hope you do, otherwise I won't know!)
Puzzles and challenges are useful when the candidate doesn't have any good examples of their work that they're at liberty to share with you. I'm very wary of hiring someone without seeing any of their work. We've been taking this approach for awhile now, and it's worked out well for us.
To clarify, I wasn't discussing recruiting Linus or Guido.
I'm actually more likely to apply if there is a puzzle. I've sent a few resumes out blindly, it usually goes into the black hole. If there is a puzzle, I figure my odds of getting a response (assuming I get it right) go way up.
Of course, if a correct solution results in a recruiter replying and asking me to submit to resumator, I'm probably ignoring your company in the future. (Yes, this happened to me once.)
"Not trying to hire Linus" is a bit of a straw-man. There are a lot of folks that fall between "Linus" and "have to actively look for a job". Probably tens of thousands of people in the programming field.
The important point that I'm getting at, however, is that the dynamics of hiring shift dramatically as you move up the skill / connectedness scale. As you move towards the higher percentages potential employees are not competing for employers, but employers are competing for hires. And the dynamics of hiring are dramatically different at that point. If you're designing heuristics for hiring, you don't want to create a low-pass filter, and I believe that synthetic puzzles are exactly that.
The point of raganwald's post was on the surprisingly high cost of false-negatives. What I'm postulating is that your suggestion is just yet another way to reduce the false-positives rather than a way to reduce the false-negatives.
If it increased the number of overall respondents as well as the number of respondents who were above a certain bar (passed a phone screen), then even if it created other false negatives, it still seems to have helped.
Good developers don't just go around submitting their resumes to every job in existence, and not filtering out false negatives was insufficient in and of itself to produce a good applicant pool. If doing something that produces false negatives (which, yes, are undesirable) nevertheless convinces enough developers to consider your company who never would have considered it otherwise, then it seems like the advantages have outweighed the disadvantages.
Obviously, an even better scenario would be to produce the same level of developer interest without also producing any false negatives. Perhaps by investing sufficient resources in networking or in making his company famous to other developers, he could accomplish this. This would still be a possibility to explore in the future.
But just because his tactic has not produced the best possible outcome does not mean it has not produced a positive one.
Most top level folks rarely respond to job ads, as they tend to get hired away before they'd realized they were ready to move, so I wouldn't worry too much about filtering them out in the first place.
I can't tell if you're being intentionally dense or just naive.
Linus copied the kernel tree to GitHub when the kernel git repo was compromised. It is not the main repo. Of the others, only Rails and JQuery could even be reasonably be called "top" projects (Bootstrap? Seriously, you list a 7 month old project with 2000 commits?) and even those are a stretch. Major projects are well into the hundreds of thousands of commits. It's ludicrous to even muse of Bootstrap being in the same league as e.g. Mozilla, WebKit, et al.
"Top" projects are better defined by the number of users than the number of commits. By this standard, Bootstrap is a very successful project.
If you define success by the number of people who have actually checked out and built the code, Bootstrap is vastly more successful than Mozilla & WebKit.
I don't understand. He tried one approach that didn't get many good candidates as a percentage of total applicants. He tried another approach and got more candidates with a higher percentage of good ones. One sample is far from conclusive but I don't see any BS here. Maybe its not an optimal solution for all top-level candidates but that doesn't mean its not a sufficiently good one for many.
Exactly. There's nothing stopping OP from using this method to get a bunch of applicants while also using other networking methods to reach out to the current top talent.
Why do these "top applicants" not want to do coding pre-screens? If I do a coding pre-screen, it means I will get to work with only people who are capable of doing it. That bar may be high or low, but it is often better than a bar that does not involve any actual programming. It also means that I won't spend a huge proportion of my time with the company talking on the phone to people who can't fizzbuzz, which is a pretty good perk if you ask me.
So far, 2 of 4 companies I have applied to have had a coding pre-screen and 1 of 2 companies that I have worked for have had a coding pre-screen.
Part of the trouble is that the best people are well employed and busy working. They are in high demand. They aren't going to do puzzles or beg you for a job. You have to find some way to meet them and convince them to work with you.
It is an interesting idea, but this may be limited in its use for small startups that hire only 1 or 2 programmers or for someone willing to wait. My reasons being: Only a small sliver of developers visit sites like HN. They are mostly busy working, going home to kids and such. Getting the news out that you are hiring, to this larger set of candidates is a real problem. Which is where the traditional methods (including job web sites) become crucial. If you use any job borads (like Monster), or a headhunter, it is hard to pipe a problem through them. They are not setup for this. If you had asked me to solve a problem just to find the email to apply to a job, I would have not done so, just because I do not have time to do it. If I were a kid out of college, or did not have time constraints, I would have loved to. But where I am today, I would rather send the resume in and talk to the hiring manager.
No discussion of rockstar ninjas, just saying exactly what we are looking for.
Like I posted in the "who's hiring", that doesn't work for me because it doesn't tell me whether you are what I'm looking for. What do you do? Why should I care about you?
As someone who occasionally solves hiring problems with little intention of applying, it was nice to hear that my hobby is less annoying than I had imagined.
How to hire C players: Do what everyone else, including recruiters and HR departments, are doing.
How to hire B players: Implement serious recruiting processes and plan to spend a lot of time on it. Get creative with hacks like OP's.
How to hire A players: Identify them through never ending networking, research, and keeping your finger on the pulse of the industry. You'll never find them the same way you find B and C players.
I think hiring A players is actually more complex than that even. To get A players, or even the best B players, the most important thing you can do is to make your company somewhere those people will want to work in terms of the work, the culture and the facilities.
If you're doing that then it's possible they'll find you, if you're not then nothing you can do is going to help.
It greatly depends on what you call an A player. Linus is extremely talented and well known, but in some tiny organisations his actual programming talent may not be above average. Most programmers working for the NSA are probably about what you would expect at a large organization, but I suspect they have a few people that would rock your world.
And the thing is Google, Microsoft, and GM also have some amazing teams simply though blind luck. The kind of thing where a few friends all end up working for a manager they like on some interesting little problem.
Some people have only worked on such teams and don't understand why so many projects fail, and others have never even seen them and wonder how complex software ever get's written. But, so few people have a truly understand the range from inspiring though dysfunctional people and teams out there that talk of A, B, and C coders / rock stars becomes almost meaningless.
PS: One of my favorite Real Programmers was a systems programmer for Texas Instruments. One day, he got a long distance call from a user whose system had crashed in the middle of saving some important work. Jim was able to repair the damage over the phone, getting the user to toggle in disk I/O instructions at the front panel, repairing system tables in hex, reading register contents back over the phone.http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/R... or the modern equivalent http://bellard.org/jslinux/tech.html
Yes, exactly. My advisor in grad school also managed 100+ PhDs in one of the national labs, and that was one lesson he imparted in almost the same words.
He went one step farther: If you have B and C leaders in your org, it's basically all over because few organizations are capable of taking the measures to correct that problem.
there are two classes of boss, those you look to for technical leadership and some degree of mentoring, and those you expect to do the actual "management" stuff so that you don't need to. for the latter category, i'm not looking for sheer brains so much as i am looking for good leadership qualities.
> and those you expect to do the actual "management" stuff so that you don't need to.
Sounds like your answer is: When there are items that smart people want to outsource, they often go work for people stupider than them rather than hire people to take care of it.
no, i think you're falling into the common technical-person trap of assuming that raw intelligence is the only measure of competence. i don't want to work for myself, and i don't want to hire people - i want to be competent at my job, and work for someone who is competent at theirs. that someone may or may not be more intelligent than me; it's largely irrelevant as long as they're smart enough to do their job well. (and i may or may not be more intelligent than they are; it should likewise be irrelevant to them as long as i do my job well)
Your boss can be smarter than you at his job, and successfully hire you, smarter than he is at your job. That's an odd way to word it, but the fact remains -- there are generally different skillsets and competencies involved, and you are looking to build or be a part of a team with complementary talents.
Not really. Leadership is about giving clear direction and encouraging others to rise to the occasion. That can happen if the "leader" is not as good technically as his subordinates.
More than it can happen, if your programmers are good it should happen.
A leader is spending a fraction of the time hands on that a programmer is. Given the developments in technology and the additional time the developers are getting with it, if they're not better than their leader in a few years, you've got the wrong programmers.
Smarter does not mean better leader. Often great leaders are indeed intelligent, but the skills that make someone a better leader do not necessarily correlate with being "smarter". Instead, great leaders possess the ability to motivate and read people in an effective way.
Conversely, you've probably encountered highly intelligent people that have the people skills of a bug.
In my experience it does. "Hire people smart than you" is a management ethos, which translates to delegate the hell out of a task.
But software development specially in small teams doesn't work that way. Many a times, junior programmers require direction and even help in completing their tasks. So it comes down to two problems (three, third one later)
1. Either you give someone sub-standard work to maintain that you be able to help them. This will lead to demotivation eventually
2. You accept you are not in position to help them. This generally leads to friction and a bad boss will try to hide incompetence by architecture discussions, meetings etc
Smartness is subjective. Which leads to the third problem. How can you assess skills of someone better than you? You can only go as far as grilling a candidate to your own competence. So a B, C player will only be hiring other B, C players and that will soon degenerate the organization.
What is 'smart' though? I think the mantra is basically saying "hire people who fill in the gaps". A good leader is someone who facilitates identifying and filling in the gaps, which has little to do with "smarter".
Management is orthogonal to code design/dev/maint.
No one (ok... almost no one) has no management nor coding skill, rather each of us has a mix of the two. The great organizations I've worked in had extremely smart people in management and in development, but the type of brilliance was different.
I am surprised how many people completely ignore this. I see tons of people blog on and on about the bright eyed, bushy tailed genius programmer out there who needs to be acquired. I see very few about companies actually trying to make an effort to look into their organizations and seeing how they can, if not try to bring their "C" players to par, at least bring their "B" players to "A" level[1]. Although I don't have any experience training other people (apart from improving myself), I am not especially convinced that software engineering and programming skills are hard enough or impossible that relatively smart people can't be with proper coaching be helped to excel at it [2].
Caveats:
[1] Implicit assumption that the person is actually interested in becoming better.
[2] I understand that this takes a "toll" in productivity because you could be out there building stuff instead of helping these people improve. However, once a company reaches a certain size, I should think there is enough "boring" work for the company to staff "B" players to handle those, while simultaneously making them spend another chunk of their time working on becoming better.
I don't know what GCHQ pays for programmers or mathematicians. I imagine that Cliff Cocks (invented public key crypto before RSA) didn't get the many millions of dollars that RSA did.
So; offer an environment where people want to work. Give them interesting work. Add a dollop of patriotism.
Work environment definitely matters. If the environment isn't any good, then the pay had better be great.
Ultimately, if you had an interesting product, a great environment and offered extremely competitive salaries then you would most likely end up with interest from more people in the "A" range.
The finance industry tends to offer way more for people with a strong technical background to do quantitative analysis work.
To use real world examples, no amount of money you offer Salman Khan now would pull him away from his non-profit Khan academy to go back to the hedge fund industry.
If you're going to point to your step 2, then that sounds like a tautology. "Attract good people who would make a move for more money. Offer them more money"
Silly money isn't a good recruiting tactic in my experience (from the original dot com boom days - which was the only time I ever had silly money available :-):
* It motivates B and C players more. So you get a lot more chaff to filter out in your application process. You also get many more D players (the ones who are knowingly incompetent and are actively trying to deceive you into recruiting them).
* The major motivation of A players usually isn't money (once you reach the "fair wage" stage). So you don't attract that many extra A players.
* Some of the A players who do get tempted by the extra cash discover that it's not quite as wonderful as they expect and leave for more interesting work. Wasting large chunks of your time and money.
* It may turn some A players off. There's a unwritten rule that you get more cash to make up for dull work. So offering silly money implies to some that the work is going to be exceptionally uninteresting.
The absolute best way I know to recruit A players is:
0) Understand the role that needs filling and the kind of person that needs to fill it
1) Have a really interesting job
2) Know how to access the A players
(I'm amazed at the number of organisations I come across that skip step 0...)
Silly money may not be a good recruiting tactic, but the importance of understanding what the market actually is for the A players cannot be understated. So to add to your excellent list
3) Be prepared to be truly market competitive in salary - your target A player may have a 20 year consistent track record of getting sh*t done, respect their sense of self worth.
Being immodest I consider myself an A player, when a startup offering "competitive salary and benefits" low-balls my current income by 50% they've wasted my time and theirs
Indeed I'd argue that it's an implicit part of performing step zero - understanding the role and the person. If you don't understand what your potential employee is looking for in a role - be it money, responsibilities, working environment, or whatever - then you're never going to find 'em.
For example I'm continually amazed at the number of companies who decide that they need an uber-geek with a ten year track record of delivering e-commerce platforms (or whatever) - and then go about recruiting them as if they were unemployed. Sigh.
Hmm. I remember telephone interviewing for this position. As one of the applicants in the first group(i.e No puzzle just a simple job description posted :-)). I don't remember it verbatim, but the couple of questions i remember was (1. Why did you decide to apply?(i.e: from the post) I mentioned that the post referred to a entropy reduction algo and implementation. 2. Do you like HPMOR? :-P
Just a reference. I don't have a strong opinion about the whole experience.
Before I look at a resume I send every applicant a coding problem with simple instructions. It looks very simple at first, but actually requires some thought. It is fairly real-world, but unusual. I even provide a sample test case, and the instruction that the solution must pass all test cases. It would take the ideal candidate less than half an hour to code and test. Acceptable candidates less than an hour. Despite having all the time and resources they need to solve it, 85% to 93% of the candidates do not get it. I don't even glance at these resumes.
It's not 85-93% not responding, but 85-93% responding with crap that took some effort to produce. Also if you're actually interested in the position, you're going to be doing more than an hours work (if that's what you want to call it) in the interview process. Lastly, it would be work if I were tricking people into doing something I used as R&D or actual production end-product. This exercise simply establishes qualification.
Lately these are contract positions, and being in a corporate environment here I'm dealing with an agency that deals with other agencies. I still do some direct (usually by Craigslist ad) recruiting. I only submit the screening task to people who have already responded to the position posting. Very few people, maybe 5-7% do not complete the task, and I assume they found another position in the interim.
195 comments
[ 2.6 ms ] story [ 211 ms ] threadI already ask about 10 technical questions in each of the interviews I conduct, and having a problem online for each candidate to hack on would be really useful.
Now that I think about it, I may even consider using something like this to drive internal improvement projects. Some people just love the challenge, and the 'a' people always step up, you just have to ask.
http://www.chrisstucchio.com/blog/2012/leaving_academia.html
But I'd suggest that except in special cases, posts with titles like mine are probably not worth reading if you are uninterested in hiring or finding a job.
I am apparently puzzle obsessed today. Slow day at work.
I have no idea if they are currently hiring, so please don't bother them with job applications unless you see a job ad.
(http://imgur.com/pwzgA)
((First Row and Fifth Row are not well defined, but that's not particularly relevant to the problem.))
But if you're throwing out maths puzzles when you actually want people to write Java business logic then I can see you ending up with the wrong people.
FizzBuzz is useful for finding the people who cannot even begin the problem; the people who couldn't craft even a vague approach in pseudocode. Some people can't sketch out a few loops and modulus tests.
You know that the person who isn't sure of the precise syntax of a modulus operator can pick it up. You also know that the person who is baffled by FizzBuzz probably will need too much hand-holding.
-(void)fizzbuzz { int i = 1;int t = 3; int f = 5;int h = 101; int z = 0;
Does it work is the most important thing by a mile but if you've asked them to wrie production ready code then you've got to ask "how would I feel if one of my developers checked this into version control and how would it fair in a code review?"
Maintainability, readability, efficiency and most significantly the candidates ability to explain why they took the approach they did.
I wouldn't hold any approach against someone so long as they could talk intelligently about the trade offs and why they'd done what they'd done.
Even really simple problems are a chance to see whether someone thinks about things.
Your comment supposes that the loop will increment without interference. While you'd hope that was the case, it's not necessarily true.
Obviously with the loop as coded your comment is correct (that'll be the first condition that terminates the loop) but it could raise a question about whether comments should reflect the routine as written or just the section they're immediately referring to.
Or indeed whether comments for something so self evident are even useful or necessary... Something I'd definitely ask in an interview if I was presented with this code, not looking for a specific answer, just looking for some indication that the candidate had (or could come up with) some vaguely informed thoughts on the subject.
However, yes it is huge it's Objective C with whitespace what do you expect? ;)
Why they do it one way or another is an occasionally interesting follow up question.
[Edit: Found a way to shave off three chars]
Pertinent passage: FizzBuzz was presented as the lowest level of comprehension required to illustrate adequacy. There's no glory to be had in writing code that establishes a minimum level of competency. Even if you can write it in five different languages or in under 50 bytes of code.
We did get one answer that was both incorrect, too slow, and massively over-engineered. I got emailed a multi-megabyte Rails project, boilerplate, tests for correctness and all. Bit It Didn't Solve The 20-line Problem. He was the guy we wrote the test to avoid!
http://haacked.com/archive/2004/07/12/google-mysterious-bill...
I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)
I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.
A good way to do this is a humans.txt, whose purpose is not just to credit the developers but also the technologies in use. It's funny that a lot of companies are crazy about hiring right now, but haven't taken 30 minutes to add this simple file to their web server.
I suspect that statement is more absolute than you intend it to be. Would you not work on a new language if the compiler for that language generated C++, which meant that the language's runtime was implemented in C++? Which, of course, implies that you will have to write a decent amount of C++ code.
If you would work on such a project, then I suspect you mean "If the problem and solution itself are not novel, then I have no interest in using Java and C++."
My experience is that language adjustment is up to a 0.5-point drop (the scale is here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) in an engineer's ability. This is on a 3.0 scale where most engineers are between 1.0 and 2.0, so that's a huge change. Compensation tends to be linear in proficiency but exponential in value-add. 0.5 points is about $50,000 in market pay but a 2-4x (depending on the type of company) difference in productivity.
Sometimes it is worth the drop, such as when one's learning a dramatically new and more effective approach, or solving a fresh class of problem (e.g. real-time software, which is done in C). That isn't the case with enterprise, "pattern"-afflicted Java. Instead of solving the problem appropriately and well because I know how to do it, I have solve problems inappropriately and badly because I don't know the crappy idioms that people invented because they don't have appropriate tools.
On that scale above, I put myself around a 1.7. If I set foot in a typical C++ or Java shop, I'd drop to 1.2 or 1.3. Which means I'd risk falling to the level of a junior programmer and being typecast that way. Which means I wouldn't get the benefit of the doubt if I say, "This project wouldn't be complete shit if we'd done it in an FP language" (in some nicer words, of course). Your first 6 months determine whether you have technical credibility or not. If you don't, you can't blame the tools, even if bad tools are (as they often are) the cause of the problem. The unenlightened tend to think of all tools as equally good (all languages are Turing complete). No, not so. A 3x difference in CPU performance may not matter for most problems, but the 3x difference in human performance that hinges on whether one uses the wrong tools certainly does.
The drop to 1.2 is something I'd tolerate if moving to something genuinely interesting, knowing it'd only take a year to pop back up to the same or even higher level. An example might be high-performance C++ in weather forecasting, bioinformatics, or artificial intelligence. I'd gladly become a junior again for a year for that kind of stuff. On the other hand, most enterprise Java/C++ bullshit isn't really worth learning, not for a creature that has 60 years left to live instead of 600,000 (#ShitYouCantSayAtWork).
So yes, my language bigotry is more directed at the typical C++ or Java job than at the language itself.
I assume my reaction to C++ is that way because my C++ experience is entirely in embedded software / consumer electronics (Strongly C-driven but way more C++ than you might imagine).
It turns out that if you implement a bunch of Callables and ExecutorServices with LinkedBlockingQueues or SynchronousQueues.. it's basically functional programming. Don't get too hung up on the syntax.
You're talking about at least 3 distinct models: direct thread management, with improved locking routines (java.util.concurrent, see recent Prismatic thread on HN), actors/green threads (Akka and erlang BEAM), and STM/guarded read/writes (clojure, and you should read about Microsoft's attempt to graft STM onto C#, Joe Duffy's blog)
Actors and STM are great thought experiments but real-world production stuff tends to always circumvent them for the java.util.concurrent stuff.
Its a mixed picture, if you want to be honest. There are places like bofa where I've pushed into production a cvar processing module on 500,000 loans which uses only actors. No low level tinkering with util.concurrent, no Threadpool setting, no Executors, no synchronized anywhere in my code. In the past at GS, I wrote concurrent code where I explicitly specified the number of threads & we had a ThreadPool manager & internally it used the util.concurrent classes. At LinkedIn, I believe they started with actors & switched parts of it to Doug Lea's framework ( which was the sort of the foundation of util.concurrent). I have seen financial shops where they do half & half - the quants seem to like actors but the swing programmers like to have control on threadcounts, & they tend to do things like naming threads & creating parent-child relationships so swing can paint reliably & they can debug threads by name when it doesn't. I did the same in the past when most of my work was on the swing side, creating custom blotters to show trade tickets & the like. But now I primarily do quant model building work with actors & no more util.concurrent stuff. If you want to construct a lattice model to price structured options on a 100 different strikes per month for 5 years in advance, you can find no better friend than an actor. Actors are certainly not a thought experiment, very much a reality.
For why, I'd assume that they found the abstractions were getting in their way. java.util.concurrent is sort of the minimum abstraction (IMO) between nicer constructs and OS-level threads.
The platform is excellent technology. The language is mind-numbingly boring.
When Scala and Clojure projects go "straight to Java", it's generally for specialized purposes (where those languages don't have native maturity) or in already-discovered bottlenecks. I think some degree of that's inevitable. Like database tuning, it's just something you have to do when you get to a certain point where (a) your problem is well-defined, and (b) shaving 20% off latency is worth doing. I consider these good problems to have. If I have to rewrite a class in Java for performance reasons, fine. I'd never start a multi-thousand-line project in Java, though.
Also, I really don't think anyone mature minds writing code in underpowered or difficult languages. C++, I find, is actually a lot of fun to write. It's reading other peoples' code, in large amounts, that becomes intolerable in these languages. If I have to spend a day to answer a simple question like "how do I initialize this object?", that's bad.
Interesting fact: Closures were cut from the 1.0 release for time. Gosling wanted them in, but Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes", so they were far from essential for that domain. The folk history (which I once believed) of Java being deliberately underpowered so that teams of hundreds of mediocre programmers could be stapled together in it is false. It was, however, co-opted by the "let's commoditize developers" crowd in the late '90s, and we've had to deal with FactoryVisitorDelegates ever since.
from my time at sun, i remember speeches by Gosling that included microwaves and refrigerators ( i vaguely recall an example of the fridge telling you via java applet that you had run out of milk ). the hotjava thing ( web browser written entirely in java http://en.wikipedia.org/wiki/HotJava) created some anticipation around Word & Photoshop & Powerpoint written in java, which were brought up when Sun hosted a gigantic industry conference in Vegas. The Java Chip ( picojava http://www.jopdesign.com/perf.jsp ) was supposed to revolutionize the industry by running java bytecode natively. Honestly I think the dumbing down began once EJB's crept in. There was some nasty visual shit called the Beantool which automatically inserted getters & setters by mandating every EJB to be a Bean, and from then on things got downhill really fast.
Mine had already been primed with XML "XSLT can map anything to anything!", and Enterprise Java was just more of the same.
It's funny that Javascript and WebGL are cleaning up in the browser, and Java applets are almost nowhere to be seen.
You know how awesome that type inference is in Scala, and how stupid and annoying it is in Java to type stuff like Map<K,V> myMap = new HashMap<K,V>? I totally agree with how annoying that is. Here's a related example in scala.
def myThing = something.getSomeThing()
What is the type of myThing?
Java has weaknesses for sure but reading code is actually one of its strengths, precisely because of the boilerplate. Note that I'm not defending EJB or any other atrocities here, just the core language.
Lastly, I agree that syntactically-nicer closures would be better for Java. But it already has closures,
final Something variableImClosingAround;
Runnable myClosure = new Runnable() { public void run() { variableImClosingAround.doSomething(); }};
exec.submit(myClosure);
For that problem, I'd just do it serially, because it's only 10 numbers.
Oh, it was an example for really big sets of numbers? Like really really big? Then I'd skip the .par.foreach and deal with java.util.concurrent directly, because it's worth my while if I have a serious problem worth optimizing.
"A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency."
A newbie in concurrency needs to understand how many threads are in their threadpool, or else they should just do it serially. I'm not talking rocket science here, it's just a single integer argument.
Tried that. Now I'm back coding Java, Groovy, JS and looking into Scala.
The perfect management role for me is one where I can shape product direction, mentor some younger guys, and still code. The problem is that in all the companies I've seen, management ends up being I sit in meetings all day and get completely cut off from coding or even having deeper tech discussions. I just don't feel like I'm still creating at that point.
I think it's the 2 surfers outpaddling the shark scenario: you don't have to be great at hiring, just slightly better than other companies in your demographic(geographic and "itch yoru scratching". You do have to be great at dev retention.
Not that rare, actually. I have a hard time finding companies that want to use Scala.
Manageable parallelism is nice, though.
Regarding ArrayBuffer: It is not as easy as it looks like, because of the conflicting nature of Arrays vs. Generics (yet another JVM problem). Amusingly I tried using Manifests to do exactly that last week. I hit the wall because the Manifests seemed to spread arbitrarily. Will try with TypeTags/macros when they become available.
I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)
I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.
The message was that anybody who has the common sense to google an unknown string is qualified duh?
The first puzzle job ad is nice, but it quickly wears off, and would start to bring bored "oh, not again" feelings. And many people just aren't good at making puzzles.
It's the same as with paint-drawn NFS ad.
For example, if you are looking for a Front-End guy, why not look around web magazines, Web 2.0 products and services, Github, Dribbble and pick the guys you like and contact them.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but still below the .50 level, standing alone). One is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Another is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Each of these kinds of tests has about the same validity in screening applicants for jobs. Neither is perfect (both operate at about .4x level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court
http://scholar.google.com/scholar_case?case=8655598674229196...
held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Companies outside the United States are regulated by different laws.
Schmidt, F.L. & Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings,” Psychological Bulletin,
http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...
via Google Scholar.
Yeah, I'd pretty much say that the IQ test was pointless in this case and luckily that was the only company I had run into using it.
EDIT: Forgot to mention that the test also took 3 hours to go through and had no limit on the questions that you would get. There was no possible way you could finish every question on the test.
Could someone do an (awesome) academic study which shows that all programmers benefit from g, or at least that all developers in x language do? Or add some specific problem where g is hugely useful (algorithm design?) and then make that problem a requirement for many programming positions?
There would be a huge benefit to technology hiring if we could start using a cheap, standard test result vs. relying on proxies like school admittance (where they can screen for IQ), or expensive work-sample tests (which burn more of the applicant's time, selecting against busy/good candidates).
I'd rather advantage poor, possibly non-US, high-IQ, busy applicants over either graduates of elite schools (correlated with rich) or people willing to spend the time to take a 4-40 hour unpaid assignment.
So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.
The key takeaway for me is not how people filter their applicants on real criteria, it's what they do when they have a large amount of seemingly equally skilled applicants. The original article demonstrated the 'unlucky' solution, I'm not sure what this article demonstrates since they still got too many applicants and they're spending time responding to all of them.
This article demonstrates a way to get a lot of highly skilled applicants-that's extremely useful.
The original article talks about finding 4-5 people worth talking to out of 100 people, 50 of whom can't fizzbuz.
We were in the situation of 20 interesting people out of 40 candidates, all of whom could fizzbuz.
If it were truly unmanageable, we'd probably have used weak Bayesian filters. E.g., degree, rockstar ninja, etc, all the things Raganwald originally advocated against using. I gather this is what Google and Facebook do.
If you give me a synthetic puzzle to solve I wouldn't bother applying to your organization. It's much better to craft a problem to the specific domain that your company is dealing with. Then you can test for both domain expertise and problem solving ability in one fell swoop with the extra bonus of it not seeming pointless.
† The important addendum on this is that you're not going to have the top level of folks sending out blind applications unless that person is in the process of making a lateral jump into another sub-field. And even then probably not. But if you give a top applicant a synthetic puzzle after your recruiting contacts them, expect a large portion of them to thumb their nose at you. Similar note on top folks, though this is mostly pedantry -- the top OSS projects aren't on GitHub and similar. The largest and most prestigious of the OSS projects host their repos themselves.
True, however if you're a contributor to a top OSS project, you'd tell me that when you apply, and send me an example of one of your contributions that you're particularly proud of (at least I hope you do, otherwise I won't know!)
Puzzles and challenges are useful when the candidate doesn't have any good examples of their work that they're at liberty to share with you. I'm very wary of hiring someone without seeing any of their work. We've been taking this approach for awhile now, and it's worked out well for us.
I'm actually more likely to apply if there is a puzzle. I've sent a few resumes out blindly, it usually goes into the black hole. If there is a puzzle, I figure my odds of getting a response (assuming I get it right) go way up.
Of course, if a correct solution results in a recruiter replying and asking me to submit to resumator, I'm probably ignoring your company in the future. (Yes, this happened to me once.)
The important point that I'm getting at, however, is that the dynamics of hiring shift dramatically as you move up the skill / connectedness scale. As you move towards the higher percentages potential employees are not competing for employers, but employers are competing for hires. And the dynamics of hiring are dramatically different at that point. If you're designing heuristics for hiring, you don't want to create a low-pass filter, and I believe that synthetic puzzles are exactly that.
The point of raganwald's post was on the surprisingly high cost of false-negatives. What I'm postulating is that your suggestion is just yet another way to reduce the false-positives rather than a way to reduce the false-negatives.
Good developers don't just go around submitting their resumes to every job in existence, and not filtering out false negatives was insufficient in and of itself to produce a good applicant pool. If doing something that produces false negatives (which, yes, are undesirable) nevertheless convinces enough developers to consider your company who never would have considered it otherwise, then it seems like the advantages have outweighed the disadvantages.
Obviously, an even better scenario would be to produce the same level of developer interest without also producing any false negatives. Perhaps by investing sufficient resources in networking or in making his company famous to other developers, he could accomplish this. This would still be a possibility to explore in the future.
But just because his tactic has not produced the best possible outcome does not mean it has not produced a positive one.
Hmm?
https://github.com/torvalds/linux
https://github.com/rails/rails
https://github.com/ry/node
https://github.com/jquery/jquery
https://github.com/twitter/bootstrap
https://github.com/symfony/symfony
Linus copied the kernel tree to GitHub when the kernel git repo was compromised. It is not the main repo. Of the others, only Rails and JQuery could even be reasonably be called "top" projects (Bootstrap? Seriously, you list a 7 month old project with 2000 commits?) and even those are a stretch. Major projects are well into the hundreds of thousands of commits. It's ludicrous to even muse of Bootstrap being in the same league as e.g. Mozilla, WebKit, et al.
If you define success by the number of people who have actually checked out and built the code, Bootstrap is vastly more successful than Mozilla & WebKit.
So far, 2 of 4 companies I have applied to have had a coding pre-screen and 1 of 2 companies that I have worked for have had a coding pre-screen.
Like I posted in the "who's hiring", that doesn't work for me because it doesn't tell me whether you are what I'm looking for. What do you do? Why should I care about you?
How to hire B players: Implement serious recruiting processes and plan to spend a lot of time on it. Get creative with hacks like OP's.
How to hire A players: Identify them through never ending networking, research, and keeping your finger on the pulse of the industry. You'll never find them the same way you find B and C players.
If you're doing that then it's possible they'll find you, if you're not then nothing you can do is going to help.
And the thing is Google, Microsoft, and GM also have some amazing teams simply though blind luck. The kind of thing where a few friends all end up working for a manager they like on some interesting little problem.
Some people have only worked on such teams and don't understand why so many projects fail, and others have never even seen them and wonder how complex software ever get's written. But, so few people have a truly understand the range from inspiring though dysfunctional people and teams out there that talk of A, B, and C coders / rock stars becomes almost meaningless.
PS: One of my favorite Real Programmers was a systems programmer for Texas Instruments. One day, he got a long distance call from a user whose system had crashed in the middle of saving some important work. Jim was able to repair the damage over the phone, getting the user to toggle in disk I/O instructions at the front panel, repairing system tables in hex, reading register contents back over the phone. http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/R... or the modern equivalent http://bellard.org/jslinux/tech.html
One thing missing in your list: Have 'A' leaders.
He went one step farther: If you have B and C leaders in your org, it's basically all over because few organizations are capable of taking the measures to correct that problem.
Sounds like your answer is: When there are items that smart people want to outsource, they often go work for people stupider than them rather than hire people to take care of it.
Do I have that right?
No, I am looking at the logical consequence of "hire people smarter than yourself". If A is smarter than B, then B is less smart than A.
Therefore: "hire people smarter than yourself" implies hiring people who are willing to work for someone dumber.
Your boss can be smarter than you at his job, and successfully hire you, smarter than he is at your job. That's an odd way to word it, but the fact remains -- there are generally different skillsets and competencies involved, and you are looking to build or be a part of a team with complementary talents.
A leader is spending a fraction of the time hands on that a programmer is. Given the developments in technology and the additional time the developers are getting with it, if they're not better than their leader in a few years, you've got the wrong programmers.
Conversely, you've probably encountered highly intelligent people that have the people skills of a bug.
But software development specially in small teams doesn't work that way. Many a times, junior programmers require direction and even help in completing their tasks. So it comes down to two problems (three, third one later)
1. Either you give someone sub-standard work to maintain that you be able to help them. This will lead to demotivation eventually
2. You accept you are not in position to help them. This generally leads to friction and a bad boss will try to hide incompetence by architecture discussions, meetings etc
Smartness is subjective. Which leads to the third problem. How can you assess skills of someone better than you? You can only go as far as grilling a candidate to your own competence. So a B, C player will only be hiring other B, C players and that will soon degenerate the organization.
No one (ok... almost no one) has no management nor coding skill, rather each of us has a mix of the two. The great organizations I've worked in had extremely smart people in management and in development, but the type of brilliance was different.
So yes, hire someone smarter than yourself.
If you're a decent company, you shouldn't hesitate to invest in B,C players so that they become A players.
[1] - Of course I'm talking about humans who are mentally healthy.
Caveats:
[1] Implicit assumption that the person is actually interested in becoming better.
[2] I understand that this takes a "toll" in productivity because you could be out there building stuff instead of helping these people improve. However, once a company reaches a certain size, I should think there is enough "boring" work for the company to staff "B" players to handle those, while simultaneously making them spend another chunk of their time working on becoming better.
How to hire B players: Offer market value.
How to hire C players: Offer below market value.
You do two things when you offer more money.
1) You are more careful about who you hire. 2) You actually attract good people who would make a move for that kind of money.
I would love to hear an inverse argument against this.
So; offer an environment where people want to work. Give them interesting work. Add a dollop of patriotism.
Ultimately, if you had an interesting product, a great environment and offered extremely competitive salaries then you would most likely end up with interest from more people in the "A" range.
To use real world examples, no amount of money you offer Salman Khan now would pull him away from his non-profit Khan academy to go back to the hedge fund industry.
If you're going to point to your step 2, then that sounds like a tautology. "Attract good people who would make a move for more money. Offer them more money"
Isn't that what acquihiring is about though? Purchase the talent that wont make the move for money.
* It motivates B and C players more. So you get a lot more chaff to filter out in your application process. You also get many more D players (the ones who are knowingly incompetent and are actively trying to deceive you into recruiting them).
* The major motivation of A players usually isn't money (once you reach the "fair wage" stage). So you don't attract that many extra A players.
* Some of the A players who do get tempted by the extra cash discover that it's not quite as wonderful as they expect and leave for more interesting work. Wasting large chunks of your time and money.
* It may turn some A players off. There's a unwritten rule that you get more cash to make up for dull work. So offering silly money implies to some that the work is going to be exceptionally uninteresting.
The absolute best way I know to recruit A players is:
0) Understand the role that needs filling and the kind of person that needs to fill it
1) Have a really interesting job
2) Know how to access the A players
(I'm amazed at the number of organisations I come across that skip step 0...)
3) Be prepared to be truly market competitive in salary - your target A player may have a 20 year consistent track record of getting sh*t done, respect their sense of self worth.
Being immodest I consider myself an A player, when a startup offering "competitive salary and benefits" low-balls my current income by 50% they've wasted my time and theirs
This can't be emphasized enough. What you consider "silly money" probably isn't silly money to an A player. In fact they may even find it low.
Indeed I'd argue that it's an implicit part of performing step zero - understanding the role and the person. If you don't understand what your potential employee is looking for in a role - be it money, responsibilities, working environment, or whatever - then you're never going to find 'em.
For example I'm continually amazed at the number of companies who decide that they need an uber-geek with a ten year track record of delivering e-commerce platforms (or whatever) - and then go about recruiting them as if they were unemployed. Sigh.
Any thoughts/feedback welcome.