1) I think a lot of start-ups want to hire "smart" people. Because they expect the new person to eventually wear many hats. Objective-C, Java, Android, CSS, server side concurrency, monitoring. An we've all seen Hunter and Schmidt reference that tokenadult usually posts when talk about interviewing comes around and it does seem that a general mental ability test (like an IQ test) combined with a work samples seem to predict future performance of that employee. Well except that one can't just straight up give IQ test to job applicants (there is a court case about that). So we are left with a job sample (which many forget to give, as is the point of the author). But instead many focus on the GMA and create proxies for it -- cute little puzzles about blenders, round manhole covers, and other such silly things.
2) Those interviewing don't know the technical stuff and are afraid you'd out-bullshit them. "How does an Ajax request work" well if the interviewer themselves doesn't quite know the details the might not be able to evaluate it properly. They could have it written down but well, some technical questions have many different levels of depth that a candidate might descent to. So a quick written answer to the question might seem wrong but it is really because the candidate is more advanced. So puzzles seems to be a generic and "easier" to handle.
I think (1) is probably the primary reason, along with the fun factor I mentioned above.
But honestly, why can't we just start giving IQ tests? Or at least asking for SAT scores? We shouldn't have to run through training puzzles just to prove we're smart enough to build a site.
What kind of IQ tests? Math thinking? Language? Other? These tests are kinda old and don't prove anything besides the fact that the person has a certain level of intelligence but that doesn't mean they can code.
> What kind of IQ tests? Math thinking? Language? Other?
Both. They're not perfect, but I'm going to be picky and look for people who can generally reason well (both mathematically and verbally).
> but that doesn't mean they can code.
That's why you pair it with work samples.
The ideal hire would be an impressive GitHub + evidence of intellect. The GitHub shows they have coding chops, while an IQ test usually means they can react quickly and learn new skills.
What if all your most relevant work is owned by previous employers? Restricting hires to people with decent personal projects and/ or extensive open source contributions isn't ideal either.
I'm fine with only hiring people who have done some work outside their workplace. If they're really great at programming, they have the drive to try new things outside work. Indeed, a couple good personal projects is probably the best predictor of skill you could find.
(Skill and interest tend to be highly correlated.)
I've heard one too many times that I have to react now and voice my confusion. When you have students and you want then to have had some work done outside of coursework, I'm perfectly ok with that as I understand the merits but I wonder is it true for people 10+ years into their career. I see great people having great jobs who, if they have free time, code something that can help their current job but that code also ends up being own by the employer and is very rarely released as open source.
I used to believe that as well. Now I'm not so sure, mainly through my own personal experience.
For 5.5 years (2004-2009), I was a core developer and maintainer over at Xfce. Over that span of time I wrote tens of thousands of lines of code, perhaps more. I did this in addition to my job, which morphed from engineering project management to programming tasks that I didn't find very challenging.
After I left that job, I joined a tiny, early startup. The programming work there I found very challenging, and the insane hours quickly ate up any spare time I might put toward Xfce. Eventually I realized that lacking a life outside of work made me sad, so I left. Fast forward a little bit, and now I'm at another startup, which (while demanding, challenging, and rewarding) gives me time for outside pursuits and a social life. But I don't feel the desire to work on personal coding projects anymore. The job fulfills my desire to write code, and outside of that I'd rather be, well... outside.
It's interesting, because all my Xfce contributions are on git.xfce.org, and not on GitHub. My GitHub account contains 2 personal projects: a semi-finished Android UI library (which was really only relevant and useful in the 2.x days), and a script for building a Gentoo image for a Raspberry Pi (which I've since abandoned as I mainly run Debian nowadays). Looking at my GH account would be fairly unimpressive, and might even be worse than not having a GH account at all.
So I'd easily fail the GitHub test. If a particular company was making that a no-go for getting through an interview pipeline, well... their loss, I guess.
I'm responsible for a fair bit of hiring at my current company, and I rarely ask about personal projects anymore, unless a candidate has one listed on their resume. While I'm not always happy with the coding problems I usually give (they tend to be data-structure/algo problems that I'd probably have trouble solving in an interview setting), they do at least clearly show whether or not the candidate can write code, and I also get a rough idea of how cleanly they code, not to mention insights in how they approach and solve problems. And not being able to completely solve the problem isn't an auto-fail in my book either, as long as they tried to work on it and their approach (and whatever code they did write) was sound.
I do very strongly believe in the aptitude test + work sample formula as the best way to evaluate candidates, but the problem is more that it's not always so easy to apply those tests in a traditional interview setting. Maybe that just means we need to come up with a different way of interviewing, though.
Sure. The intent of that statement was kinda muddled. I was more referring to the fact that my "main body" of open source code, to which my contributions stopped 4 years ago is all on a non-GH site, so it's interesting to look at what is on GH, which represents what I've done out in the open since then, which is... not much.
My assumption (which maybe is wrong) is that 5 years of OSS contributions that ended 4 years ago may not be all that impressive. Why did I abandon it? etc.
>> For 5.5 years (2004-2009), I was a core developer and maintainer over at Xfce.
>> But I don't feel the desire to work on personal coding projects anymore. The job fulfills my desire to write code, and outside of that I'd rather be, well... outside.
Another thing to remember is that you're 5-10 years older then you were at that point. Life priorities have changed. You might not be a caffeine-fueled 19 year old anymore, instead you're 28 with 2 kids and a mortgage. Or 34 and just have other things that interest you than tweaking Linux or exploring video drivers.
I have a bunch of personal projects, but few of them are on github, and the ones on github are not done under my real name, because I find the concept of me having to work on nights and weekends to impress future employers offensive.
My personal projects also include things like my very playing around with a new language (so it is freaking ugly and so I would never want it used as a job indicator) and old abandoned projects that I've salvaged and updated as a favor to friends, which means the entire architecture is alien and "I Don't Care If It's Ugly Just Make It Work Because Right Now It Does Not Work At All" is exactly what the 'client' wants.
I also reverse-engineer game protocols and find vulnerabilities in them. There is no social good to me releasing that stuff. (No, I really don't feel like being a soldier in your Full Disclosure army.)
I do lots of coding stuff in my spare time. I don't do it to get a job, I do it because it's fun.
The github test isn't about your what you have on github. It's about what code you can show, out in the public. Github just happens to be the common way to do that now.
If a company requires Github specifically, than it's just a stupid company you wouldn't want to work for anyways, so it saves you the trouble.
You'll probably never get anyone who used to work at Google/Facebook/Apple/Twitter then.
The problem is that IP agreements in the state of California say that the employer is not entitled to work done "on the employee's own time, without using the resources of their employer, and unrelated to the lines of business of their employer". Google claims to be in basically every tech-related business there is. Do something as simple as a mobile app or casual game and you can potentially get in trouble for it because it may compete with a division you've never heard of.
There's a procedure to get around this, but it's fairly slow and bureaucratic, and so many Googlers find it more efficient to do their "for fun" coding as 20% projects. Facebook has a similar system with their hackathons. Apple just works its devs so hard they don't have time for independent coding products.
I think that GitHub profiles are a good way to identify good devs who don't work for the leading tech companies, but once you're in one of them you a.) don't have time for personal programming projects anymore and b.) find that the stuff that your coworkers are doing is more interesting than pretty much anything you could work on as a solo dev.
I like spending time with my friends and family, some of which I have to travel far to see, for one.
I spend 40+ hours a week in front of a computer, when I get a chance to go out in the world, I want to GO OUT.
None of these things don't mean I don't love software (I do) or I'm not interested in it (I am). I just have varied interests.
Disregarding that, I used to work for a Big Nameless Faceless Corporation, and the legalize in employment agreement said basically that because I was a software engineer for them, they owned all the software I wrote while employed, even if I wrote it on my own time and with my own computer.
That is an extremely long post that says "hire the rock stars". Except, of course, we all cannot only hire the very best, transformative people (like Jeff Dean at Google). And, he has no way to actually identify them, except "ask around".
It's entirely unrealistic as a business strategy. It sure is nice if you luck into it.
I agree with the principle of wanting to see code but keep in mind that expecting Github (or similar) is a bit ageist. I definitely went through a phase of my life where I coded like crazy and produced lots of interesting side projects, but that was long before Github or even Sourceforge existed. So I could point you towards some interesting stuff I passed around the BBS world in 1985, except that even I didn't bother to keep a copy after its usefulness faded.
I still have a bit to show on Github and mean to put up more when I have a chance but still, most developers find that when you've got a decade or more of experience, you start to want to take on new challenges and experiences (whether family or woodworking) in your free time rather than more programming.
GitHub is a false profit. Almost every excellent programmer I know has almost nothing in their public GitHub. I've been doing this for a great many years however going off my GitHub I'm barely a programmer if that's the metric you're tracking.
> But honestly, why can't we just start giving IQ tests?
It's illigal. Or at least very very legally questionable.[1] Theoretically good proxies could be attacked in the same way, but there's a difference between doing something that could conceivably be the basis of a lawsuit and doing something that the Supreme Court has specifically ruled on.
I meant that mostly rhetorically in that I think that was a poor decision, or at least applied poorly. (Correct me if I'm wrong, but if you can show a high IQ is necessary for the job, isn't an IQ test acceptable?)
Still, asking for SAT scores is still legal and they correlate decently with intelligence.
I'd guess that there exists a causal relationship between "intelligence" as measured by standardized tests and the sum of dollars poured into someones education.
I suspect the actual correlation is not with wealth (i.e. giving money so somebody with low SAT scores would not actually improve their scores - unless he would have enough money to bribe whoever is overseeing the test of course ;) but with some things that correlate with wealth.
It sounds like you heard "causation" when parent said correlation. Correlation is observed in past samples; what you describe is a mechanism for trying to disprove causation, but it won't remove any correlation from the past samples, and it probably won't remove the correlation with historical wealth of the family etc.
Parent said "primary correlate". Primary implies some special-ness, that is not like the others.
I do not see any way "historical wealth" could directly - without intervening variables - influence one's SAT scores. Of course, having affluent parents may mean certain value given to quality education, certain amount of care and access to development tools and so on - but then those should be primary factors considered, not wealth per se.
Money buys access to tutors and training materials. It buys access to free time for practicing for your SAT. It also buys opportunity to repeatedly retake the SAT until you get a grade you like.
"Does test preparation help improve student performance on the SAT and ACT?
For students that have taken the test before and would like to boost their scores, coaching
seems to help, but by a rather small amount. After controlling for group differences, the
average coaching boost on the math section of the SAT is 14 to 15 points. The boost is
smaller on the verbal section of the test, just 6 to 8 points. The combined effect of
coaching on the SAT for the NELS sample is about 20 points."
Maybe they aren't as dumb - at least in the area that made them rich - as it seems. I can imagine someone bing smart in the university hall and dumb on the street, or vice versa.
oh really if the US had a system like we have in the UK the house of commons would have eaten quite a few high profile US polticians (including some presdents) alive.
Id love to see Paxman interview sarah palin or Ted Cruz for example.
So in fact, it is more likely that people from wealthy backgrounds are able to better utilize their intelligence, and thus retain their wealth and status, whereas people from poor backgrounds are at a disadvantage when it comes to escaping their position in life.
Did you read more than half of one sentence that I wrote? The poor are at a disadvantage when it comes to escaping their position in life; if you need this spelled out for you, the poor are disadvantaged when it comes to making decisions about saving money that could help them improve their financial situation. Wealthy people are better able to make rational decisions about their money, which helps in retaining wealth.
Maybe you should actually read the article I linked to, if it would help you understand what I wrote.
If you are hiring for start up, you do not care why they are dumb. Potentially you can save money by hiring non traditional candidates and paying them less, but any advantage you get this way will be short lived. Unfortunately diamonds in a rough are rare and finding them cost more than their value.
I was originally replying to a comment that claimed that people became wealthy because of their intelligence. My point does not have much to do with the issue of startup hiring (admittedly making the thread a bit off-topic for the article).
I understand the position very well, and I think most people with common sense do. Of course it's harder to become wealthy than it is to achieve other positions in life.
I just think it's a stupid use of "disadvantaged." It implies someone else has an advantage, in to quote you, "escaping their position in life." It's like saying a bad baseball player is at a disadvantage when trying to become a good baseball player. It almost doesn't make any sense.
Same goes with the idea that poor people are disadvantaged about making decisions wrt money. Of course they are, or they quite likely wouldn't be poor any longer. To use baseball again, or any skill really, the experts of course have advantages in deciding the best course of action, and typically have more ways to achieve it.
What I've gleaned here is that you consider poor people to be novices at the skill of acquiring wealth. I agree. For some reason people seem to think it's far different from rookies in other skills, because acquiring wealth is perceived by most to be a vital part of success, happiness, health, etc.
Aren't intelligence and wealth correlated with each other, and both are correlated with SAT and IQ tests? I thought the explanation for the mysterious "g-factor" in intelligence that IQ is supposed to measure isn't that it exists in and of itself, but that a number of measurable outcomes (income, wealth, test scores, GPA, job prestige, etc.) are all positively correlated, and then that correlation is called g.
We need to define some terms. Education != intelligence. The IQ tests are silly and heavily judged as nonsense. These tests are focused on math mostly, but math alone is hardly the only area that matters. However, in this context, math matters, because in many ways it is correlated to programming.
Management consulting firms and Wall Street banks have employed somewhat strict GPA and SAT/GMAT cutoffs for quite some time, but even they can't resist the allure of brainteasers during the interview process.
Standardized tests like SATs and school grades are a terrible way of ranking a potential employee and have significant biases such as income and age.
I didn't have very good grades and my SATs were ok but not great, both primarily because I didn't study or apply myself. I was completely uninterested in school. Get me to write code and you'll see what I'm capable of. I'm certainly not unique in this regard.
Because SAT scores are a very very terrible indicator of if you have 1) the ability and 2) the work ethic to build a site.
And how old are you? SAT scores? I'm only in my 30s and I don't remember my what my SAT scores were or remember if they were "good" or "bad" I only know that they changed the SATs after I took them because my youngest sibling took a different one than I did, with some crazy scoring. I wouldn't be able to find my SAT scores even if I wanted to.
In what way will an IQ or SAT test give you an indication if the applicant even knows what a compiler is? Your comment is cute but I'm not sure how useful it is ... :(
Oh I agree, I am just saying they miss both the part about how GMA is best when paired with a work sample and that those puzzles are not necessarily good equivalents to an IQ test.
1) I think a lot of start-ups want to hire "smart" people.
What does it really mean to be smart? Lately I cannot stop thinking about it. I have always been considered a 'smart person'. I am a self-taught freelance developer now - it used to be my hobby and somehow (mostly because I needed location-independent work quickly) it became my profession. I get by because everybody thinks I am smart but I feel like an impostor because the more I think about myself the more I realize that intelligence is not some general ability to solve problems - it's more just a set of very different skills that corelate to much lesser extent than people usually think and you can be really good at something that people use to judge your abilities and at the same time really bad at something else that is actually required to get the job done.
I studied sociology and I shortly worked as a data analyst. It seems to me that this kind of work requires... ehm... a different intelligence than programming. You need to be good at connecting the dots, noticing things, seeing patterns. This is my kind of thinking and I have always been good at this - doubting everything, seeing pure assumptions where other people saw 'truths', permanently creating hypotheses and alternative theories, trying to spot logical fallacies in prevailing theories... basically trying to spot things.
Programing is very different (at least it seems to me - so different tat it's even difficult do describe it). I guess it's about creating stuff, not just observing stuff. You need to build very complex and abstract mental models, keep them in your head and be able to operate with them - and this is the part of intelligence that I seem to be lacking. It just does not feel natural. I try to solve some problem and I am thinking... if this condition and that condition but at the same time not that condition... and bang!, suddenly I am lost and I don't even remember what I am doing. I cannot keep it in my head. I totally get what OP was saying about passing anonymous functions in JavaScript - I had the same experience. The first time I encountered something so simple as JavaScript closures it took me two hours to get it. And the day after that I had to repeat the whole mental process to get there again because I somehow lost it over night. This is simply not how my brain works and I think I am really bad at this. Yet people pay me for this... which is just depressing (and you understand why I write this under throwaway).
I remember our 'statistical analysis 101' professor always telling us 'remember, you are not really testing hypotheses, you are only testing indicators!' - if HR department picks wrong indicators for the skills that they actually need both company and employee are going to be unhappy - and I think this is very common because our understanding of the indicators for different kinds of 'being smart' is still poor.
You're conflating intelligence and knowledge. Intelligence is a matter of capacity and capability; knowledge is one of the things intelligent people have a great capacity for attaining and retaining. The main thing, in fact.
Knowledge, unlike intelligence, is in practice infinite. And now you can understand the inverse Dunning-Kruger effect: intelligent people have the capacity to recognize how little knowledge they have, and can ever have, about any given topic.
I don't think I am. I guess that you are probably referring to that stuff about JavaScript because that is the only part of my post where I mention something knowledge-related. But I am not talking about the lack of knowledge of JavaScript closures or functions passed as arguments - I was talking about the fact that it was surprisingly difficult for me to grasp that concept while trying to acquire that knowledge. Let me use your own words - it was 'a matter of capacity and capability'.
If you think I am wrong please explain how I am conflating intelligence and knowledge.
The first time I encountered something so simple as JavaScript closures it took me two hours to get it.
Closures are not simple. They are a powerful concept, one which introduces a whole new way of thinking about programming compared to what most people start out doing. It's normal for it to take a long time, over many sessions, for the concept to sink in and become a part of your programming vocabulary.
Freshmen year of undergrad, it took me well over a week to wrap my head around this "public" and "private" concept of classes. I had only written C++ that was mostly C with structs and C++ input/output. I had never had to reason in an object-oriented way before, and it was different. It was completely alien to me that I would intentionally "hide" some parts of my code from myself. Now, it's completely natural to me.
What the author did not see was those same developers learn about closures for the first time. When she sat down with them, they had already reasoned with the concept many times. Don't confuse initial difficulty with some inherent deficiency in your mental ability.
> * I get by because everybody thinks I am smart but I feel like an impostor*
That's a real thing. Almost all of the wildly talented people in my life feel Impostor Syndrome[0] at some level. Three Panel Soul even did a spot-on comic about it [1]
I've been thinking about the same problem for a while, and I'm also a self-taught developer that looks way towards sociology. I have a little different spin on the main hurdles of doing programming. With proper tools and the right mindset, the entire goal of the organization process is that you DON'T have to think and hold 100 things in memory at once. It works okay for some problems, but any problem with a sufficient complexity (like, say a full rich website with lots and lots of features) will be nearly impossible to work with if everything was intertwined.
I find the hardest park of programming is getting over hurdles as they come up, over and over and over again. It takes a while to get over fear of small failure. Also, if you're constantly reanalyzing and have a perfectionism streak you can be paralyzed quite frequently. Have you run into the same problems?
Exactly! I would say that complexity is the biggest problem form me. It makes it impossible to keep the whole mental model in my memory and I need to reanalyze and rebuild it constantly depending on the part of the problem that I am trying to solve right at the moment.
It is rather unfortunate how little correlation most tech interviews have with their respective jobs. It's largely a lose-lose situation for everyone. Developers who could easily build great systems but aren't experts in graph theory get passed over while brilliant mathematicians who can't necessarily code get hired. Result? Companies simultaneously having to fire employees while facing a supposed talent crunch. Given that this hurts everyone, how did we even get into this situation?
Probably because the only person who doesn't lose from this is the interviewer: they get to have fun. Honestly, when you spend all day buried in code, it's fun to play with puzzles for a change.
Perhaps it's time we started optimizing interviews for hiring success rather than interviewer happiness.
Generally speaking, high quality people won't take a short-term contracting gig. They are probably working elsewhere and don't need that level of uncertainty; they're looking for something better, not just anything.
I'd hope that by that point I'd have sold them on my company enough that they really thought it was better.
Also, I don't think it needs to be a long gig. Maybe even just a day, which isn't much longer than the gauntlet of technical interviews some companies will put you through. Even that short period should be enough to see if someone works well.
Doing a day's contracting elsewhere would violate my current employment contract. (And I think strict adherence to the letter of the rules will be reasonably common among good developers)
Generally speaking, most programming position don't need to be filled by high quality people. There are lots of smart, reasonably competent programmers out there, but who don't have years of experience on their CV, whom are eager to learn and would happily take a day or two off for a chance to work at an interesting company.
Truly high quality people are generally known by their reputation and are headhunted rather than expected to answer job ads.
Best people I know don't even give or take interviews.
They generally have a long history of awesome projects they have worked on, and they've built years of reputation through working on such projects.
And they just don't pick up any other project in any other company. They are pretty much clear about the best kind of problems they want to work on, and they don't wish to waste their time other than that.
Every time a company boasts about having a process to hire only 'A' candidates I chuckle. The 'A' or even 'B' candidates aren't even up for hire. The interview process begins with 'C' people.
Sounds good except for every job opening, I get 40 resumes, interview 15 people and of those 5 look reasonable. Which of those 5 do I give the short term contract gig to? So you're back to the same problem.
> Which of those 5 do I give the short term contract gig to? So you're back to the same problem.
Ideally, all of them. It's not hard to find a couple one-off tasks that you'd like to see done in a day. Or could could proceed sequentially until you find one whose work you like.
My opinion on this is that most of the interview processes is pretty old(over 20-30 years) and back then a good programmer was also a pretty good mathematician, and now most of the people that do interviews just use the same old patterns because, maybe, some of them don't know any better or because that is what they found in some books they have read.
I tend to hate the interviews that ask me to solve math and logic brainteasers because I don't see the value in them regarding my knowledge of programming.
This may be another reason people are eager to start their own company in lieu of working for someone else. If the questions are rubbish and completely unrelated to the actual job, then there's a huge disconnect between the interviewer (or HR company, as a lot of places outsource that) and where the actual work is to take place. I blame both.
The irony is that, in an effort to hire the "smartest" people, they leave out the wisest. Which is arguably more useful.
Well, if they really enjoy the challenge (and it is challenging), then yes. But I think there's something sad about being forced into it because hiring practices are archaic elsewhere. It doesn't have to be this way.
I don't like it that so many new startups fail and I have a feeling many of these (besides lacking an idea, failure in implementation etc...) are due to a lack of other options. If you can't get hired with the skillset you have, even though it should be more than enough, it can drive you to do desperate things. Including becoming a founder.
I'm sure there are a certain amount of people who have simply given up on or refuse to subject themselves to the standard interview process and will either become freelancers or get their jobs through networking.
Depends. They will likely work on the sort of problems they are good at solving. But at the expense of spending a large part of their time running a business, rather than solving the problems they are good at.
I actually like asking math questions on interviews. It shows how people approach a problem. Asking code questions in an arbitrary interview setting shows just about nothing - no access to a reference doc, somebody peering over your shoulder. Heck, I couldn't code my way out of a wet paperback in that setting.
Certainly, asking only math questions is stupid as well, people should know at least a little about the stuff they're supposed to work with, but teaching an actual language to a smart person eager to learn is a breeze compared to teaching problem solving to someone who memorized the reference manual.
> I would guess that there is actually significant overlap between these two groups.
I wouldn't. Smart people get bored sitting around memorizing things. They'd rather be thinking. Purely anecdotally, the smartest people I know rarely have encyclopedic knowledge of anything.
But thinking has the side effect of storing lots of stuff in your memory. So you can end up with encyclopedic knowledge of subjects that you've spent a lot of time thinking about. It just isn't knowledge that got stored by explicitly trying to "memorize" things (which means it's more reliable anyway, since it's knowledge that's connected to other things you know).
I store a lot of "I've read the solution for that problem somewhere while researching something else." I still can't recite the stdlib doc for any of the programming languages I work with. There's only so much stuff I can cram in my head and reference docs are fairly easy to look up.
Not necessarily; an encyclopedia isn't just an unconnected catalogue of facts--at least, it's not supposed to be. An encyclopedia article about a given subject is supposed to show the subject as a connected whole; it will contain facts, but will also contain important relationships between the facts, general principles, theories that explain the facts, etc. If you have that kind of knowledge of a subject, you don't have to memorize all its facts, because you can easily get to them from the facts you do have memorized via one of many interconnections.
Memorising the manual is the strategy of a surface learner. Surface learners can make pretty good PHP programmers, but they'll struggle with, say, passing Javascript closures.
A deep learner will look for the underlying principles and abstractions. They can quickly get an overview that, even when it's fuzzy, is still accurate enough for them to know where the gaps in their knowledge are, so they can fill those gaps quickly when they need to. They're like Mendeleev with his first periodic table. He didn't need somebody to show him a sample of gallium to know that it existed: he could inferred its presence and properties from the overall structure of the system.
Of course, a really advanced programmer will have done both. They will know and understand everything. But those guys are few and far between.
What I mean is that people who enjoy programming probably tend to be smart. I haven't done any statistical surveys, but the motivational feedback loop for learning programming rewards intelligence.
I doubt that. There's a lot of people going into programming because they think
a) IT is akin to playing computer games all day.
b) It's easy work in a climatized office with solid pay
c) The jobs are relatively secure and abundant.
I've seen many people go into IT that would much better have been employed elsewhere, so I think that we, as programmers are not more intelligent on average than the rest of the population. I might agree that people who enjoy programming have a knack for a certain type of intelligence and problem solving ability, but I don't think that programmers are limited to the group of people that enjoy programming.
looks like the one part you said you might agree with in your very last sentence is 100% of the group that they were talking about: people who enjoy programming.
Actually, since I'm the author of the post they're referring to I can say that the group we're referring to is "programmers" or rather "people you'd hire as a programmer" which is a superset of "people who enjoy programming." Even if you only consider people who enjoy programming I'd be very careful about calling them "more intelligent". They're probably good solvers of a certain kind of logical puzzle, but intelligence encompasses much much more than that.
*presuming that you went to an undergraduate institution where the variance in student intelligence is large enough to detect an appreciable difference between the avg intelligence in these two majors.
The phenomenon of the Computer Science graduate who can't write a FizzBuzz program, or even the post-graduate who can't write a simple recursive function, is well attested. But a Math student who can't handle a recursive definition is unlikely to make it through the first term.
I think you don't get the point. Learning a new language after you programmed for 5 years in a variety of paradigms like assembler, object-oriented, purely functional, will take you between a couple of hours and a week. But if you have NOT had those 5 years of varied programming experience, then the new language is not your problem. Learning the concepts is, and that will take time.
I think you don't get my point: I expect a domain expert to be a domain expert, but I don't expect a trainee to know anything and I'll structure my interview accordingly. I'll ask the domain expert questions about his domain. But I don't expect a domain expert to know the programming language that we work with just because he's a domain expert. And seriously, I'll prefer the eager-to-learn math graduate with little coding experience over the bored cs graduate that knows the java reference doc by heart and now thinks he knows how to build stuff. In any case coding examples in your interview will show you little beyond the fact that somebody memorized the docs.
The actual pain point I read from the article is a different one: There's a mismatch between hiring process and expectations. If I need a lead programmer for the iOS project that I'm about to start next week then I can't hire anyone that doesn't know Objective C and then I absolutely need to structure my interview accordingly. But if I have a couple of weeks more I'll happily teach him. And if I'm hiring somewhat smart I try to avoid those "we need someone urgently" situations.
I also think that there might be a mismatch between hiring process and expectations. When you say "But if I have a couple of weeks more" then you obviously underestimate the time it takes to learn if we are actually talking about somebody without much programming experience. Try "if I have a couple of years more".
See, there's so much more that you need to learn when you start on a project in a company. There's the framework they use, their libraries, the requirements for the project, the problem domain, the people you work with, the process they use, ... just to list the few that come to the top my head. Especially learning the problem domain can be very hard and challenging to programmers with little or no knowledge of the field. It can easily take a couple of month, if not years to become reasonably proficient.
So yes, "a couple of weeks" is certainly not enough to transform a graduate into a project lead for bespoke iOS project, but it could easily be enough to transform a decent java programmer with solid project lead experience. Transforming a crack Objective C Programmer with no lead experience can take easily as much time, if not more.
My gist is that programming knowledge will only get you so far and depending on what position you're hiring for other experience combined with the ability and willingness to learn may be much more interesting. And whatever I do, I tend to hire for that trait since that allows people to pick up other abilities when required.
I think you vastly underestimate the time it takes to learn these sorts of things. Sure, you can teach someone to write a simple, passable iOS app that does a few basic things in a couple weeks. But they're still going to be a raw-novice iOS developer. Maybe the app you need them to build is super simple, but if not, you're doing yourself and your company a disservice by not hiring someone who's done iOS before.
I'm speaking from experience here: I learned iOS (even after having previous MacOS X desktop devel experience) on the job, when a friend asked me to write an iOS app for her startup. I learned quickly, but made a lot of mistakes in how I structured the app that came back to bite me months later. If I'd had the time to start over from scratch, I would have done things quite differently and the whole thing would have been a lot easier.
And I was slow. Every new framework I had to learn slowed me down and added days to implementing the part of the app that needed it. A seasoned iOS developer wouldn't have run into problems like that.
I think you vastly underestimate the value of "other knowledge" for any kind of complex undertaking. Unless you get a domain expert who happens to be a rock star programmer and a ninja architect and top-notch devops, you'll have to compromise somewhere. Take an iOS game: Depending on the game it can be vastly more value to have knowledge of game programming and game AI than having knowledge about iOS frameworks. If there's a team on the project I'd even go as far and say that I would pick the domain expert over the framework expert as team lead, given that all other capabilities were on par.
Sure, there are always compromises. I wasn't claiming otherwise. I was merely pointing out that ramping up on a particular platform is nowhere near as easy as the parent suggests.
Games are a bit of a special breed, though, and there are several game engines out there that hide the platform pretty well. I draw a bit of a distinction between "building and iOS app" and "building an OpenGL app that runs on iOS". The latter is much easier if you lack iOS experience but have the required OpenGL skills.
Learning a new language is fairly easy. Learning the language's standard library to the point that you don't have to look at the reference docs every 30 seconds takes a bit more time.
The author clearly showcases how this is not true in his (and probably others') case. Anonymous functions, function pointers, etc. are not language-specific features, but rather something you learn when learning to program.
I think the point of the article is that being smart at puzzles is not enough. It may not matter whether you have 3 or 10 years of programming experience, but if you have 20 years of puzzle solving experience with only 6 months of (real-life) programmering experience you have a lot of non-trivial learning left to do.
Being in a startup setting where everyone (usually) is expected to be a bit of jack-of-all-trades, I would argue that having programming experience trumphs being smart, at least for the first period. You may be the master of finding smart algorithms to design your application, but if you can't build the first CRUD website it does you little good.
So true. I've been a professional developer for 15 years and I've worked with several languages (C++/Java/PHP/Python/Perl/C#/Ruby/JS/Groovy), but man, does this whole "functional" stuff is being hard to get a hold of.
Specially Scala, for me, is very hard to understand the syntax, even after a year of working with Play. The most frustrating part is understanding the flow of the data in chained calls like in map-reduce.
Asking math questions might exclude some good candidates. I know more than a couple very productive programmers that did not go through a formal CS education. Asking math questions to those candidates could even scare them for no reason. I hired people who had no idea about Lagrange Multipliers but were able to ship code in various languages and even learn new paradigms when necessary.
There are not only smart people and persons referencing reference manuals. Being a programmer often means solving bugs in messed-up codebases, build web apps using the technology du jour, or making data go from one place to another, and asking math questions does not help a lot to find people able to do this. This blog post resonates with some people I met.
I have been programming for a while, and went through a CS education, but my experience with hiring made me realize that being good with maths and being a productive programmer are not necessarily two things that always come together.
Hell, even those can exclude a good candidate, especially at larger companies. Candidate not a good fit for the team? Probably won't do as well as one who is.
I agree - whenever interview processes come up, commenters mostly criticise the interview processes for excluding good candidates.
But that's only one part of the equation - the number of unsuitable candidates that slip through is normally more important.
Suppose that somehow we magically know that 20% of candidates would be good hires - and the other 80% are unsuitable. But we don't know which are which!
As an interviewer, I'd be very happy with an interview process that discards 50% of the good candidates and 99% of the unsuitable candidates, because that leaves me with 10 good hires and 1 unsuitable hire for every 100 candidates.
On the other hand, if a different interview process discarded 20% of the good candidates and 80% of the unsuitable candidates, that would result in 16 good hires and 16 unsuitable hires - which would be disastrous!
Even from the point of the interviewee, one probably wouldn't want to work somewhere where 50% of your colleagues are not suited to their jobs!
Summary - it's a shame to discard good candidates, but it's worse to let too many unsuitable candidates slip through ...
> Asking math questions might exclude some good candidates.
Every question can exclude a good candidate, especially if you only ask the question and tick the "correct/incorrect answer" box. However, often you can learn the most interesting trait from asking a question which the candidate can't answer right off the bat: How does the candidate deal with failure or lack of knowledge. Does she/he start guessing? Does she/he ask the right questions moving in the general direction of solving the problem?
I'm not checking for academic knowledge in interviews.
> and even learn new paradigms when necessary.
This often requires knowledge about the stuff you don't know. That is a value of formal education: Not the stuff that you memorize, the bigger value I derive from my formal education is all that "I know that there's a solution to the problem but can't remember exactly" kind of knowledge. I can't remember all the sort algorithms I had to code, but I remember there's more than one and that there are tradeoffs between all of them. So if I'm constrained on memory and have a pre-sorted list I can go luck up how bubblesort is implemented exactly. That's a knowledge that self-taught programmers often lack [1].
> being good with maths and being a productive programmer are not necessarily two things that always come together.
No, nobody proclaimed so. But having a trait for problem solving and logical puzzles certainly helps :)
[1] n.b: often. Some of them have read and digested tons of theory books which could count as formal education.
> However, often you can learn the most interesting trait from asking a question which the candidate can't answer right off the bat: How does the candidate deal with failure or lack of knowledge.
Some research suggests that tests are much better at predicting performance than informal grading.
Did you read the OP? The point of the article was that one can be great at that sort of thing and yet quite terrible as a production programmer. You are not measuring the skill you want with math problems, you are measuring a proxy.
Furthermore, I've known plenty of smart math people that just never seemed to be able to program (well). I think they are different skill sets, certainly with plenty of overlap, but plenty of differences as well, and those differences matter. I laughed out loud at the "variables, variables everywhere!" answer in the OP - I've had to deal with so much code written that way. Some people, very smart people, just don't 'get' design in that way. I worked with a guy that used to run around the office, asking brain teasers, sharing tidbits of knowledge, but he couldn't execute a basic project - couldn't plan what to do, couldn't do things in a rational order, couldn't experiment and gather data, couldn't incrementally design, develop, refactor, nor do a big-bang waterfall kind of design, and so on. He was not sorely missed when let go. Smart as a whip, and useless (for programming).
I've worked at several companies with staff mathematicians. Sooner or later they got their hands on a compiler. Oh my. No, let me do that. You tell me what is wrong with my Kalman filter, but I'll take care of the implementation, thank you.
Its easy to kvetch at somebody else's answer without offering an alternative (I do agree whiteboard progamming is disastrous). So, instead of asking math, why not ask them to write a simple routine, but then start asking real world problems about the code they would face - how would you make this API interface robust? What kind of documentation would you write. How would you handle errors? Is this code exception safe? Thread safe? How would you make it either/both of those. Suppose your problem size was n=100MM, how might you need to change this (say they have a data structure that loaded everything into memory)? Ask them some problems they will see in production - what is the network delay, or whatever your problem case is. You still get to see how they approach problems, but in the context of the actual decisions they will be making while programming for you.
Anyway, that is what I try to do. I am revising my thoughts even on that, because I find people flopping on the 'code the simple problem (and, it is simple)' yet doing great on all the engineering questions, and doing fine if we hire them.
> Certainly, asking only math questions is stupid as well
So what I actually advocate is asking questions about the field that the person is going to work in. I still like to ask some math questions - a lot of the stuff that programs deal with basically fall back to math. Heck, all relational database stuff falls back to set theory, so knowledge in that area certainly helps a lot. I'm not advocating employing pure theoretical mathematicians to develop contact forms in php. However, from my experience the following two statements hold true:
* Studying CS does just about nothing good to your actual programming skills and your knowledge of real-world problems. So just don't expect recent graduates to be able to develop a program in a rational order, using a reasonable process, refactor or any of the skills you're asking for just by virtue of having "programmed". If you're looking for an experienced programmer hire for an experienced programmer, but the OPs tale clearly shows that she was no experienced programmer - why would I quiz her in the way I'd quiz a 10-years-learned ruby programmer?
* If you have time to teach and educate a programmer, prefer a smart and eager type over someone who spools down memorized knowledge. While memorized knowledge can sometimes help, the ability to learn and educate yourself is much more helpful once your memorized stuff is not saving you.
Yes, I did read your post. The point is whether asking math questions at all is valuable compared to the other questions you asked. I do not believe you particularly addressed that in your post or in this one. It is a proxy for what you are actually trying to measure - why not measure what you want to measure?
Your first asterisked point does not at all square with my experience. It depends on the program of study; I've certainly come across kids with no real pragmatic ability, and other programs that turned out, as much as you can in that environment, very pragmatic and skilled programmers. Certainly this is a skill set that improves over time. But let's not quibble over that; you raise a larger, valid point about interviewing recent grads vs more experienced people.
As far as that goes, I try to question them about school projects. "So, if I was to try to take that and do Y with it, what would be the consequences" type questions. Like you w/ the math questions, I do not expect any particular expertise and experience in actually solving the problem. But, I can start to see how they think about things. If they aren't thinking that clearly, throw them a bone and see what they do with it. Does it give them an 'aha' moment that then leads them to a better answer (I infer, perhaps incorrectly, that they can learn and be mentored), or do they just stonewall, not make the connection, or what have you. My suggestion is pretty simple. Measure what you want measured, not some proxy. I will point out that recent data suggests I am right. Google has admitted that all of their algorithm type questions are not good predictors of on the job performance, but questions about experience and "how would you X" are. I don't consider their data the last word on the subject (their hiring is quite narrow after all), but certainly suggestive.
I absolutely agree with asterisk 2, so I didn't address it.
> The point is whether asking math questions at all is valuable compared to the other questions you asked.
The author uses a graph search problem as an example - which is a very typical problem in IT. How do you approach such a problem? This is a valid math question that certainly adds value compared to other questions I might ask. It actually checks for multiple things: Does the candidate have a grasp of the underlying mathematical concepts. How does the candidate approach a problem decoupled of the actual real-world constraints of a programing language? Can the candidate describe a problem in clear, concise terms? Same for set theory: intersection, union, functions mapping input to output. Complexity analysis of algorithms - all of that is valuable knowledge and it's absolutely valid to ask people for that. How does binary AND/OR/XOR work? How does an exponential decay curve look like compared to a gauss or linear curve? This GH issue https://github.com/elasticsearch/elasticsearch/issues/3423 was posted on HN as example of a great feature description and it's full of formulas describing how it works. Vectors, matrix multiplication is a fundamental thing when you're doing natural language processing. Statistical problems are not exactly uncommon in programming either. Map/reduce are mathematical concept. What's wrong asking for that? Failing the answer does not mean that the interview is over, but to assert a candidate I need to know what she knows. As annoying as it sometimes is, math is _the_ _fundamental_ underpinning of what we do every day.
> As far as that goes, I try to question them about school projects.
Totally fine and I agree. Still - what's wrong with asking math questions?
> Google has admitted that all of their algorithm type questions are not good predictors of on the job performance
You're running into a problem if you hire only for math knowledge. But -repeat- that's not what I've been advocating.
What kind of math questions do you ask? Are then relevant to the job? Is it knowledge that the employee will need to have for their day-to-day tasks?
Asking math questions that are unrelated to the employee's tasks has strong bias for recent graduates. Ask a 40 year old a question about an equation they haven't used since they were 19 and they won't remember.
I was once asked to do binary math on a phone interview. I hadn't had to do binary math since I was in school 10 years prior. I was pretty much guaranteed to fail. This was not a good test. If I needed it I could easily relearn binary math in a few hours at most.
The interview questions cited in this article are more CS questions than math questions. These questions are perfectly valid if the job will require lots of coding using said CS concepts. But if the job DOESN'T use those CS concepts on a regular basis, the graph algorithm test will generate some false negatives.
On the other extreme, testing knowledge of language syntax is just as bad. A smart programmer will be able to figure out the syntax of any language in short order, especially if it belongs to the same class of a language with which they are already familiar (such as functional vs imperative, dynamic vs static).
An effective interview process would test both reasoning and coding capabilities, and it can be done simultaneously -- i.e. code the solution to reasoning puzzles.
There can be an air of intellectual masturbation to some programming interviews. Lots of otherwise reasonably smart people are close-minded about what is predictive of productivity on the job, and I think that is the real issue that this article should address.
I believe this is deeply valuable. For some roles, I would much prefer to hire someone who can quickly see the value of breadth-first search from both ends.
If he/she doesn't happen to know the syntax of Ruby, or Java, etc. it's less important to me.
I agree as such. Given the the choice between someone who's really smart and can solve hard problems, but a mediocre programmer or someone who's a good programmer, but sucks at problem solving, I'd chose the first one. That being said, if you hire someone in the first category, don't expect them to be happy and competent at writing an iPhone CRUD app in two weeks after seeing Objective-C for the first time.
I think that was the thrust of the article. If you're hiring based solely on someones math and algorithm skills with zero concern about their coding skills, you cannot turn around and be angry when their coding skills aren't what you needed. It's not math vs programming as such, but more generally about tuning your interviews to finding the skills you actually need (as opposed the skills you think you need).
I think you're still suffering from same mindset that led to the problem described in the article though. He wasn't a mediocre programmer - he was a bad programmer. If you make a lot of assumptions about people in an interview you will probably be disappointed.
This is your company, correct? (I stalked your Github profile)
Seems like the types of problems you're solving are exactly those which require far more domain experience with Ruby/HTML5/Javascript/whatever than the ability to see the value of various graph-searching techniques.
Would you hire this guy even though he's said quite plainly that he lacks the experience with these technologies (and has difficulty picking up new ones owing to that lack of experience)?
Some of our projects have us on site, side-by-side with a client's staff programmers. These projects involve millions of dollars, hundreds of stakeholders, and years of existing code. The programmers have a wide range of skill levels.
The work in these projects involves figuring out the project's objectives, goal decomposition, some agile/lean PoCs, then developing the BDD, MVC, DCI, API, CQRS, SQA, etc. Much of this can happen in pseudocode.
We also do pair programming, code reviews, brown bag demos, cross-training, and the like. I believe all these can help with developers getting up to speed with language syntax.
That said, choose the right person for the job. YMMV.
This is great. Now what would you do if after three months, your new hire can still not ship code that benefits your business? Fire the person? Keep teaching?
I also believe algorithmic knowledge is important, and tend to give algorithm questions to my candidates, but it matters more for those who will write databases and game engines than for those who will write CRUD apps.
I was good at the algorithms and data structures part of my university course, but as a developer with ten years experience, I find it is very rare to actually use these now.
Far more important is how / when to index an database table, how to design the tables in the first place.
If I need an efficient algorithm, in most cases someone has written (and tested) it and put it in a library.
Bringing it full circle, a puzzle is just a peculiar form of algorithm, most of the time. You don't want to hire someone who will sit around thinking about graph puzzles and how to measure 4 liters when you only have a 5L and 3L beaker. Google it and get on with the job.
If the person wrote the code in such a way that no one else could understand that it was a breath-first search, is that valuable? If they didn't even leave comments when they write code? If they didn't realise that you're using some slightly arcane feature of your language to keep a library of useful tree operations, and was expected to use the pre-existing function through macros/generics? Or was expected never to use such a technique because half the team wouldn't understand the code?
Hey, I know the theory on how to bake a large amount of cakes, breads, cookies, etc. I learned it by watching it on TV, and from cookery books. But actually it turns out that I'm a pretty crap baker, because I lack experience to know when I'm over-working/under-working dough, how to adjust cooking time with an unknown oven, etc. And I make a huge mess when I'm doing this, and take a lot longer than anyone with basically any experience of doing it at all.
Totally agree, asking math puzzles (sometimes really hard ones) to develop a copycat iphone app? Interviewing like this is really off the rails.
I really understand that a startup with scarce resource would like to do its best shot. However as discussed long ago (https://news.ycombinator.com/item?id=2385424), it is really frustrating that asking math puzzles are assumed as the best way to hire the best for the job.
After reading this I have a dumb question: The person behind the post is a CS major but only played a little bit with the C programming language in college — is this pretty common these days?
We definitely do C programming at my program (shell, malloc implementation, thread pool, web server, MIPS assembler/interpreter etc.). I go to Virginia Tech, a public university.
I go to an Engineering school, and there are only two classes that use C and are required for my CS major.
The first one is the second programming class we take and is three quarters java and one quarter C. Pretty basic stuff.
The second class is Operating Systems, and is C the whole quarter.
I'm pretty sure those are the only two required classes that use C. And I didn't have to take the first one cause I tested out of it with AP credit, and I didn't do C in HS. So some people will only take one class in C when they graduate.
Certainly at the school I went to you had quite a lot of choice over what courses you took. I imagine it would be quite possible to get a CS degree without ever touching C, especially if you took either a more math/theory approach or a more UX/HCI approach to your degree. Of course it was equally possible to spend half your degree writing operating systems and programming micro-controllers in C and assembly if that was your cup of tea.
Basically CS is a huge field and any assumption you make about someones skills just because they have a "CS degree" is almost certainly false to a greater or lesser degree.
Yes, I have seen many CS graduates who couldn't write C or C++ if their life depended on it. In fact, I have seen CS graduates who could explain the theory behind, say, parallel computing in detail, but could barely write scripts.
Knowing everything about painting doesn't make you a good painter :).
You've just shocked me! I went to art school and back in the day even if you studied say graphic design or fashion they actually taught you basic foundation skills which included painting and life drawing. Of course that was in the better schools, so maybe CS faces the same issues.
Yes, I've interviewed plenty of people with degrees from very prestigious Universities that evidenced no real programming skills, and a wonderful (truly) grasp of algorithms. Trouble is, the former is necessary on a day to day basis, and the second is always available from some contractor when you need it.
It depends on the quality of the university, I graduated in 1999, but from quick glance at the CS department web site, you still get to use at least C, C++, Prolog, ML, Java, C#, depending on the project assignments.
I started uni in 2002 and only did half a semester in C, and the other half on C++. I picked up a game dev elective for a semester, and did that in C++. Other than that, it was Java and PHP mostly, with smatterings of Lisp and Prolog for our AI classes.
Hiring engineers is hard, and companies haven't really figured it out yet. Even the best companies rely on puzzles and gimmicks that often have little to do with day-to-day programming.
At one company I interviewed with, I was asked to implement a queue using two stacks. At that time in my programming career, I had worked with C, C++, Obj-C, Lua, Python, JavaScript, SQL, and a handful of DSLs developing games, game development tools, and web applications. Want to know what I had never done? Written a queue using two stacks. My immediate response to the question was, "Why would you want to do that?"
If you really want to know if someone has the capacity to pull their weight as an engineer, ask them about what they've built. Even if they are fresh out of college, the best engineers will have projects they can talk about and explain. Ask how they approached/solved specific problems. Ask what they're most proud of building. Ask what was most frustrating.
Those are the kind of questions that will provide insight into a person's problem solving capabilities and offer a decent picture of what they're capable of doing.
I definitely agree with a model that places emphasis on past work/projects.
"To find out if they can get stuff done, I just ask what they’ve done. If someone can actually get stuff done they should have done so by now. It’s hard to be a good programmer without some previous experience and these days anyone can get some experience by starting or contributing to a free software project."
http://www.aaronsw.com/weblog/hiring
Best way to go is to ask them about their projects and hear them as to how they will explain them and how enthusiastic they are about building new things and ask them to probably build one in 2 days with the help of google. Then i think the quality would just come out naturally and now you can decide whether to hire this guy or not.
Exactly. "Tell me about your last project" can easily turn into a 2 hour conversation with any developer who has built anything of substance. It will tell you everything you need to know about them.
If you're wondering, the "two stacks queue" is an easy way to write a simple immutable queue for a functional programming language. You use one functional stack for enqueueing and one for dequeueing. When the dequeue stack is exhausted the enqueue stack is "flipped" into the dequeue. Amortized runtime is O(1) per operation.
Once they told me to use one stack for enqueueing and one for dequeueing, I immediately solved the problem. What I didn't understand is why you would a.) implement your own queue structure and b.) implement it in such a non-intuitive way.
Now that I know this would be for a functional language, it makes way more sense. At the time I couldn't get over why you wouldn't just use a doubly-linked list like the standard implementation in most OO languages.
The problem is, you often want to know what they can build for you in the future. And old projects where someone worked on something very specific for 5 years and exercised a small subset of a particular language or general algorithmic tool kit may not be predictive of their ability to work on YOUR project
- After a first non-technical call, we ask the candidate to create a very small project based on our SDK. We send him the documentation and a very small sample. He can almost use every tools he wants to create that small project and, of course, we do not set any deadlines. It allows us to see how the candidate architecture his applications and it gives us a project to discuss during the following call.
- If all goes well, we invite the candidate on site to present our code/project and eventually brainstorm together. So that both parties can see if they can work together and the candidate has an insight about how we work, how our code looks like.
Clearly, it's far from perfect and we are often considering changing it. Imagine if every company where you are applying would ask you to create an app from scratch with their SDK? We may lose some candidates, but at least we hire only people that fit the company's culture.
I have experienced "interviews" like this from the other end. I told YC funded founders I'm not interested in their job if this is how they interview. In my opinion, it shows inability to make a decision.
I am an excellent developer, with years of experience in the industry. I know lots of technologies, and already have a great job. There is no reason for me to spend personal time writing your projects, when I would be rewarded by spending personal time on my employer's projects.
"Interviews" like this will only grab candidates with nothing better to do than to fulltime interview with your company. In my opinion, the best people already have jobs, and you're excluding them from the process.
"I am an excellent developer, with years of experience", but seemingly unable or unwilling to complete a small practical test. Sounds to me like they are good at weeding out the wrong people.
What makes you think "I work at Google" provides sufficient data to an interviewer such that they should exempt you from their interviewing process? The fact that you have a job somewhere else says very little.
I'm not saying I'm immune from interviews. I'm saying it's proof I have chops.
Technical interviewing in a broken process. I've given almost 200 technical interviews at Google, and I've seen all kinds of results. But, I believe Google's results. Having worked at Google longer than anywhere else I've ever worked, I can say that the people are incredible, and it's a direct result of the interviewing process. We interview someone for a set number of interviews (N≤8 nowadays) and we make a decision. I can count on one hand the number of people I think are deadwood.
All I'm saying is the "interview" process of having people do projects for you is broken. You will filter out a lot of people with jobs they are kicking butt at.
Math puzzles are great if the problem is easily understood, the solution achievable without a math degree, and you ask them to solve it by writing code.
For example: "This database contains 100,000 problems with standardized parameters. The problem definition is defined in the file spec.txt which you can grab from our code repository. Write the code to solve these problems efficiently, passing each solution to a remote service via POSTing to a REST API, the documentation for which you can find here. Bonus points for parallel execution. Feel free to use any editor/IDE and reference online documentation, Stack Overflow, etc. that you want. If anything's not clear or you need a hand with something, just ask as you would if you were an employee already. Ready to get started?"
The great thing is that once you've identified a candidate, you can do remote screen sharing and have them write code before they even have to come into the office. I've interviewed a fair number of remote people this way and it's excellent for weeding out the people who can talk the talk but can't program worth a damn. And it limits bias because you don't care about much beyond their communication ability plus their technical ability.
Here is an interesting idea that I had reading this. As a startup, what if you were to create a simple computer language that looked different from most other computer languages, at least somewhat different. Alternatively, just use one of the many really obscure programming languages out there, just make sure the applicant does not know it ahead of time. Give the applicant a 10-20 page reference manual for the language and ask them to make a simple program of some sort. Have them read the manual and write the program, hopefully while not looking over their shoulder, so they can relax. In the manual you give them omit one critical function or API reference, but make sure that info is available online (make it available if you made up the language). Then see what happens.
This would test programmers ability to learn a new language.
You would be testing them on the ability to learn a new language. You could test the same abilities using just a custom library. This is a test I used to use on intern candidates. I didn't even provide documentation, just examples of API usages, and was testing their ability to learn from example.
Hm, maybe if I ever have to hire somebody I'll challenge them to a Corewar duel :-)
Before the internet my friends and I sometimes used to get to together, everybody got some time to write a Corewar warrior on paper, and then we'd watch the tournament together. It was great fun.
I guess that proposed interview is supposed to get people out of their comfort zone and test whether how quickly they can adapt and learn from a manual and simple examples and then at least be able to write simple programs.
It is a good strategy, if the company is interviewing freshers as programming is teachable and the assumption is that new inductee will take few months to become productive. If you can't wait, the best strategy is to give a live coding problem and test the person's proficiency in the required language/technology. I invariably do the latter as my requirements are always very specific. Most start ups I suppose, are themselves undecided on product/market/technology choice and thus the former strategy.
> Spoiler alert: to solve this problem, you need to know how to enumerate the rationals.
This problem was addressed nicely in this functional pearl by Jeremy Gibbons, et al.: http://www.cs.ox.ac.uk/jeremy.gibbons/publications/rationals... . As interesting as the result is, however, it's a pretty well-made point that research-level ideas from the programming languages community are not really software engineering interview material in the vast majority of cases.
This is yet another example of "rockstar developer"-itis, wherein startups are given to believe that they need the best of the best when in fact they do not. This particular example is entirely egregious because they asked her about something that requires enumerating the rationals when what they really wanted was an iOS code monkey. Then they fired her, based on their own shoddy interview.
The worst part of it is that it's easy to feel like it's your own fault when something like this happens. The real problem was the hiring manager misleading the applicant about what the job would be, but the OP clearly felt responsible for it.
I had a similar experience (working for family, at that), where I joined their R&D group and it went well for a while, but eventually they decided they wanted to get into selling solar power and didn't need a software/electrical engineer, so I was lectured about this or that bullshit and eventually brought into a meeting with the IT director and offered a position on his team. I realize now that they were trying to make me feel unwelcome, but I felt entirely responsible at the time. I never felt like I had a choice in taking the IT position (or lose my employment there entirely), and I hated it. The new boss was, for the most part, a nice guy, but he turned into a major nitpicking asshole over little things.
I'm at a new place now and it's still hard to kick the habit of trying to avoid coworkers because you know they don't like you and want you gone.
Culture has a huge impact on hiring and working. I wish I had been taught about culture when I was in school, I could have avoided the problems I faced had I realized the warning signs earlier.
The need the best of the best but they are identifying the best the wrong way. Maybe they need the best CRUD developer, well if all you look at are candidates with strong research-level knowledge then they will not be the best at building CRUD apps. Sure we can all say that the research-level code is harder and naturally that makes that candidate smarter, but it's like trying to find the best doctor for a nursing position.
IMHO, the post is more about the interviewers not understanding what is important for success in the job they are interviewing for than about anything else. If you need a person that will have to switch technologies, languages and paradigms, you have to test for that, make sure a candidate has done it before or is capable of doing it in expected time with expected depth.
If one is good and quick in problem solving or has high GMA, that does indicate that he has the capacity to handle new and difficult things in general, but says nothing about the speed with which he can handle a particular new thing. Author's example with JavaScript is very good illustration how difficult can it be to learn a new paradigm for the first/second time.
The author was not saying she was above simple coding tests but that she was an inexperienced programmer who was GOOD at the maths puzzles and got the job then struggled and the interviewer's assessment of her software development ability would have been better with some software related questions.
I agree. But for a different reason: I'm shit at maths puzzles.
I just don't have the experience or tools or interest for them.
And yet, somehow, in 20 years of business geekery I've never come across a problem I can't solve.
Maybe when writing Tetris for J2ME I would have saved myself 10 minutes googling if I'd had the experience to realise that right angle based matrix translations don't require fp maths and maybe when writing financial indicators, I'd have saved myself half a day if I hadn't had to look up integrals but this sort of stuff is definitely in the minority as far as my experience goes.
I don't play Sudoku, I don't solve puzzles. I don't want to solve mathematical puzzles. I just want to deal with system and deal with security and none of them requires me to understand the tricks to solve a puzzle. Sure I can learn some cool algorithms but no thanks. I don't want to solve puzzles. Exactly.
After much experimentation giving interviews for server side positions, I've come to favor questions that involve routine real world problems that can be handled in increasingly sophisticated ways.
One example I use is getting the candidate to write crud, list, and search controller actions for a simple category data structure. Given a basic category data model (e.g. Name, Parent), the candidate starts with the crud actions.
Crud actions aren't meant to be difficult to solve and serve as a basic screener to verify the candidate has working knowledge of the basics. The only edge case I look for the candidate to ask about is if orphaning child nodes is allowed (I.e updating parent node, deleting a node with children)
List action(s) start getting more interesting since recursion comes into play. A basic implementation of an action that can load the tree given an arbitrary category as a starting point is expected. If the candidate has some prior experience, a discussion of what performance concerns they may have with loading the category tree is a follow up question. The tree loading algorithm is then expected to be revised to handle an optional max depth parameter. An edge case I look to be considered is how to signify in the action response that a category has one or more child nodes that weren't loaded due to a depth restriction.
The search action implementation has a degree of difficulty scaled to the candidates experience level. All candidates have to write an action that returns a collection of categories matching a search string. Those with previous experience are asked about a paging solution. Senior level candidates are asked to return matching categories in a format that indicates all ancestors ( for instance: "Category 1 -> Category 1.1 -> Category 1.1.1" result for search string "1.1.1")
For an added degree of difficulty, candidates can be asked to recommend data model tweaks and algorithms supporting tree versioning requirements necessary to allow for loading the category tree's state at a given point in time.
The candidate's performance to this exercise seems to give some insight into their level of experience and ability to implement algorithms from a common real world example without having to ask much trivia or logic problems.
And even better - knowing that quite a few database engines have extensions to help handling hierarchical data and it probably depends on the details as to which is actually better in a particular circumstance.
Definitely. Nested sets have a lot of weaknesses, and any decent modern database will have CTEs so you can just use an adjacency list with a recursive query, or materialize out the closure table.
I am not able to understand any of the questions because the jargon used is unfamiliar although the concepts look simple. Are all these terms specific to a particular framework ? Genuine question, thanks.
No, from what I see they are not. CRUD relates to "Create, Remove, Update, Delete" which is the most common set of actions when dealing with database records.
Then the GP goes into detail about a tree-like structure of categories and their parents. If you're not familiar with trees and graph algorithms this might be quite daunting.
Do some googling on the terms I mentioned and please ask about specific things you do not understand.
1. Most IQ tests are Bullshit
2. We all know what happened to the company famous for " Who moved mount fuji"
3. Math Puzzles are good if they are of the IMO level- but these things need a lot of concentration and joy to solve- Not under stress interview conditions.
4. Expecting someone to show brilliance by solving a math puzzle in under ten twenty minutes is a lot like a public willy wagging competition
5.Even more disgusting is the semi dumb questions at Mckinsey inerviews like - "Estimate the number of mineral water bottles in London"
6.
7.In 'Jobs', Walter Issacson says Steve was never into much of these puzzles- I can understand the reason.
8. ' I think a lot of what people call intelligence just boils down to curiosity'-(great quote from an inspirational friend-http://www.flickr.com/photos/elizabethbw/8373942339/)
9. People who ask these kind of puzzles end up creating a lot of CPU without any GPU. Very Little beauty. Very Little love.
Disclosure- Im a member of Mensa Inernational. No Offense meant.
I can relate on the opposite. I am not great at those complex math problems. But, I have been coding for 15 years at >20 hours a week average. Mostly web stuff. I've built dozens of full products, that we're complex, and I generally feel like I could build anything I wanted. Every time I use a new site I can visualize how I would have built it, usually not a question of if I could; time permitting.
Yet, I have never had the balls to pursue it professionally. I build stuff and usually never launch it. I have learned several times over that marketing is not my strong suit.
That said, I'd actually like to work for a startup. Hit me up if anyone wants to talk.
Dear god, what kind of startup hires a person with only basic Java and Python knowledge, then hands them K&R and expects them to churn out production-quality code?! That's unfair.
353 comments
[ 4.4 ms ] story [ 187 ms ] thread1) I think a lot of start-ups want to hire "smart" people. Because they expect the new person to eventually wear many hats. Objective-C, Java, Android, CSS, server side concurrency, monitoring. An we've all seen Hunter and Schmidt reference that tokenadult usually posts when talk about interviewing comes around and it does seem that a general mental ability test (like an IQ test) combined with a work samples seem to predict future performance of that employee. Well except that one can't just straight up give IQ test to job applicants (there is a court case about that). So we are left with a job sample (which many forget to give, as is the point of the author). But instead many focus on the GMA and create proxies for it -- cute little puzzles about blenders, round manhole covers, and other such silly things.
2) Those interviewing don't know the technical stuff and are afraid you'd out-bullshit them. "How does an Ajax request work" well if the interviewer themselves doesn't quite know the details the might not be able to evaluate it properly. They could have it written down but well, some technical questions have many different levels of depth that a candidate might descent to. So a quick written answer to the question might seem wrong but it is really because the candidate is more advanced. So puzzles seems to be a generic and "easier" to handle.
But honestly, why can't we just start giving IQ tests? Or at least asking for SAT scores? We shouldn't have to run through training puzzles just to prove we're smart enough to build a site.
Both. They're not perfect, but I'm going to be picky and look for people who can generally reason well (both mathematically and verbally).
> but that doesn't mean they can code.
That's why you pair it with work samples.
The ideal hire would be an impressive GitHub + evidence of intellect. The GitHub shows they have coding chops, while an IQ test usually means they can react quickly and learn new skills.
(Skill and interest tend to be highly correlated.)
For 5.5 years (2004-2009), I was a core developer and maintainer over at Xfce. Over that span of time I wrote tens of thousands of lines of code, perhaps more. I did this in addition to my job, which morphed from engineering project management to programming tasks that I didn't find very challenging.
After I left that job, I joined a tiny, early startup. The programming work there I found very challenging, and the insane hours quickly ate up any spare time I might put toward Xfce. Eventually I realized that lacking a life outside of work made me sad, so I left. Fast forward a little bit, and now I'm at another startup, which (while demanding, challenging, and rewarding) gives me time for outside pursuits and a social life. But I don't feel the desire to work on personal coding projects anymore. The job fulfills my desire to write code, and outside of that I'd rather be, well... outside.
It's interesting, because all my Xfce contributions are on git.xfce.org, and not on GitHub. My GitHub account contains 2 personal projects: a semi-finished Android UI library (which was really only relevant and useful in the 2.x days), and a script for building a Gentoo image for a Raspberry Pi (which I've since abandoned as I mainly run Debian nowadays). Looking at my GH account would be fairly unimpressive, and might even be worse than not having a GH account at all.
So I'd easily fail the GitHub test. If a particular company was making that a no-go for getting through an interview pipeline, well... their loss, I guess.
I'm responsible for a fair bit of hiring at my current company, and I rarely ask about personal projects anymore, unless a candidate has one listed on their resume. While I'm not always happy with the coding problems I usually give (they tend to be data-structure/algo problems that I'd probably have trouble solving in an interview setting), they do at least clearly show whether or not the candidate can write code, and I also get a rough idea of how cleanly they code, not to mention insights in how they approach and solve problems. And not being able to completely solve the problem isn't an auto-fail in my book either, as long as they tried to work on it and their approach (and whatever code they did write) was sound.
I do very strongly believe in the aptitude test + work sample formula as the best way to evaluate candidates, but the problem is more that it's not always so easy to apply those tests in a traditional interview setting. Maybe that just means we need to come up with a different way of interviewing, though.
In this case, just mention it. It's not about github literally.
My assumption (which maybe is wrong) is that 5 years of OSS contributions that ended 4 years ago may not be all that impressive. Why did I abandon it? etc.
>> But I don't feel the desire to work on personal coding projects anymore. The job fulfills my desire to write code, and outside of that I'd rather be, well... outside.
Another thing to remember is that you're 5-10 years older then you were at that point. Life priorities have changed. You might not be a caffeine-fueled 19 year old anymore, instead you're 28 with 2 kids and a mortgage. Or 34 and just have other things that interest you than tweaking Linux or exploring video drivers.
My personal projects also include things like my very playing around with a new language (so it is freaking ugly and so I would never want it used as a job indicator) and old abandoned projects that I've salvaged and updated as a favor to friends, which means the entire architecture is alien and "I Don't Care If It's Ugly Just Make It Work Because Right Now It Does Not Work At All" is exactly what the 'client' wants.
I also reverse-engineer game protocols and find vulnerabilities in them. There is no social good to me releasing that stuff. (No, I really don't feel like being a soldier in your Full Disclosure army.)
I do lots of coding stuff in my spare time. I don't do it to get a job, I do it because it's fun.
> So I'd easily fail the GitHub test.
The github test isn't about your what you have on github. It's about what code you can show, out in the public. Github just happens to be the common way to do that now.
If a company requires Github specifically, than it's just a stupid company you wouldn't want to work for anyways, so it saves you the trouble.
The problem is that IP agreements in the state of California say that the employer is not entitled to work done "on the employee's own time, without using the resources of their employer, and unrelated to the lines of business of their employer". Google claims to be in basically every tech-related business there is. Do something as simple as a mobile app or casual game and you can potentially get in trouble for it because it may compete with a division you've never heard of.
There's a procedure to get around this, but it's fairly slow and bureaucratic, and so many Googlers find it more efficient to do their "for fun" coding as 20% projects. Facebook has a similar system with their hackathons. Apple just works its devs so hard they don't have time for independent coding products.
I think that GitHub profiles are a good way to identify good devs who don't work for the leading tech companies, but once you're in one of them you a.) don't have time for personal programming projects anymore and b.) find that the stuff that your coworkers are doing is more interesting than pretty much anything you could work on as a solo dev.
I like spending time with my friends and family, some of which I have to travel far to see, for one.
I spend 40+ hours a week in front of a computer, when I get a chance to go out in the world, I want to GO OUT.
None of these things don't mean I don't love software (I do) or I'm not interested in it (I am). I just have varied interests.
Disregarding that, I used to work for a Big Nameless Faceless Corporation, and the legalize in employment agreement said basically that because I was a software engineer for them, they owned all the software I wrote while employed, even if I wrote it on my own time and with my own computer.
http://www.joelonsoftware.com/articles/GuerrillaInterviewing...
http://www.amazon.com/Smart-Gets-Things-Done-Technical/dp/15...
http://steve-yegge.blogspot.com/2008/06/done-and-gets-things...
http://steve-yegge.blogspot.com/2008/06/done-and-gets-things...
It's entirely unrealistic as a business strategy. It sure is nice if you luck into it.
I still have a bit to show on Github and mean to put up more when I have a chance but still, most developers find that when you've got a decade or more of experience, you start to want to take on new challenges and experiences (whether family or woodworking) in your free time rather than more programming.
It's illigal. Or at least very very legally questionable.[1] Theoretically good proxies could be attacked in the same way, but there's a difference between doing something that could conceivably be the basis of a lawsuit and doing something that the Supreme Court has specifically ruled on.
[1] See Griggs v. Duke Power Co. (1971)
http://finduslaw.com/griggs-v-duke-power-co-1971-401-us-424-...
Still, asking for SAT scores is still legal and they correlate decently with intelligence.
I do not see any way "historical wealth" could directly - without intervening variables - influence one's SAT scores. Of course, having affluent parents may mean certain value given to quality education, certain amount of care and access to development tools and so on - but then those should be primary factors considered, not wealth per se.
"Does test preparation help improve student performance on the SAT and ACT? For students that have taken the test before and would like to boost their scores, coaching seems to help, but by a rather small amount. After controlling for group differences, the average coaching boost on the math section of the SAT is 14 to 15 points. The boost is smaller on the verbal section of the test, just 6 to 8 points. The combined effect of coaching on the SAT for the NELS sample is about 20 points."
http://nepc.colorado.edu/files/Briggs_Theeffectofadmissionst...
I would bet that's far more likely.
Id love to see Paxman interview sarah palin or Ted Cruz for example.
http://articles.washingtonpost.com/2013-08-29/national/41584...
So in fact, it is more likely that people from wealthy backgrounds are able to better utilize their intelligence, and thus retain their wealth and status, whereas people from poor backgrounds are at a disadvantage when it comes to escaping their position in life.
Maybe you should actually read the article I linked to, if it would help you understand what I wrote.
I just think it's a stupid use of "disadvantaged." It implies someone else has an advantage, in to quote you, "escaping their position in life." It's like saying a bad baseball player is at a disadvantage when trying to become a good baseball player. It almost doesn't make any sense.
Same goes with the idea that poor people are disadvantaged about making decisions wrt money. Of course they are, or they quite likely wouldn't be poor any longer. To use baseball again, or any skill really, the experts of course have advantages in deciding the best course of action, and typically have more ways to achieve it.
What I've gleaned here is that you consider poor people to be novices at the skill of acquiring wealth. I agree. For some reason people seem to think it's far different from rookies in other skills, because acquiring wealth is perceived by most to be a vital part of success, happiness, health, etc.
Google, on the other hand, appears to have done away with them, and now prefers to evaluate potential hires without giving too much weight to GPA or test scores: http://qz.com/96206/google-admits-those-infamous-brainteaser...
I didn't have very good grades and my SATs were ok but not great, both primarily because I didn't study or apply myself. I was completely uninterested in school. Get me to write code and you'll see what I'm capable of. I'm certainly not unique in this regard.
And how old are you? SAT scores? I'm only in my 30s and I don't remember my what my SAT scores were or remember if they were "good" or "bad" I only know that they changed the SATs after I took them because my youngest sibling took a different one than I did, with some crazy scoring. I wouldn't be able to find my SAT scores even if I wanted to.
What does it really mean to be smart? Lately I cannot stop thinking about it. I have always been considered a 'smart person'. I am a self-taught freelance developer now - it used to be my hobby and somehow (mostly because I needed location-independent work quickly) it became my profession. I get by because everybody thinks I am smart but I feel like an impostor because the more I think about myself the more I realize that intelligence is not some general ability to solve problems - it's more just a set of very different skills that corelate to much lesser extent than people usually think and you can be really good at something that people use to judge your abilities and at the same time really bad at something else that is actually required to get the job done.
I studied sociology and I shortly worked as a data analyst. It seems to me that this kind of work requires... ehm... a different intelligence than programming. You need to be good at connecting the dots, noticing things, seeing patterns. This is my kind of thinking and I have always been good at this - doubting everything, seeing pure assumptions where other people saw 'truths', permanently creating hypotheses and alternative theories, trying to spot logical fallacies in prevailing theories... basically trying to spot things.
Programing is very different (at least it seems to me - so different tat it's even difficult do describe it). I guess it's about creating stuff, not just observing stuff. You need to build very complex and abstract mental models, keep them in your head and be able to operate with them - and this is the part of intelligence that I seem to be lacking. It just does not feel natural. I try to solve some problem and I am thinking... if this condition and that condition but at the same time not that condition... and bang!, suddenly I am lost and I don't even remember what I am doing. I cannot keep it in my head. I totally get what OP was saying about passing anonymous functions in JavaScript - I had the same experience. The first time I encountered something so simple as JavaScript closures it took me two hours to get it. And the day after that I had to repeat the whole mental process to get there again because I somehow lost it over night. This is simply not how my brain works and I think I am really bad at this. Yet people pay me for this... which is just depressing (and you understand why I write this under throwaway).
I remember our 'statistical analysis 101' professor always telling us 'remember, you are not really testing hypotheses, you are only testing indicators!' - if HR department picks wrong indicators for the skills that they actually need both company and employee are going to be unhappy - and I think this is very common because our understanding of the indicators for different kinds of 'being smart' is still poor.
Knowledge, unlike intelligence, is in practice infinite. And now you can understand the inverse Dunning-Kruger effect: intelligent people have the capacity to recognize how little knowledge they have, and can ever have, about any given topic.
I don't think I am. I guess that you are probably referring to that stuff about JavaScript because that is the only part of my post where I mention something knowledge-related. But I am not talking about the lack of knowledge of JavaScript closures or functions passed as arguments - I was talking about the fact that it was surprisingly difficult for me to grasp that concept while trying to acquire that knowledge. Let me use your own words - it was 'a matter of capacity and capability'.
If you think I am wrong please explain how I am conflating intelligence and knowledge.
Closures are not simple. They are a powerful concept, one which introduces a whole new way of thinking about programming compared to what most people start out doing. It's normal for it to take a long time, over many sessions, for the concept to sink in and become a part of your programming vocabulary.
Freshmen year of undergrad, it took me well over a week to wrap my head around this "public" and "private" concept of classes. I had only written C++ that was mostly C with structs and C++ input/output. I had never had to reason in an object-oriented way before, and it was different. It was completely alien to me that I would intentionally "hide" some parts of my code from myself. Now, it's completely natural to me.
What the author did not see was those same developers learn about closures for the first time. When she sat down with them, they had already reasoned with the concept many times. Don't confuse initial difficulty with some inherent deficiency in your mental ability.
That's a real thing. Almost all of the wildly talented people in my life feel Impostor Syndrome[0] at some level. Three Panel Soul even did a spot-on comic about it [1]
[0] http://en.wikipedia.org/wiki/Impostor_syndrome
[1] http://threepanelsoul.com/2013/10/14/on-impostor-syndrome/
edited for sane line breaks
I find the hardest park of programming is getting over hurdles as they come up, over and over and over again. It takes a while to get over fear of small failure. Also, if you're constantly reanalyzing and have a perfectionism streak you can be paralyzed quite frequently. Have you run into the same problems?
Probably because the only person who doesn't lose from this is the interviewer: they get to have fun. Honestly, when you spend all day buried in code, it's fun to play with puzzles for a change.
Perhaps it's time we started optimizing interviews for hiring success rather than interviewer happiness.
Then do a short-term contracting gig (maybe just 1 day). If it works well, hire them.
Edit: clarify length which even the best people could do.
Also, I don't think it needs to be a long gig. Maybe even just a day, which isn't much longer than the gauntlet of technical interviews some companies will put you through. Even that short period should be enough to see if someone works well.
Truly high quality people are generally known by their reputation and are headhunted rather than expected to answer job ads.
They generally have a long history of awesome projects they have worked on, and they've built years of reputation through working on such projects.
And they just don't pick up any other project in any other company. They are pretty much clear about the best kind of problems they want to work on, and they don't wish to waste their time other than that.
Every time a company boasts about having a process to hire only 'A' candidates I chuckle. The 'A' or even 'B' candidates aren't even up for hire. The interview process begins with 'C' people.
Ideally, all of them. It's not hard to find a couple one-off tasks that you'd like to see done in a day. Or could could proceed sequentially until you find one whose work you like.
I tend to hate the interviews that ask me to solve math and logic brainteasers because I don't see the value in them regarding my knowledge of programming.
The irony is that, in an effort to hire the "smartest" people, they leave out the wisest. Which is arguably more useful.
If it is so, then it is a good thing. Right?
I don't like it that so many new startups fail and I have a feeling many of these (besides lacking an idea, failure in implementation etc...) are due to a lack of other options. If you can't get hired with the skillset you have, even though it should be more than enough, it can drive you to do desperate things. Including becoming a founder.
Certainly, asking only math questions is stupid as well, people should know at least a little about the stuff they're supposed to work with, but teaching an actual language to a smart person eager to learn is a breeze compared to teaching problem solving to someone who memorized the reference manual.
I would guess that there is actually significant overlap between these two groups.
I wouldn't. Smart people get bored sitting around memorizing things. They'd rather be thinking. Purely anecdotally, the smartest people I know rarely have encyclopedic knowledge of anything.
Yes, this is more what I was thinking of as "encyclopedic knowledge" (which may not be what the parent post to mine meant by the term).
Memorising the manual is the strategy of a surface learner. Surface learners can make pretty good PHP programmers, but they'll struggle with, say, passing Javascript closures.
A deep learner will look for the underlying principles and abstractions. They can quickly get an overview that, even when it's fuzzy, is still accurate enough for them to know where the gaps in their knowledge are, so they can fill those gaps quickly when they need to. They're like Mendeleev with his first periodic table. He didn't need somebody to show him a sample of gallium to know that it existed: he could inferred its presence and properties from the overall structure of the system.
Of course, a really advanced programmer will have done both. They will know and understand everything. But those guys are few and far between.
a) IT is akin to playing computer games all day.
b) It's easy work in a climatized office with solid pay
c) The jobs are relatively secure and abundant.
I've seen many people go into IT that would much better have been employed elsewhere, so I think that we, as programmers are not more intelligent on average than the rest of the population. I might agree that people who enjoy programming have a knack for a certain type of intelligence and problem solving ability, but I don't think that programmers are limited to the group of people that enjoy programming.
The phenomenon of the Computer Science graduate who can't write a FizzBuzz program, or even the post-graduate who can't write a simple recursive function, is well attested. But a Math student who can't handle a recursive definition is unlikely to make it through the first term.
The actual pain point I read from the article is a different one: There's a mismatch between hiring process and expectations. If I need a lead programmer for the iOS project that I'm about to start next week then I can't hire anyone that doesn't know Objective C and then I absolutely need to structure my interview accordingly. But if I have a couple of weeks more I'll happily teach him. And if I'm hiring somewhat smart I try to avoid those "we need someone urgently" situations.
So yes, "a couple of weeks" is certainly not enough to transform a graduate into a project lead for bespoke iOS project, but it could easily be enough to transform a decent java programmer with solid project lead experience. Transforming a crack Objective C Programmer with no lead experience can take easily as much time, if not more.
My gist is that programming knowledge will only get you so far and depending on what position you're hiring for other experience combined with the ability and willingness to learn may be much more interesting. And whatever I do, I tend to hire for that trait since that allows people to pick up other abilities when required.
I'm speaking from experience here: I learned iOS (even after having previous MacOS X desktop devel experience) on the job, when a friend asked me to write an iOS app for her startup. I learned quickly, but made a lot of mistakes in how I structured the app that came back to bite me months later. If I'd had the time to start over from scratch, I would have done things quite differently and the whole thing would have been a lot easier.
And I was slow. Every new framework I had to learn slowed me down and added days to implementing the part of the app that needed it. A seasoned iOS developer wouldn't have run into problems like that.
Games are a bit of a special breed, though, and there are several game engines out there that hide the platform pretty well. I draw a bit of a distinction between "building and iOS app" and "building an OpenGL app that runs on iOS". The latter is much easier if you lack iOS experience but have the required OpenGL skills.
I think the point of the article is that being smart at puzzles is not enough. It may not matter whether you have 3 or 10 years of programming experience, but if you have 20 years of puzzle solving experience with only 6 months of (real-life) programmering experience you have a lot of non-trivial learning left to do.
Being in a startup setting where everyone (usually) is expected to be a bit of jack-of-all-trades, I would argue that having programming experience trumphs being smart, at least for the first period. You may be the master of finding smart algorithms to design your application, but if you can't build the first CRUD website it does you little good.
There are not only smart people and persons referencing reference manuals. Being a programmer often means solving bugs in messed-up codebases, build web apps using the technology du jour, or making data go from one place to another, and asking math questions does not help a lot to find people able to do this. This blog post resonates with some people I met.
I have been programming for a while, and went through a CS education, but my experience with hiring made me realize that being good with maths and being a productive programmer are not necessarily two things that always come together.
But that's only one part of the equation - the number of unsuitable candidates that slip through is normally more important.
Suppose that somehow we magically know that 20% of candidates would be good hires - and the other 80% are unsuitable. But we don't know which are which!
As an interviewer, I'd be very happy with an interview process that discards 50% of the good candidates and 99% of the unsuitable candidates, because that leaves me with 10 good hires and 1 unsuitable hire for every 100 candidates.
On the other hand, if a different interview process discarded 20% of the good candidates and 80% of the unsuitable candidates, that would result in 16 good hires and 16 unsuitable hires - which would be disastrous!
Even from the point of the interviewee, one probably wouldn't want to work somewhere where 50% of your colleagues are not suited to their jobs!
Summary - it's a shame to discard good candidates, but it's worse to let too many unsuitable candidates slip through ...
Every question can exclude a good candidate, especially if you only ask the question and tick the "correct/incorrect answer" box. However, often you can learn the most interesting trait from asking a question which the candidate can't answer right off the bat: How does the candidate deal with failure or lack of knowledge. Does she/he start guessing? Does she/he ask the right questions moving in the general direction of solving the problem?
I'm not checking for academic knowledge in interviews.
> and even learn new paradigms when necessary.
This often requires knowledge about the stuff you don't know. That is a value of formal education: Not the stuff that you memorize, the bigger value I derive from my formal education is all that "I know that there's a solution to the problem but can't remember exactly" kind of knowledge. I can't remember all the sort algorithms I had to code, but I remember there's more than one and that there are tradeoffs between all of them. So if I'm constrained on memory and have a pre-sorted list I can go luck up how bubblesort is implemented exactly. That's a knowledge that self-taught programmers often lack [1].
> being good with maths and being a productive programmer are not necessarily two things that always come together.
No, nobody proclaimed so. But having a trait for problem solving and logical puzzles certainly helps :)
[1] n.b: often. Some of them have read and digested tons of theory books which could count as formal education.
Some research suggests that tests are much better at predicting performance than informal grading.
Furthermore, I've known plenty of smart math people that just never seemed to be able to program (well). I think they are different skill sets, certainly with plenty of overlap, but plenty of differences as well, and those differences matter. I laughed out loud at the "variables, variables everywhere!" answer in the OP - I've had to deal with so much code written that way. Some people, very smart people, just don't 'get' design in that way. I worked with a guy that used to run around the office, asking brain teasers, sharing tidbits of knowledge, but he couldn't execute a basic project - couldn't plan what to do, couldn't do things in a rational order, couldn't experiment and gather data, couldn't incrementally design, develop, refactor, nor do a big-bang waterfall kind of design, and so on. He was not sorely missed when let go. Smart as a whip, and useless (for programming).
I've worked at several companies with staff mathematicians. Sooner or later they got their hands on a compiler. Oh my. No, let me do that. You tell me what is wrong with my Kalman filter, but I'll take care of the implementation, thank you.
Its easy to kvetch at somebody else's answer without offering an alternative (I do agree whiteboard progamming is disastrous). So, instead of asking math, why not ask them to write a simple routine, but then start asking real world problems about the code they would face - how would you make this API interface robust? What kind of documentation would you write. How would you handle errors? Is this code exception safe? Thread safe? How would you make it either/both of those. Suppose your problem size was n=100MM, how might you need to change this (say they have a data structure that loaded everything into memory)? Ask them some problems they will see in production - what is the network delay, or whatever your problem case is. You still get to see how they approach problems, but in the context of the actual decisions they will be making while programming for you.
Anyway, that is what I try to do. I am revising my thoughts even on that, because I find people flopping on the 'code the simple problem (and, it is simple)' yet doing great on all the engineering questions, and doing fine if we hire them.
Did you read my post? May I quote myself:
> Certainly, asking only math questions is stupid as well
So what I actually advocate is asking questions about the field that the person is going to work in. I still like to ask some math questions - a lot of the stuff that programs deal with basically fall back to math. Heck, all relational database stuff falls back to set theory, so knowledge in that area certainly helps a lot. I'm not advocating employing pure theoretical mathematicians to develop contact forms in php. However, from my experience the following two statements hold true:
* Studying CS does just about nothing good to your actual programming skills and your knowledge of real-world problems. So just don't expect recent graduates to be able to develop a program in a rational order, using a reasonable process, refactor or any of the skills you're asking for just by virtue of having "programmed". If you're looking for an experienced programmer hire for an experienced programmer, but the OPs tale clearly shows that she was no experienced programmer - why would I quiz her in the way I'd quiz a 10-years-learned ruby programmer?
* If you have time to teach and educate a programmer, prefer a smart and eager type over someone who spools down memorized knowledge. While memorized knowledge can sometimes help, the ability to learn and educate yourself is much more helpful once your memorized stuff is not saving you.
Your first asterisked point does not at all square with my experience. It depends on the program of study; I've certainly come across kids with no real pragmatic ability, and other programs that turned out, as much as you can in that environment, very pragmatic and skilled programmers. Certainly this is a skill set that improves over time. But let's not quibble over that; you raise a larger, valid point about interviewing recent grads vs more experienced people.
As far as that goes, I try to question them about school projects. "So, if I was to try to take that and do Y with it, what would be the consequences" type questions. Like you w/ the math questions, I do not expect any particular expertise and experience in actually solving the problem. But, I can start to see how they think about things. If they aren't thinking that clearly, throw them a bone and see what they do with it. Does it give them an 'aha' moment that then leads them to a better answer (I infer, perhaps incorrectly, that they can learn and be mentored), or do they just stonewall, not make the connection, or what have you. My suggestion is pretty simple. Measure what you want measured, not some proxy. I will point out that recent data suggests I am right. Google has admitted that all of their algorithm type questions are not good predictors of on the job performance, but questions about experience and "how would you X" are. I don't consider their data the last word on the subject (their hiring is quite narrow after all), but certainly suggestive.
I absolutely agree with asterisk 2, so I didn't address it.
The author uses a graph search problem as an example - which is a very typical problem in IT. How do you approach such a problem? This is a valid math question that certainly adds value compared to other questions I might ask. It actually checks for multiple things: Does the candidate have a grasp of the underlying mathematical concepts. How does the candidate approach a problem decoupled of the actual real-world constraints of a programing language? Can the candidate describe a problem in clear, concise terms? Same for set theory: intersection, union, functions mapping input to output. Complexity analysis of algorithms - all of that is valuable knowledge and it's absolutely valid to ask people for that. How does binary AND/OR/XOR work? How does an exponential decay curve look like compared to a gauss or linear curve? This GH issue https://github.com/elasticsearch/elasticsearch/issues/3423 was posted on HN as example of a great feature description and it's full of formulas describing how it works. Vectors, matrix multiplication is a fundamental thing when you're doing natural language processing. Statistical problems are not exactly uncommon in programming either. Map/reduce are mathematical concept. What's wrong asking for that? Failing the answer does not mean that the interview is over, but to assert a candidate I need to know what she knows. As annoying as it sometimes is, math is _the_ _fundamental_ underpinning of what we do every day.
> As far as that goes, I try to question them about school projects.
Totally fine and I agree. Still - what's wrong with asking math questions?
> Google has admitted that all of their algorithm type questions are not good predictors of on the job performance
You're running into a problem if you hire only for math knowledge. But -repeat- that's not what I've been advocating.
Asking math questions that are unrelated to the employee's tasks has strong bias for recent graduates. Ask a 40 year old a question about an equation they haven't used since they were 19 and they won't remember.
I was once asked to do binary math on a phone interview. I hadn't had to do binary math since I was in school 10 years prior. I was pretty much guaranteed to fail. This was not a good test. If I needed it I could easily relearn binary math in a few hours at most.
On the other extreme, testing knowledge of language syntax is just as bad. A smart programmer will be able to figure out the syntax of any language in short order, especially if it belongs to the same class of a language with which they are already familiar (such as functional vs imperative, dynamic vs static).
An effective interview process would test both reasoning and coding capabilities, and it can be done simultaneously -- i.e. code the solution to reasoning puzzles.
There can be an air of intellectual masturbation to some programming interviews. Lots of otherwise reasonably smart people are close-minded about what is predictive of productivity on the job, and I think that is the real issue that this article should address.
I believe this is deeply valuable. For some roles, I would much prefer to hire someone who can quickly see the value of breadth-first search from both ends.
If he/she doesn't happen to know the syntax of Ruby, or Java, etc. it's less important to me.
I think that was the thrust of the article. If you're hiring based solely on someones math and algorithm skills with zero concern about their coding skills, you cannot turn around and be angry when their coding skills aren't what you needed. It's not math vs programming as such, but more generally about tuning your interviews to finding the skills you actually need (as opposed the skills you think you need).
http://sixarm.com/
This is your company, correct? (I stalked your Github profile)
Seems like the types of problems you're solving are exactly those which require far more domain experience with Ruby/HTML5/Javascript/whatever than the ability to see the value of various graph-searching techniques.
Would you hire this guy even though he's said quite plainly that he lacks the experience with these technologies (and has difficulty picking up new ones owing to that lack of experience)?
Some of our projects have us on site, side-by-side with a client's staff programmers. These projects involve millions of dollars, hundreds of stakeholders, and years of existing code. The programmers have a wide range of skill levels.
The work in these projects involves figuring out the project's objectives, goal decomposition, some agile/lean PoCs, then developing the BDD, MVC, DCI, API, CQRS, SQA, etc. Much of this can happen in pseudocode.
We also do pair programming, code reviews, brown bag demos, cross-training, and the like. I believe all these can help with developers getting up to speed with language syntax.
That said, choose the right person for the job. YMMV.
I also believe algorithmic knowledge is important, and tend to give algorithm questions to my candidates, but it matters more for those who will write databases and game engines than for those who will write CRUD apps.
Far more important is how / when to index an database table, how to design the tables in the first place.
If I need an efficient algorithm, in most cases someone has written (and tested) it and put it in a library.
Bringing it full circle, a puzzle is just a peculiar form of algorithm, most of the time. You don't want to hire someone who will sit around thinking about graph puzzles and how to measure 4 liters when you only have a 5L and 3L beaker. Google it and get on with the job.
If the person wrote the code in such a way that no one else could understand that it was a breath-first search, is that valuable? If they didn't even leave comments when they write code? If they didn't realise that you're using some slightly arcane feature of your language to keep a library of useful tree operations, and was expected to use the pre-existing function through macros/generics? Or was expected never to use such a technique because half the team wouldn't understand the code?
Hey, I know the theory on how to bake a large amount of cakes, breads, cookies, etc. I learned it by watching it on TV, and from cookery books. But actually it turns out that I'm a pretty crap baker, because I lack experience to know when I'm over-working/under-working dough, how to adjust cooking time with an unknown oven, etc. And I make a huge mess when I'm doing this, and take a lot longer than anyone with basically any experience of doing it at all.
I really understand that a startup with scarce resource would like to do its best shot. However as discussed long ago (https://news.ycombinator.com/item?id=2385424), it is really frustrating that asking math puzzles are assumed as the best way to hire the best for the job.
We definitely do C programming at my program (shell, malloc implementation, thread pool, web server, MIPS assembler/interpreter etc.). I go to Virginia Tech, a public university.
Basically CS is a huge field and any assumption you make about someones skills just because they have a "CS degree" is almost certainly false to a greater or lesser degree.
Knowing everything about painting doesn't make you a good painter :).
Then again no one automatically expects an art history major to know how to paint (even though there no doubt is quite a bit of overlap).
I don't think I could write a simple echo clone in C anymore.
I miss C, but since entering the "real world" I haven't written a line of C for work purposes.
At one company I interviewed with, I was asked to implement a queue using two stacks. At that time in my programming career, I had worked with C, C++, Obj-C, Lua, Python, JavaScript, SQL, and a handful of DSLs developing games, game development tools, and web applications. Want to know what I had never done? Written a queue using two stacks. My immediate response to the question was, "Why would you want to do that?"
If you really want to know if someone has the capacity to pull their weight as an engineer, ask them about what they've built. Even if they are fresh out of college, the best engineers will have projects they can talk about and explain. Ask how they approached/solved specific problems. Ask what they're most proud of building. Ask what was most frustrating.
Those are the kind of questions that will provide insight into a person's problem solving capabilities and offer a decent picture of what they're capable of doing.
"To find out if they can get stuff done, I just ask what they’ve done. If someone can actually get stuff done they should have done so by now. It’s hard to be a good programmer without some previous experience and these days anyone can get some experience by starting or contributing to a free software project." http://www.aaronsw.com/weblog/hiring
Now that I know this would be for a functional language, it makes way more sense. At the time I couldn't get over why you wouldn't just use a doubly-linked list like the standard implementation in most OO languages.
- After a first non-technical call, we ask the candidate to create a very small project based on our SDK. We send him the documentation and a very small sample. He can almost use every tools he wants to create that small project and, of course, we do not set any deadlines. It allows us to see how the candidate architecture his applications and it gives us a project to discuss during the following call. - If all goes well, we invite the candidate on site to present our code/project and eventually brainstorm together. So that both parties can see if they can work together and the candidate has an insight about how we work, how our code looks like.
Clearly, it's far from perfect and we are often considering changing it. Imagine if every company where you are applying would ask you to create an app from scratch with their SDK? We may lose some candidates, but at least we hire only people that fit the company's culture.
I am an excellent developer, with years of experience in the industry. I know lots of technologies, and already have a great job. There is no reason for me to spend personal time writing your projects, when I would be rewarded by spending personal time on my employer's projects.
"Interviews" like this will only grab candidates with nothing better to do than to fulltime interview with your company. In my opinion, the best people already have jobs, and you're excluding them from the process.
I hope no company will make their decision just because a candidate says that he is an excellent developer ;)
Technical interviewing in a broken process. I've given almost 200 technical interviews at Google, and I've seen all kinds of results. But, I believe Google's results. Having worked at Google longer than anywhere else I've ever worked, I can say that the people are incredible, and it's a direct result of the interviewing process. We interview someone for a set number of interviews (N≤8 nowadays) and we make a decision. I can count on one hand the number of people I think are deadwood.
All I'm saying is the "interview" process of having people do projects for you is broken. You will filter out a lot of people with jobs they are kicking butt at.
Also, who says "I am an excellent developer"?
Also, please avoid ad hominem attacks.
This should be easy to settle and show up the other guy. ;)
For example: "This database contains 100,000 problems with standardized parameters. The problem definition is defined in the file spec.txt which you can grab from our code repository. Write the code to solve these problems efficiently, passing each solution to a remote service via POSTing to a REST API, the documentation for which you can find here. Bonus points for parallel execution. Feel free to use any editor/IDE and reference online documentation, Stack Overflow, etc. that you want. If anything's not clear or you need a hand with something, just ask as you would if you were an employee already. Ready to get started?"
The great thing is that once you've identified a candidate, you can do remote screen sharing and have them write code before they even have to come into the office. I've interviewed a fair number of remote people this way and it's excellent for weeding out the people who can talk the talk but can't program worth a damn. And it limits bias because you don't care about much beyond their communication ability plus their technical ability.
This would test programmers ability to learn a new language.
Before the internet my friends and I sometimes used to get to together, everybody got some time to write a Corewar warrior on paper, and then we'd watch the tournament together. It was great fun.
How long do you expect your interview to last?
This problem was addressed nicely in this functional pearl by Jeremy Gibbons, et al.: http://www.cs.ox.ac.uk/jeremy.gibbons/publications/rationals... . As interesting as the result is, however, it's a pretty well-made point that research-level ideas from the programming languages community are not really software engineering interview material in the vast majority of cases.
This is yet another example of "rockstar developer"-itis, wherein startups are given to believe that they need the best of the best when in fact they do not. This particular example is entirely egregious because they asked her about something that requires enumerating the rationals when what they really wanted was an iOS code monkey. Then they fired her, based on their own shoddy interview.
I had a similar experience (working for family, at that), where I joined their R&D group and it went well for a while, but eventually they decided they wanted to get into selling solar power and didn't need a software/electrical engineer, so I was lectured about this or that bullshit and eventually brought into a meeting with the IT director and offered a position on his team. I realize now that they were trying to make me feel unwelcome, but I felt entirely responsible at the time. I never felt like I had a choice in taking the IT position (or lose my employment there entirely), and I hated it. The new boss was, for the most part, a nice guy, but he turned into a major nitpicking asshole over little things.
I'm at a new place now and it's still hard to kick the habit of trying to avoid coworkers because you know they don't like you and want you gone.
Culture has a huge impact on hiring and working. I wish I had been taught about culture when I was in school, I could have avoided the problems I faced had I realized the warning signs earlier.
If one is good and quick in problem solving or has high GMA, that does indicate that he has the capacity to handle new and difficult things in general, but says nothing about the speed with which he can handle a particular new thing. Author's example with JavaScript is very good illustration how difficult can it be to learn a new paradigm for the first/second time.
E.g. if somebody hire John Carmack (ID Software), nobody will let him do some math test or ask him trivial programming questions.
But you are not John Carmack ;-)
It is like in every other job: if you are not a rockstar you are nobody.
It is reasonable.
The author was not saying she was above simple coding tests but that she was an inexperienced programmer who was GOOD at the maths puzzles and got the job then struggled and the interviewer's assessment of her software development ability would have been better with some software related questions.
We life in a world of casting shows, now we (the programmers) are in that shows too. They run only in companies.
I just don't have the experience or tools or interest for them.
And yet, somehow, in 20 years of business geekery I've never come across a problem I can't solve.
Maybe when writing Tetris for J2ME I would have saved myself 10 minutes googling if I'd had the experience to realise that right angle based matrix translations don't require fp maths and maybe when writing financial indicators, I'd have saved myself half a day if I hadn't had to look up integrals but this sort of stuff is definitely in the minority as far as my experience goes.
One example I use is getting the candidate to write crud, list, and search controller actions for a simple category data structure. Given a basic category data model (e.g. Name, Parent), the candidate starts with the crud actions.
Crud actions aren't meant to be difficult to solve and serve as a basic screener to verify the candidate has working knowledge of the basics. The only edge case I look for the candidate to ask about is if orphaning child nodes is allowed (I.e updating parent node, deleting a node with children)
List action(s) start getting more interesting since recursion comes into play. A basic implementation of an action that can load the tree given an arbitrary category as a starting point is expected. If the candidate has some prior experience, a discussion of what performance concerns they may have with loading the category tree is a follow up question. The tree loading algorithm is then expected to be revised to handle an optional max depth parameter. An edge case I look to be considered is how to signify in the action response that a category has one or more child nodes that weren't loaded due to a depth restriction.
The search action implementation has a degree of difficulty scaled to the candidates experience level. All candidates have to write an action that returns a collection of categories matching a search string. Those with previous experience are asked about a paging solution. Senior level candidates are asked to return matching categories in a format that indicates all ancestors ( for instance: "Category 1 -> Category 1.1 -> Category 1.1.1" result for search string "1.1.1")
For an added degree of difficulty, candidates can be asked to recommend data model tweaks and algorithms supporting tree versioning requirements necessary to allow for loading the category tree's state at a given point in time.
The candidate's performance to this exercise seems to give some insight into their level of experience and ability to implement algorithms from a common real world example without having to ask much trivia or logic problems.
Although if you've been doing this interview for a while, you're bound to have come across someone who mentioned it, so I'm sure you're familiar.
Then the GP goes into detail about a tree-like structure of categories and their parents. If you're not familiar with trees and graph algorithms this might be quite daunting.
Do some googling on the terms I mentioned and please ask about specific things you do not understand.
Minor correction: create, READ, update, delete
This is like solving your submarine problem. Jeese.
Yet, I have never had the balls to pursue it professionally. I build stuff and usually never launch it. I have learned several times over that marketing is not my strong suit.
That said, I'd actually like to work for a startup. Hit me up if anyone wants to talk.