> “I think it’s offensive and I don’t like how the industry has standardized on basically assuming everyone’s a bullshitter,”
I think it's weird for people to complain about whiteboards and technical questions, given the implied alternative. There are many other industries if you prefer the traditional hiring model.
That is, if you can get an interview. While you complain about the awkwardness of a whiteboard, people in other industries are quite happy to get a call. And in the interview, they compete in terms of poise, diction, rapport, and other intangibles, without the chance to repair any lack thereof by demonstrating technical competence. And they get 1 hour to make their case, not 3-6 hours.
Did you try to interview at companies paying $80k to senior engineers? Low status companies tend to have more normal interview processes, many don't even ask you to code before hiring you. This way you can get similar pay and interviewing as people get in other fields.
I think it’s interesting when people characterize feedback about technical interview shortfalls as “complaining” ... reveals a lot of the pre-existing bias in this very contentious topic.
That's a solid point. Yeah, engineering interviewing could be better, a lot better, but as we push to improve it, it's really important to keep in mind how privileged we are. I remember having a discussion with one of my friends in a different field who was thrilled to get a recruiter pinging them on Linkedin, for example.
Also, given how chaotic the field is, and how people come from all sides (some with CS degrees, some self taught, some who worked at great tech companies, some who didn't, etc), assuming everyone's a bullshitter is unfortunately kind of expected. We absolutely should treat people with respect and dignity, but that doesn't mean being naive either.
And in other fields getting fired and even demotions are pretty common, while in big tech hubs thats nearly unheard of, except for extreme cases or very very entry level people.
Wasn't about you in particular to be honest (but I like how you owned it). Personally, my official position is "software engineer" and I hate it. We have nowhere near the culpability engineers in other disciplines do.
Ironically it took me a while to even start using that terminology. Where I'm from, "engineer" means something specific, even in software, and you can't just call yourself an engineer (well, there's no LAW against it, but people will call bullshit pretty fast if you didn't go through the right channels). In the US, everyone's an engineer. Im surprised the person who serves me my burger at the counter isn't a Bovine Product Assembly Engineer.
I haven't had much trouble sifting out the bullshitters. There are patterns to their resume and their conversation. You just need to ask the right questions and be able to understand the answers. No coding problems required. I mean, if you can't tell weather a person can actually do what they claim to do when you work in the same field, you have to work on your people skills.
Heard of a PIP or forced grading to the curve at say Microsoft back in the day.
You cant directly compare a professional job to say an skilled worker in a corner store / bodega - we may have similar problems but they express them selves in very different ways.
In other industries their past experience is also believed, rather then tested by random facts. I’ve had an interviewer ask me how to change Swift method names when exposing them to Objective C and then take that as an indicator of my whole iOS skill set.
Most fields where experience is "believed" fall in one of a couple of categories:
1) Extremely regulated fields with strict education, training and certification requirements (eg: a physician)
2) Fields where you have to show something. Designers frequently need to have extensive portfolios. Carpenters need both that and real world referrals of happy customers (not just nearly automated HR checks we have in software dev where no one will say anything bad out of fear of getting sued)
3) Trial period followed by sink or swim. Come work for us and if it doesn't work out we'll demote you or fire you. There's a few places like that (I think its Netflix's model?), but generally firing someone has everyone around calling foul.
The trial period is not without its flaws but I’ve always preferred that over take home tests and puzzles because it gives both parties time to truly assess one another. I’ve done it twice in my career and I appreciated getting paid like a contractor and giving more time to understand whether or not I really fit at the company I was joining. Likewise, the company’s weren’t 100% assured that I was good but I was able to prove my value both times.
I wouldn't deal with the contractor paperwork myself, but in at will employment states that's not even needed. Hire by W-2 and let go after a few months if it doesn't work out.
Thats absolutely my personal preference. The argument against it (very valid), is that if someone with a family relying on their paycheck quits their job to go work at another with that policy, and gets let go after 3 months, they're kind of fucked.
I think its state by state, but even where it is, companies get a pretty bad rep for actually using it regularly on software engineers. In the current political climate it if the company involved isn't one where the koolaid is being drank by the gallon but big enough that it has a reputation, it would quickly end up on Glassdoor and social medias as a big deal.
"Wrongful termination" lawsuits are still a thing. If you fire someone, they can file a lawsuit that is written to make it look like they have a case. You can fight it, you can win, but it still costs you lawyer time, which means it costs you money.
We could have this for our field - and to a certain extent we already do, at least via a github repo. Only problem is, nobody seems to look at them.
People have mentioned that SWEs should have a portfolio - but I've never worked for a company they let me take home code and put it in my portfolio. For web development, I may or may not be able to point to a specific site and show someone "see this here, I developed this part" - and even if I could, how would I prove it? Most of the time, it doesn't matter - as that code/feature/website is likely gone or changed by that time.
So you only have a couple of options to develop a portfolio, and those options only work for some people, not all: Either contribute to an open source project, or work on your own side-projects, putting them all up on github or similar repository you can reference.
But not everyone has the time or inclination to do this.
> and to a certain extent we already do, at least via a github repo.
There are significant pushbacks in some communities that this isn't inclusive, because people can't spend that much time outside of work hours to build one (and as you pointed out, internal code can't really be included).
Graphic designers and stuff are always sketching/drawing stuff on the side for their portfolios though, and that seems pretty expected, so... /shrugs.
>And in the interview, they compete in terms of poise, diction, rapport, and other intangibles
Don't forget the interviewer's arbitrary "system" for weeding out applicants that has no basis in research and varies from one hiring manager to the next.
If you give someone gold long enough, eventually they'll complain how heavy and ugly gold is.
Whiteboards are an unbelievable utopia compared to so many other disciplines. It's an amazing stroke of luck that software developers don't have to do medical-style "residencies". We don't have to rely on letters of recommendation from our professors (hope that professor you slaved away for for six years doesn't have a bad day when he writes your letter!) We don't have to pay thousands of dollars to take excruciating standardized tests to obtain certifications that expire in a decade. We don't have to do unpaid internships, we don't even have to wear business suits to the interview.
Software devs don't realize how many people in the world would kill to have things as good as we have them.
But I'm not sure it's that developers are being given gold and don't recognize it. It's that developers are being given mud and recognize it, while most other people are being given shit instead.
I'd rather be handed a pile of mud than a pile of shit, but it's not hard to feel that there must be some alternative that's better than either.
(Maybe there isn't, though. Matching candidates to jobs is just hard, I think; everyone's incentives are misaligned, performance is difficult to predict, it's amazingly difficult to get past your initial snap judgements or even to realise that you're failing to do so, etc.)
Well, some developers feel like they are being given mud. But developers who are really good at whiteboard interviews feel like they are being given gold.
I'd be thrilled with that. I've got a great resume, great references from former bosses, and have plenty enough confidence in my ability to work contract for a while to prove myself.
The problem is those things only get you in the door, then it 100% comes down to your whiteboard under-fire skills - which admittedly have gotten worse for me the older I get. Although I know I'm a better programmer now.
The problem isn't technical interviews it's bad technical interviews. I think this is pretty "regional", with many people reporting big tech companies in e.g. the Bay Area doing things like have people code textbook algorithms on whiteboards.
Technical interviews where you look at some code and review it, or is asked to broadly describe how to design a system etc isn't unreasonable, and as you say it's better than judging only on poise...
Let’s begin with a technical interview problem. Consider the following coding question from LeetCode, an online platform for preparing software development candidates for interviews:
[statement of the Maximum Subarray Problem]
Before going further—and regardless of your coding
proficiency—we’d like you to spend a few minutes and take
a stab at this question.
From Wikipedia:
The maximum subarray problem was proposed by Ulf Grenander in 1977 as a simplified model for maximum likelihood estimation of patterns in digitized images. ... Grenander derived an algorithm that solves the one-dimensional problem in O(n2) time, improving the brute force running time of O(n3).
Jay Kadane of Carnegie Mellon University soon after designed an O(n)-time algorithm for the one-dimensional problem,[1] which is clearly as fast as possible. The same O(n)-time algorithm was later automatically derived by algebraic manipulation of the brute-force algorithm using the Bird–Meertens formalism.[2]
If you really think anyone -- short of a faculty-level algorithm specialist at places like CMU -- can be reasonably expected to derive an optimal solution to problems like these on the spot (as opposed to the way candidates are actually forced to prove their ability to "solve" them: by binge-cramming on dozens and dozens of problems like these for weeks on end, so that they have at least a 50 percent change of passing your "filter") --
the you're either very naive, or deliberately kidding yourself.
I've looked at your test results and Im sorry to say you did not pass the test. You need to work on your algorithms. Can we keep your CV? /automatically generated message
> If you really think anyone (..) can be reasonably expected to derive an optimal solution to problems like these...
I can see an usefulness to questions like these. Maybe you are precisely not looking for reasonable but for exceptional people. Or maybe you want to see how the candidate behaves in front of a difficult and concrete problem, even if you do not expect them to solve it.
The unstated implication in most cases is that you damn sure are supposed to solve it - or get "flushed".
As to "seeing how the candidate behaves in front of a difficult and concrete problem" - I'm sure this is what a lot people who ask questions like these think they're achieving by asking them. But again, you have to ask yourself if that's what the questions actually do permit you to assess.
My sense is that it's not - and that questions of this sort are mostly, well - cheap gimmicks, basically.
Think about the potential hidden states (already knows the answer vs blank) vs the evidence you could get from the person answering right or wrong. What should your prior be? Surely that there's a very large chance this person is not a genius. Now how much is that going to move, given that by far most people are not geniuses?
To put it another way, what would you do to impress someone following your reasoning? I would pretend like I didn't know it, and then act out the miraculous direct line to the answer.
As for looking at how people behave in front of a hard problem, what on earth do you know about that? Do you have evidence connecting people's behaviour to subsequent on-the-job performance, both for people you hired and those you didn't?
Presumably asking questions like these would be fine if the expectation to solve wasn't there, and instead their performance was evaluated based on how their reasoning/approach changes with hints and discussion with the interviewer. But instead gotta be able to solve at least 1 question and make good progress on a follow up to be considered for an offer from a FAANG company.
Now if you'll excuse me I need to grind out some dynamic programming problems.
> to be considered for an offer from a FAANG company.
I don't disagree with the main point, but FAANG companies' interview questions are available all over the net. In many cases (most?) they'll literally tell you what they are ahead of time, so it's a lot more of a filter for grit than a filter for computer science aptitude.
Eg: I know for a while Google was pretty keen on asking about A* algorithms. A friend who worked there mentioned that to me. Not having a CS degree myself, my first reaction was that was a very "either you know it or you don't...maybe you can figure it out, but that's not gonna be fun" kind of situation.
At some point i saw the material Google gives out. A* was literally listed there as something that they were likely to ask.
Now, it does mean preparing for a very specific interview, which not everyone (or even most people) would be willing to do. And today I don't think Google has the reputation to pull that stunt off for much longer. But a couple of years ago, if you really wanted that free cafeteria? Its a small price to pay.
When I last interviewed with Google (which was about 8 years ago, I think), they explicitly told me not to post my questions to the internet, and if I recall correctly, had me sign an NDA about the questions because they want to reuse them. I don't know if that has changed in the intervening years.
They'll generally give a list of topics but not specific questions. For example facebook may put an emphasis on graph and tree data structures and search algorithms
My info is also pretty out of date, but for a long time at least they would give a PDF/flier thingy with a "how to prepare for your interview" that had all that stuff listed, and it went in a lot of details.
This is one of the problems with sites like HackerRank that lots of companies seem to be moving to for testing applicants. I know almost all of us hate whiteboard problems, but I would definitely prefer the opportunity to talk through my thought process on a problem rather than having X number of minutes to get my code to pass a unit test or I fail the interview.
Presumably asking questions like these would be fine if the expectation to solve wasn't there
What I would usually do with a programming question is ask a number of my coworkers to solve it and use that to evaluate what an "average good performance" looked like. So if most of your coworkers can solve a problem in 10 minutes, it's reasonable to expect that from someone you're interviewing.
> gotta be able to solve at least 1 question and make good progress on a follow up to be considered for an offer from a FAANG company
It's a little harder than that. That would be for the screening interview. But on-site, you'll get several algorithm interviews (I had 3 in two such companies, plus 2 system designs). The recruiter told me that you could fail one of the interviews, but you have to nail the four others.
Even if you are well-prepared, I find it quite hard to have consistant results on these algorithmic questions, especially considering the stress of the situation and the fatigue of going through several interviews in a raw. Practicing leetcode problems for months is fun, up to a point (especially if you already have a job).
That being said, I like that the rules are clearly stated and everyone is given a chance. They don't discriminate as much on your background/educations as some other companies.
Another good one is the "does linked list have loop" question. If you don't know the answer, you have to figure it out from inspiration. Took over a decade for the first guy to do it.
but that one should be, in fact, well known. i've been out of school forever but surely it has to be taught now?
so, this question isn't testing if you can find a loop, it's testing whether you have enough experience to have heard of it. which is a fine enough thing to test for.
i understand your point, that questions are often designed poorly, but your example isn't necessarily a good one.
I think this is overstating things. Because of the way bleeding edge research gradually trickles down to being generic undergrad content, things like this that would've been the domain of algorithms researchers in the 80s are now tractable for smart undergrads who are into competitive programming.
Now tractable for smart undergrads who are into competitive programming.
Even if we leave aside the question of to what extent facility at competitive programming contests actually correlates with success in real engineering environments (my own sense is: yes it can help; but at the end of the day, just not all that much) --
if that's a skill you consider to be important, then for gosh sakes, put it in your job ads. Something like "We typically hire CS olympiads, and folks who have at one point or another have been fascinated by competitive programming contests. If this doesn't describe you, then this probably isn't the right company for you."
This also happens in math and physics. One of my homework assignments was solving the quantum harmonic oscillator via path integral formalism. Something that took Feynman a modest amount of time.
Just as a data point: I read the paper and thought "that sounds like it might be fun, so let's have a go" and wrote down a (correct aside from an off-by-one error) O(n) solution in a couple of minutes. I am not a faculty-level algorithm specialist at CMU. I don't recall seeing a solution to this particular problem before. I am a mathematician working in industry doing kinda-algorithm-y things, though. There are at least two other people working at my (small) employer whom I would expect to be able to find an O(n) solution fairly quickly. (To quantify: >=75% chance that they find one, given 20 minutes to think.)
The fact that the first couple of people to consider the problem didn't find efficient solutions doesn't mean that finding efficient solutions is a super-hard problem. Grenander wasn't really interested in the 1-dimensional problem but in the (I think distinctly harder) 2-dimensional problem, and I don't think he was really an algorithms guy. (That's not a polite way of saying he wasn't very smart; he clearly was, but I don't think finding optimal algorithms for things was what he was most interested in.)
For the avoidance of doubt, I don't think you're likely to get much insight into interview candidates by seeing whether they spot the most efficient solution to a problem like this unaided -- even an extremely strong candidate might just happen not to think of a good approach, especially under stressful interview conditions. (It might still be a useful question if a less adversarial approach is taken -- discuss the problem with the candidate, get a sense of how they think, and nudge them in a helpful direction if they get stuck -- or if the only question is whether the candidate can find and implement any approach, which for this problem I think any reasonably competent software developer should be able to do since you can basically just translate the question into straightforward but inefficient code that does the job.)
I'm not sure whether we actually disagree or not, but I do think you may be overstating the difficulty of this particular problem.
It's possible to use dynamic programming but it's such a simple example you don't really have to. If you're familiar with dynamic programming at all, you should be able to get it - it would be pretty reasonable to assign this as a introductory homework program for the dynamic programming section of an algorithms class.
Another data point: but I checked my LeetCode history and I actually solved this problem[1] a year ago. It's flagged as "easy" on LeetCode and I have one submission (which was accepted) but I don't even remember solving it, indicating it was probably routine. The problem also has 4809 "thumbs up" vs 177 "thumbs down" which is a strong indicator that others didn't find the problem too hard either; problems that don't match their stated difficultly get downvoted hard.
According to the Wikipedia page[1], the brute force solution is O(n^3) for the one-dimensional case. They don't give an algorithm, but the obvious pseudo-code is:
highest_total = 0
for all i from 0 to length-1:
for all j from i+1 to length-1:
total = 0
for all k from i to j-1:
total += data[k]
highest_total = max(total, highest_total)
return highest_total
It takes three nested loops, with the third being the sum from i to j. This is obviously a very naive algorithm, but that's why it's called brute force. According to Wikipedia, Grenander apparently improved this to O(n^2), and then Jay Kadane came up with the first O(n) solution. For the 2D case, brute force is actually O(n^6).
Huh! That's a bit more brutishly forceful than I was expecting, and I concede the point. It hadn't occurred to me to stick that innermost loop in there rather than keeping a running sum in the second loop.
The brute force solution that seems most obvious to me is to pick a starting index and an ending index out of all possible indexes such that start < end, and calculate the sum of that sub-array (keeping the biggest result).
This is O(N^3) because iterating over index pairs is O(N^2) and doing the sum is O(N).
You're right and I stand corrected - it's not that hard of a problem. I found this out when (sometime later) I actually sat down to solve it, without considering any of the hints posted elsewhere in this thread.
At least there seems to be some general agreement (within the thread) that, while not super-hard -- it's still basically a "eureka problem" -- that is, even among what we may consider to be adequately good engineers -- some will get the visual solution more or less right away; but a fair percent (25 or more) won't.
And that's given ideal circumstances -- working alone and in unhurried circumstances. Throw in the (considerable) distraction of having to think aloud in front of other people (who themselves may not exactly be providing the most relaxing vibe, to say the lease), and all the other stress that comes with interviewing -- and I'll bet that 25 percent failure rate rises quite considerably, in turn.
And in many cases -- if you fail just one of these "eureka" problems -- and maybe fail to provide the expected canned response to a "behavioral" question or two -- than that's pretty much all it takes. To be considered, you know, "not a fit".
As to why I jumped the gun -- fatigue, probably, from being fed a few too many "cute" problems like these.
If you really think anyone -- short of a faculty-level algorithm specialist at places like CMU -- can be reasonably expected to derive an optimal solution to problems like these on the spot
Eh, you really don't need to be a "faculty-level algorithm specialist" to solve this problem. I wouldn't even really call it dynamic programming.
Just make one pass to calculate the sum of the first n numbers for each n. Call that array prefixSum. Then you make a second pass calculating the lowest prefixSum[i] for i <= n. Call that smallestPrefixSum. The maximum subarray sum is then defined by the largest prefixSum[i] - smallestPrefixSum[i], and you can get the subarray itself with a bit more tracking.
I don't know if it's a great interview question but I think most of the engineers I have hired (at Google or Facebook) would be able to solve this problem on the spot, given 15-20 minutes.
Well, kind of. I would call it dynamic programming if you thought of the algorithm as calling some function many times in an inefficient version, and then caching that result to make it efficient. That is probably how the textbook would describe it, too. I don't think you need to understand dynamic programming to get this, is all.
I used to work with a guy who tech-screened applicants by demanding that they come up with the binary-search algorithm: “given a sorted list of numbers, find the fastest way to locate an element in the array”. Of course, if you had any university-level exposure to CS, you would have already seen the solution - I sort of suspected this was his tricky way to filter out non-CS graduates.
> “When Jon Bentley assigned binary search as a problem in a course for professional programmers, he found that ninety percent failed to provide a correct solution after several hours of working on it, mainly because the incorrect implementations failed to run or returned a wrong answer in rare edge cases. A study published in 1988 shows that accurate code for it is only found in five out of twenty textbooks. Furthermore, Bentley's own implementation of binary search, published in his 1986 book Programming Pearls, contained an overflow error that remained undetected for over twenty years. The Java programming language library implementation of binary search had the same overflow bug for more than nine years.”
The bug only manifests if the range you're searching is within MAX_INT/2 (or whatever numeric datatype you're using). I'm not sure I understand the point you're trying to make.
Yet you focus only on one aspect of the linked discussion on the high rate of misimplementations, an aspect that represents the least relevant part, and then act as if this invalidates the point?
I didn't say it invalidates the point, because I honestly don't know what your point was to begin with. Which is why I asked the question originally. I'm still no closer to knowing, and it seems like you're not interested in elaborating. Suit yourself.
This is an extremely disingenuous reply as well. You started off your first reply by saying “the bug only manifests ...” already pre-supposing that the scope of my original comment was restricted to that one thing, and tacitly ignoring all the other aspects of the link.
The (obvious) point is that binary search is widely misimplemented even by professionals writing textbooks (and this had little to do with the specific Java bug responsible for that one example), and this is all stated directly in the link you didn’t read.
It’s comically ill-suited for time based tech screens as even experts teaming with editors often still get it wrong.
I didn't understand what you were talking about offhand and thought range might be the value of the stored/searched data, not a pointer arithmetic issue.
When the array/list consumes a memory footprint larger than ptrdiff_t it was possible that the specific implementations might fail. It is crucial to remember that the difference between memory address is signed and that the size of objects (even if bare machine integers of some size) in the set might be displayed as a number smaller than one of the powers of two many recall.
On a 32 bit machine this wouldn't be approx 2 billion but rather often 500M 32 bit words or 250M-ish 'double' precision numbers. Let alone if the search is over more complex structures representing objects. For a smaller 16-bit embedded system I could even more readily see this occurring.
You make a good point. I was actually speaking about the general case, where you're simply pulling the bounds of x_min and x_max closer until f(x) reaches a target value or the bounds converge to the same value. f could be an array and x could be an index, but it doesn't have to be. The bug still exists in this general case. All you need is a numerical datatype that can overflow, and most of them can.
Divide and conquer? I don't have a cs degree, but this was thought in highschool in Romania. If you went to the right kind of high school.
Reading the other comments I'm not sure d and c is the answer. Might be terminology, but I would split the array in two at the middle, see if the value I'm om is the one I'm searching for. If not, recursively do the same thing on the left or right side of the array.
I wouldn't characterize it that way, I don't have a CS degree and could solve this trivially.
I think the question first asks the interviewee to have cracked an algorithms textbook or be clever enough to come up with the method on the spot.
Second, it asks if you can explain the concept to another engineer. I've met engineers in their 40s who can implement something like depth first search but cannot explain it. That's an immediate red flag to me that, at the very least, the person I'm talking to is not able to meet the responsibilities of a senior engineer.
It seemed like a bit much to me at the time. It's not something that I've ever had occasion to actually use (I don't search sorted lists much...). I wondered if I would have been able to come up with it if I hadn't already seen it in data structures class years ago - I mean, of course I figured out the "algorithm" when I was a little kid playing "guess the number", but I'll never know if I would have come up with it in that context if I hadn't already seen it. The consensus here seems to be that it's a reasonable question, though - but I do remember that the only people who ever got it were people with CS degrees.
I was a TA in a few undergraduate algorithms classes in at UIUC. This level of problems can show up in the exams.
Although I hold UIUC students in high regard, certainly we don't expect undergrads to be "faculty-level algorithm specialist."
The discovery of many things is done by people with a genius-level intellect. Like calculus, probability, etc. but I can certainly ask mechanical engineers calculus problems, or data scientist probability problems.
It is certainly difficult for someone who never seen anything related to it. But for people who had the background, it is not unreasonable. I think this kind of problem is biased toward people who
1. had a good (theoretical) CS education, or
2. solved lots of leetcode problems so they can just do "pattern matching".
I was absolutely rekt by this exact question several years back for a Microsoft internship interview. I had just finished my first CS class where I learned binary search, heaps, hash tables, but unfortunately not cumulative sums.
I don't think this problem is THAT difficult -- I did manage to solve this on my own after a semester of algorithms class and a bit of practice from DP problems. The intuition is that at each index you're either continuing a previous subarray or starting a new one, depending on which one results in a larger sum. It's pretty standard DP stuff, and I'm sure there's many undergrads out there who can bang it out in an interview. Though I personally did it in the comfort of my home and probably wouldn't have come up with it in an interview.
Honest question, which I can't really answer myself: Do people think the hiring process for developers is more broken as compared to other industries? My personal experience says that interviewing as a developer is often more straightforward and less arbitrary than what my non-tech friends have experienced.
Can anyone who has made career changes to or from tech give their experience?
All hiring is extremely arbitrary. The reason coding is so bad is because there's this huge myth of lower class workers becoming coders overnight, just by studying for a few months of leet code.
There is an essay about the feminist movement in the 70s (https://www.jofreeman.com/joreen/tyranny.htm). It makes the reasonable point that people are likely to hire/recruit people similar to themselves. The code interviews are expected to remove this, if you stick a bunch of CS people in a room and tell them to make an unbiased interview process; you'll get the hell world we live in today. To get a job on a sales floor, most of it is just impressing the existing gate-keepers.
Coders live in a fantasy world where gate keeping isn't real. Except it isn't possible to remove that from hiring, no one wants to hire someone who doesn't integrate well in their team. So we have this hellworld where people can study for 3 months on leet code and nail the tech interview, they still won't get the job. Even though the "unbias" process says they should.
Most of other high skill professionals in other engineering fields that I know have a fairly easy hiring process: their degree + their license + their previous working experience is enough. The interview is basically just an hr screening to avoid jerk personalities.
If we could just accept that not all companies should cater to everyone, and each can pick their methods that work for them (and if you don't like it as an applicant, go elsewhere), things would be a bit simpler.
When Im looking for a job, I make it pretty clear what Im ok and not ok with, which will dictate which companies Ill interview at or not. If we were in a dotcom crash it would be a different story, but right now its definitely something people can do.
"- Credentials, including graduate degrees, are no guarantee that someone is a good engineer"
Too true, I've started to recruit non-CS people and teaching them how to code. The state of CS education in the US is insanely inconsistent. But I'm lucky, I have lots of time to train people; most companies can't handle months of startup time.
Want to hire me? I’m a reasonably competent programmer already, pretty good designer. Most recently I wanted to familiarize myself with the basics of C++ so I wrote my own big integer library.
I'm OK with whiteboarding. I don't object to take-home assignments in principal, but I do object to them in practice -- they take up way too much time, and then part of the time you get ghosted after submitting them.
I think the right answer here, to the extent that there is a right answer, is to give applicants a choice.
It’s pretty hard to have a reliable interview process if you don’t ask all of the candidates to do the same thing. Comparing how a candidate performed compared to other candidates on the same set of tasks is an important part of the evaluation.
Consistent processes are also important for avoiding bias. For example, take-home projects are more likely to make sense for young candidates who aren’t raising families and have free time after work. If you have most of your young candidates doing take-home projects and the older candidates doing whiteboarding, you will end up evaluating candidates differently based on age.
Candidates have different strengths and weaknesses. Sure, I'll ask the base questions but in terms of assessing their coding ability I'll go with whatever they feel the most comfortable with.
If they have github, I'll ask them to walk me through a few files, asking their reasons for doing X, why not Y - what does this do?
If they prefer take-home, I'll give them something to create/improve.
Want to do something now? knock yourself out - with or without me watching.
I don't need to sort the candidate into an ordered list, just filter out the "over confident".
The questions I want answering is: Can they actually code? Does their skill match their experience? Is their skill level similar to the codebase?
The only time I've ever told a recruiter off was when I was asked to "spend about a half an hour to write a solution for [problem]" for Microsoft and it turned out to take about that long for me to set Visual Studio up on my laptop at home, and then an equal amount of time to figure out how to navigate the project files, etc. etc. and I could see that the interviewer A) had not given this any thought and B) had no idea how long this would actually take for someone who doesn't develop software at home.
That was assuming you even had a Windows laptop to set VS on. I know a lot of people have to say no these days since they just have a MBP running OS X.
> I know a lot of people have to say no these days since they just have a MBP running OS X.
That's where the person who knows how to set up a VirtualBox VM running Windows and VS would have a large advantage; when they brought in the result, showing their MBP running VS in a virtualized manner, and explained what they had to do - that's a bit more of an impression than someone who says "they can't do it because MBP".
You can actually run Windows just fine on a mac using bootcamp. The big problem is just getting a legit Windows license, and then the hurdle is just a few hundred bucks (or whatever a non-OEM non-upgrade license costs these days).
Also, I'm nowhere near ops jobs, so I don't think those people interviewing me would care much about those kind of logistics (anyways, they wouldn't want to know the details). It wouldn't score any "points" beyond just getting the task done at all.
> for someone who doesn't develop software at home.
Are you saying in general - or just for Windows-based software (since you mentioned needing to set up Visual Studio)?
The latter I can understand; but the former makes me want to ask, "Who applies for an SWE position who doesn't write code at home?"
I've been an SWE professionally since 1991 when I was 18; I got my first computer in 1984, and have been "coding at home" ever since. I'm not saying "to be an SWE you need a similar experience" (I know that's untrue) - I'm just relating that I've been coding for a long time, and "at home" is very much a part of it.
Then again, I've heard of people who absolutely hated coding, but were extremely good at it (and I assume made good money doing it), but when they went home, they didn't even want to think about such a thing. Maybe you are similar in that regard?
I guess I am just curious and a bit fascinated by your comment...
I'm someone who does it for 8-10 hours a day at their day job. My fiance works in a retail store. She doesn't come home and immediately start doing retail sales out of her house. Another friend is a teacher and she doesn't run a school out of her house.
Professional baseball players rest after games. They don't finish up and then immediately start practicing again. And soldiers and police get down-time.
Just because I'm doing something I enjoy and that I want to do, doesn't mean I should do it all of my waking hours. I'm not doing this to code, I'm doing this because my employer is paying me to build cool shit and I like doing that. I don't lift weights 24/7, I don't run 24/7, I don't hike 24/7, and I don't program 24/7.
Frankly your attitude is one of the reasons why we don't have any diversity in this profession at most self-described tech companies, and it also underlies a lot of the seeming ageism today. All of these self-described non-conformists expect everyone in the profession to conform to their attitude and it's sickening.
True, they could be optional. You may not get the job if you don't do it. Honestly, if you can't spend one or two hours on a take home assignment during a 7 day period, you certainly have different problems than applying at this company.
> Pair programming during interviews makes people uncomfortable
I agree, pair programming during an interview is mostly a no go. Not because of the issues you mentioned but because most interviewers are totally unqualified to do this properly (unfortunately, most interviewers are also unqualified in general to conduct interviews). It takes a lot of skill to conduct pair programming interviews and draw the correct conclusions.
> People who currently have good jobs are unwilling to consider contract-to-hire
In US, there is practically no difference between those three. You can be fired any day anyway. But yes, if you have a secure job that is good, the bar for switching is high, but so what? Why even switch? It's your choice then... Luxury problem!
> Credentials, including graduate degrees, are no guarantee that someone is a good engineer
I would go as far as saying your credentials and degrees are horrendously irrelevant in predicting your skills as an engineer. The only thing you might be able to conclude is that a GPA 4 M.I.T. graduate probably can pick up missing knowledge quickly, as opposed to a no degree dish washer who learned "how to code" in a 6 week bootcamp.
On that note I have experienced first hand, Berkeley and Georgia Tech CS majors, who were absolutely incompetent on the job and just had no idea what the hell they were doing, with little improvement in sight...
> Honestly, if you can't spend one or two hours on a take home assignment during a 7 day period, you certainly have different problems than applying at this company.
That's an extremely privileged thing to say. And what a terrible way to start out your working relationship. The very first thing you ask of me is to do unpaid work in my free time? IF you're willing to do that, how else will abuse me in the future?
Also, why should they spend time on your homework before they know if you're a company they would even want to work for?
I mostly agree, but attending any interview is also doing unpaid work in your free time, and often takes a comparable amount of time to the sort of take-home assignments used for recruiting.
I think the reason why take-home assignments tend to feel more unreasonable is the asymmetry. If I attend an interview, it's costing a pile of my time, but it's also costing a similar pile of time for multiple people at the company where I'm interviewing, which gives me some reason to think that my application's going to be considered seriously and therefore that the effort I'm putting in isn't going to be totally wasted.
Whereas a company can hand out a take-home assignment with no effort at all, so the fact that I've been given one doesn't indicate that the prospective employer sees me as a serious prospect or that there aren't hundreds of other candidates in the same position as me, so it's no guarantee that my chances of getting hired aren't miserably low, so there's more risk that I'm wasting my time.
As someone who uses take-home assignments as part of my hiring process, I disagree that it is no effort at all. We still have to review submissions, get the code running (we usually let the candidate do the problem with the tools of their choice). You won't believe the number of candidates we receive with great looking CVs who struggle with a warm-up coding question like this: https://www.hackerrank.com/challenges/python-quest-1/problem
If anything, completing the assignment and at having some working code already gets you above 90% of applicants.
One of the sibling comments said it well. It's not the money, it's the fact that giving me take home work shows a lack of seriousness on the part of the company.
Although perhaps an escrow system would work. The company pays me a large sum (say $10,000), which I then pay back if they call me in for an interview. That way I know they are serious, and it actually forces them to consider the cost to themselves.
It's not perfect, but personally far preferable to having to perform with a gun to the head in an interview. Which has nothing in common with how software is written in the real world.
The start of your working relationship is really important. I'll get interview offers all the time and if the compensation is off by more than 3% I just tell them no, due to compensation. Usually I get a contact back asking what I was looking for and I just tell them that I am not in the market to be making my compensation an issue each time I want a raise. They had an idea what they wanted to pay and I am not who they are looking for.
One of the sibling comments said it well. It's not the money, it's the fact that giving me take home work shows a lack of seriousness on the part of the company because my time investment is much greater than theirs.
I just can't understand this mentality. You are the one who wants the job, doing the bare minimum of effort to prove that you can perform the job is too much? Why should they review your code before they know if you're someone they'd want to work with? It's part of the mating dance, and it goes both ways. Presumably you would have read up on the company or spoken to people in the industry? Unless you have some other way of proving your ability (Github projects, Kaggle, research) don't expect a job to just land in your lap.
> credentials and degrees are horrendously irrelevant in predicting your skills as an engineer
I'd suggest that an MSEE or MSCS degree from a good school tends to be pretty good training for working in the industry. The projects you complete are usually challenging, relevant, and intentionally educational.
For example, an MSEE might learn how design analog amplifiers and filters, program a DSP, lay out VLSI circuits for an ASIC, implement a RISC CPU in verilog, implement an internet router on an FPGA, and implement Wi-Fi on a software radio. I'd be happy to hire (or work with) someone who can do all that!
Yeah except that for 99.9999% of software engineering jobs, that knowledge is irrelevant. Also, to be a good engineer, you need to have MUCH more than just some math and raw computer science knowledge. I would go as far as saying that all scientific tasks are mostly carried out by research scientists these days anyway. The skills you need at companies like Google or Amazon, as a regular engineer, have VERY little to do with what you learned at University, which is why we hire even PhDs as junior engineers, and they usually are. They know a lot of stuff that doesn't matter (like how to color a graph), sure, but they also don't know a lot of stuff that matters (how to build production quality software, without shooting yourself into the foot at every turn).
I really wish university CS programs would have a course on how to use source control (probably Git), and CI (probably Jenkins).
I'm sure some programs teach this stuff already. But I've hired people with graduate degrees in CS from well-known schools who didn't even use source control for their group projects in grad school -- they just emailed the source files to each other.
>> People who currently have good jobs are unwilling to consider contract-to-hire
> In US, there is practically no difference between those three. You can be fired any day anyway. But yes, if you have a secure job that is good, the bar for switching is high, but so what? Why even switch? It's your choice then... Luxury problem!
It seemed weird to me that we are one of the only industries (well I guess engineering in general) who gets a contract that is contingent on each day until it is completed. At any given point you could be let go and yet if I contract someone to work on my home to legal work you setup a contract that states you owe them the money unless you are willing to go to court to contest their work.
When I last did a contract-to-hire it was basically "you can be let go for any reason, from slow output to your boss having a headache." Unless you get a sweet deal its difficult to pull a job with a set start / end time with guaranteed payout regardless of employment. I busted my hump at that last job so instead of doing my 6 months for the hire option they offered me the job before then 2nd month was over.
Not only that, but the GP misses the point that, while yes I can work for myself and do contract work, I then also have to find health insurance, pay taxes monthly, do a bunch of extra accounting that does nothing to move my business forward and is boring, etc. That's fine if you get more benefit that the resource sink of doing those things, but for most people that's only additional stress and no additional value. So saying they're equivalent is patently absurd.
Yeah, maybe degree isnt a great indicator of a programmer's skill, what about their past experience and projects? I think that at would at least make someone more confident in their abilities.
Paying doesn’t really change the dynamic very much. If the issue is that people don’t have a bunch of time to do the problem, paying them a few bucks doesn’t really address that.
Even small-mid size companies can see thousands of applicants for a single position. You cannot pay every one of them at market rates for even a day of work without an unlimited recruiting budget.
The code you write for these take-home tests will be evaluated and thrown away. It isn't actually making its way into a product or feature the company releases.
I'm sure it's being done in some cases (but then any shady thing you can think of is being done by at least one company out there). But it is definitely very far from the norm.
I wouldn't offer a take-home task to anyone who wasn't already on my short-list. When I recruited interns at job fairs there were plenty that were just passing resumes out to whomever would take them.
How significant is that take home? I immediately reject such requests of unpaid work if it takes more than 10-15 minutes.
If in your case, you demand allocating more time for that step, consider adding additional filter (requiring 10-15 mins) after the resume review. And then redirect [part of] resources (money) from the onsite part to pay for that take home task.
Another red flag I see in your process - asking for allocating time before even talking to a human being - that would be another reason for an immediate rejection. Consider moving your phone part before the take home task.
Every company that has given me a take home exercise gave me a phone screen first. I don't think your order is at all typical.
When there is something before the phone interview, it is a coding assessment; these are generally different from a take home assignment in that they are generally timed and automatically scored, e.g. you hit start and have two hours and then a computer scores you. With take home exercises, I've always had engineers at the company read my code and discuss it with me.
I have to say that from all of those, the least annoying has been the pair programming.
But to be fair the problems were not too hard, the interview was set-up properly (problem files on place and a good code editor at hand, no tricks or gotchas)
And the worse of course is having to write fizzbuzz in a whiteboard, I just have to roll my eyes then
Honestly, I really like whiteboarding. I liked doing programming contests too, which are like whiteboarding except harder and you spend more time at them. There are times when I have gotten together with other like-minded software engineers and given each other whiteboard programming problems for fun. (Also to test whether problems are good interview problems.)
A lot of people like whiteboarding problems and like interviewing with whiteboarding problems. Those are usually also the people who pass whiteboarding interviews. Just because there are a lot of complaints about them, don't assume that represents everyone.
I like whiteboarding too. It's stupid to expect people to write syntactically correct code on a whiteboard, but totally reasonably to use it to discuss a problem. I mean, that's why we have whiteboards in our offices in the first place, right?
First, you’re not going to get your resume in front of me without a college degree, CS degree or equivalent experience is a bar that recruiters apply well before I’m involved.
But as far as I can tell, Google CodeJam is an individual sport (I may be mistaken about that, but Google’s page won’t load in my browser, so I could only read the Wikipedia page about it).
The key thing about the ACM contest is it requires BOTH effective cooperation with a teammate AND the desire to solve programming puzzles.
Now, just to be clear, I’ve fired people who passed this bar; being a smart puzzle solver who’s social enough to collaborate is a great start, but it turns out lots of people just aren’t interested in shipping software. I still think ACM team participation is a strong signal, it just doesn’t cover the fact that being an adult means 90% of every job is boring, and not everyone is ready to do the boring stuff when they’re fresh out of college.
They are also indicators of over qualification; e.g. will that PhD from CMU really be that interested in working on your CRUD app, will they find the work fulfilling enough to hang around?
Candidates should just be given a horrible, undocumented, broken piece of legacy code and asked to fix it. Select the person who comes up with the quickest solution. That's who you want on your team when you find out there's a production issue at 4:30pm on a Friday afternoon.
The best interview experience I had was when I was asked to bring in my laptop with some code. I had a very unfinished side project, so I talked the guy through it, he asked questions then asked me to add a simple feature. It didn't take a huge investment of my time, its code that I am familiar with, so no nerves problems, no clever algorithmic stuff.
The real solution is to allow me create a portfolio of work, which would be a total of about 100 hours and then let me use that _everywhere_. I have easily spent 5k hours getting the degree, the 100 hours over the ears it took to do that isn't a big deal. We could even have some sort of offical stamp that "this guy can code".
Doing 10 hours of work to apply for every random "we make another business PHP/crud website" company is not worth it.
Your company isn't Google, don't interview like it is. I am not going to write custom code for you, but I am willing to flash my credentials.
Last time I was looking for a job (about 3 years ago in Seattle) I had a lot of trouble getting through the initial screening to even get to a technical interview. I didn't have a lot of luck until I started applying through AngelList. One notable thing about AngelList applications is that there was no HR/recruiter screening step you had to get through first.
I just went through the interview process with multiple Bay Area startups. Competency and experience don't really account for much and it is much more about the companies internal politics. People are thrown in a room without any evidence based HR training to wing it. The processes are ridiculously bureaucratic, in most scenarios you need approval from 5 different people who at any time can decide they don't like you for their own personal reasons. Given a 50/50 chance with each person, that is a 3% chance of getting an offer. People have no idea what they are doing.
Just for some friendly and non-judgemental feedback from a different point of view, to me that sounds like more of your own fear coming out than reality.
Everyone who interviewed you also said the same things out of fear before they were hired, and then they got in and didn't manage to change it. That could mean it's not viewed as a problem from the inside...
Companies are hiring people onto teams, not to hero-solve pure code problems in a vacuum, but to work together with other people. Communication and personality and attitude are all super important, usually more important than technical skill. (That doesn't mean technical skill is unimportant, it means precisely that people skills are more important, for the average programmer candidate.) It's a feature, not a bug, that team consensus is needed. And there's rarely 50/50 chances, using that to speculate wildly on your chances will prove incredibly inaccurate... the most common case behind the scenes is that more or less everyone agrees that either the candidate is right, or that they're not right, or that everyone is meh on the candidate so nobody is pulling for them.
Chances are determined by how many people are applying for the job versus how many jobs are available, with a side of how long the company wants to take to fill those positions. The company has the upper-hand knowledge of what the candidate pool looks like, and they can hire the top 10% of those candidates who apply. As an applicant, you have no idea and no control over that. All you can do is improve yourself, research as much as you can, try many options, and try again when it doesn't work out.
I'm working at a unicorn. It's a relatively unknown one. They have no idea what they're doing. I've talked to peers across the bay and with my various companies I've worked at - they almost universally don't know what they're doing.
I've interviewed at over 100 places in the bay area. I'd say that from internally and externally, most of the companies here have no idea what they're doing. Most don't use rubrics, most don't have a standardized set of questions that have been heavily reviewed and tested, there is either no training or inadequate training, and most are winging it completely. Many ask whiteboard questions to interview candidates and they never had to do them for the job they got (got in through referral or at a different stage when they weren't doing such questions). Resume, referrals, and charisma will get you into positions very easily in many of these companies even if you're very technically challenged compared to others with less resume+referral+charisma.
Being liked is one of the best ways to get into positions you are terrible at but compensated so well for.
I hear you and I can very much agree ... especially for startups. And the majority of companies are startups. But! I don’t know exactly what “they don’t know what they’re” doing even means, it’s not well defined. That sentence implies you have some specific unstated expectations. What are you expecting, and why? What I’ve seen a lot of in interview and hiring threads is a lot of expectations that aren’t fully realistic.
Does it matter if someone who didn’t get asked whiteboard questions is asking you whiteboard questions? That question is implying some vague concern about fairness, and isn’t focused on whether whiteboard questions are a good idea. On the positive side, at least they’re asking coding questions and not hiring 100% based on resume and charisma, which definitely happens in other industries.
Younger programmers especially seem to think more often that highly reliable technical evaluation in interviews is both achievable and desirable above soft skills. In my experience that view may be too narrow.
I’ve interviewed and hired many people over the years for several companies, large and small, and no matter how much advice I offer about what is actually important when interviewing, it seems like candidates rarely listen, and instead complain that “they don’t know what they’re doing”, as if they know better.
I honestly think interviews are legitimately scary, and that fear comes out in various ways with programmers suggesting their ideas for “improvements”, but by and large the system is working, good companies are finding good people. I know there are a lot of interviews that are stupid, and I know there are bad companies out there. The suggestions in the article won’t fix that.
Being liked is a good way to get hired period. Being liked and being good technically is better. Being technical and not likeable is a good way to not get the job. It has very little to do with the occasional cases of incompetent people being hired and paid a lot.
The problem with hiring in our industry is the lack of licensing/certification that everyone trusts. When a doctor interviews for a job, they don't ask the doctor to perform six surgeries in a day for free and then say, "we'll get back to you". That's because they know that if the doctor holds a license and a degree, they have the necessary skills.
We don't have that in our industry. I interviewed countless "senior engineers" with degrees from well regarded universities who couldn't write a simple loop. Therefore I must now conclude that a degree from that university means nothing.
There are some folks like Tripplebyte who are trying to solve this by pre-qualifying candidates, but I don't think the problem will really be solved until we form a software engineering licensing board. Take a test once, and then take a refresher course every couple of years to maintain your license.
> When a doctor interviews for a job, they don't ask the doctor to perform six surgeries in a day for free and then say, "we'll get back to you"...have interviewed countless "senior engineers" with degrees from well regarded universities who couldn't write a simple loop.
I think this is fundamental to the discussion. Are we really sure they couldn't write a simple loop, or were there other factors that prevented them from doing so? For comparison with your other analogy, keep in mind that even surgeons who have amputated the wrong limb are defended by their colleagues, saying “Well, it should have been amputated anyway.” (https://www.nytimes.com/1995/09/17/us/doctor-who-cut-off-wro...)
I understand that you specifically might not have used that as your only filter, but a lot of people do – and I think a more holistic approach that takes into account education, training so far, and overall experience with tech. Certifications might be one part of this too.
> Are we really sure they couldn't write a simple loop
I mean I suppose the stress under pressure could have made them forget the loop syntax in their favorite language, but that is unlikely. I literally asked them to implement FizzBuzz in their favorite language during a phone screen where they even had internet access to look up syntax. But since I was interviewing them for an SRE job, coding under pressure is actually part of the job anyway.
Regardless, I'd expect someone who claims to have 3+ years of Python experience to be able to write a python loop.
I am a developer, not HR. But I do play a role in hiring. We already have an "aptitude" style test, and an SQL test. So I wanted to make my addition to be limited to 15 minutes. I ask candidates to write Fizz Buzz but with the addition writing at the end the average of Fizz, average of Buzz, and average of FizzBuzz.
Pretty much every candidate has "failed" the test. More than half have something incorrect or not quite matching the instructions. But we have hired some of those people and they are great programmers.
It actually does give a surprising insight into the developer's style. It's not pleasant to see all of their code stuffed into main(). It's not pleasant to see every single line of code commented. But it is refreshing to see Enumerable.Range(1, 100) instead of C style for loop.
I could do that on a whiteboard, in a notepad in psudeocode, and possibly for real in a language I'd recently used (so the syntax is in mind) and recognize the output is correct or needs a tweak or two...
However I don't think I could do this at all over the phone.
I'd also, for a throw away like this, knowing it's a throw away stuff everything in to main / global and use that c-style loop BECAUSE it's portable to literally every language.
If you're looking for a FizzBuzz class/object/module specify that. If you've got a specific language with nice features ask for that, BUT don't demerit an individual too much for not knowing the day to day features of the coding environment in your company when they've not been part of it for months.
> Are we really sure they couldn't write a simple loop, or were there other factors that prevented them from doing so?
what's the difference? I believe that getting hired by a company is the easiest thing you will have to do for that company. I believe that if you can't perform a basic task at interview time, you should not be working there.
Either because you can't perform.
Or
The mood is not right for you. If this is the case, why would you want to work there anyway?
I see so many people that thing getting hired is the end goal. Mate, you still have to work for them. Those people that can bullshit their way through a job, have no issues getting hired to begin with.
I think there are additional subjectivities at play in the marketplace.
A "senior software engineer" at a cardboard box factory might not be the same role as one with the same name at an aerospace firm doing government contracts.
Be careful what you wish for. I have friends that are lawyers and they talk about how terrible the bar is for assessing real world skill. You need to be fairly well versed in 20+ forms of law (admin, criminal, civil etc) independent of what form of law you practice. Most of the test is irrelevant to what you do, but you need to know it anyways.
I guess the advantage is once you pass you don't need to re-certify. Which also seems incorrect.
I’m shocked at how important software development is to the 21st century economy while we still don’t have anything even remotely resembling a licensing exam. I’d honestly welcome one, if for no other reason than to be able to make some assumptions about what my co-workers already know vs. what I’m going to have to explain: do they know what a socket is? Do they know how public key cryptography works? Do they know the difference between a unit test and an integration test? It might not be perfect, but it would be a heck of a lot better than what we have now.
> Do they know the difference between a unit test and an integration test?
Funny when you think of how many blog posts, flame wars, SO questions exist on the internet just to debate what is indeed a unit test, and what is an integration test :)
Who would have the legitimacy to design such an exam?
>>> Who would have the legitimacy to design such an exam?
If it's like civil engineering, it would be the regulatory body that determines the approved technologies and design formulas for your field. The exam would be based on those standards. I'm not sure most governments would even approve public key cryptography.
How do you propose making the licensing exam harder to game than a well respected university's degree program? I would think anyone who can game four years of exams and projects at a good university would also be able to game a single-session licensing exam.
> There are some folks like Tripplebyte who are trying to solve this by pre-qualifying candidates, but I don't think the problem will really be solved until we form a software engineering licensing board. Take a test once, and then take a refresher course every couple of years to maintain your license.
It's risky. Licensure often becomes a wage-protection cartel, and if so, does little to guarantee the quality of licensees.
Another problem is that there are many levels of skill demanded by the industry, though maybe you can have a bevy of licenses that qualify a person for various roles.
That's my principle issue with the OP: they don't attempt to control for the possibility that people who hate technical interviews are most likely the people who didn't get the job they wanted.
If you're working at company X and think, "I hate this shitty IT job, I'm smart enough to get a real developer position" and interview at company Y and learn that, no, you aren't, that's a hard pill to swallow.
Yes, we all ought to find our zen and accept that no one has all the talent they wish they had. But people are human, so we're liable to blame the messenger, which is the technical interview.
It’s a hard pill to swallow because you don’t get feedback from rejections. It doesn’t matter what stage of the interview process you reached, companies very very very rarely give feedback when asked. Because of this, those people never get to hear why they aren’t good enough. Instead, it’s all left up to their imagination which creates the scenario you describe.
Do you realize how intensive the interview process for a doctor is? You work as an intern and essentially "perform surgeries" over the course of a year.
>Therefore I must now conclude that a degree from that university means nothing.
Degrees always mean nothing but the educations might still be useful, depending on what the individual took advantage of.
Do you realize how intensive the interview process for a doctor is? You work as an intern and essentially "perform surgeries" over the course of a year.
For the first job as a licensed physician.
After the internship period, it's generally one or two in-person interviews (possibly more if you're joining a private practice), followed by a period of weeks to months where they check on your licensure and credentials.
"We don't have that in our industry. I interviewed countless "senior engineers" with degrees from well regarded universities who couldn't write a simple loop. Therefore I must now conclude that a degree from that university means nothing."
The hiring timeline for Doctors is in the months -- almost six months to a year -- because of all of the due diligence that is involved: it is sometimes more so for specialties like OB/GYN.
Even to be brought on as a "Doc in a box" type of situation, you're still looking at a minimum of 90 days.
Folks want to "move fast and break things," and "hire fast, fire faster," but those two are not conducive for doctors.
The reason that's not going to work is how diverse and specific skillsets there are, and how fast the progression is (at least so far) in what's in demand.
Imagine the resources needed to maintain tests that should work for low-level embedded developers, data engineers, video game coders, smart contract programmers, AI implementors, etc..? Yes, there are fundamentals and Computer Science, but most of us agree that you can have the greatest craftspeople perform poorly there.
Though, I guess the certifications from private companies like Microsoft in Cisco actually work as some kind of license
> He has Aspergers and locks up and fails miserably in the interviewing process, but the honest, reality is, he is10 times the developer I am, the guy sees patterns instantly and has a knack for code organization.
I don't think I can get hired without a referral - haven't managed to land a cold interview once so far. So at least, there's no evidence of it.
Interviews are a performance act, they are the packaging that you throw away after buying a product for the cover.
The interview "suit" I wear is actually one of someone who's calm and smiling - instead of stimming with my hair and dead-pan with a smirk (if I was a woman, I'd have resting b-tch face).
I feel for this guy, because I figure out how to wear that suit, but I still need to thumb the scales to get hired, because the mask I'm wearing is effort & it does show to the perceptive HR person (& it doesn't matter what you are faking - if you're faking it, it counts against you).
However, here's where everything is unfair in this world.
I have a pretty decent job because I ran into the right people at the right conference back in 2004, where someone noticed that I might be able to do hard things with minimal effort (and fail at easy things, simultaneously).
There must be thousands of people in my state who never got that "come to Yahoo, have a coffee" invitation & get plugged into a network of people who have kept me gainfully employed for more than a decade.
I definitely had fewer challenges there because I wasn't an outright minority in the industry - I didn't have anything to prove beyond the performance.
All things being equal, I'd rather hire someone that seemed nervous in an interview than someone that seemed relaxed, as the former is a good indicator that at least they are excited about the position.
Nervousness is also often an indicator that you may not feel fully prepared. The best way I find to relieve nervousness in these situations is to be so prepared that there's really no reason at all to be nervous.
> how did that person at the 2004 conference know you had the right stuff?
I was there talking about my FOSS work, that probably helped.
toolz asked me if the "Portable .NET" platform was portable enough to run on some new hardware & gcc patches he had got before the conference (not insane, but cross-compile & port the project from a Zaurus/iPAC to run on a PicoPeta Simputer).
Anyway, it must've looked more impressive than 2 days of work, because when toolz died, someone else remembered that incident[1].
(as a side note, I had already interviewed at IBM, Oracle Platforms team and several other places with no luck at that point ... so I had already spent 4 months prepping & interviewing in 2004.)
And after the conference, I met Swaroop [2] & Philip, who were at Yahoo then.
Current recommendations on discussion boards for job seeking are to complete 300 medium-to-hard Leetcodes in the 60 days before your interview. This is up from 200 1-2 years ago.
If you won't do it there are many others who will.
If I'm not going to do a 6 hour take-home assignment for you, I'm definitely not going to do 300 Leetcodes to "prepare". I've been doing this for 34 years. That's my preparation. Either you want what I've got, or you don't.
I will never forget my interview at the 2 best fashion houses in London. Both failed for what I still consider was a massive loss to them as I had all the skills they needed and more, but either management or the devs had no idea. Im not bitter for I earn much more now than I would have, it was just crazy to see the disconnect between management and programmers, in opposing ways.
Fashion house 1:
2 guys sat me down and ask me to answer a question about some node.js code, which took around 25 minutes to get running on my laptop. Once it was they were hoping for progress, when it took me a few minutes to read through several config files ( over a hundred lines each).
After 30 minutes they gave up and asked me what library I should use for a certain task. I said I had built alot of code in that domain didn't recall which library I used for that specific task.
Suddenly I was walked out, but not before one of them took pity and tried to explain to me how, when you make a request you can write code which waits for a response from a server (async programming).
I ofcourse thanked him and left.
This was after I spent 3 years developing a successful networked engine in my spare time ( with a custom 3d engine bolted on it ) which enabled 20 asynchronous connections for a war simulation I built. They both had 30 minutes and hadnt thought of reading my CV or asking me about any of code / experience. Just drank their coffees looked at their mobiles and waited for me to somehow decipher their 10k line codebase. (Management had already OK'ed me but were not around for this part of the interview and blind to what was happening ( or the lack of it.)
To this day I dont think the engineers even looked at my CV. Just thought I was some confused kid who didn't understand how basic network code worked.
Fashion house 2
I also interviewed at their competitor, who had looked at my CV, lead engineer was about to extend an offer, CEO walked in, not knowing who I was, asked a casual question ( culture fit ) type, but I was unable to answer, I had trouble understanding what he was saying. Failed interview, but that is ok atleast I was not as offended this time as there were no quips.
The icing on the cake, was that both companies had a 20+ engineering team, that relied on an app, which just framed a webpage.
A few years back I interviewed for a software architect position at a major US airline. I actually had experience with every one of the 20+ things they listed on their availability description, including the “nice to haves”, some of which was sort of obscure but I had actually done quite a bit of work with anyway. So they phone screened me, I got feedback from the recruiter that the phone screen went well and they wanted to bring me in for a face-to-face. I took a day off of my then-current job to drive out there and spend nearly a whole day (four hours) talking to them. I got word back from the recruiter that things went great, but a week later they asked me to come out for ANOTHER face-to-face, so I took ANOTHER day off to spend the whole day talking to them. I got feedback from the recruiter that things went great, and I should hear back from them any time. A week passed, nothing. I called the recruiter. He said he hadn’t heard back from them. Another week passed, nothing. I never heard back from them.
That sucks. I recently had a similar experience. Onsite interview went well with some coding and design questions and then got a rejection a day later saying I don't have networking experience which was quite clear from my resume.
I was only asked how DNS works in the interview and I answered that. I told the recruiter this wasn't mentioned in the job description nor was I asked those questions in the interview, but he didn't have anything to say.
I've had some good technical interviews and some bad technical interviews. The worst I dealt with recently was the technical phone interview process for a FAANG company (not Amazon) that only provided a basic text editor (requiring me to tab over after every newline) for writing full working solutions to problems, and unhelpful and disinterested interviewers. This was a couple years ago, so hopefully it's improved since then. But it left a sour taste in my mouth.
I once interviewed at a company that had you write a traveling salesman program where you were planning the shortest path of flights between cities. It was a take home test where you were given at least a week to complete it. I completed the test in record setting time (3 days earlier than most), mine was the only one that was fully documented with comments (no one else bothered to do this), they said I had the most visually stunning user interface (most were ugly or unusable; I even animated the plane flying between cities), the program came to the outcome in the shortest runnable time, out of all the submitters they said my program was the most optimized solution (not the best ever made, but the best submitted to this company so far), and mine was the only one that actually worked correctly and ran without flaws. They then proceeded to have me come in for the 6 hour interview in which they asked questions like, "Write a working hash table from scratch on the whiteboard." I got nervous and passed the question, they ended the interview early and rejected my application.
I once interviewed at a company that had me write a couple styles of matrix multiplication in C as a take home test, as well as give a one-page description of a project I had done. The project I described heavily involved graph searches.
Then I had a phone interview where they started off by commenting that I was the only one to include tests and address undefined behaviour. Then I had to write a piece of code to traverse a tree and add numbers, I can't remember what it was exactly, but I got nervous and messed up somehow.
Then I waited for 2 weeks just to hear back that they wanted someone more experienced with graph theory.
Expecting perfection on a white-board test is setting everyone up for failure. At best that's a large napkin exercise. Though having a failure, or a point where for whatever reason production is a bottleneck (if they happen to do it without obvious faults) is an opportunity to identify that either thing is the case in that section and move forward with how they'd react; what should be changed, etc.
We're all still human; expect failure to happen. See what happens next.
100% agree. I interviewed at Microsoft nearly a decade ago now. I was told I'd be interviewing for the company, and then if I was hired, based on preference and ability and need I'd be placed with a team
When I got there I was told I'd be interviewing for the SQL Server team, interesting work, but not something I'm interested in. I had never taken operating systems in university because it was always available at a time that another class I HAD to take occurred.
So by some random chance, I'd never heard of a semaphore. Perhaps inexcusable at this point in my career, but I was ignorant of the topic at that time.
My first interview question was to write a way to let many readers on a resource, and only 1 writer, with no readers reading at the time of the writer writing. I decided to use lists of something or another, ProcessId, or something similar.
Every time I would go to write the code, i'd get a few characters in, and the interviewer would interrupt me and ask me questions to help me along. Since I had no idea what a semaphore was, much less that I was supposed to use it, we spent like 30 minutes of this back and forth, me trying to write, him interrupting me. By the end of the session I had a single line initializing a list written.
Some of my other interviews during that battery of interviews went better than that one for sure but, needless to say, I didn't get the job.
I've worked on pretty cool projects since then at other companies. And, it isn't the only interview I've ever had like that, it is more common than I would like. It is sometimes very hard for me to prove myself. Even though I've had plenty of jobs in the past with proof I can code and with good references. I sometimes feel like I'm an imposter, even while kicking ass on a project.
The problem with take-home tests is you may have had help, or someone else did the test for you. If you didn't do well in the in-person test, they may have reasonably assumed that was the case. Especially if the home test was unusually good.
I was aware candidates would do things like this when I applied for job long ago. I also was aware that employers were suspicious of these sorts of things. So I brought along listings of code I'd written, pulled them out, and walked the interviewer through how they worked to show I knew my stuff. Note that they didn't ask for this, I just did it proactively. I got an offer, too, and was told some time later by the interviewer that that was why.
Why bother giving a take-home test if you don't trust the results?
You can have one-way trust in a take-home. If they don't meet your quality bar, you can discard the resume. And it takes near zero work on the company's part, that's why so many companies like it.
However, some people will just get a lot of help on the take-home. So it isn't fair to the other applicants if you treat a take-home as anything other than a filter.
> The problem with take-home tests is you may have had help
I think we over focus on these scary edge cases where some trickster is trying to con their way into the job. The worst case is you just fire them when they are unable to do the work, and that's on them.
Besides, what environment discourages help? I ask friends for help with my job all the time.
The cost of over optimizing for avoiding those false positives (ie: bad candidate marked good) is a huge drop in true positives (ie: good candidate marked good). It's silly.
>I think we over focus on these scary edge cases where some trickster is trying to con their way into the job. The worst case is you just fire them when they are unable to do the work, and that's on them.
The whole reason for the long hiring process is that firing is hard (legally, financially, emotionally). If you're willing to dismiss this difficulty, you're assuming away the core problem.
I'm not saying to not interview people I'm saying that the existing process is petrified of a bad apple making their way through it, at almost all costs.
How many people can truly, reliably, "fake" their way through a take home? How many people could then do an in person code review and fake their way through that as well?
I doubt it's that many, but you'll get way better returns on good candidates, in my opinion.
Firing somebody can set you back $100,000 (or more if they're in a protected class). Worse, a bad hire can wreck your schedule.
I've encountered programmers who know their stuff, can talk about it intelligently and knowledgeably, but cannot write code worth spit. It's not as rare as you might think. It's sort of like someone with an art degree who has no talent for actually painting.
So, to be clear, firing is not the goal, it's just the worst case. 100k sounds high but honestly, not something I'm familiar with. I wonder how much money is lost on recruiting being so ineffective? Given that the current process is about ~6-7 hours of eng time per on-site candidate, I'm not convinced that the cost of firing is going to outway the costs of recruiting being considerably optimized.
I'm not sure I see how someone is going to:
a) Write a take home project
b) Explain their design decisions, respond to code review, etc, in person
and somehow not be able to "write code worth spit" or otherwise game the system consistently.
edit: Would love to hear (in the US) where that 100k comes from. At-will state means I can fire you for whatever reason, can't imagine it really comes close to 100k tbh.
Don''t know why you're being downvoted, but 100k is definitely not the cost of firing a just-hired employee in the US.
If you're paying a new-hire-fast-fire more than the salary due to them for the period worked (and any other amounts listed in their offer letter, if one existed), then you need to talk to a labor lawyer because you're definitely doing something wrong...unless you're paying them to shut up about the experience.
6. it can take a while to determine they're incompetent if they're good at the facade
7. if they're in a protected class, you have to go through a period (often 6 months or more) of documenting their incompetence and leaving a trail of trying to help them improve
8. firing someone always comes with it the risk of a lawsuit, and meritless lawsuits are commonplace, but they still cost a LOT of money to defend.
9. disruption to your team, scheduling slippage, having to allocate someone else to fix the incompetent work, etc.
Think of it this way. If I sell my house, I pay a 6% commission. If I reject the offer, I still owe the commission. I pay the commission. Then if I sell it again, I pay another 6% commission.
If you sell your house, you pay a 6% commission on closing. If you reject the offer, there is no closing, and you don't owe the commission. Because the house didn't sell. If you sell the house, you can't sell it again unless you're committing fraud...But assuming you meant you find another buyer, and successfully close with the second buyer, you would then owe a single 6% commission to your real estate broker, not two 6% commissions.
This is also not how recruitment bonuses work, and I say that as someone that works closely with HR on the tax/legal side of things.
If you pay a recruiter a recruitment bonus, their recruited employee has to remain with the company for X amount of time (aka "trial period"). Usually at least 3 months, but it varies. Once that happens, the bonus "vests" and is due to the recruiter. (Many recruiters will try to get paid before the trial period ends, but you don't actually have to pay them yet.) If you terminate the employee before the trial period ends, the recruiter is not owed a recruitment bonus. These are all standard provisions for recruitment contracts involving 3rd party recruiters. (Note: this is for the contract with the recruiter, not the recruitment employee. The employee's employment contract, if any, may not include any language about a trial period.)
Of course, it's very possible for a startup without an HR department to leave out the trial period language from their recruiter contracts. Because they didn't know any better because they didn't think that HR personnel provided useful functions. Now you know why HR departments exist.
If you actually think those are the costs of firing a new hire, you need to talk to a labor lawyer immediately, because 1, 2, and 3 are hiring costs which don't apply to most jobs--startups included. Only in the rarefied world of FAANG and VC-funded money-losing unicorns do new hires get relocation expenses and a starting bonus. Even associates starting new jobs at BigLaw firms don't get paid relocation expenses or starting bonuses. And if you're doing the hiring in-house why are you paying a headhunter fee?
5. COBRA is paid by the employee, not the employer.
6. Somehow every other industry except programming manages to do this just fine.
7. This is completely and utterly false. If a new hire doesn't work out you can just let them go. Somehow, companies do this all the time without issues, including companies with much cushier jobs than startups.
8. Everything comes with the risk of a lawsuit. You're prematurely optimizing for the 0.0001% case, for an imagined scenario that isn't that expensive to defend.
9. Most of these apply to leaving the position open for months without filling. So let me add one to your list that applies if you don't fill the job: employee burnout leaving to existing employees leaving before the open position is filled, leaving you with more positions to fill than you started with.
I agree with you for the most part, however, you keep claiming that relocation are not usually paid. I know that relocation packages are very commonly offered to programmers, even in entry-level positions, in the finance and oil industries.
Paid relocation expenses is fairly common outside of SV for highly compensated employees, however, they are structured so that you must pay it back if you don't stay employed for a certain time (usually a year). I've never heard of a relocation bonus that wasn't structured this way.
Retention bonuses and starting bonuses are structured the same, you sign an agreement to get that bonus that says you pay it back if either you or the company decides to end employment within a certain timeframe.
Unless this is a contracted for benefit, this isn't a separate cost from lawsuit risk, but a mitigation of lawsuit risk: severance pay is paid to people on condition of waiving claims.
> if they're in a protected class
If you start a sentence this way, it's a clear sign you don't know what you are talking about. Except for a few special cases (“age over 40” and “veteran status”), every employment protected class is a dimension on which discrimination against any value is equally restricted, and everyone has some value on each axis (race, sex, etc.)
So, everyone is “in a protected class”. Well, several of them, actually.
> you have to go through a period (often 6 months or more) of documenting their incompetence and leaving a trail of trying to help them improve
No, you don’t, and you can’t really count this “requirement” and lawsuit risk as separate costs, since this is simply a policy companies adopt to mitigate lawsuit risk.
> you can’t really count this “requirement” and lawsuit risk as separate costs, since this is simply a policy companies adopt to mitigate lawsuit risk.
I understand, it isn't intuitive. I hope my analogy of someone well trained in art, yet still can't paint worth a damn, makes sense.
Or pilots who ace all the pre-flight training, yet simply are too uncoordinated to fly well. (That's me. I know all about flying, yet should never be in the pilot's seat.)
Some people are just unable to synthesis good code out of all their knowledge.
If they understand how it works well enough to go through a code review and modify it based on that code review, what's the problem? That just... sounds like what your work is going to be.
Good question. Like I said elsewhere in this thread, there are many people who are knowledgeable and intelligent about coding, but cannot code. They are different skills. I've met many people like this.
I could study Romeo+Juliet and know everything about it that lit professors know. But I could never write it.
For another example, from my time at Boeing, the engineers were divided into two independent groups. One group designed, the other group analyzed and checked the designs. They were different skills, and the engineers would gravitate to whichever path best suited them.
It's just silly that startups will go months without filling a position so they can avoid spending a week on a person that they might need to fire. This is an example of a premature optimization. A problem faced by less than 1% of companies nationwide is not something you need to worry about. It would be like leasing the top floor of an office building in Silicon Valley for top dollar because you heard that NYC occasionally floods in the winter.
If someone is in a protected class and they get fired within the first week, does anyone really think they'll win a lawsuit alleging discrimination? Does anyone actually think they'd get beyond a motion to dismiss (usually the first motion filed by a defendant)? If the employer was discriminating against protected classes they wouldn't have hired the employee in the first place. These types of claims are quickly dismissed and usually incur minimal legal fees.
FAANG companies have this problem: when you hire thousands of people a year, a higher false-positive rate is a big deal.
When you are startupbro.com, you don’t have this problem, but you’re desperately cargo-culting your way to success, and doing anything in a way that goes against the grain causes the other bros to accuse you of “bad signaling”.
See also: why tech companies cram together in San Francisco.
Couldn't agree more. Almost every startup in LA I interviewed with seemed to suffer from some sort of imposter syndrome. Even a hint of a question about their technology choices or their business model and they look at you like you just ran over their dog. HOW DARE YOU QUESTION THE GREAT AND WISE WIZARD?!??
>when you hire thousands of people a year, a higher false-positive rate is a big deal.
My friend's employer is currently hiring thousands of people a year with a single interview lasting 1-2 hours without problems. Some highly technical and some manual laborers.
Then do this by putting new employees on a probationary period for the first six months. Many of their employees are union, and it still works out, the unions agree to this arrangement.
It's really not that hard at all to fire someone, at least in the U.S. I think most companies do it because they don't know any better and they heard that's what Google does.
In the US, where most startups are based, firing is actually quite easy. For a brand new employee, it's especially easy.
All that BS about it being hard financially or legally? Not true. At all. It's just an excuse startups use because they don't understand basic HR functions. And that's why every company of a reasonable size (i.e., more than a few dozen people) has an HR department, even if it's just one person. Because they know how to do this stuff.
My comments are based on what nearly every major company in the US actually does, not what people in Silicon Valley think other companies do. Since all those other companies have HR departments and most of the startups in SV don't, I'm pretty comfortable with my stance.
After all, managing hiring and firing are literally the primary reasons HR departments exist in every company making more than a few million $/year. Especially the profitable ones.
Every company in the US has an aversion to firing that leads them to use expensive defensive measures (in the case of the larger ones: severance pay, PIPs, standardized performance reviews, diversity training) for when they feel they have to go though with it.
Whether or not you’re talking about Silicon Valley, you’ve endorsed a much stronger claim, that such measures are pointless because firing is just so easy.
If you’re not ready to defend that claim, then maybe you should walk back on your original. Or at least show some some sign that you recognize things aren’t as easy or simple as your original blithe dismissal implies.
Remember, you didn’t even think the emotional side merited an acknowledgement.
This is simply not true. Employers have been hiring like crazy all across the country for the past few years.
And severance pay is not a thing for most employees, just the ones, like programmers, that have fairly cushy white collar jobs.
You need to stop seeing everything through a rose-tinted SV lens and look at it the way the rest of the country does. I'm ready to defend my claims. Are you?
But we aren't talking about non programming jobs. We are talking about programming jobs.
What everyone else does is completely irrelevant.
In the programming world, specifically, which is what we are talking about, firing is extremely costly.
This is proved by what companies in the programming world are doing right now.
If you disagree, then you are disagreeing with basically every major tech company in the world. And personal, my bet on who is right would be those tech companies, not you.
Oh if we're going to limit this to programming jobs then I know for a fact that the majority of programming jobs do not pay for relocation or a signing bonus. Especially not in the startup world.
And when they do, it's standard practice to require that relocation and signing bonuses be repaid if the employee resigns or is terminated within a certain time frame after joining the company.
I have worked at 5 different companies over the years, and Every single one of them was absolutely terrified of firing engineers.
Not only have I worked at a bunch of different companies, but I have also straight up failed at some of the earlier jobs, and have been in situations where I absolutely deserved to be fired, yet it took literally months for anyone to even approach the topic of my performance.
I am talking, situations of me not doing any work, for months, because I didn't care and was looking for new jobs during that time period.
My experience has shown to me that engineers are pretty damn untouchable. And yes, these have been (mid sized) startups.
The one time I had a reloc bonus, of a job that I left after 5 months, they did not ask for it back, and simply never talked to me again.
It is not about signing bonuses though. It is mostly about things like bad Glassdoor reviews. Many engineers would never ever work at a company where the engineers were constantly terrified of, and complaing about being fired quickly.
Also, disgruntled ex employees are something that you don't ever want to deal with.
> Every company in the US has an aversion to firing that leads them to use expensive defensive measures (in the case of the larger ones: severance pay, PIPs, standardized performance reviews, diversity training) for when they feel they have to go though with it.
Not really. It's just harder to hire again usually than figure out how to coerce the problem you have already to be less of a problem. That's the nature of the market right now.
If developers were easy to find and hire then they would be getting fired left and right like happens with just about any other commodity job.
> I think we over focus on these scary edge cases where some trickster is trying to con their way into the job. The worst case is you just fire them when they are unable to do the work, and that's on them.
The problem is once you offer them a job you stop searching for a candidate. Then you get a bad hire and even if it's quick you won't really know for a while, so then you're really behind.
It's bad enough when you hire someone who really wasn't qualified but didn't do anything bad. Now you have to get rid of someone who probably left a previous position and, like above, you have to go get someone to do the job you were hiring for in the first place. Worst of all possible worlds.
Perhaps the calculus is different in a big company, but for a startup your approach sounds pretty bad.
For what it's worth, I'm not convinced that my approach will actually let more "bad" people through. My personal opinion is that you will improve both your rate of good candidate hiring and lower your rate of good candidate rejection.
I believe that the existing system intends to optimize heavily for "reject bad candidates" and actually fails at that too.
Give candidates a take home, then review it with them.
I have a lot of reasons for believing this approach would be far superior to the "sit in a room and get interrogated for 7 hours" approach.
My company gives a take-home test. When the candidate comes in for the interview I go over their code with them and ask questions about it: Why did you take this approach? How could you test this method?
I don't care how much help they got if they can discuss their solution coherently. This also shows the candidate that we took their solution seriously.
And how many candidates spend hours of their valuable time in those take-home assignments, only to get beaten by a purchased solution?
It's wonderful you don't care about the fact that these solutions are purchased. I'm sure your attitude is a great comfort for all the honest professionals who worked hard to find the time to do a 5 hour take-home, only to see their job taken away by dishonest candidates who paid for the solution.
Taken to its logical conclusions, if your attitude is representative, then everyone should just buy take-home solutions and just spend a bit of time understanding them before the onsite. Now we're back to square one, except at least we're not actively screening for dishonesty anymore.
That's not the problem that take home assignment is facing right now. As you said, it's the experience issue, people feel shit after doing it, rejected without any reason.
If copying/buying solution is actually a problem, that means take home project is a thing that's happening, and copying is the new problem to solve along the way.
> If copying/buying solution is actually a problem, that means take home project is a thing that's happening, and copying is the new problem to solve along the way.
In case you didn't know this: take-home is very much happening. I was job-searching not too long ago, and roughly 10% of the places I talked to insisted on take-home assignments. I declined all these requests because I'm a senior engineer with many options, but clearly many candidates are taking them.
There really is a problem with people buying solutions for take-home assignments, I saw that when I was hiring. It just isn't acknowledged, so effectively it has become a competitive advantage for dishonest candidates who are willing to cheat.
I was worried about that for senior candidates, and if I had a good recommendation or had some sort of screen call with them I’d usually skip the takehome.
Very occasionally the candidate came in and just was wildly off, but usually it was fine.
Later on we changed to having all candidates submit the takehome under the logic that if they’re a, “positive” candidate that they’ll be inclined to see this as an opportunity to strut their stuff. I was a little unsure about this approach but we went with it.
I’m curious how senior folks feel about these different attitudes.
> I’m curious how senior folks feel about these different attitudes.
It's quite simple: I'm a senior engineer and I will never do a takehome assignment.
Every time I'm on the job market, there's a certain percentage of positions that require a take home. I just skip them.
I don't explain my position, since I learned long ago that it just leads to arguments and confrontations with the very same recruiter who just a hours ago cold-called you begging you to interview.
I just terminate the process as quietly and unobtrusively as I can.
I wish I could comfort you by saying I ever regret this decision, but the truth is that the best jobs out there - the Google, Netflix, Airbnb jobs - do not require a take-home assignments. I guess they know better than to restrict their candidate pool in this competitive market.
It tends to be the below-average and (often) crappiest jobs that require take-homes, to the point it became a red flag to me. Now I often wonder "what kind of a desperate crappy senior engineer had to jump through all your hoops in this market, especially when you're just one anonymous company out of tens of thousands to be hiring engineers, with nothing special to recommend you this early in the process?".
In short, to insist on take-home tasks in this market is to give up on your strongest candidates. Neither me nor many senior engineers I know will at all consider spending 5+ hours on takehomes when we can easily get dozens (!) of onsites with short, fun remote screens. I will also be very skeptical of any engineer who in this market is willing to do these hours of takehomes after a long, hard day of coding at work.
One last point: it seems quite unfair rude to me for a potential employer to ask me to unilaterally dedicate 5+ hours of my rare valuable time, when we barely begun the process. It's especially out of place and comical when the same recruiter who literally begged you to interview in the first cold-call, is suddenly demanding you do 5+ hours of unpaid work. At least in remote screens, I know the employer is investing some of their own resources into this process as well.
Thanks for your candor, I generally had this sense. If we had something like, “we usually ask for a take-home but a pointer to a code sample you’d be willing to discuss will work.” Would that be acceptable?
My logic on the take home was that interviews are hard without something to talk about and contrived problems are not super reflective of real work. Therefore, I’d send a dumb version of a simple task we have, then we’d walk through trade offs and such.
If I have some code you wrote and can walk me through that’s a very useful datapoint not just for competence (if you’ve been around a while you probably are good) but for personality and how we’d work together.
Does that seem more reasonable as a trade off for your time?
I think in general all of these things just start to come off as dishonest, when senior engineers think about what makes them feel "senior," or what they respect and value in "senior" colleagues.
At some point I think senior engineers realize that coding is the absolute least of your worries as someone expected to do things like interface with the business side, plan medium/long-term roadmaps, improve dev tools or general architecture to make juniors more productive, or mentor/train/manage juniors.
When the entire process seems to be heavily weighted towards coding, it gives off the vibe that this is a company that recognizes overworking and churning out code as traits of a senior engineer, instead of shrewd decision-making, efficient delegation, smart scoping of projects, strong leadership skills, etc etc etc.
I don't want to spend my free time building a portfolio of random bullshit code any more than I want to spend time on your take home test. Because I can't for the life of me see why having an impressive (or average) github profile should be the leading indicator for senior engineering skills.
What is wrong with simple live-coding sessions on very small pieces of code while discussing objectives, edge cases, extensions, etc? Coupled with a high-level systems design session with whiteboard diagrams but no code, and a behavioral interview assessing leadership skills, extracurriculars, personality type, etc. Sane companies seem perfectly fine with sticking to this formula.
Where is this idea that we need more and more, deeper and deeper assessment of candidates coming from? What problem(s) is this solving to look at large amounts of pre-written code from candidates?
That's all very fair. The Q I usually ask of all candidates onsite is a design, "give-me-pseudocode/prototypes/class outlines" question which I find infinitely more useful in probing how someone thinks than whiteboard algorithm/coding qs. And to be clear, I wouldn't want to see BS code, but if someone had contributed to an open source project or had some sort of side project that they would have done anyway, that would have been useful.
I think our focus on code was based on the fact that everyone needed to write code. We were a small company so there was less of the architecture work. We may well have benefited from doing more of it, but we still needed people to ship lots of code.
You do ask a very important question for which I don't really have an answer: why do we care so much about these take-homes or coding interviews for people who have been around for a while?
For me I'd say because I want to have a point of comparison with all other candidates, but that's a pretty weak answer. I'll think more about it.
> In short, to insist on take-home tasks in this market is to give up on your strongest candidates.
This assertion makes absolutely no sense at all. Not being able to pass a simple fizzbuzz test doesn't mean the candidate is among the strongest there is. It just means that the candidate was either incapable of taking such a simple test or unwilling to provide any assurance that his alleged seniority made him a paper tiger.
I've personally interviewed a candidate who had a double major in math and CS, had a PhD, declared having over 10years of experience as a senior software dev, and on the first interview he showed he didn't even knew how to include a third-party library on what he claimed to be his programming language of choice he used most of his career.
That's the sort of stuff that take at home tests help weed out. If he took the test he wouldn't have wasted hours of his and our time.
Honestly it's annoying and does not tell that much. We recruted once a guy that passed algorithm tests on site. We had to fire him 9 month later. He wasn't abble to handle code he had not written and was disappearing every time there was a production issue. Tottally unreliable.
If I'm doing homework I expect an office visit. And pizza, or lunch, or something. I expect to be a serious candidate.
I code and troubleshoot for $$$, and unless there is a direct route to interview + hire (or at least lunch) I'm not interested. I understand there are sorting methods needed to do hiring, and I'll play that game if I have to, but if there is no direct, up front expectation that I'm in the running then it's just kind of patronizing without payoff.
> And how many candidates spend hours of their valuable time in those take-home assignments, only to get beaten by a purchased solution?
In the few hiring processes where I've participated as a hiring consultant, take at home tests were only used in the initial stages of a hiring process to filter out blatantly incompetent candidates, who in some cases represented about 90% of all applicants. We're talking about basic fizzbuzz stuff.
Even so, in subsequent stages the candidates who made the cut were again evaluated based on their technical skills, mostly to weed out candidates who might have cheated their way through the first stage.
This meant that it was impossible for dishonest candidates to eat the lunch of candidates who were cut due to their performance of the take at home test because anyone who failed that test was obviously not a competent developer. I mean, are you expecting to be hired as a professional software developer if you can't write a for loop?
Have you read any of the comments in this thread? We are talking about take-home tasks that take 5 to 10 hours or more, not fizzbuzz.
You are naive if you think you solved cheating by checking technical skills on the onsite. What if I'm a 7/10 candidate, and someone else is a 9/10, but I cheat and submit the solution of my friend who is a 10/10. I'll get the flight ticket to the onsite, the better qualified guy will not, and you will have to choose from a smaller pool of candidates, in which I might be the best one.
I think the previous post wasn't literally saying they would be okay with someone purchasing a solution. Just that it's okay if they had help with the problem, even it's quite a bit, as long as they actually understand the code.
I can't imagine a scenario where a person can't solve the problem, so hires someone to do it, then explains it as well as the person that actually wrote the code. If you can explain it that well, with reasoning for tradeoffs and design decisions, you might as well have written the code yourself and saved some money.
My companies do something a bit different: we ask people to pick any issue from any opensource project in our tech stack and try to land a patch closing it.
We think this is better then wasting the candidate's time on a toy project - at least the can write off their time on the opensource-karma account.
Also, we can check not only their code but also their communication style and other skills that matter in the real world (projects on our tech stack have a high threshold for documentation, unit tests and other stuff that matters).
We call it "social code challenge" and it is working wonderfully for our recruiting process.
Love this, especially with evaluating not only code but how well they collaborate. Ideally if the candidate had already submitted such a patch they could just explain it.
Exactly. And it is way easier to fake a resume than fake public reputation. For example, looking at a stackoverflow profile I can tell if someone knows how to ask and how to answer (a good question is 90% of the answer).
I basically like the approach, but I'm a bit wary after playing devils advocate with it for a moment.
A candidate might reasonably assume that they can look better than the other candidates by spending more time on the problem and tackling a larger or more impressive ticket. If you imagine a world where all interviews were like this, candidates could end up working 60 hours a week on these projects in the normal course of job hunting (easier to imagine if you assume that the talent shortage isn't permanent). Probably, since they can see the size of the contributions made by previous candidates, the pressure increases over time as well.
edit: it would be amazing for FOSS in general though. Imagine how much faster GNOME would be if hiring committees required you to write bug fixes for popular projects.
> A candidate might reasonably assume that they can look better than the other candidates by spending more time on the problem
The same is true with a take-home, candidates see the (often underestimated) statement "this should not take more than X hours" and assume they will have a better shot if the invest more than the X hours.
I forgot to mention that we also accept past contributions - in the end the hypothetical candidate can spread this 60 hours investment over any other hiring processes using the same method.
I'm thinking about writing a "social code challenge manifesto".
Many employers coming looking for D programmers to hire have looked at accepted D contributions to the Dlang repository. It's a great way to build one's resume and find great people.
Exactly, many people complain about about not being hired because they lack experience and they can't build experience if they are not hired.
My advice is always "build experience contributing to FOSS projects in your spare time".
We are pushing the envelope and often find bugs or lacking features. I want to hire people that can fix/implement those and send the PRs upstream instead of nagging the project maintainers.
Seems like a good approach but beware "any issue" that's really a decoy filed by applicants aware of your hiring process. Unlikely yes, but worth considering, if only to appreciate the initiative behind such ruse.
>The problem with take-home tests is you may have had help, or someone else did the test for you
Ask about the code. Ask about the thought process. Ask how to debug it. Ask how to change it or improve it. Put one of the company's team members with them and pair code for half an hour. Find a bug or two and fix it.
There's no lack of questions to confirm if someone did the work themselves. This is easy to figure out once you have working code in front of everyone.
I've had candidates who can explain it all, and aced the take home. Failed all interviews. I spent 20 minutes chatting amicably with them letting them enthusiastically ask questions and then asked them to iterate an array and PR nt the elements. Print function was given. Array was given. They could not write the code in 40 minutes. I did this make good interview after the candidate had tanked their other interviews on tech and soft skills. Even if your Terrible on a white board you should be able to write a for loop.
Had other candidates totally ace the take home and flat out state first thing that they can't code and ask to switch to a non coding interview or to go home.
Some people spend a lot of time practicing to cheat. Given how much this industry pays it shouldn't be surprising.
Both of these happened 3 weeks ago when I was interviewing 32 candidates in a event. But several times in the last few years as well.
No I don't have statistics on this or pure confirmation... Kinda hard to get that info from suspected cheaters.
There are problems with every type of interviews a d they're hueristics for skill not a judge of skill.
I would love to meet someone who was at ease, could explain code in detail (including talking about looping over arrays) and then NOT be able to produce that code.
The second one flat out cheated, but I bet they wouldn't have been able to explain the take home so you might have caught them.
Kinda weak speculation on my part but I guess I could be wrong. I've literally never met anyone like that.
What is the take home like? Maybe the code/answers have leaked somewhere?
You can cheat by paying someone for the solution, and it also comes with a clear verbal explanation that you can read.
What GP describes is more common than you'd think. Not surprising when the prize for cheating is a 6 figure job, and there's no penalty at all for getting caught.
I've seen cases where people had a friend, someone they were dating or a cs major roommate who it was very likely did the code for them (they mentioned these people encouraging them in interviews)
In the gp scenarios they said they were able to Google almost all of the take home and stich the parts together. The overall solution/problem isn't Googleable, we checked.
The second one could infact explain it all. And could solve design problems. This happens a lot actually, rusty former coders or people who learned in college and never actually did it so it stuck.
The first one could also explain the code fine. Interviews 2 and 3 they were comfortable and able to design decent systems but fell over on code. This why I was given the task of "get any code whatsoever after you put them at ease"
If I were ever to give a coding interview, I wouldn't be asking the candidate to implement a certain data structure, but rather explain the pros and cons of using say, an array vs a linked list and when you would use one over another. Because at the end of the day, you will not be implementing them but using them.
At a previous education startup I did a lot of interviewing of candidates. The question I feel gave me the most insight into candidates was very, very broad. After explaining (generally) how the company’s product worked, I’d ask them something like, “Let’s have a conversation about how you would design a system to distribute e-textbooks to students.”
I would reassure them there is not a right answer, that the exercise was just asking them to engage with the question. It was so effective as an indicator of future performance, and, according to those who eventually were hired, felt fair to them.
Of course this implies all my biases to people who can have a 1:1 convo about a very vaguely specified problem and so on. No idea if it’d be positive for everyone, but I wish someone would interview me like that.
Exactly, after years and years of interviewing and building teams I learned that the best way to interview about data structures is to do Q&A related to them.
Even if you are going to work in a project that requires building such structures, I demand that you don't do it from memory, but make sure to get help from resources to ensure that your structures are sound.
0. The actual job of a software developer is usually to develop software with some real-world use and benefit, not to write cs101 code on a whiteboard while someone watches you and grades you for speed and accuracy.
1. Software is what Google does, and Google does tons of software from tools to infrastructure to services to games. If you created a key software tool that tons of people use at Google to do their jobs, it's pretty relevant.
2. Software isn't like doing routine maintenance on an existing A/C system - it's usually somewhere between inventing A/C and creating a new design for data center A/C systems then building, deploying, and operating it.
Without making any judgement about this particular situation, this logic isn't sound:
1. I develop tool X, which is a simple popular solution.
2. Some company Y is hiring engineers to develop solutions that are far more complex and demanding than what my tool does, for instance because they have to support billions of concurrent users.
3. I deserve the job because most of Y's engineers use the simple tool X that I developed, even though the job requires far more advanced skills than I have demonstrated writing X, and I actually don't have these skills and will totally fail at the job if hired.
I think Noah's point was that homebrew wasn't a simple tool. And, not only was it evidence of the developer's capabilities but it was also used extensively by all of the employer's own engineers. A tool they all relied heavily on to get their own jobs done. Making it clear he could create software they found important for their own production. And, made it possible for them to deliver service to billions of users.
He got nervous on the whiteboard when asked a relatively simple question. They proceeded to shut him out of the position, completely disregarding his ample qualifications signified by their extensive use of his product. Which is a kind of tragedy.
Comparing an A/C unit to rocket science isn't an equivalent comparison. I can't think of a good analogy to be honest, but that isn't even close.
No other field I can think of really has the problem ours does. You can either make A/C units and rockets, or you can't.
But, in software engineering, it is highly possible to be able to say..... develop high quality GPS mapping software from scratch used by the Navy. Then turn around and have a company making word processing software reject you because you couldn't write a Mandelbrot fractal generator from memory on a whiteboard with a marker.
The fractal generator has zero to do with their word processing product and will never be a useful test of your abilities. Clearly the guy who made GPS mapping software can figure out how to write a word processor. Not to mention, they already have the word processor written so he'd only be assisting them in maintaining their existing code. In other words, he's already demonstrated his ability to write and maintain code of a high caliber. None of it makes any sense.
I really don't think any other field has this issue. It is weird as hell, and unusually pervasive in our culture.
Edit: It won't let me hit reply on your comment below. I think we've hit the maximum length for this comment thread. But, I'll respond by saying I don't know what position he was applying for. I'm not sure he ever explained beyond that tweet. The first I heard about it was Noah mentioning it. But, if it made their lives easier so they can deliver products on time to their own customers it makes sense they have a position somewhere in their company that would be a fit for him. At the very least it proves he is a good developer. Massive systems like the ones Google creates are made by many good engineers working together to make it possible.
Our field isn't quite so unique, and the rocket vs AC example is actually perfect.
Both are engineering projects. But one is simple and a solved problem, the other is far more complex.
The fact that many people at Google use Homebrew doesn't automatically qualify a Homebrew developer to work at Google. Maybe Google would develop Homebrew by itself if there was no tool like that available. Perhaps it would do a better job than said developer. Either way, it doesn't automatically imply that you can do whatever it is Google wants you to do just because you developed a tool that some of their engineers use.
Also, let's get real, to say that Google engineers use a package management system to "get their work done" and then describe said PMS as integrally related and essential to their work... more than a little bit of a stretch.
I'm just making a point about the abstract argument. I have no idea about whether Homebrew is technically impressive or not, but it sounds like people think that it is?
I'd think that writing an extremely widely used package manager is good evidence of your competency as a software engineer. Whereas in your analogy the qualification is totally irrelevant to the position.
"I developed a simple popular tool that many of your employees are using, therefore you must offer me a job that requires far more complex skills that I cannot demonstrate."
Simple solutions for simple problems. More complex solutions for more complex problems. It's quite possible that the problem Homebrew solves is much simpler than the problems Google is hiring engineers to solve.
I always felt like this meant that google dodged a bullet. Not being entitled is perhaps the most important thing I want when hiring. I really don't see why developing homebrew should guarantee a job at google.
It doesn't guarantee a job at google, but I don't think that was the point. If he feels that the reasons for his disqualification are absurd, how should he voice that frustration while not being perceived as entitled? Are hiring practices beyond critique?
If they were willing to give you a top compensation, then OK, writing hash table from the scratch is a pretty common question at Google/FB. If it was for some generic job with generic pay, they did you a great service by rejecting you.
> "Writing a hash table from scratch" is a programming 101 problem just slightly harder than FizzBuzz, but not by much.
No, it's bloody not.
What's your hash function? Why? What do you do on hash collisions. How many buckets and how big? Does your hash function map to identity or not? Do you exclude certain fields from hashing? How do you handle contention from multiple threads? I can go on and on.
The fact that you think writing a hash is easy betrays a stunning amount of ignorance.
That's the difference between a hashtable you might want to use in practice and a toy example to have someone write some code while quizzing some CS theory.
To be fair, the interview question a few posts up mentioned a working hash table, saying nothing about efficiency or thread safety. (though such a question could be gamed with a linked list as a "well akshuolly" a single-bucket hash table...)
I wouldn't do that to them. They have a right to interview how they want. I just wasn't good enough. They were looking for someone who could think on their feet under pressure in the moment and I'm not that guy.
I need a quiet place to sit and think something through and plan it out. It is why I did well on the test portion but not on site surrounded by interrogators. In those kind of situations I kind of feel like I'm the in the movie Swordfish where they're like, "Hack that server you have sixty seconds! (gun to head)" The difference is, I would have been dead.
I went back and looked at the emails. I can't edit my original comment so I'll jot the corrections here. They gave 3 days not a week. I completed the submission in 7 hours, they required you track how many hours you spent and when. I used a genetic algorithm to solve the problem. And, I took advantage of multiple cores if they were available to divide the work load.
Looking at the email there were signs even in their review of my submission that their outlook wasn't completely rosy. They mentioned things like how I failed to correct for differential conversion factors based on global location and stuff like that.
Skipping the question obviously didn't work out for me as I didn't get the position.
It was a weird experience to be put on the spot with such great skepticism with all my past accomplishments and knowing how much they respected my test submission. In a way it was humiliating. I didn't leave that interview feeling very good about myself. I still regret it to this day. In some ways I wish I could go back in time and pass on the interview, I think I would be more confident today.
I bombed an interview where I couldn't understand the prompt, literally. Like the interviewer spoke it to me, and I restated the problem, and he said "no it's…" and drew something on the white board, and I listened and restated it, and we went back and forth like that for about 15 minutes. To this day I don't understand what he was asking for, only that involved a rate of bank customers and a Greek lambda.
Getting good jobs means pushing your limits and it's therefore a numbers game. Feeling shame from rejection is natural, but also generally unhelpful these days, so it's worth unlearning.
I remember doing a take home and adding it to github as requested by the company. Then I realized every one else who applied did the same thing and left up their solution and found all the other candidate answers using a simple search. This was prior to github making private repos free.
My favorite interview experience was when I passed a live coding exercise on Monday and had time to optimize, then completely failed the exact same problem on Friday with a different company because my nerves hit me.
I feel for you man, I drove 4 hours from Sacramento to Palo Alto for a great start up job. I nailed the phone interview and thought all was going to go well. The first onsite interview went pretty well. The second part went south and my nerves got me. I really wanted the job but I blew it. I got good feedback and I'm working on interview problems. It was my first interview in 20 years but again I really wanted the job.
There is a lot of discussion in the comments on this thread about take home tests not being useful. I believe we've have a pretty good hiring process that makes significant use of take home tests (16-18 hours of them in fact). At the same time, I assure you that we care deeply about making the best use of our and the applicants time. This is what we do to make that happen:
- We make a commitment to reply to every candidate who applies as instructed. Not every reply can be detailed, but candidates are never applying to a black hole.
- We have a very detailed job description that answers a lot of the questions many developers have about a company up front. There is always a chance that we are lying or don't live up to what we say, but the info is at least there so the applicants can make an informed decision about whether they are really interested in working for us. The details about our application process are also stated clearly up front so that candidates know what they are getting into.
- We ask pretty in-depth follow questions about technical experience to make sure it aligns with what we are looking for. This filters out a decent amount of candidates who would be unlikely to pass the rest of our process saving them and us time. Other than roughly validating that the candidates work experience falls within the range we are looking for, we find resumes to be mostly useless.
- The initial "take home" exercise we ask applicants to complete takes 60-90 minutes. It's not a coding exercise and they are given a document that helps them prep for the work they are going to be asked to do. We have found over time that this is more predictive of success in our organization that doing an initial phone screen. Candidates are given relatively detailed feedback on this exercise, usually within 3 business days of submitting their work.
- The next step is a Zoom interview. We have a few very basic coding tasks that we ask them to do as part of this interview. These tasks are representative of real world skills a developer would need and not in any way convoluted or academic. The environment could be challenging for some due to the fact that we are watching over their shoulder, but again the tasks are very basic and the goal is simply to see that the tasks are able to be finished in a reasonable time. This isn't to tell us if the candidate can do the job, it just helps us filter out candidates who are likely to fail badly on the next part of our interview process. It saves them time and us time and money. We have occasionally passed candidates on this step who seemed like their bad performance might have been from nervousness and, to date, those candidates have never performed well on the rest of our skills tests.
The candidate has an opportunity to ask any questions they would like during this interview.
We also talk about money at this stage. Our salary range and benefits are published on the job description so we ask the candidate what their expectations are from a compensation perspective. If they are reluctant to share details, that's fine, but we at least confirm that they understand that our published compensation is what we are actually planning on paying and ask that if that is not suitable to them, that they let us know that now before moving along in the process.
Only if everything is aligning at this stage, do we then ask them to commit to a significant more time working through our skills tests process. Our increasing "ask" of the candidate's time is deliberately progressive. We are trying to only ask more of them when we have had the chance to vet them for obvious mismatches and vice versa.
- We then move into a two part skills test process (16-18 hours). All candidates who move on to these tests are compensated at around $50 per hour. All tests are timed and represent real world tasks our developers do on a daily basis. The first three have to be scheduled and proctored but the fourth is really "take home" and...
Is this really normal for web developer? I don't work in web dev, but I would absolutely not put up with this for only a 100k salary and getting paid roughly one grand for an interview process that takes in total days.
Reason why is because I've had interviews at other companies which are far less interrogative from the company towards me. For example, for my current job (embedded systems) I had:
- 30 minute phone interview with a technically competent hr rep
- a quick very easy online quiz (multiple choice/few sentence answers)
-one more 30 minute phone interview with a developer (open ended technical discussion)
-onsite for roughly 2 to 3 hours with a dew developers, all very open ended technical discussions
It was honestly the best hiring process I encountered. I didn't feel like I was even tested during the interviews, it was genuinely enjoyable technical conversations.
As best I can tell, our process is not typical. But it's also not fundamentally a bad thing. It obviously doesn't interest you and I'm sure there are others who wouldn't apply for the same reason.
But, there are others out there who appreciate a more thorough approach and might view the process you described as inadequate.
Also, our salary range is higher for senior devs.
I think what matters is whether the process being used by the organization is resulting in the hiring of developers that are a good match. I'm glad we are doing things this way even though it makes hiring take longer and likely filters out candidates who could do the work. It's an imperfect balancing act, one I'm always looking to improve. The main point of the post I made above was to show how IMO take home exams can be executed well, not to argue that everyone needs such an elaborate application process.
As best I can tell, our process is not typical. But it's also not fundamentally a bad thing. It obviously doesn't interest you and I'm sure there are others who wouldn't apply for the same reason.
But, there are others out there who appreciate a more thorough approach and might view the process you described as inadequate.
Path of least resistance my friend, path if least resistance.
The key thing is that you filter out anyone you aren't willing to risk paying $800 or more to do the testing before asking them to commit to the extra time.
This is pretty onerous, but it should be pretty damn good at rejecting bad candidates.
I do wonder how many good or great candidates you lose with this method, though.
I've wondered that myself. I don't know of a good way to evaluate that. I've thought about looking at bounce rate on the job description page or even showing a popup on bounce detection asking for feedback on the job description and application process, but haven't executed on either yet because I'm not convinced the effort would be worth it. If someone is going to bounce, how likely are they to take the time to tell us why.
It's also more than just the $800 we pay out. I personally proctor and grade the tests. That takes me 4-6 hours. Given my billing rate and roll in the company, that's arguably a greater cost to us than what we pay out to the candidate.
They'e also likely filtering out anyone who currently has a job, or a family, or even is interviewing at other companies. The process described here and in the posting at https://www.level12.io/careers/full-stack-web-app-developer-... is on the order of 30 to 40 hours. That's a massive time commitment.
The company chose to spend a lot of time evaluating all these take home projects, in the first step, and then reject people for silly reason afterwards? It's a lose-lose situation.
Sounds like their hiring procedure is deeply broken.
Let me give a charitable and slightly less negative interpretation:
It may have looked like the applicant had someone else do the take-home assignment for him.
That said, I imagine that they could have interviewed in a more constructive way that may have been less stressful (e.g., walk us through your thought processes as you completed this take home assignment — most frauds can’t do this convincingly).
The company definitely could have done better. If you have someone hitting a home run on a take home assignment, at that point a no-hire decision better have some very strong justification behind it.
I went back and looked through my emails with them. When I asked why they rejected not only did the cite the interview questions as a problem they didn't like that I was vague in response to "What is your GPA?" I had responded "Average"
> I got nervous and passed the question, they ended the interview early and rejected my application.
When interviewing, we test for presentation, self-control and anxiety as well. For me it's not a big deal, but for the guys that were interviewing you it seems it was.
Software development is a very stressful job, if you can't even handle the interview, how would you handle real life (c) at this corporation?
Software development does not have to be a stressful experience. I'm sorry that you believe it does as that implies that your experience has largely been stressful.
In my mind, that perspective is a huge red flag for me when I'm on the job hunt. If you are trying to stress me out in an interview rather than make me comfortable then I have to thank you for letting me know before I started working there that you would be exhausting both physically and emotionally.
I genuinely don't understand what this industry is looking for in an interview process.
Some background, about 2 years ago I started teaching myself to program full time. About a year ago, I felt confident that I knew enough to get a job, taught myself all of this leetcode stuff, and started applying.
People told me that if I could crush leetcode, I'd have no problem getting a job. What they didn't say was how impossible it was to get an interview when transitioning fields.
I was also told that people would look at my GitHub code, but considering recruiters spend about 15 seconds on a resume screen, this is more of a meme than reality.
In the last year in Bay Area, I've managed a total of 2 on-sites with well over a hundred applications, all of them quite targeted. Maybe 8 companies total have engaged with me, and I have not missed a single one of these types of questions at any point in the process, yet it hasn't been enough.
It seems that there are so few entry level jobs that that market is extremely saturated. I'm not sure how this industry expects to address it's serious man power shortage if no one wants to hire juniors.
I have not missed a single one of these types of questions at any point in the process
The most likely problem is simply that you are getting questions wrong, but you don't realize it. Companies don't often give candidates whiteboard questions, have the candidates nail those whiteboard questions, and then drop the applicants.
I mean that's a fair take, and one I've tried really hard to examine, but I feel as though the questions have a very obvious answer and you either get it or don't. And to reply to that, no one actually gives feedback so you'll never actually know.
Looking at your profile, you seem well suited to be able to evaluate this, so let me give an example from an interview I just had.
Looks good to me. You are right, it's hard to say what's going wrong when nobody gives feedback. Often if you do well on 4 problems and then tank the 5th, that can be enough to miss out. I think if you can regularly solve this difficulty of problem you should be able to get a job though.
Would you mind emailing me whatever resume-ish stuff you have at my hn username at gmail? Maybe I can give more specific help.
Python: uses underscores. Java, Javascript: camelcase. .NET: Capitalize. And so forth, those are the conventions that professionals use, so stick to them for an interview.
`biggestSize = 99999999 #MaxInt` This is returned if the caller passes you 0 for slicesToFind, so you're returning a nonsense value in that case. Either raise an exception (preferred as it catches errors fast) or return None.
`def area(size)` You're using pi r^2, so size is a radius, but it seems you call it with a diameter. This is why choosing distinct names is very important.
`newSize = (originalSize / newSlicesInCake)` Again, using "size" all over the place is confusing. I'd write something like `new_slice_area = (original_cake_area / new_slice_count)`.
I personally think you had a strong answer despite the criticisms mentioned.
But, yeah. Casing convention was surprisingly important in one of my interviews. They wanted to know that I was able to adapt to a new language as the job was for a language I didn't have on my resume. It's weird, but I understand where they're coming from. Seeing improper styling in a code review is an immediate red flag and an interview problem is ultimately a code review.
FWIW, if you want feedback on working code, Stack Overflow has a sister site Code Review[0] where people will critique your code and give lots of useful feedback. It's a pretty welcoming community so long as you're looking for a code review and not debugging help.
Companies are all over the place all the time. And I have seen this from both sides of the isle.
For example, at one company I was at, we said we were hiring for a full stack position, and we had a candidate come in that did well on all the questions, but then afterwords, we decided that we actually wanted a dedicated front end engineer. So no job for them!
It was no fault of the candidate. We just didn't know what we're were doing and screwed up by interviewing someone who was better suited for a different position.
Stuff like this happens all the time. Even if you blow the interview out of the water, there is still only like a 50/50 chance that you'll get the job, due to no fault of your own.
> What they didn't say was how impossible it was to get an interview when transitioning fields.
It's tough to get a foot in the door. I managed a transition largely thanks to preferential hiring for veterans, but even there I had a CS degree and some prior tech experience before going into the military.
The other problem is that "junior" really means "fresh out of college." Transitional resumes look weird and they get thrown out in favor of people doing the normal thing.
> I was also told that people would look at my GitHub code
Recruiters, probably not, but if I'm doing an in-person interview with you, I've definitely read your github page.
Tip: I'm looking for a. high quality code, and b. a delivered product.
What's high quality? Find code written by the best people (e.g. your standard library, a major framework) and aim to make your code look that good.
What's a delivered product? It has thorough, well-written documentation, and I can install it with a single command. If you set up a demo on a website, that's obviously better.
I'm 46 and still employed as an SWE - got my current position 3 years ago. The key is to not let your skills stagnate, and to always be playing with something new - even stuff that you may not think you'll use in your career. For instance, for the past several years I've been playing with machine learning with a focus on self-driving vehicles. Do I think I'll ever get a job working in that field? Not likely, but the skills I have learned may be useful down the line (and personally, I've been pursuing such an education for my hobbyist aspirations).
No one wants to address the elephant in the room, and that's that "leetcode" or "codefight" tests have nothing to do with your ability to design and build complex systems of interacting parts.
Why anyone puts a high degree of credence into these tests and whether someone can solve one in a high-pressure interview is beyond me.
Give them a take-home assignment where they can develop something similar to what they would be at the company, and decide if they wrote efficient, elegant code that's easy to follow.
Also, if you're self-taught and applying to a team of CS majors, may the force be with you. No one's more insecure than a CS major interviewing a guy who might have more development experience than he does.
I've put a line in the sand that I won't do take-home projects anymore for anything short of a "dream job". They can be extremely time-consuming, and it's entirely possible that they'll decide that they don't like you anyway. If the project doesn't result in an offer, and if I don't really have a need for another "Roman Numeral to Arabic Number" converter, then the time is completely wasted.
I mean, they can be time consuming - yes - but so can "grinding leetcode" for 3 months be too.
Have you seen some of the study guides / roadmaps people make for FAANG interviews? I've seen people claim they've coded hackerrank / leetcode / CTCI questions for multiple hours a day, for months.
Well, I actually work for a FAANG company (not going to say which one), and I didn't have to spend hours a day for months to do so; granted, I'm a bit of a math/theory junky.
EDIT:
Also, at least with the Hackerrank for hours a day, it's not for any specific company. Getting good at these problems is useful for getting a job at virtually any big company. Contrast this to the take-home project, where (in order to make sure you don't just copypaste someone else's from the internet), they make these things kind of specific, making the transferability to other things substantially less likely.
Even if you do have to "grind leetcode", at least that gives you the skills to get almost any tech job with very small marginal investment.
Once you get very good at leetcode-style questions, just apply anywhere and get an onsite with a one-hour interactive technical screen, that will also tell you about the sort of people and culture that employer has.
Conversely, homework assignments can easily take 5-10 hours. There's a lot of desperate unemployed people out there taking 10+ hours for these tasks, and setting the bar really high. A busy professional can be hard pressed to invest 5 hours they can ill-afford coding this task after a full day at work, then get rejected because some unemployed applicants dedicated 20+ hours for the same task.
There are also services that solve these tasks for you, so now you - a busy professional - are competing against hordes of people who do these tasks all day every day for a living.
So you hard-scramble 4-5 hours to finish a take home assignment to apply to just one place (!), and then you send it in and get a meaningless rejection note. Sometimes you don't get anything at all.
These are some of the reasons I'm never completing a homework assignment again.
I agree; in NYC, it's not uncommon for a software engineer to make $50+/hour; if you value your time at that rate, that means that some of these take home tests are effectively costing you $250-$500, when there's no guarantee that it will even result in a job.
What also frustrates me the most is when they say "use whatever language you want", I use Scheme or Haskell, and I get feedback to make it more "modular" by making it more object-oriented. The reviewers of the take-home assignment will be too afraid to admit that they don't understand Haskell, too afraid to admit that they made a mistake making the language a wildcard, and automatically assume that it's less scalable because it's not Java.
If you're willing to go uptown to Washington Heights or Inwood, you can get a decent two-bedroom for around $2,000-$2,500 a month, at least I was able to without too much effort. I survived for about a year with a wife (who wasn't working at the time) at $90,000/year (I was working for a university).
Definitely became about a million times easier to live when I left academia and started making six figures.
> I guess, but then your commute will become a nightmare.
I mean, not really? I just hopped onto the A train straight down to NYU; it took me like 40 minutes door to door, and being on the train allowed me to read, so it's much better than driving.
I make substantially more than that now, but have moved to Brooklyn since then; door to door it takes me about 50 minutes, but I wouldn't classify that as "a nightmare".
> a software engineer is a highly trained professional working in an high-demand field.
At the time I was a college dropout whose only professional experience was in startups, 4/5 of which had gone bankrupt, though the university actually paid their actually-qualified people pretty poorly. My coworker had a masters degree in data-science, and was only making $75,000/year.
> I make substantially more than that now, but have moved to Brooklyn since then; door to door it takes me about 50 minutes, but I wouldn't classify that as "a nightmare".
Depends on your perspective, I suppose. Personally, spending 2 hours every day in commute is definitely not how I want to spend my life. A major reason I would move to a city like NYC is to eliminate such a long commute.
Last few times I took the train to Brooklyn, the train was always delayed by several minutes. Sometimes 15+ minutes. Trains were packed, especially during rush hour, and there was nowhere to sit. Occasionally they'd just stop place for technical reasons for as much as 10-20 minutes. Definitely not a smooth or pleasant experience.
Regarding compensation: $120k is becoming entry-level compensation in the Bay right now.
The express trains will get you from inwood to columbus circle in... 5 minutes? 10?
I have no desire to live north of 59th st, but lets be real here.
Anyway, NYC software engineers do not need to compromise on their living situation. Google alone has bought two billion dollar blocks of Manhattan pays just as much as in the bay area.
Here's an ad from someone looking to hire a programmer in Nassau county:
VOIP company looking for a technician to assist with programming back end systems. Should be knowledgeable with various database programs and voice services and applications.
Primary experience: Windows Environment - IIS / ASP.Net, VB.Net, MS-SQL, XML, Json.
Primary helpful: PHP, MySQL
Secondary helpful: Google SDK, ffmpeg / sox, SIP Voip , basic CentOS / Debian navigation
Nassau county seems to be just barely onto Long Island, near JFK.
Here's the list of skills wanted from another NYC junior programmer want ad that says $35-40K/yr:
Programming Skills Desired:
Transact SQL
Database Design & Theory
HTML
XML
CSS
JavaScript
VB Script
Auto Hot Key (AHK)
PHP
C#
All of these (except for AHK which I've never heard of) look trivial, but I'd expect the interview would be impossible, because the interviewer is going to be primed to expect people with no skills and therefore assume the worst of anyone who doesn't have 110% current experience in each.
All of those look trivial for a Junior Programmer to have experience in?
VB Script and PHP and T-SQL and C# along with the rest?
I'd expect a college student not to be actively taught any of these (T-SQL maybe if they took a database class), and they're not exactly the new hotness for languages to learn.
I'm saying this as a dev that works for a company that uses a Microsoft stack right now too (I almost never see anyone talk about C# on this site or in the Who's Hiring threads.).
This seems like a laundry list of that employer's specific tech stack hoping they can get some sucker that has their exact stack because they happened to experiment with all of those languages in their free time, which is unlikely. Anyone who does have this stack can probably get a much better salary almost anywhere else.
It does say "desired" at least, and not "required", so they might not expect applicants to match this completely, but I bet some of the interviewers haven't gotten that memo (which seems to be what you're saying in the last paragraph also).
Also you mention you haven't heard of AHK, and it can be pretty useful. It's basically a macro program for doing things in Windows and assigning a hotkey combination to it. I've scripted some things in it before, although I'm not currently using it. It's not that hard to learn, and I can't imagine that any programmer couldn't pick up the basics after a few hours on the job, so I don't know why anyone would include that in a list of technology, thereby adding one extra thing for an applicant to think they need to have experience in to apply and not apply to their company.
No, a junior programmer probably wouldn't have experience that would satisfy them. And they probably wouldn't hire someone with years of experience. I meant trivial to learn in a few months.
CUNY is looking for a Research Assistant - starting pay of $19/hr, and the only technical skills requested are that you know Stata and R. But you do need a Master's, preferably a PhD, and lots of knowledge of statistics. Basically what I imagine people are claiming equates to a six-figure data scientist for other employers.
Like a lot of ads, they mix ridiculously specific qualifications with ridiculously generic ones.
For example, one important skill is:
- "robust practices" such as "loops"!
and another is:
- "knowledge of City University of New York Institutional Research Data"
People are advertising for "Data Science & Python Instructors" in NYC starting at $15/hr.
Requirements are "in-depth knowledge in big data, data science, Python programming, object-oriented programming, machine learning, or natural language processing (NLP)."
Responsibilities are:
"Lead technical workshops and training sessions in different areas of data science and Python programming
Help clients and students understand the immense career options for Python programmers and data science knowledge
Motivate clients about the career change and career boost that upskilling with Python, data science, and blockchain can provide."
I think it stands to reason that if big data, etc. generally paid much more in the NYC area, nobody would take such a position, so this must be representative of the job market.
I've had very similar experiences like this; the most similar was a supposed '2 to 3 hour' problem that had no time limit and no language restriction. I rushed through it (had an offer for another position with a decision deadline coming up), and it still took 12 hours. I made an error where I didn't capture all cases, which I was fine to be rejected on those grounds, but I got the exact same feedback for more modularity, and when the interviewer pointed out my 'mistake', it was clear they had no idea what the code was doing.
Let's think about it for a second: reading code is hard. We all know that. Reading code in "whatever language" that you may not know at all is nigh impossible.
These companies get a ton of candidates and tell them all to submit homework assignments that take at least 5 hours (in your case 12).
Reading 12 hours worth of coding work is intense effort even when you know the language well. If you don't, it can take a multiple of that number of hours.
Some programmer got 10-20+ of these completed tasks dumped in their lap.
I can pretty much guarantee they never really took the time to read and review the code with anything close to sufficient care or attention. Doubly so if it's in a language they didn't know very well.
> But on the other hand, I think good professional like you, don't need to take 4-5 hours for a take home, since you'll be fine without doing it :)
Thanks, but I'm a programmer, not a magician. A good programmer might work faster, but he's not able to reduce any amount of work to a constant.
A good programmer may accomplish in 4 hours what an average one will accomplish in 7-8, but he will not be able to do it in 1 hour. Also, he will still seem inferior to an average programmer who took 20-25 hours to perfect this project, especially if those take-home tasks are all that other programmer does.
> So you hard-scramble 4-5 hours to finish a take home assignment to apply to just one place (!), and then you send it in and get a meaningless rejection note. Sometimes you don't get anything at all.
Do you mean you've gotten rejection letters straight after submission? In anything but exceptional cases I thought it was standard practice to always have an interview after the submission and talk about the solution - how you talk at the code, design choices, possible optimizations and issues should weigh heavier than the quality of the submission itself.
So I don't think anyone using services doing it for them has any success with it. And if they do, you may have dodged a bullet.
> Do you mean you've gotten rejection letters straight after submission?
Yes, that definitely happens. In fact, from what I've seen, it's the norm: companies who give out homework projects only bother spending any time with you if you completed them to their satisfaction.
Otherwise you just get a meaningless auto-generated that they're rejecting you.
> So I don't think anyone using services doing it for them has any success with it.
Why not?
If I'm a mediocre programmer in the US, I can go online and buy a solution with a full, detailed verbal explanation attached. Then I'll take maybe 40 minutes to read through the solution and explanation, and voila, I'm ready for any followup session you may have.
Not to criticize other people's preferences, but I have read people's gripes about things like take-home assignments and fermi estimation for years and years, and what I want to know is how I can find an employer that likes them, and hires based on them, because I both enjoy them and would like to work with people who like them.
There's quite a few employers who like take-home assignments. However, just like GP experienced, that doesn't mean they will hire you based on take-home assignments alone.
So they'll have you do one before you even come onsite. Then if you pass, they'll run you through the same whiteboard loop we all know and love. Even if your take-home performance was fantastic, they'll still reject you if your onsite performance was anything short of.
There is, if they want to hire strong candidates in this competitive market.
A candidate like that will have 10+ companies trying to recruit them. Out of these, 9 will require 1-2 hours of remote screens at most for an onsite. The 10th will require a 5-10 hour take-home project for the same onsite.
I don't know, but if 10 companies are competing for one employee, then the 9 that inevitably fail obviously were using the wrong approach. You can't point to the similarities as evidence of the correct strategy, because by definition the differences were probative.
I've done one take-home project for a job and it was a terrible experience. They gave me an overnight challenge that in their words was meant to confuse you and was impossible to complete. They informed me I should have realized this and focused on one specific area of the challenge. I spent all night trying to complete it, focusing on a "pace over perfection" approach and then was torn about by their dev team. Of course I explained there would be a difference between a DB, Server, & UI I implemented overnight and what I would do at work. They informed me I was too defensive, attempted more of the challenge than anyone, and that I wouldn't get the job.
tl;dr: I feel you completely and will never waste my time like this again.
They give you impossible requirements and expect you to come to conclusion that these are impossible requirements AND zero in one specific part of the impossible requirements and implement that bit?
Come on. It's not completely unheard of to be given confusing or impossible requirements while working for some companies and while in those situations I would probably just communicate back to the Business Analysts what's broken about the ask, you aren't given that opportunity in a take-home with a new company. Presumably a bunch of other people have been given the task and successfully completed it, so I would probably try to power through it.
I'm sorry, but that's an asshole thing to do to someone. In an interview setting it's more okay, because you have a chance to mention it after a few minutes and talk it out and get feedback on it or they can stop you if it's clear you're not pointing that out and you waste a bunch of time, but with a take-home you won't get that quick feedback, even if you send them an email, and can waste hours and hours on that thing.
Interviewers reading this, don't do this, please. It would leave a very bad impression on me and probably a bunch of other people about your company.
Engineers: coding exotic data structures on a whiteboard in a time-crunched interview is high-pressure and unrealistic!
...how about we give you a project that more closely reflects problems we solve here day-to-day that you can solve in the comfort of your home on your own hardware in whatever IDE makes you comfortable?
Engineers: That's absurd! I've drawn a line in the sand!
But, software engineering interviews are pretty rough. It is a uniquely miserable experience. Since I'm a temp worker I have a lot of them. They don't get easier.
I don't know of anyone who likes high pressure time crunch interrogations except maybe the ones getting to do the interrogating part.
A take home test isn't so bad. But, ya if you asked me today if I'd be willing to do one like that again I'd say probably not unless I was desperate. The reality is, it is a lot of effort to put in for it to all come crashing down with a rejection anyway. When you're applying at dozens of companies if they're all requiring that level of time and mental commitment from you, you can't swing it really and it's stressful. Interviewing and being under financial pressure at the same time is already a stressful position to be in, let alone being mentally and emotionally flogged for it.
All the jobs I was ever hired onto the hiring manager just had a good gut feeling about me and the tests never mattered. And, they were happy with my work.
The ones that never hire me usually rely on all the employees to vote on you, if even one says no you're out. Or, the ones that are very fixated on your GPA or whether you can invert a binary tree on the whiteboard while they yell at you "Wrong!" along the way.
Early in my career, I went to an open house sort of thing. Hit it off chatting with one of the chief architects who basically cut through all the red tape and got me a position. A couple weeks after starting work, I get a rejection letter from HR. Was a bit awkward conversation when I went up to their office with the letter to check if I still had a job. (I did, but boy was I sweating at the time)
> I don't know of anyone who likes high pressure time crunch interrogations except maybe the ones getting to do the interrogating part.
As someone who has been through this early career I agree with the sentiment. But being mid-career it is kind of fun to go through one now, the catch being that you shouldn't give a sh/*t if you get through or not. Think of it more as two-way courting instead of a one-way interview, not all of them pan out.
I actually don't have a huge issue with whiteboarding, but I don't think it's a fair comparison.
Something that "reflects problems we solve here day-to-day" is missing one feature: you're not paid for it. I have a pretty good job right now, but it's not like I'd keep doing it if they decided to stop paying me.
I had an interview a few years ago that I thought did something cool: you spent half a day working with people, and they paid you for your time (around $40/hour, if I recall), and made an offer based on how well you worked with people. I ended up not getting the job, but at least I didn't feel used or like I wasted me time in the process.
...how about we give you a project that more closely reflects problems we solve here day-to-day that you can solve in the comfort of your home on your own hardware in whatever IDE makes you comfortable?
Me: How about we do something that more closely reflects problems you solve there day-to-day there and during the day? Your interviewer being present throughout means you'll get to see how I work and you'll be able to provide clarifications or answer questions as well. As a convenient side-effect, we'll both know that the exercise is a sufficiently valuable use of everyone's time to give up the equivalent amount of salary over it.
> "...how about we give you a project that more closely reflects problems we solve here day-to-day that you can solve in the comfort of your home on your own hardware in whatever IDE makes you comfortable?"
"How about you, a highly trained professional, do 5-10+ hours of unpaid work for us, that will be judged against desperate candidates who invested 25+ hours into them and/or bought a solution online, so you will get nothing for hours of work, except a meaningless rejection notice by some overworked programmer who didn't even read much of your code since he had 50 candidates' solutions dumped on his plate?"
How about working to improve those take home projects since it seems to be more promising than the whiteboard interview which is doomed to suck. For example, my company has a take home project that involves hitting some APIs and doing some stuff with the JSON they return. It shouldnt take more than an hour and we give a time limit of 3 hours just so people don't spend days on it.
Who decided that whiteboard interviews are inferior to takehome assignments?
Personally, I've hired many great candidates, and got several great jobs, via whiteboard interviews. It's not perfect but it's far superior to takehomes, which are completely one-sided, very hard to get right (by the recruiting company), and often are boring.
Whiteboard interviews and (even better) pair-programming with an interviewer are reciprocal and allow me as a candidate to learn about the company rather than just do some work for them for free. Often the tasks are more interesting too.
In my opinion, it is the takehome tasks that are "doomed". Their premise is flawed: "we don't want to invest an hour of our engineers' time in talking you worthless candidate, so we'll instead have you spend several hours coding up a task that our engineers will never take the time to check properly, since their time is more valuable."
In my opinion, the future is platforms interviewing.io, where candidates are vetted and top candidates get the minimal investment of an hour of the company's engineer time to pair-program, which is far superior and more reliable way of screening.
In my experience, most people who are hired by Google or Facebook consider the interview process to be good, and generally approve of applying that same interview process to find new candidates.
However, that set of people is outnumbered by the set of people who applied to Google or Facebook and got rejected.
I had one great interview once that worked out very well; it's only downside was the length of time it took.
I went to the interview, did a sit-down back-and-forth in-person discussion about my past work and resume; the usual. No whiteboard.
Then I was told I had to do a practical exam; they basically gave me a laptop (MBP with MAMP - this was for a PHP/MySQL position), sat me in a cube with the rest of their development team, and gave me a test to work thru. First half was just question-and-answer (multiple choice and t/f - maybe some written too IIRC). The second was to code up a simple CRUD app using PHP and MySQL.
They said I could use any resource I wanted - google, etc. This was about 3pm, and I had until 5pm (quittin' time) to finish. During this time I was working in the area I would be normally working, among potential future colleagues, with all the office distractions normally found at this place.
Oh - I forgot to mention where: Fender, in their marketing department.
So - randomly - people would start to play amplified bass guitars, or a drum solo out of nowhere. At one point, someone rang a cowbell and stuck a triangle. Fortunately, there wasn't any artists being introduced, but that was a possibility too. Various people on the team came up and introduced themselves. Distraction galore!
But that was how it was going to be - so that's how they tested you.
5pm rolled around, and they came by asked how I was doing - I let them know things were coming along well, but that if I had another 15 minutes I could get things more solid. They said, sure - take your time, we sometimes stay late - no problem. So I finished, and gave it back to them.
Went into a separate f2f with the director of the department and some of the team, chatting about my experience and such. The SWE supervisor and lead SWE came over, shut the door - and basically made me an offer right there.
I worked there for a couple of years before they did a downsizing and I had to "move on". But during that time, I found that this kind of test was actually very, very difficult for more than a few people. Some didn't even do anything - they just sat there. One guy sat for a few minutes, then got up and left, never to be seen again. Some would complete the first half of the practical, but completely blank on the second half. All of this, despite having a completely set up system in front of them, plus ability to reference the entire internet. If they didn't know PHP or MySQL, it wasn't for a lack of resources that they couldn't or didn't write anything...
Then again - I've heard of people failing to write fizzbuzz, so...
But overall - that was one of the top interviews I've ever had for an SWE position. I guess it was partly the practical testing, but also putting you "on the floor" to see how you'd fair in that environment (and yes, the whole instrument playing thing was a daily thing). I guess some people couldn't handle it. Others might've just not wanted to do the practicals feeling it was beneath them or something? And still others might have actually just "frozen up" for one reason or another - or really couldn't do what they claimed.
For me, it was one of the easiest and relaxing interviews I had ever done, and I would do again - and would love to give a similar interview to someone, if I were in that position.
1. A phone conversation with a manager to see if we were on the same page.
2. A one-hour on-site coding test, writing a fairly simple class using very strict TDD (i think we did this in the evening, after my day at my previous job had finished).
3. A whole day on site, spending each half of the day pairing with someone on something. In the morning we wrote a little demo app for using Redis on Cloud Foundry (in Ruby, which i didn't know!), and in the afternoon it was some made-up webapp (using Spring, which i also didn't know). Both things either were or were very like real development (on the Spring thing, we spent half an hour setting up the environment, fifteen minutes coding, then two hours debugging and reading Spring docs to find out why it wasn't working, so exactly like real Spring development).
The first two stages were pretty undemanding, which was good for me. The final stage took a lot of time and energy, but i got to spend the whole day with two current employees, both of whom really impressed me (shouts out to Chris B and Nick S), and that was really good for me.
In general, i really like to be given a stringent interview, because it indicates i'll be working with high-calibre people who have passed it. But it does have to be effective in its stringency, not just unpleasant. Amazon Instant Video's hiring process was the latter (and not just because it took a whole day with no break for lunch).
Yes - my best interview was with an online flower shop. We walked though building a "like" widget that other sites could embed - from the front end architecture, to middle layer to database choices. It was exactly like something you'd expect a lead engineer/architect to design.
One of them that worked out very well: the hiring manager was highly technical (very strong developer, who later got an applied-mathy PhD), looking for a sort of technical partner on a big new technical system. He knew of me from some open source, had seen my resume with lots of experience. When we met, he said "obviously, you know how to program", and we went immediately to talking about what he was building, asked my ideas about it, etc. We covered working style and values and such, too. That turned into several years of very successful work.
I'd be happy to interview people that way. When you can't talk enough about your project, nor about stuff they've worked on before, you can still engage them in a discussion of a hypothetical but real-world problem (not CS101 problem sets), and go from there.
If you know your own technical stuff, IME, it's pretty easy to tell from talking with someone that they can fizzbuzz, without a hazing gauntlet of whiteboard exercises that give little sense of anything (other than that the company thinks hazing experienced people is a good idea, and that the person will submit to hazing).
I'm sure I'm not the only one here in a position where I can re-shape my company's interview process. And it's loud and clear in the comments here that people are unhappy with the status quo. So, if you were making a hiring process, what would you ask candidates to do?
> if you were making a hiring process, what would you ask candidates to do?
Review code.
It would be at least a few pages, in the language they would be working in, with a few bugs that you know of in advance. Depending on what the code does and the particular bugs you introduce you can get a pretty good idea of the candidates' comprehension of the things you need. (Bonus points if they find bugs you didn't introduce.)
One problem with this is that both the code itself and the evaluation of the candidates review have to be done by sharp folk who know what they are about. Otherwise GIGO.
I'll second this. This is mostly a copy paste of my comment on another one of these interview topics:
"One of my best interview experiences was exactly that. The interviewer handed me a printed out class and said "tell me what this does, any mistakes you see, and any little improvements that you think could be made."
I got the job and later confirmed it was a class that was actually in the software I would be working on, not some toy class he came up with.
He also asked me a handful of technical questions and had me go over some source code I brought in, but that was about it. At one point he even said, "I know enough to know you can do the job, but now I'm curious what you really know." And asked me some no pressure really low level questions. When I said I didn't know the answer to something he'd spend a couple of minutes explaining the concept to me.
I walked out of that interview having learned a few new things about my field. A couple of them have stuck with me over a decade later.
And the guy was a grizzled veteran who was the lead on some massive mainstream video games I pretty much guarantee you've heard of. He used to code them in Assembly, even. He's probably the most knowledgeable programmer I've ever met.
I've never had anyone try going over actual code with me since, in probably a couple dozen interviews, and I don't understand why. It seemed very effective."
I am surprised how many people here considers take home tasks as an "unpaid work". You are already doing other interviews with them which costs you time as much as an assignment does. Why it becomes a problem when it is a home assignment.
Honestly I would take home assignment over any technical interview. Even if it will cost me 3x time. At least then I will be working in a relaxed state and will do my best, compared to an interview where anything can go wrong.
Interviewing is somewhat symmetrical in that the employer also wastes resources if they don't hire you. It's no cost to ask 1000 candidates to submit homework.
Homework feels really unnecessary too. If the interviewer would like to see code I have written I have a USB stick full of old university assignments and some public projects on Github.
Coding tests are completely useless unless the company doing the assessment can clearly explain up front their method for assessing the code.
I did a coding test once that was organised, met the requirements of the test spec, was documented, included tests and in identified and solved an inconsistency in the test spec.
The feedback from the employer "they didn't like your code" - right there is the point at which I realised that coding tests are, without a quantifiable, systematic assessment method, complete garbage.
Imo the industry should consider a more cooperative process that involves larger real world problems with the interviewer working with the interviewee to work through the problem, teaching them even, if need be, to get to an acceptable solution. Big tech interviews can already run almost a day long and the phone test is just a whiteboard lite. I feel like the face to face time is kind of wasted on an open ended examination format rather than trying to find out a developer's temperament, behavior, and openness in a real work environment where they will be expected to work constructively with peers. Given a lot of companies have separate design and behavioral interviews, with the former probably being close to my idea already.
I just hate the technical beat stick that is the whiteboard test, feels like going up in front of the class in data structures and algorithms classes
As a fan of algorithmic problems and websites like leetcode, I'm totally fine with this kind of interview challenges, it's a good way to select candidates, and anyone can prepare for it
I see what you mean, but totally disagree, the important is not necessarily to pass those challenges, if at least there are interesting ideas presented.
You could say, why not giving Sudokus, or chess boards challenges to candidates? It's not totally bad either, it's just further from the context
All this is not incompatible with knowing other engineering fields, networking, security, data structures, code patterns, which are indeed more useful in day-to-day work.
Students are often selected the same way in exams, which won't really matter in their future job/life if you look at their content. But the process matters, school teaches you to learn mostly, like those small logical/algorithmic challenges trains you to think, visualize, reflect on data and more
474 comments
[ 4.2 ms ] story [ 282 ms ] threadI think it's weird for people to complain about whiteboards and technical questions, given the implied alternative. There are many other industries if you prefer the traditional hiring model.
That is, if you can get an interview. While you complain about the awkwardness of a whiteboard, people in other industries are quite happy to get a call. And in the interview, they compete in terms of poise, diction, rapport, and other intangibles, without the chance to repair any lack thereof by demonstrating technical competence. And they get 1 hour to make their case, not 3-6 hours.
All of the complaining is tiresome. Some companies have difficult interviews, many do not. Who cares?
Guess the gist is that bottom of the barrel jobs have easier interviews. I could buy that.
That's a solid point. Yeah, engineering interviewing could be better, a lot better, but as we push to improve it, it's really important to keep in mind how privileged we are. I remember having a discussion with one of my friends in a different field who was thrilled to get a recruiter pinging them on Linkedin, for example.
Also, given how chaotic the field is, and how people come from all sides (some with CS degrees, some self taught, some who worked at great tech companies, some who didn't, etc), assuming everyone's a bullshitter is unfortunately kind of expected. We absolutely should treat people with respect and dignity, but that doesn't mean being naive either.
And in other fields getting fired and even demotions are pretty common, while in big tech hubs thats nearly unheard of, except for extreme cases or very very entry level people.
When was the last time you saw a news story about a Ceng / PE getting disbarred? Stories about doctors are much more common.
A lot of people can speak (very deeply) about many topics, but as soon as they're in front of a keyboard it falls apart.
You cant directly compare a professional job to say an skilled worker in a corner store / bodega - we may have similar problems but they express them selves in very different ways.
1) Extremely regulated fields with strict education, training and certification requirements (eg: a physician)
2) Fields where you have to show something. Designers frequently need to have extensive portfolios. Carpenters need both that and real world referrals of happy customers (not just nearly automated HR checks we have in software dev where no one will say anything bad out of fear of getting sued)
3) Trial period followed by sink or swim. Come work for us and if it doesn't work out we'll demote you or fire you. There's a few places like that (I think its Netflix's model?), but generally firing someone has everyone around calling foul.
Thats absolutely my personal preference. The argument against it (very valid), is that if someone with a family relying on their paycheck quits their job to go work at another with that policy, and gets let go after 3 months, they're kind of fucked.
We could have this for our field - and to a certain extent we already do, at least via a github repo. Only problem is, nobody seems to look at them.
People have mentioned that SWEs should have a portfolio - but I've never worked for a company they let me take home code and put it in my portfolio. For web development, I may or may not be able to point to a specific site and show someone "see this here, I developed this part" - and even if I could, how would I prove it? Most of the time, it doesn't matter - as that code/feature/website is likely gone or changed by that time.
So you only have a couple of options to develop a portfolio, and those options only work for some people, not all: Either contribute to an open source project, or work on your own side-projects, putting them all up on github or similar repository you can reference.
But not everyone has the time or inclination to do this.
There are significant pushbacks in some communities that this isn't inclusive, because people can't spend that much time outside of work hours to build one (and as you pointed out, internal code can't really be included).
Graphic designers and stuff are always sketching/drawing stuff on the side for their portfolios though, and that seems pretty expected, so... /shrugs.
Don't forget the interviewer's arbitrary "system" for weeding out applicants that has no basis in research and varies from one hiring manager to the next.
Whiteboards are an unbelievable utopia compared to so many other disciplines. It's an amazing stroke of luck that software developers don't have to do medical-style "residencies". We don't have to rely on letters of recommendation from our professors (hope that professor you slaved away for for six years doesn't have a bad day when he writes your letter!) We don't have to pay thousands of dollars to take excruciating standardized tests to obtain certifications that expire in a decade. We don't have to do unpaid internships, we don't even have to wear business suits to the interview.
Software devs don't realize how many people in the world would kill to have things as good as we have them.
But I'm not sure it's that developers are being given gold and don't recognize it. It's that developers are being given mud and recognize it, while most other people are being given shit instead.
I'd rather be handed a pile of mud than a pile of shit, but it's not hard to feel that there must be some alternative that's better than either.
(Maybe there isn't, though. Matching candidates to jobs is just hard, I think; everyone's incentives are misaligned, performance is difficult to predict, it's amazingly difficult to get past your initial snap judgements or even to realise that you're failing to do so, etc.)
The problem is those things only get you in the door, then it 100% comes down to your whiteboard under-fire skills - which admittedly have gotten worse for me the older I get. Although I know I'm a better programmer now.
As if it's a given the whiteboard process actually provides a useful measure of technical competence.
Technical interviews where you look at some code and review it, or is asked to broadly describe how to design a system etc isn't unreasonable, and as you say it's better than judging only on poise...
[statement of the Maximum Subarray Problem]
Before going further—and regardless of your coding proficiency—we’d like you to spend a few minutes and take a stab at this question.
From Wikipedia:
The maximum subarray problem was proposed by Ulf Grenander in 1977 as a simplified model for maximum likelihood estimation of patterns in digitized images. ... Grenander derived an algorithm that solves the one-dimensional problem in O(n2) time, improving the brute force running time of O(n3).
Jay Kadane of Carnegie Mellon University soon after designed an O(n)-time algorithm for the one-dimensional problem,[1] which is clearly as fast as possible. The same O(n)-time algorithm was later automatically derived by algebraic manipulation of the brute-force algorithm using the Bird–Meertens formalism.[2]
If you really think anyone -- short of a faculty-level algorithm specialist at places like CMU -- can be reasonably expected to derive an optimal solution to problems like these on the spot (as opposed to the way candidates are actually forced to prove their ability to "solve" them: by binge-cramming on dozens and dozens of problems like these for weeks on end, so that they have at least a 50 percent change of passing your "filter") --
the you're either very naive, or deliberately kidding yourself.
I can see an usefulness to questions like these. Maybe you are precisely not looking for reasonable but for exceptional people. Or maybe you want to see how the candidate behaves in front of a difficult and concrete problem, even if you do not expect them to solve it.
The unstated implication in most cases is that you damn sure are supposed to solve it - or get "flushed".
As to "seeing how the candidate behaves in front of a difficult and concrete problem" - I'm sure this is what a lot people who ask questions like these think they're achieving by asking them. But again, you have to ask yourself if that's what the questions actually do permit you to assess.
My sense is that it's not - and that questions of this sort are mostly, well - cheap gimmicks, basically.
Think about the potential hidden states (already knows the answer vs blank) vs the evidence you could get from the person answering right or wrong. What should your prior be? Surely that there's a very large chance this person is not a genius. Now how much is that going to move, given that by far most people are not geniuses?
To put it another way, what would you do to impress someone following your reasoning? I would pretend like I didn't know it, and then act out the miraculous direct line to the answer.
As for looking at how people behave in front of a hard problem, what on earth do you know about that? Do you have evidence connecting people's behaviour to subsequent on-the-job performance, both for people you hired and those you didn't?
Now if you'll excuse me I need to grind out some dynamic programming problems.
I don't disagree with the main point, but FAANG companies' interview questions are available all over the net. In many cases (most?) they'll literally tell you what they are ahead of time, so it's a lot more of a filter for grit than a filter for computer science aptitude.
Eg: I know for a while Google was pretty keen on asking about A* algorithms. A friend who worked there mentioned that to me. Not having a CS degree myself, my first reaction was that was a very "either you know it or you don't...maybe you can figure it out, but that's not gonna be fun" kind of situation.
At some point i saw the material Google gives out. A* was literally listed there as something that they were likely to ask.
Now, it does mean preparing for a very specific interview, which not everyone (or even most people) would be willing to do. And today I don't think Google has the reputation to pull that stunt off for much longer. But a couple of years ago, if you really wanted that free cafeteria? Its a small price to pay.
What I would usually do with a programming question is ask a number of my coworkers to solve it and use that to evaluate what an "average good performance" looked like. So if most of your coworkers can solve a problem in 10 minutes, it's reasonable to expect that from someone you're interviewing.
It's a little harder than that. That would be for the screening interview. But on-site, you'll get several algorithm interviews (I had 3 in two such companies, plus 2 system designs). The recruiter told me that you could fail one of the interviews, but you have to nail the four others.
Even if you are well-prepared, I find it quite hard to have consistant results on these algorithmic questions, especially considering the stress of the situation and the fatigue of going through several interviews in a raw. Practicing leetcode problems for months is fun, up to a point (especially if you already have a job).
That being said, I like that the rules are clearly stated and everyone is given a chance. They don't discriminate as much on your background/educations as some other companies.
Similarly for many other tricks.
I have never used it in real-life development, though.
> I have never used it in real-life development, though.
I'd wager both of these are true for the majority of technical interview problems, and I think that's exactly the point.
so, this question isn't testing if you can find a loop, it's testing whether you have enough experience to have heard of it. which is a fine enough thing to test for.
i understand your point, that questions are often designed poorly, but your example isn't necessarily a good one.
Even if we leave aside the question of to what extent facility at competitive programming contests actually correlates with success in real engineering environments (my own sense is: yes it can help; but at the end of the day, just not all that much) --
if that's a skill you consider to be important, then for gosh sakes, put it in your job ads. Something like "We typically hire CS olympiads, and folks who have at one point or another have been fascinated by competitive programming contests. If this doesn't describe you, then this probably isn't the right company for you."
The fact that the first couple of people to consider the problem didn't find efficient solutions doesn't mean that finding efficient solutions is a super-hard problem. Grenander wasn't really interested in the 1-dimensional problem but in the (I think distinctly harder) 2-dimensional problem, and I don't think he was really an algorithms guy. (That's not a polite way of saying he wasn't very smart; he clearly was, but I don't think finding optimal algorithms for things was what he was most interested in.)
For the avoidance of doubt, I don't think you're likely to get much insight into interview candidates by seeing whether they spot the most efficient solution to a problem like this unaided -- even an extremely strong candidate might just happen not to think of a good approach, especially under stressful interview conditions. (It might still be a useful question if a less adversarial approach is taken -- discuss the problem with the candidate, get a sense of how they think, and nudge them in a helpful direction if they get stuck -- or if the only question is whether the candidate can find and implement any approach, which for this problem I think any reasonably competent software developer should be able to do since you can basically just translate the question into straightforward but inefficient code that does the job.)
I'm not sure whether we actually disagree or not, but I do think you may be overstating the difficulty of this particular problem.
[1]: https://leetcode.com/problems/maximum-subarray/
[1]: https://en.wikipedia.org/wiki/Maximum_subarray_problem
This is O(N^3) because iterating over index pairs is O(N^2) and doing the sum is O(N).
At least there seems to be some general agreement (within the thread) that, while not super-hard -- it's still basically a "eureka problem" -- that is, even among what we may consider to be adequately good engineers -- some will get the visual solution more or less right away; but a fair percent (25 or more) won't.
And that's given ideal circumstances -- working alone and in unhurried circumstances. Throw in the (considerable) distraction of having to think aloud in front of other people (who themselves may not exactly be providing the most relaxing vibe, to say the lease), and all the other stress that comes with interviewing -- and I'll bet that 25 percent failure rate rises quite considerably, in turn.
And in many cases -- if you fail just one of these "eureka" problems -- and maybe fail to provide the expected canned response to a "behavioral" question or two -- than that's pretty much all it takes. To be considered, you know, "not a fit".
As to why I jumped the gun -- fatigue, probably, from being fed a few too many "cute" problems like these.
Eh, you really don't need to be a "faculty-level algorithm specialist" to solve this problem. I wouldn't even really call it dynamic programming.
Just make one pass to calculate the sum of the first n numbers for each n. Call that array prefixSum. Then you make a second pass calculating the lowest prefixSum[i] for i <= n. Call that smallestPrefixSum. The maximum subarray sum is then defined by the largest prefixSum[i] - smallestPrefixSum[i], and you can get the subarray itself with a bit more tracking.
I don't know if it's a great interview question but I think most of the engineers I have hired (at Google or Facebook) would be able to solve this problem on the spot, given 15-20 minutes.
If I ask them to draw a picture, they get your solution pretty much immediately.
So I put this problem in the “gotcha” drawer, because it basically depends on having a eureka moment to get the easy solution.
Still fun to pull it out, but more as extra credit than as a useful interview question.
- https://en.m.wikipedia.org/wiki/Binary_search_algorithm#Impl...
> “When Jon Bentley assigned binary search as a problem in a course for professional programmers, he found that ninety percent failed to provide a correct solution after several hours of working on it, mainly because the incorrect implementations failed to run or returned a wrong answer in rare edge cases. A study published in 1988 shows that accurate code for it is only found in five out of twenty textbooks. Furthermore, Bentley's own implementation of binary search, published in his 1986 book Programming Pearls, contained an overflow error that remained undetected for over twenty years. The Java programming language library implementation of binary search had the same overflow bug for more than nine years.”
That’s not sincere, it’s disingenuous.
The (obvious) point is that binary search is widely misimplemented even by professionals writing textbooks (and this had little to do with the specific Java bug responsible for that one example), and this is all stated directly in the link you didn’t read.
It’s comically ill-suited for time based tech screens as even experts teaming with editors often still get it wrong.
When the array/list consumes a memory footprint larger than ptrdiff_t it was possible that the specific implementations might fail. It is crucial to remember that the difference between memory address is signed and that the size of objects (even if bare machine integers of some size) in the set might be displayed as a number smaller than one of the powers of two many recall.
On a 32 bit machine this wouldn't be approx 2 billion but rather often 500M 32 bit words or 250M-ish 'double' precision numbers. Let alone if the search is over more complex structures representing objects. For a smaller 16-bit embedded system I could even more readily see this occurring.
Reading the other comments I'm not sure d and c is the answer. Might be terminology, but I would split the array in two at the middle, see if the value I'm om is the one I'm searching for. If not, recursively do the same thing on the left or right side of the array.
I guess this is the old “there are two clusters of tech jobs” problems, where I can barely fathom the techniques of the other cluster.
I think the question first asks the interviewee to have cracked an algorithms textbook or be clever enough to come up with the method on the spot.
Second, it asks if you can explain the concept to another engineer. I've met engineers in their 40s who can implement something like depth first search but cannot explain it. That's an immediate red flag to me that, at the very least, the person I'm talking to is not able to meet the responsibilities of a senior engineer.
This is on the order of fizzbuzz +1 level maybe. It's elementary stuff.
Or was he looking for best-case optimization or something more than just binary search.
The discovery of many things is done by people with a genius-level intellect. Like calculus, probability, etc. but I can certainly ask mechanical engineers calculus problems, or data scientist probability problems.
It is certainly difficult for someone who never seen anything related to it. But for people who had the background, it is not unreasonable. I think this kind of problem is biased toward people who
1. had a good (theoretical) CS education, or 2. solved lots of leetcode problems so they can just do "pattern matching".
Can anyone who has made career changes to or from tech give their experience?
There is an essay about the feminist movement in the 70s (https://www.jofreeman.com/joreen/tyranny.htm). It makes the reasonable point that people are likely to hire/recruit people similar to themselves. The code interviews are expected to remove this, if you stick a bunch of CS people in a room and tell them to make an unbiased interview process; you'll get the hell world we live in today. To get a job on a sales floor, most of it is just impressing the existing gate-keepers.
Coders live in a fantasy world where gate keeping isn't real. Except it isn't possible to remove that from hiring, no one wants to hire someone who doesn't integrate well in their team. So we have this hellworld where people can study for 3 months on leet code and nail the tech interview, they still won't get the job. Even though the "unbias" process says they should.
- Nobody likes whiteboarding
- Take-home assignments are unpaid work, and candidates who are raising children don't have time to do homework after they get home from work
- Pair programming during interviews makes people uncomfortable because of the pace or because it's not how they are used to working
- People who currently have good jobs are unwilling to consider contract-to-hire or "tryout" periods
- Credentials, including graduate degrees, are no guarantee that someone is a good engineer
The current interviewing situation in the industry sucks but all of the alternatives are worse in important ways.
When Im looking for a job, I make it pretty clear what Im ok and not ok with, which will dictate which companies Ill interview at or not. If we were in a dotcom crash it would be a different story, but right now its definitely something people can do.
Too true, I've started to recruit non-CS people and teaching them how to code. The state of CS education in the US is insanely inconsistent. But I'm lucky, I have lots of time to train people; most companies can't handle months of startup time.
I think the right answer here, to the extent that there is a right answer, is to give applicants a choice.
Consistent processes are also important for avoiding bias. For example, take-home projects are more likely to make sense for young candidates who aren’t raising families and have free time after work. If you have most of your young candidates doing take-home projects and the older candidates doing whiteboarding, you will end up evaluating candidates differently based on age.
If they have github, I'll ask them to walk me through a few files, asking their reasons for doing X, why not Y - what does this do?
If they prefer take-home, I'll give them something to create/improve.
Want to do something now? knock yourself out - with or without me watching.
I don't need to sort the candidate into an ordered list, just filter out the "over confident".
The questions I want answering is: Can they actually code? Does their skill match their experience? Is their skill level similar to the codebase?
That's where the person who knows how to set up a VirtualBox VM running Windows and VS would have a large advantage; when they brought in the result, showing their MBP running VS in a virtualized manner, and explained what they had to do - that's a bit more of an impression than someone who says "they can't do it because MBP".
Also, I'm nowhere near ops jobs, so I don't think those people interviewing me would care much about those kind of logistics (anyways, they wouldn't want to know the details). It wouldn't score any "points" beyond just getting the task done at all.
Are you saying in general - or just for Windows-based software (since you mentioned needing to set up Visual Studio)?
The latter I can understand; but the former makes me want to ask, "Who applies for an SWE position who doesn't write code at home?"
I've been an SWE professionally since 1991 when I was 18; I got my first computer in 1984, and have been "coding at home" ever since. I'm not saying "to be an SWE you need a similar experience" (I know that's untrue) - I'm just relating that I've been coding for a long time, and "at home" is very much a part of it.
Then again, I've heard of people who absolutely hated coding, but were extremely good at it (and I assume made good money doing it), but when they went home, they didn't even want to think about such a thing. Maybe you are similar in that regard?
I guess I am just curious and a bit fascinated by your comment...
Professional baseball players rest after games. They don't finish up and then immediately start practicing again. And soldiers and police get down-time.
Just because I'm doing something I enjoy and that I want to do, doesn't mean I should do it all of my waking hours. I'm not doing this to code, I'm doing this because my employer is paying me to build cool shit and I like doing that. I don't lift weights 24/7, I don't run 24/7, I don't hike 24/7, and I don't program 24/7.
Frankly your attitude is one of the reasons why we don't have any diversity in this profession at most self-described tech companies, and it also underlies a lot of the seeming ageism today. All of these self-described non-conformists expect everyone in the profession to conform to their attitude and it's sickening.
That's why companies should allow laptops.
> Take-home assignments are unpaid work
True, they could be optional. You may not get the job if you don't do it. Honestly, if you can't spend one or two hours on a take home assignment during a 7 day period, you certainly have different problems than applying at this company.
> Pair programming during interviews makes people uncomfortable
I agree, pair programming during an interview is mostly a no go. Not because of the issues you mentioned but because most interviewers are totally unqualified to do this properly (unfortunately, most interviewers are also unqualified in general to conduct interviews). It takes a lot of skill to conduct pair programming interviews and draw the correct conclusions.
> People who currently have good jobs are unwilling to consider contract-to-hire
In US, there is practically no difference between those three. You can be fired any day anyway. But yes, if you have a secure job that is good, the bar for switching is high, but so what? Why even switch? It's your choice then... Luxury problem!
> Credentials, including graduate degrees, are no guarantee that someone is a good engineer
I would go as far as saying your credentials and degrees are horrendously irrelevant in predicting your skills as an engineer. The only thing you might be able to conclude is that a GPA 4 M.I.T. graduate probably can pick up missing knowledge quickly, as opposed to a no degree dish washer who learned "how to code" in a 6 week bootcamp.
On that note I have experienced first hand, Berkeley and Georgia Tech CS majors, who were absolutely incompetent on the job and just had no idea what the hell they were doing, with little improvement in sight...
That's an extremely privileged thing to say. And what a terrible way to start out your working relationship. The very first thing you ask of me is to do unpaid work in my free time? IF you're willing to do that, how else will abuse me in the future?
Also, why should they spend time on your homework before they know if you're a company they would even want to work for?
I think the reason why take-home assignments tend to feel more unreasonable is the asymmetry. If I attend an interview, it's costing a pile of my time, but it's also costing a similar pile of time for multiple people at the company where I'm interviewing, which gives me some reason to think that my application's going to be considered seriously and therefore that the effort I'm putting in isn't going to be totally wasted.
Whereas a company can hand out a take-home assignment with no effort at all, so the fact that I've been given one doesn't indicate that the prospective employer sees me as a serious prospect or that there aren't hundreds of other candidates in the same position as me, so it's no guarantee that my chances of getting hired aren't miserably low, so there's more risk that I'm wasting my time.
Although perhaps an escrow system would work. The company pays me a large sum (say $10,000), which I then pay back if they call me in for an interview. That way I know they are serious, and it actually forces them to consider the cost to themselves.
I'd suggest that an MSEE or MSCS degree from a good school tends to be pretty good training for working in the industry. The projects you complete are usually challenging, relevant, and intentionally educational.
For example, an MSEE might learn how design analog amplifiers and filters, program a DSP, lay out VLSI circuits for an ASIC, implement a RISC CPU in verilog, implement an internet router on an FPGA, and implement Wi-Fi on a software radio. I'd be happy to hire (or work with) someone who can do all that!
I'm sure some programs teach this stuff already. But I've hired people with graduate degrees in CS from well-known schools who didn't even use source control for their group projects in grad school -- they just emailed the source files to each other.
0. You seem to have misunderstood my electrical engineering example.
1. Why would an MSEE not qualify you to be hired as a junior electrical engineer?
2. Why would a PhD in the appropriate area not qualify you to be hired as a junior research scientist?
> In US, there is practically no difference between those three. You can be fired any day anyway. But yes, if you have a secure job that is good, the bar for switching is high, but so what? Why even switch? It's your choice then... Luxury problem!
It seemed weird to me that we are one of the only industries (well I guess engineering in general) who gets a contract that is contingent on each day until it is completed. At any given point you could be let go and yet if I contract someone to work on my home to legal work you setup a contract that states you owe them the money unless you are willing to go to court to contest their work.
When I last did a contract-to-hire it was basically "you can be let go for any reason, from slow output to your boss having a headache." Unless you get a sweet deal its difficult to pull a job with a set start / end time with guaranteed payout regardless of employment. I busted my hump at that last job so instead of doing my 6 months for the hire option they offered me the job before then 2nd month was over.
Resume review -> take home exercise -> phone interview -> onsite interview -> offer.
If in your case, you demand allocating more time for that step, consider adding additional filter (requiring 10-15 mins) after the resume review. And then redirect [part of] resources (money) from the onsite part to pay for that take home task.
Another red flag I see in your process - asking for allocating time before even talking to a human being - that would be another reason for an immediate rejection. Consider moving your phone part before the take home task.
When there is something before the phone interview, it is a coding assessment; these are generally different from a take home assignment in that they are generally timed and automatically scored, e.g. you hit start and have two hours and then a computer scores you. With take home exercises, I've always had engineers at the company read my code and discuss it with me.
Isn't that the point of a resume? What am I missing?
But to be fair the problems were not too hard, the interview was set-up properly (problem files on place and a good code editor at hand, no tricks or gotchas)
And the worse of course is having to write fizzbuzz in a whiteboard, I just have to roll my eyes then
Honestly, I really like whiteboarding. I liked doing programming contests too, which are like whiteboarding except harder and you spend more time at them. There are times when I have gotten together with other like-minded software engineers and given each other whiteboard programming problems for fun. (Also to test whether problems are good interview problems.)
A lot of people like whiteboarding problems and like interviewing with whiteboarding problems. Those are usually also the people who pass whiteboarding interviews. Just because there are a lot of complaints about them, don't assume that represents everyone.
It’s probably the most strongly correlated with liking and being able to do the job signal I’ve ever seen.
First, you’re not going to get your resume in front of me without a college degree, CS degree or equivalent experience is a bar that recruiters apply well before I’m involved.
But as far as I can tell, Google CodeJam is an individual sport (I may be mistaken about that, but Google’s page won’t load in my browser, so I could only read the Wikipedia page about it).
The key thing about the ACM contest is it requires BOTH effective cooperation with a teammate AND the desire to solve programming puzzles.
Now, just to be clear, I’ve fired people who passed this bar; being a smart puzzle solver who’s social enough to collaborate is a great start, but it turns out lots of people just aren’t interested in shipping software. I still think ACM team participation is a strong signal, it just doesn’t cover the fact that being an adult means 90% of every job is boring, and not everyone is ready to do the boring stuff when they’re fresh out of college.
[0] -- https://news.ycombinator.com/item?id=9324209
Some people even suggest that advanced degrees are an anti-indicator of ability:
http://blog.alinelerner.com/how-different-is-a-b-s-in-comput...
And mostly they aren't wrong.
Doing 10 hours of work to apply for every random "we make another business PHP/crud website" company is not worth it.
Your company isn't Google, don't interview like it is. I am not going to write custom code for you, but I am willing to flash my credentials.
Just for some friendly and non-judgemental feedback from a different point of view, to me that sounds like more of your own fear coming out than reality.
Everyone who interviewed you also said the same things out of fear before they were hired, and then they got in and didn't manage to change it. That could mean it's not viewed as a problem from the inside...
Companies are hiring people onto teams, not to hero-solve pure code problems in a vacuum, but to work together with other people. Communication and personality and attitude are all super important, usually more important than technical skill. (That doesn't mean technical skill is unimportant, it means precisely that people skills are more important, for the average programmer candidate.) It's a feature, not a bug, that team consensus is needed. And there's rarely 50/50 chances, using that to speculate wildly on your chances will prove incredibly inaccurate... the most common case behind the scenes is that more or less everyone agrees that either the candidate is right, or that they're not right, or that everyone is meh on the candidate so nobody is pulling for them.
Chances are determined by how many people are applying for the job versus how many jobs are available, with a side of how long the company wants to take to fill those positions. The company has the upper-hand knowledge of what the candidate pool looks like, and they can hire the top 10% of those candidates who apply. As an applicant, you have no idea and no control over that. All you can do is improve yourself, research as much as you can, try many options, and try again when it doesn't work out.
I've interviewed at over 100 places in the bay area. I'd say that from internally and externally, most of the companies here have no idea what they're doing. Most don't use rubrics, most don't have a standardized set of questions that have been heavily reviewed and tested, there is either no training or inadequate training, and most are winging it completely. Many ask whiteboard questions to interview candidates and they never had to do them for the job they got (got in through referral or at a different stage when they weren't doing such questions). Resume, referrals, and charisma will get you into positions very easily in many of these companies even if you're very technically challenged compared to others with less resume+referral+charisma.
Being liked is one of the best ways to get into positions you are terrible at but compensated so well for.
Does it matter if someone who didn’t get asked whiteboard questions is asking you whiteboard questions? That question is implying some vague concern about fairness, and isn’t focused on whether whiteboard questions are a good idea. On the positive side, at least they’re asking coding questions and not hiring 100% based on resume and charisma, which definitely happens in other industries.
Younger programmers especially seem to think more often that highly reliable technical evaluation in interviews is both achievable and desirable above soft skills. In my experience that view may be too narrow.
I’ve interviewed and hired many people over the years for several companies, large and small, and no matter how much advice I offer about what is actually important when interviewing, it seems like candidates rarely listen, and instead complain that “they don’t know what they’re doing”, as if they know better.
I honestly think interviews are legitimately scary, and that fear comes out in various ways with programmers suggesting their ideas for “improvements”, but by and large the system is working, good companies are finding good people. I know there are a lot of interviews that are stupid, and I know there are bad companies out there. The suggestions in the article won’t fix that.
Being liked is a good way to get hired period. Being liked and being good technically is better. Being technical and not likeable is a good way to not get the job. It has very little to do with the occasional cases of incompetent people being hired and paid a lot.
We don't have that in our industry. I interviewed countless "senior engineers" with degrees from well regarded universities who couldn't write a simple loop. Therefore I must now conclude that a degree from that university means nothing.
There are some folks like Tripplebyte who are trying to solve this by pre-qualifying candidates, but I don't think the problem will really be solved until we form a software engineering licensing board. Take a test once, and then take a refresher course every couple of years to maintain your license.
I think this is fundamental to the discussion. Are we really sure they couldn't write a simple loop, or were there other factors that prevented them from doing so? For comparison with your other analogy, keep in mind that even surgeons who have amputated the wrong limb are defended by their colleagues, saying “Well, it should have been amputated anyway.” (https://www.nytimes.com/1995/09/17/us/doctor-who-cut-off-wro...)
I understand that you specifically might not have used that as your only filter, but a lot of people do – and I think a more holistic approach that takes into account education, training so far, and overall experience with tech. Certifications might be one part of this too.
I mean I suppose the stress under pressure could have made them forget the loop syntax in their favorite language, but that is unlikely. I literally asked them to implement FizzBuzz in their favorite language during a phone screen where they even had internet access to look up syntax. But since I was interviewing them for an SRE job, coding under pressure is actually part of the job anyway.
Regardless, I'd expect someone who claims to have 3+ years of Python experience to be able to write a python loop.
Pretty much every candidate has "failed" the test. More than half have something incorrect or not quite matching the instructions. But we have hired some of those people and they are great programmers.
It actually does give a surprising insight into the developer's style. It's not pleasant to see all of their code stuffed into main(). It's not pleasant to see every single line of code commented. But it is refreshing to see Enumerable.Range(1, 100) instead of C style for loop.
However I don't think I could do this at all over the phone.
I'd also, for a throw away like this, knowing it's a throw away stuff everything in to main / global and use that c-style loop BECAUSE it's portable to literally every language.
If you're looking for a FizzBuzz class/object/module specify that. If you've got a specific language with nice features ask for that, BUT don't demerit an individual too much for not knowing the day to day features of the coding environment in your company when they've not been part of it for months.
what's the difference? I believe that getting hired by a company is the easiest thing you will have to do for that company. I believe that if you can't perform a basic task at interview time, you should not be working there.
Either because you can't perform.
Or
The mood is not right for you. If this is the case, why would you want to work there anyway?
I see so many people that thing getting hired is the end goal. Mate, you still have to work for them. Those people that can bullshit their way through a job, have no issues getting hired to begin with.
That doesn't describe any of the five job offers I've taken since college. Well, maybe the case where I was rehired.
A "senior software engineer" at a cardboard box factory might not be the same role as one with the same name at an aerospace firm doing government contracts.
I guess the advantage is once you pass you don't need to re-certify. Which also seems incorrect.
Funny when you think of how many blog posts, flame wars, SO questions exist on the internet just to debate what is indeed a unit test, and what is an integration test :)
Who would have the legitimacy to design such an exam?
If it's like civil engineering, it would be the regulatory body that determines the approved technologies and design formulas for your field. The exam would be based on those standards. I'm not sure most governments would even approve public key cryptography.
Better watch what you ask for. ;-)
It's risky. Licensure often becomes a wage-protection cartel, and if so, does little to guarantee the quality of licensees.
Another problem is that there are many levels of skill demanded by the industry, though maybe you can have a bevy of licenses that qualify a person for various roles.
That's my principle issue with the OP: they don't attempt to control for the possibility that people who hate technical interviews are most likely the people who didn't get the job they wanted.
If you're working at company X and think, "I hate this shitty IT job, I'm smart enough to get a real developer position" and interview at company Y and learn that, no, you aren't, that's a hard pill to swallow.
Yes, we all ought to find our zen and accept that no one has all the talent they wish they had. But people are human, so we're liable to blame the messenger, which is the technical interview.
It’s a hard pill to swallow because you don’t get feedback from rejections. It doesn’t matter what stage of the interview process you reached, companies very very very rarely give feedback when asked. Because of this, those people never get to hear why they aren’t good enough. Instead, it’s all left up to their imagination which creates the scenario you describe.
>Therefore I must now conclude that a degree from that university means nothing.
Degrees always mean nothing but the educations might still be useful, depending on what the individual took advantage of.
For the first job as a licensed physician.
After the internship period, it's generally one or two in-person interviews (possibly more if you're joining a private practice), followed by a period of weeks to months where they check on your licensure and credentials.
Seems like baloney
Even to be brought on as a "Doc in a box" type of situation, you're still looking at a minimum of 90 days.
Folks want to "move fast and break things," and "hire fast, fire faster," but those two are not conducive for doctors.
Imagine the resources needed to maintain tests that should work for low-level embedded developers, data engineers, video game coders, smart contract programmers, AI implementors, etc..? Yes, there are fundamentals and Computer Science, but most of us agree that you can have the greatest craftspeople perform poorly there.
Though, I guess the certifications from private companies like Microsoft in Cisco actually work as some kind of license
I don't think I can get hired without a referral - haven't managed to land a cold interview once so far. So at least, there's no evidence of it.
Interviews are a performance act, they are the packaging that you throw away after buying a product for the cover.
The interview "suit" I wear is actually one of someone who's calm and smiling - instead of stimming with my hair and dead-pan with a smirk (if I was a woman, I'd have resting b-tch face).
I feel for this guy, because I figure out how to wear that suit, but I still need to thumb the scales to get hired, because the mask I'm wearing is effort & it does show to the perceptive HR person (& it doesn't matter what you are faking - if you're faking it, it counts against you).
However, here's where everything is unfair in this world.
I have a pretty decent job because I ran into the right people at the right conference back in 2004, where someone noticed that I might be able to do hard things with minimal effort (and fail at easy things, simultaneously).
There must be thousands of people in my state who never got that "come to Yahoo, have a coffee" invitation & get plugged into a network of people who have kept me gainfully employed for more than a decade.
I definitely had fewer challenges there because I wasn't an outright minority in the industry - I didn't have anything to prove beyond the performance.
I was there talking about my FOSS work, that probably helped.
toolz asked me if the "Portable .NET" platform was portable enough to run on some new hardware & gcc patches he had got before the conference (not insane, but cross-compile & port the project from a Zaurus/iPAC to run on a PicoPeta Simputer).
Anyway, it must've looked more impressive than 2 days of work, because when toolz died, someone else remembered that incident[1].
(as a side note, I had already interviewed at IBM, Oracle Platforms team and several other places with no luck at that point ... so I had already spent 4 months prepping & interviewing in 2004.)
And after the conference, I met Swaroop [2] & Philip, who were at Yahoo then.
[1] - https://news.ycombinator.com/item?id=5812177 [2] - https://python.swaroopch.com/
If you won't do it there are many others who will.
If I'm not going to do a 6 hour take-home assignment for you, I'm definitely not going to do 300 Leetcodes to "prepare". I've been doing this for 34 years. That's my preparation. Either you want what I've got, or you don't.
Fashion house 1: 2 guys sat me down and ask me to answer a question about some node.js code, which took around 25 minutes to get running on my laptop. Once it was they were hoping for progress, when it took me a few minutes to read through several config files ( over a hundred lines each).
After 30 minutes they gave up and asked me what library I should use for a certain task. I said I had built alot of code in that domain didn't recall which library I used for that specific task.
Suddenly I was walked out, but not before one of them took pity and tried to explain to me how, when you make a request you can write code which waits for a response from a server (async programming).
I ofcourse thanked him and left.
This was after I spent 3 years developing a successful networked engine in my spare time ( with a custom 3d engine bolted on it ) which enabled 20 asynchronous connections for a war simulation I built. They both had 30 minutes and hadnt thought of reading my CV or asking me about any of code / experience. Just drank their coffees looked at their mobiles and waited for me to somehow decipher their 10k line codebase. (Management had already OK'ed me but were not around for this part of the interview and blind to what was happening ( or the lack of it.)
To this day I dont think the engineers even looked at my CV. Just thought I was some confused kid who didn't understand how basic network code worked.
Fashion house 2 I also interviewed at their competitor, who had looked at my CV, lead engineer was about to extend an offer, CEO walked in, not knowing who I was, asked a casual question ( culture fit ) type, but I was unable to answer, I had trouble understanding what he was saying. Failed interview, but that is ok atleast I was not as offended this time as there were no quips.
The icing on the cake, was that both companies had a 20+ engineering team, that relied on an app, which just framed a webpage.
And I cant help feeling I know the reason why.
I was only asked how DNS works in the interview and I answered that. I told the recruiter this wasn't mentioned in the job description nor was I asked those questions in the interview, but he didn't have anything to say.
1. It hires me
2. It can be used to unhire others thereafter
Then I had a phone interview where they started off by commenting that I was the only one to include tests and address undefined behaviour. Then I had to write a piece of code to traverse a tree and add numbers, I can't remember what it was exactly, but I got nervous and messed up somehow.
Then I waited for 2 weeks just to hear back that they wanted someone more experienced with graph theory.
We're all still human; expect failure to happen. See what happens next.
When I got there I was told I'd be interviewing for the SQL Server team, interesting work, but not something I'm interested in. I had never taken operating systems in university because it was always available at a time that another class I HAD to take occurred.
So by some random chance, I'd never heard of a semaphore. Perhaps inexcusable at this point in my career, but I was ignorant of the topic at that time.
My first interview question was to write a way to let many readers on a resource, and only 1 writer, with no readers reading at the time of the writer writing. I decided to use lists of something or another, ProcessId, or something similar.
Every time I would go to write the code, i'd get a few characters in, and the interviewer would interrupt me and ask me questions to help me along. Since I had no idea what a semaphore was, much less that I was supposed to use it, we spent like 30 minutes of this back and forth, me trying to write, him interrupting me. By the end of the session I had a single line initializing a list written.
Some of my other interviews during that battery of interviews went better than that one for sure but, needless to say, I didn't get the job.
I was aware candidates would do things like this when I applied for job long ago. I also was aware that employers were suspicious of these sorts of things. So I brought along listings of code I'd written, pulled them out, and walked the interviewer through how they worked to show I knew my stuff. Note that they didn't ask for this, I just did it proactively. I got an offer, too, and was told some time later by the interviewer that that was why.
Why compare the results to work done in a completely different medium with no opportunity to iterate on the output, take a break or work comfortably?
If you assign a take-home test, make the interview focus on discussing the product.
You can have one-way trust in a take-home. If they don't meet your quality bar, you can discard the resume. And it takes near zero work on the company's part, that's why so many companies like it.
However, some people will just get a lot of help on the take-home. So it isn't fair to the other applicants if you treat a take-home as anything other than a filter.
I think we over focus on these scary edge cases where some trickster is trying to con their way into the job. The worst case is you just fire them when they are unable to do the work, and that's on them.
Besides, what environment discourages help? I ask friends for help with my job all the time.
The cost of over optimizing for avoiding those false positives (ie: bad candidate marked good) is a huge drop in true positives (ie: good candidate marked good). It's silly.
The whole reason for the long hiring process is that firing is hard (legally, financially, emotionally). If you're willing to dismiss this difficulty, you're assuming away the core problem.
How many people can truly, reliably, "fake" their way through a take home? How many people could then do an in person code review and fake their way through that as well?
I doubt it's that many, but you'll get way better returns on good candidates, in my opinion.
I've encountered programmers who know their stuff, can talk about it intelligently and knowledgeably, but cannot write code worth spit. It's not as rare as you might think. It's sort of like someone with an art degree who has no talent for actually painting.
I'm not sure I see how someone is going to: a) Write a take home project b) Explain their design decisions, respond to code review, etc, in person
and somehow not be able to "write code worth spit" or otherwise game the system consistently.
edit: Would love to hear (in the US) where that 100k comes from. At-will state means I can fire you for whatever reason, can't imagine it really comes close to 100k tbh.
If you're paying a new-hire-fast-fire more than the salary due to them for the period worked (and any other amounts listed in their offer letter, if one existed), then you need to talk to a labor lawyer because you're definitely doing something wrong...unless you're paying them to shut up about the experience.
1. relocation expenses
2. starting bonus
3. headhunter fees
4. severance pay
5. COBRA
6. it can take a while to determine they're incompetent if they're good at the facade
7. if they're in a protected class, you have to go through a period (often 6 months or more) of documenting their incompetence and leaving a trail of trying to help them improve
8. firing someone always comes with it the risk of a lawsuit, and meritless lawsuits are commonplace, but they still cost a LOT of money to defend.
9. disruption to your team, scheduling slippage, having to allocate someone else to fix the incompetent work, etc.
And I said earlier I maintain that my method of interviewing will actually reduce those costs.
If you sell your house, you pay a 6% commission on closing. If you reject the offer, there is no closing, and you don't owe the commission. Because the house didn't sell. If you sell the house, you can't sell it again unless you're committing fraud...But assuming you meant you find another buyer, and successfully close with the second buyer, you would then owe a single 6% commission to your real estate broker, not two 6% commissions.
This is also not how recruitment bonuses work, and I say that as someone that works closely with HR on the tax/legal side of things.
If you pay a recruiter a recruitment bonus, their recruited employee has to remain with the company for X amount of time (aka "trial period"). Usually at least 3 months, but it varies. Once that happens, the bonus "vests" and is due to the recruiter. (Many recruiters will try to get paid before the trial period ends, but you don't actually have to pay them yet.) If you terminate the employee before the trial period ends, the recruiter is not owed a recruitment bonus. These are all standard provisions for recruitment contracts involving 3rd party recruiters. (Note: this is for the contract with the recruiter, not the recruitment employee. The employee's employment contract, if any, may not include any language about a trial period.)
Of course, it's very possible for a startup without an HR department to leave out the trial period language from their recruiter contracts. Because they didn't know any better because they didn't think that HR personnel provided useful functions. Now you know why HR departments exist.
5. COBRA is paid by the employee, not the employer.
6. Somehow every other industry except programming manages to do this just fine.
7. This is completely and utterly false. If a new hire doesn't work out you can just let them go. Somehow, companies do this all the time without issues, including companies with much cushier jobs than startups.
8. Everything comes with the risk of a lawsuit. You're prematurely optimizing for the 0.0001% case, for an imagined scenario that isn't that expensive to defend.
9. Most of these apply to leaving the position open for months without filling. So let me add one to your list that applies if you don't fill the job: employee burnout leaving to existing employees leaving before the open position is filled, leaving you with more positions to fill than you started with.
Retention bonuses and starting bonuses are structured the same, you sign an agreement to get that bonus that says you pay it back if either you or the company decides to end employment within a certain timeframe.
Unless this is a contracted for benefit, this isn't a separate cost from lawsuit risk, but a mitigation of lawsuit risk: severance pay is paid to people on condition of waiving claims.
> if they're in a protected class
If you start a sentence this way, it's a clear sign you don't know what you are talking about. Except for a few special cases (“age over 40” and “veteran status”), every employment protected class is a dimension on which discrimination against any value is equally restricted, and everyone has some value on each axis (race, sex, etc.)
So, everyone is “in a protected class”. Well, several of them, actually.
> you have to go through a period (often 6 months or more) of documenting their incompetence and leaving a trail of trying to help them improve
No, you don’t, and you can’t really count this “requirement” and lawsuit risk as separate costs, since this is simply a policy companies adopt to mitigate lawsuit risk.
Mitigating lawsuit risks is costly.
It certainly has a cost, but it's part of the cost of lawsuit risk, not a separate, additional cost.
I understand, it isn't intuitive. I hope my analogy of someone well trained in art, yet still can't paint worth a damn, makes sense.
Or pilots who ace all the pre-flight training, yet simply are too uncoordinated to fly well. (That's me. I know all about flying, yet should never be in the pilot's seat.)
Some people are just unable to synthesis good code out of all their knowledge.
Good question. Like I said elsewhere in this thread, there are many people who are knowledgeable and intelligent about coding, but cannot code. They are different skills. I've met many people like this.
I could study Romeo+Juliet and know everything about it that lit professors know. But I could never write it.
For another example, from my time at Boeing, the engineers were divided into two independent groups. One group designed, the other group analyzed and checked the designs. They were different skills, and the engineers would gravitate to whichever path best suited them.
If someone is in a protected class and they get fired within the first week, does anyone really think they'll win a lawsuit alleging discrimination? Does anyone actually think they'd get beyond a motion to dismiss (usually the first motion filed by a defendant)? If the employer was discriminating against protected classes they wouldn't have hired the employee in the first place. These types of claims are quickly dismissed and usually incur minimal legal fees.
FAANG companies have this problem: when you hire thousands of people a year, a higher false-positive rate is a big deal.
When you are startupbro.com, you don’t have this problem, but you’re desperately cargo-culting your way to success, and doing anything in a way that goes against the grain causes the other bros to accuse you of “bad signaling”.
See also: why tech companies cram together in San Francisco.
My friend's employer is currently hiring thousands of people a year with a single interview lasting 1-2 hours without problems. Some highly technical and some manual laborers.
Then do this by putting new employees on a probationary period for the first six months. Many of their employees are union, and it still works out, the unions agree to this arrangement.
All that BS about it being hard financially or legally? Not true. At all. It's just an excuse startups use because they don't understand basic HR functions. And that's why every company of a reasonable size (i.e., more than a few dozen people) has an HR department, even if it's just one person. Because they know how to do this stuff.
After all, managing hiring and firing are literally the primary reasons HR departments exist in every company making more than a few million $/year. Especially the profitable ones.
Whether or not you’re talking about Silicon Valley, you’ve endorsed a much stronger claim, that such measures are pointless because firing is just so easy.
If you’re not ready to defend that claim, then maybe you should walk back on your original. Or at least show some some sign that you recognize things aren’t as easy or simple as your original blithe dismissal implies.
Remember, you didn’t even think the emotional side merited an acknowledgement.
And severance pay is not a thing for most employees, just the ones, like programmers, that have fairly cushy white collar jobs.
You need to stop seeing everything through a rose-tinted SV lens and look at it the way the rest of the country does. I'm ready to defend my claims. Are you?
What everyone else does is completely irrelevant.
In the programming world, specifically, which is what we are talking about, firing is extremely costly.
This is proved by what companies in the programming world are doing right now.
If you disagree, then you are disagreeing with basically every major tech company in the world. And personal, my bet on who is right would be those tech companies, not you.
And when they do, it's standard practice to require that relocation and signing bonuses be repaid if the employee resigns or is terminated within a certain time frame after joining the company.
I work pretty regularly with HR. Do you?
Not only have I worked at a bunch of different companies, but I have also straight up failed at some of the earlier jobs, and have been in situations where I absolutely deserved to be fired, yet it took literally months for anyone to even approach the topic of my performance.
I am talking, situations of me not doing any work, for months, because I didn't care and was looking for new jobs during that time period.
My experience has shown to me that engineers are pretty damn untouchable. And yes, these have been (mid sized) startups.
The one time I had a reloc bonus, of a job that I left after 5 months, they did not ask for it back, and simply never talked to me again.
It is not about signing bonuses though. It is mostly about things like bad Glassdoor reviews. Many engineers would never ever work at a company where the engineers were constantly terrified of, and complaing about being fired quickly.
Also, disgruntled ex employees are something that you don't ever want to deal with.
Not really. It's just harder to hire again usually than figure out how to coerce the problem you have already to be less of a problem. That's the nature of the market right now.
If developers were easy to find and hire then they would be getting fired left and right like happens with just about any other commodity job.
How is it that, simultaneously, firing is hard and job security sucks?
The problem is once you offer them a job you stop searching for a candidate. Then you get a bad hire and even if it's quick you won't really know for a while, so then you're really behind.
It's bad enough when you hire someone who really wasn't qualified but didn't do anything bad. Now you have to get rid of someone who probably left a previous position and, like above, you have to go get someone to do the job you were hiring for in the first place. Worst of all possible worlds.
Perhaps the calculus is different in a big company, but for a startup your approach sounds pretty bad.
I believe that the existing system intends to optimize heavily for "reject bad candidates" and actually fails at that too.
Give candidates a take home, then review it with them.
I have a lot of reasons for believing this approach would be far superior to the "sit in a room and get interrogated for 7 hours" approach.
This sounds like a contradiction to me. How are you defining "unqualified" and "bad" here?
I don't care how much help they got if they can discuss their solution coherently. This also shows the candidate that we took their solution seriously.
It's wonderful you don't care about the fact that these solutions are purchased. I'm sure your attitude is a great comfort for all the honest professionals who worked hard to find the time to do a 5 hour take-home, only to see their job taken away by dishonest candidates who paid for the solution.
Taken to its logical conclusions, if your attitude is representative, then everyone should just buy take-home solutions and just spend a bit of time understanding them before the onsite. Now we're back to square one, except at least we're not actively screening for dishonesty anymore.
If copying/buying solution is actually a problem, that means take home project is a thing that's happening, and copying is the new problem to solve along the way.
We're far from there.
In case you didn't know this: take-home is very much happening. I was job-searching not too long ago, and roughly 10% of the places I talked to insisted on take-home assignments. I declined all these requests because I'm a senior engineer with many options, but clearly many candidates are taking them.
There really is a problem with people buying solutions for take-home assignments, I saw that when I was hiring. It just isn't acknowledged, so effectively it has become a competitive advantage for dishonest candidates who are willing to cheat.
Very occasionally the candidate came in and just was wildly off, but usually it was fine.
Later on we changed to having all candidates submit the takehome under the logic that if they’re a, “positive” candidate that they’ll be inclined to see this as an opportunity to strut their stuff. I was a little unsure about this approach but we went with it.
I’m curious how senior folks feel about these different attitudes.
It's quite simple: I'm a senior engineer and I will never do a takehome assignment.
Every time I'm on the job market, there's a certain percentage of positions that require a take home. I just skip them.
I don't explain my position, since I learned long ago that it just leads to arguments and confrontations with the very same recruiter who just a hours ago cold-called you begging you to interview.
I just terminate the process as quietly and unobtrusively as I can.
I wish I could comfort you by saying I ever regret this decision, but the truth is that the best jobs out there - the Google, Netflix, Airbnb jobs - do not require a take-home assignments. I guess they know better than to restrict their candidate pool in this competitive market.
It tends to be the below-average and (often) crappiest jobs that require take-homes, to the point it became a red flag to me. Now I often wonder "what kind of a desperate crappy senior engineer had to jump through all your hoops in this market, especially when you're just one anonymous company out of tens of thousands to be hiring engineers, with nothing special to recommend you this early in the process?".
In short, to insist on take-home tasks in this market is to give up on your strongest candidates. Neither me nor many senior engineers I know will at all consider spending 5+ hours on takehomes when we can easily get dozens (!) of onsites with short, fun remote screens. I will also be very skeptical of any engineer who in this market is willing to do these hours of takehomes after a long, hard day of coding at work.
One last point: it seems quite unfair rude to me for a potential employer to ask me to unilaterally dedicate 5+ hours of my rare valuable time, when we barely begun the process. It's especially out of place and comical when the same recruiter who literally begged you to interview in the first cold-call, is suddenly demanding you do 5+ hours of unpaid work. At least in remote screens, I know the employer is investing some of their own resources into this process as well.
My logic on the take home was that interviews are hard without something to talk about and contrived problems are not super reflective of real work. Therefore, I’d send a dumb version of a simple task we have, then we’d walk through trade offs and such.
If I have some code you wrote and can walk me through that’s a very useful datapoint not just for competence (if you’ve been around a while you probably are good) but for personality and how we’d work together.
Does that seem more reasonable as a trade off for your time?
At some point I think senior engineers realize that coding is the absolute least of your worries as someone expected to do things like interface with the business side, plan medium/long-term roadmaps, improve dev tools or general architecture to make juniors more productive, or mentor/train/manage juniors.
When the entire process seems to be heavily weighted towards coding, it gives off the vibe that this is a company that recognizes overworking and churning out code as traits of a senior engineer, instead of shrewd decision-making, efficient delegation, smart scoping of projects, strong leadership skills, etc etc etc.
I don't want to spend my free time building a portfolio of random bullshit code any more than I want to spend time on your take home test. Because I can't for the life of me see why having an impressive (or average) github profile should be the leading indicator for senior engineering skills.
What is wrong with simple live-coding sessions on very small pieces of code while discussing objectives, edge cases, extensions, etc? Coupled with a high-level systems design session with whiteboard diagrams but no code, and a behavioral interview assessing leadership skills, extracurriculars, personality type, etc. Sane companies seem perfectly fine with sticking to this formula.
Where is this idea that we need more and more, deeper and deeper assessment of candidates coming from? What problem(s) is this solving to look at large amounts of pre-written code from candidates?
I think our focus on code was based on the fact that everyone needed to write code. We were a small company so there was less of the architecture work. We may well have benefited from doing more of it, but we still needed people to ship lots of code.
You do ask a very important question for which I don't really have an answer: why do we care so much about these take-homes or coding interviews for people who have been around for a while?
For me I'd say because I want to have a point of comparison with all other candidates, but that's a pretty weak answer. I'll think more about it.
This assertion makes absolutely no sense at all. Not being able to pass a simple fizzbuzz test doesn't mean the candidate is among the strongest there is. It just means that the candidate was either incapable of taking such a simple test or unwilling to provide any assurance that his alleged seniority made him a paper tiger.
I've personally interviewed a candidate who had a double major in math and CS, had a PhD, declared having over 10years of experience as a senior software dev, and on the first interview he showed he didn't even knew how to include a third-party library on what he claimed to be his programming language of choice he used most of his career.
That's the sort of stuff that take at home tests help weed out. If he took the test he wouldn't have wasted hours of his and our time.
Nobody is talking about fizzbuzz. We are all talking about take home assignments that take hours, generally 5-10 hours and sometimes more.
I code and troubleshoot for $$$, and unless there is a direct route to interview + hire (or at least lunch) I'm not interested. I understand there are sorting methods needed to do hiring, and I'll play that game if I have to, but if there is no direct, up front expectation that I'm in the running then it's just kind of patronizing without payoff.
In the few hiring processes where I've participated as a hiring consultant, take at home tests were only used in the initial stages of a hiring process to filter out blatantly incompetent candidates, who in some cases represented about 90% of all applicants. We're talking about basic fizzbuzz stuff.
Even so, in subsequent stages the candidates who made the cut were again evaluated based on their technical skills, mostly to weed out candidates who might have cheated their way through the first stage.
This meant that it was impossible for dishonest candidates to eat the lunch of candidates who were cut due to their performance of the take at home test because anyone who failed that test was obviously not a competent developer. I mean, are you expecting to be hired as a professional software developer if you can't write a for loop?
You are naive if you think you solved cheating by checking technical skills on the onsite. What if I'm a 7/10 candidate, and someone else is a 9/10, but I cheat and submit the solution of my friend who is a 10/10. I'll get the flight ticket to the onsite, the better qualified guy will not, and you will have to choose from a smaller pool of candidates, in which I might be the best one.
I can't imagine a scenario where a person can't solve the problem, so hires someone to do it, then explains it as well as the person that actually wrote the code. If you can explain it that well, with reasoning for tradeoffs and design decisions, you might as well have written the code yourself and saved some money.
We think this is better then wasting the candidate's time on a toy project - at least the can write off their time on the opensource-karma account.
Also, we can check not only their code but also their communication style and other skills that matter in the real world (projects on our tech stack have a high threshold for documentation, unit tests and other stuff that matters).
We call it "social code challenge" and it is working wonderfully for our recruiting process.
[1] https://stackoverflow.com/users/22656/jon-skeet
[2] https://stackoverflow.com/users/444036/paulo-scardine
A candidate might reasonably assume that they can look better than the other candidates by spending more time on the problem and tackling a larger or more impressive ticket. If you imagine a world where all interviews were like this, candidates could end up working 60 hours a week on these projects in the normal course of job hunting (easier to imagine if you assume that the talent shortage isn't permanent). Probably, since they can see the size of the contributions made by previous candidates, the pressure increases over time as well.
edit: it would be amazing for FOSS in general though. Imagine how much faster GNOME would be if hiring committees required you to write bug fixes for popular projects.
The same is true with a take-home, candidates see the (often underestimated) statement "this should not take more than X hours" and assume they will have a better shot if the invest more than the X hours.
I forgot to mention that we also accept past contributions - in the end the hypothetical candidate can spread this 60 hours investment over any other hiring processes using the same method.
I'm thinking about writing a "social code challenge manifesto".
My advice is always "build experience contributing to FOSS projects in your spare time".
We are pushing the envelope and often find bugs or lacking features. I want to hire people that can fix/implement those and send the PRs upstream instead of nagging the project maintainers.
Ask about the code. Ask about the thought process. Ask how to debug it. Ask how to change it or improve it. Put one of the company's team members with them and pair code for half an hour. Find a bug or two and fix it.
There's no lack of questions to confirm if someone did the work themselves. This is easy to figure out once you have working code in front of everyone.
Had other candidates totally ace the take home and flat out state first thing that they can't code and ask to switch to a non coding interview or to go home.
Some people spend a lot of time practicing to cheat. Given how much this industry pays it shouldn't be surprising.
Both of these happened 3 weeks ago when I was interviewing 32 candidates in a event. But several times in the last few years as well.
No I don't have statistics on this or pure confirmation... Kinda hard to get that info from suspected cheaters.
There are problems with every type of interviews a d they're hueristics for skill not a judge of skill.
I would love to meet someone who was at ease, could explain code in detail (including talking about looping over arrays) and then NOT be able to produce that code.
The second one flat out cheated, but I bet they wouldn't have been able to explain the take home so you might have caught them.
Kinda weak speculation on my part but I guess I could be wrong. I've literally never met anyone like that.
What is the take home like? Maybe the code/answers have leaked somewhere?
What GP describes is more common than you'd think. Not surprising when the prize for cheating is a 6 figure job, and there's no penalty at all for getting caught.
In the gp scenarios they said they were able to Google almost all of the take home and stich the parts together. The overall solution/problem isn't Googleable, we checked.
The first one could also explain the code fine. Interviews 2 and 3 they were comfortable and able to design decent systems but fell over on code. This why I was given the task of "get any code whatsoever after you put them at ease"
I would reassure them there is not a right answer, that the exercise was just asking them to engage with the question. It was so effective as an indicator of future performance, and, according to those who eventually were hired, felt fair to them.
Of course this implies all my biases to people who can have a 1:1 convo about a very vaguely specified problem and so on. No idea if it’d be positive for everyone, but I wish someone would interview me like that.
Even if you are going to work in a project that requires building such structures, I demand that you don't do it from memory, but make sure to get help from resources to ensure that your structures are sound.
"Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard so f* off."
"NASA: 90% of our rocket scientists use the A/C you just fixed, but you can't explain how a rocket works so f* off"
Presumably what matters is whether homebrew is technically significant or relevant, not just that they use it?
0. The actual job of a software developer is usually to develop software with some real-world use and benefit, not to write cs101 code on a whiteboard while someone watches you and grades you for speed and accuracy.
1. Software is what Google does, and Google does tons of software from tools to infrastructure to services to games. If you created a key software tool that tons of people use at Google to do their jobs, it's pretty relevant.
2. Software isn't like doing routine maintenance on an existing A/C system - it's usually somewhere between inventing A/C and creating a new design for data center A/C systems then building, deploying, and operating it.
1. I develop tool X, which is a simple popular solution.
2. Some company Y is hiring engineers to develop solutions that are far more complex and demanding than what my tool does, for instance because they have to support billions of concurrent users.
3. I deserve the job because most of Y's engineers use the simple tool X that I developed, even though the job requires far more advanced skills than I have demonstrated writing X, and I actually don't have these skills and will totally fail at the job if hired.
He got nervous on the whiteboard when asked a relatively simple question. They proceeded to shut him out of the position, completely disregarding his ample qualifications signified by their extensive use of his product. Which is a kind of tragedy.
Comparing an A/C unit to rocket science isn't an equivalent comparison. I can't think of a good analogy to be honest, but that isn't even close.
No other field I can think of really has the problem ours does. You can either make A/C units and rockets, or you can't.
But, in software engineering, it is highly possible to be able to say..... develop high quality GPS mapping software from scratch used by the Navy. Then turn around and have a company making word processing software reject you because you couldn't write a Mandelbrot fractal generator from memory on a whiteboard with a marker.
The fractal generator has zero to do with their word processing product and will never be a useful test of your abilities. Clearly the guy who made GPS mapping software can figure out how to write a word processor. Not to mention, they already have the word processor written so he'd only be assisting them in maintaining their existing code. In other words, he's already demonstrated his ability to write and maintain code of a high caliber. None of it makes any sense.
I really don't think any other field has this issue. It is weird as hell, and unusually pervasive in our culture.
Edit: It won't let me hit reply on your comment below. I think we've hit the maximum length for this comment thread. But, I'll respond by saying I don't know what position he was applying for. I'm not sure he ever explained beyond that tweet. The first I heard about it was Noah mentioning it. But, if it made their lives easier so they can deliver products on time to their own customers it makes sense they have a position somewhere in their company that would be a fit for him. At the very least it proves he is a good developer. Massive systems like the ones Google creates are made by many good engineers working together to make it possible.
The point was they knew he was a good dev.
Both are engineering projects. But one is simple and a solved problem, the other is far more complex.
The fact that many people at Google use Homebrew doesn't automatically qualify a Homebrew developer to work at Google. Maybe Google would develop Homebrew by itself if there was no tool like that available. Perhaps it would do a better job than said developer. Either way, it doesn't automatically imply that you can do whatever it is Google wants you to do just because you developed a tool that some of their engineers use.
Also, let's get real, to say that Google engineers use a package management system to "get their work done" and then describe said PMS as integrally related and essential to their work... more than a little bit of a stretch.
Though I guess these days I guess 'software development' means downloading Javascript crap from github and copy-pasting CSS.
No, it's bloody not.
What's your hash function? Why? What do you do on hash collisions. How many buckets and how big? Does your hash function map to identity or not? Do you exclude certain fields from hashing? How do you handle contention from multiple threads? I can go on and on.
The fact that you think writing a hash is easy betrays a stunning amount of ignorance.
I need a quiet place to sit and think something through and plan it out. It is why I did well on the test portion but not on site surrounded by interrogators. In those kind of situations I kind of feel like I'm the in the movie Swordfish where they're like, "Hack that server you have sixty seconds! (gun to head)" The difference is, I would have been dead.
That’s the only way hard, unique problems are solved. If it can be done quickly, it’s routine.
>I got nervous and passed the question
I didn't know that was allowed. Has that worked for people in the past?
I went back and looked at the emails. I can't edit my original comment so I'll jot the corrections here. They gave 3 days not a week. I completed the submission in 7 hours, they required you track how many hours you spent and when. I used a genetic algorithm to solve the problem. And, I took advantage of multiple cores if they were available to divide the work load.
Looking at the email there were signs even in their review of my submission that their outlook wasn't completely rosy. They mentioned things like how I failed to correct for differential conversion factors based on global location and stuff like that.
Skipping the question obviously didn't work out for me as I didn't get the position.
It was a weird experience to be put on the spot with such great skepticism with all my past accomplishments and knowing how much they respected my test submission. In a way it was humiliating. I didn't leave that interview feeling very good about myself. I still regret it to this day. In some ways I wish I could go back in time and pass on the interview, I think I would be more confident today.
I bombed an interview where I couldn't understand the prompt, literally. Like the interviewer spoke it to me, and I restated the problem, and he said "no it's…" and drew something on the white board, and I listened and restated it, and we went back and forth like that for about 15 minutes. To this day I don't understand what he was asking for, only that involved a rate of bank customers and a Greek lambda.
Getting good jobs means pushing your limits and it's therefore a numbers game. Feeling shame from rejection is natural, but also generally unhelpful these days, so it's worth unlearning.
- We make a commitment to reply to every candidate who applies as instructed. Not every reply can be detailed, but candidates are never applying to a black hole.
- We have a very detailed job description that answers a lot of the questions many developers have about a company up front. There is always a chance that we are lying or don't live up to what we say, but the info is at least there so the applicants can make an informed decision about whether they are really interested in working for us. The details about our application process are also stated clearly up front so that candidates know what they are getting into.
- We ask pretty in-depth follow questions about technical experience to make sure it aligns with what we are looking for. This filters out a decent amount of candidates who would be unlikely to pass the rest of our process saving them and us time. Other than roughly validating that the candidates work experience falls within the range we are looking for, we find resumes to be mostly useless.
- The initial "take home" exercise we ask applicants to complete takes 60-90 minutes. It's not a coding exercise and they are given a document that helps them prep for the work they are going to be asked to do. We have found over time that this is more predictive of success in our organization that doing an initial phone screen. Candidates are given relatively detailed feedback on this exercise, usually within 3 business days of submitting their work.
- The next step is a Zoom interview. We have a few very basic coding tasks that we ask them to do as part of this interview. These tasks are representative of real world skills a developer would need and not in any way convoluted or academic. The environment could be challenging for some due to the fact that we are watching over their shoulder, but again the tasks are very basic and the goal is simply to see that the tasks are able to be finished in a reasonable time. This isn't to tell us if the candidate can do the job, it just helps us filter out candidates who are likely to fail badly on the next part of our interview process. It saves them time and us time and money. We have occasionally passed candidates on this step who seemed like their bad performance might have been from nervousness and, to date, those candidates have never performed well on the rest of our skills tests.
The candidate has an opportunity to ask any questions they would like during this interview.
We also talk about money at this stage. Our salary range and benefits are published on the job description so we ask the candidate what their expectations are from a compensation perspective. If they are reluctant to share details, that's fine, but we at least confirm that they understand that our published compensation is what we are actually planning on paying and ask that if that is not suitable to them, that they let us know that now before moving along in the process.
Only if everything is aligning at this stage, do we then ask them to commit to a significant more time working through our skills tests process. Our increasing "ask" of the candidate's time is deliberately progressive. We are trying to only ask more of them when we have had the chance to vet them for obvious mismatches and vice versa.
- We then move into a two part skills test process (16-18 hours). All candidates who move on to these tests are compensated at around $50 per hour. All tests are timed and represent real world tasks our developers do on a daily basis. The first three have to be scheduled and proctored but the fourth is really "take home" and...
https://www.level12.io/careers/full-stack-web-app-developer-...
Reason why is because I've had interviews at other companies which are far less interrogative from the company towards me. For example, for my current job (embedded systems) I had:
- 30 minute phone interview with a technically competent hr rep - a quick very easy online quiz (multiple choice/few sentence answers) -one more 30 minute phone interview with a developer (open ended technical discussion) -onsite for roughly 2 to 3 hours with a dew developers, all very open ended technical discussions
It was honestly the best hiring process I encountered. I didn't feel like I was even tested during the interviews, it was genuinely enjoyable technical conversations.
But, there are others out there who appreciate a more thorough approach and might view the process you described as inadequate.
Also, our salary range is higher for senior devs.
I think what matters is whether the process being used by the organization is resulting in the hiring of developers that are a good match. I'm glad we are doing things this way even though it makes hiring take longer and likely filters out candidates who could do the work. It's an imperfect balancing act, one I'm always looking to improve. The main point of the post I made above was to show how IMO take home exams can be executed well, not to argue that everyone needs such an elaborate application process.
As best I can tell, our process is not typical. But it's also not fundamentally a bad thing. It obviously doesn't interest you and I'm sure there are others who wouldn't apply for the same reason. But, there are others out there who appreciate a more thorough approach and might view the process you described as inadequate.
Path of least resistance my friend, path if least resistance.
This is pretty onerous, but it should be pretty damn good at rejecting bad candidates.
I do wonder how many good or great candidates you lose with this method, though.
It's also more than just the $800 we pay out. I personally proctor and grade the tests. That takes me 4-6 hours. Given my billing rate and roll in the company, that's arguably a greater cost to us than what we pay out to the candidate.
The company chose to spend a lot of time evaluating all these take home projects, in the first step, and then reject people for silly reason afterwards? It's a lose-lose situation.
Sounds like their hiring procedure is deeply broken.
It may have looked like the applicant had someone else do the take-home assignment for him.
That said, I imagine that they could have interviewed in a more constructive way that may have been less stressful (e.g., walk us through your thought processes as you completed this take home assignment — most frauds can’t do this convincingly).
The company definitely could have done better. If you have someone hitting a home run on a take home assignment, at that point a no-hire decision better have some very strong justification behind it.
Sounds like there are better places to work.
I hope you landed in a good place.
When interviewing, we test for presentation, self-control and anxiety as well. For me it's not a big deal, but for the guys that were interviewing you it seems it was.
Software development is a very stressful job, if you can't even handle the interview, how would you handle real life (c) at this corporation?
In my mind, that perspective is a huge red flag for me when I'm on the job hunt. If you are trying to stress me out in an interview rather than make me comfortable then I have to thank you for letting me know before I started working there that you would be exhausting both physically and emotionally.
Some background, about 2 years ago I started teaching myself to program full time. About a year ago, I felt confident that I knew enough to get a job, taught myself all of this leetcode stuff, and started applying.
People told me that if I could crush leetcode, I'd have no problem getting a job. What they didn't say was how impossible it was to get an interview when transitioning fields.
I was also told that people would look at my GitHub code, but considering recruiters spend about 15 seconds on a resume screen, this is more of a meme than reality.
In the last year in Bay Area, I've managed a total of 2 on-sites with well over a hundred applications, all of them quite targeted. Maybe 8 companies total have engaged with me, and I have not missed a single one of these types of questions at any point in the process, yet it hasn't been enough.
It seems that there are so few entry level jobs that that market is extremely saturated. I'm not sure how this industry expects to address it's serious man power shortage if no one wants to hire juniors.
The most likely problem is simply that you are getting questions wrong, but you don't realize it. Companies don't often give candidates whiteboard questions, have the candidates nail those whiteboard questions, and then drop the applicants.
Looking at your profile, you seem well suited to be able to evaluate this, so let me give an example from an interview I just had.
Here's a problem and solution from the last interview I had: https://pastebin.com/LyVYqLv4
Would you mind emailing me whatever resume-ish stuff you have at my hn username at gmail? Maybe I can give more specific help.
`biggestSize = 99999999 #MaxInt` This is returned if the caller passes you 0 for slicesToFind, so you're returning a nonsense value in that case. Either raise an exception (preferred as it catches errors fast) or return None.
`def area(size)` You're using pi r^2, so size is a radius, but it seems you call it with a diameter. This is why choosing distinct names is very important.
`newSize = (originalSize / newSlicesInCake)` Again, using "size" all over the place is confusing. I'd write something like `new_slice_area = (original_cake_area / new_slice_count)`.
As far as 0 input, they actually told me it would always be valid so edge cases were not accounted for.
But, yeah. Casing convention was surprisingly important in one of my interviews. They wanted to know that I was able to adapt to a new language as the job was for a language I didn't have on my resume. It's weird, but I understand where they're coming from. Seeing improper styling in a code review is an immediate red flag and an interview problem is ultimately a code review.
[0] https://codereview.stackexchange.com/
Here's some discussion on it https://leetcode.com/discuss/interview-question/348510/Googl...
Companies are all over the place all the time. And I have seen this from both sides of the isle.
For example, at one company I was at, we said we were hiring for a full stack position, and we had a candidate come in that did well on all the questions, but then afterwords, we decided that we actually wanted a dedicated front end engineer. So no job for them!
It was no fault of the candidate. We just didn't know what we're were doing and screwed up by interviewing someone who was better suited for a different position.
Stuff like this happens all the time. Even if you blow the interview out of the water, there is still only like a 50/50 chance that you'll get the job, due to no fault of your own.
It's tough to get a foot in the door. I managed a transition largely thanks to preferential hiring for veterans, but even there I had a CS degree and some prior tech experience before going into the military.
The other problem is that "junior" really means "fresh out of college." Transitional resumes look weird and they get thrown out in favor of people doing the normal thing.
> I was also told that people would look at my GitHub code
Recruiters, probably not, but if I'm doing an in-person interview with you, I've definitely read your github page.
Tip: I'm looking for a. high quality code, and b. a delivered product.
What's high quality? Find code written by the best people (e.g. your standard library, a major framework) and aim to make your code look that good.
What's a delivered product? It has thorough, well-written documentation, and I can install it with a single command. If you set up a demo on a website, that's obviously better.
Why anyone puts a high degree of credence into these tests and whether someone can solve one in a high-pressure interview is beyond me.
Give them a take-home assignment where they can develop something similar to what they would be at the company, and decide if they wrote efficient, elegant code that's easy to follow.
Also, if you're self-taught and applying to a team of CS majors, may the force be with you. No one's more insecure than a CS major interviewing a guy who might have more development experience than he does.
Have you seen some of the study guides / roadmaps people make for FAANG interviews? I've seen people claim they've coded hackerrank / leetcode / CTCI questions for multiple hours a day, for months.
EDIT: Also, at least with the Hackerrank for hours a day, it's not for any specific company. Getting good at these problems is useful for getting a job at virtually any big company. Contrast this to the take-home project, where (in order to make sure you don't just copypaste someone else's from the internet), they make these things kind of specific, making the transferability to other things substantially less likely.
Once you get very good at leetcode-style questions, just apply anywhere and get an onsite with a one-hour interactive technical screen, that will also tell you about the sort of people and culture that employer has.
Conversely, homework assignments can easily take 5-10 hours. There's a lot of desperate unemployed people out there taking 10+ hours for these tasks, and setting the bar really high. A busy professional can be hard pressed to invest 5 hours they can ill-afford coding this task after a full day at work, then get rejected because some unemployed applicants dedicated 20+ hours for the same task.
There are also services that solve these tasks for you, so now you - a busy professional - are competing against hordes of people who do these tasks all day every day for a living.
So you hard-scramble 4-5 hours to finish a take home assignment to apply to just one place (!), and then you send it in and get a meaningless rejection note. Sometimes you don't get anything at all.
These are some of the reasons I'm never completing a homework assignment again.
What also frustrates me the most is when they say "use whatever language you want", I use Scheme or Haskell, and I get feedback to make it more "modular" by making it more object-oriented. The reviewers of the take-home assignment will be too afraid to admit that they don't understand Haskell, too afraid to admit that they made a mistake making the language a wildcard, and automatically assume that it's less scalable because it's not Java.
Not a great income at a place where a half decent 1br can easily cost you over $40k/yr.
Definitely became about a million times easier to live when I left academia and started making six figures.
Either way, let's get real for a second: a software engineer is a highly trained professional working in an high-demand field.
There's no reason for them to be earning $120k/yr, which is a pretty low salary in a very high CoL area like NYC.
I mean, not really? I just hopped onto the A train straight down to NYU; it took me like 40 minutes door to door, and being on the train allowed me to read, so it's much better than driving.
I make substantially more than that now, but have moved to Brooklyn since then; door to door it takes me about 50 minutes, but I wouldn't classify that as "a nightmare".
> a software engineer is a highly trained professional working in an high-demand field.
At the time I was a college dropout whose only professional experience was in startups, 4/5 of which had gone bankrupt, though the university actually paid their actually-qualified people pretty poorly. My coworker had a masters degree in data-science, and was only making $75,000/year.
Depends on your perspective, I suppose. Personally, spending 2 hours every day in commute is definitely not how I want to spend my life. A major reason I would move to a city like NYC is to eliminate such a long commute.
Last few times I took the train to Brooklyn, the train was always delayed by several minutes. Sometimes 15+ minutes. Trains were packed, especially during rush hour, and there was nowhere to sit. Occasionally they'd just stop place for technical reasons for as much as 10-20 minutes. Definitely not a smooth or pleasant experience.
Regarding compensation: $120k is becoming entry-level compensation in the Bay right now.
I have no desire to live north of 59th st, but lets be real here.
Anyway, NYC software engineers do not need to compromise on their living situation. Google alone has bought two billion dollar blocks of Manhattan pays just as much as in the bay area.
Here's an ad from someone looking to hire a programmer in Nassau county:
VOIP company looking for a technician to assist with programming back end systems. Should be knowledgeable with various database programs and voice services and applications.
Primary experience: Windows Environment - IIS / ASP.Net, VB.Net, MS-SQL, XML, Json. Primary helpful: PHP, MySQL
Secondary helpful: Google SDK, ffmpeg / sox, SIP Voip , basic CentOS / Debian navigation
Job Type: Full-time
Salary: $35,000.00 to $45,000.00 /year
I guess that will be the next trend in hiring propaganda: selling us on that sweet homeless life.
Here's the list of skills wanted from another NYC junior programmer want ad that says $35-40K/yr:
Programming Skills Desired:
Transact SQL
Database Design & Theory
HTML
XML
CSS
JavaScript
VB Script
Auto Hot Key (AHK)
PHP
C#
All of these (except for AHK which I've never heard of) look trivial, but I'd expect the interview would be impossible, because the interviewer is going to be primed to expect people with no skills and therefore assume the worst of anyone who doesn't have 110% current experience in each.
VB Script and PHP and T-SQL and C# along with the rest?
I'd expect a college student not to be actively taught any of these (T-SQL maybe if they took a database class), and they're not exactly the new hotness for languages to learn.
I'm saying this as a dev that works for a company that uses a Microsoft stack right now too (I almost never see anyone talk about C# on this site or in the Who's Hiring threads.).
This seems like a laundry list of that employer's specific tech stack hoping they can get some sucker that has their exact stack because they happened to experiment with all of those languages in their free time, which is unlikely. Anyone who does have this stack can probably get a much better salary almost anywhere else.
It does say "desired" at least, and not "required", so they might not expect applicants to match this completely, but I bet some of the interviewers haven't gotten that memo (which seems to be what you're saying in the last paragraph also).
Also you mention you haven't heard of AHK, and it can be pretty useful. It's basically a macro program for doing things in Windows and assigning a hotkey combination to it. I've scripted some things in it before, although I'm not currently using it. It's not that hard to learn, and I can't imagine that any programmer couldn't pick up the basics after a few hours on the job, so I don't know why anyone would include that in a list of technology, thereby adding one extra thing for an applicant to think they need to have experience in to apply and not apply to their company.
Like a lot of ads, they mix ridiculously specific qualifications with ridiculously generic ones.
For example, one important skill is: - "robust practices" such as "loops"!
and another is: - "knowledge of City University of New York Institutional Research Data"
Requirements are "in-depth knowledge in big data, data science, Python programming, object-oriented programming, machine learning, or natural language processing (NLP)."
Responsibilities are: "Lead technical workshops and training sessions in different areas of data science and Python programming
Help clients and students understand the immense career options for Python programmers and data science knowledge
Motivate clients about the career change and career boost that upskilling with Python, data science, and blockchain can provide."
I think it stands to reason that if big data, etc. generally paid much more in the NYC area, nobody would take such a position, so this must be representative of the job market.
These companies get a ton of candidates and tell them all to submit homework assignments that take at least 5 hours (in your case 12).
Reading 12 hours worth of coding work is intense effort even when you know the language well. If you don't, it can take a multiple of that number of hours.
Some programmer got 10-20+ of these completed tasks dumped in their lap.
I can pretty much guarantee they never really took the time to read and review the code with anything close to sufficient care or attention. Doubly so if it's in a language they didn't know very well.
But on the other hand, I think good professional like you, don't need to take 4-5 hours for a take home, since you'll be fine without doing it :)
Thanks, but I'm a programmer, not a magician. A good programmer might work faster, but he's not able to reduce any amount of work to a constant.
A good programmer may accomplish in 4 hours what an average one will accomplish in 7-8, but he will not be able to do it in 1 hour. Also, he will still seem inferior to an average programmer who took 20-25 hours to perfect this project, especially if those take-home tasks are all that other programmer does.
Do you mean you've gotten rejection letters straight after submission? In anything but exceptional cases I thought it was standard practice to always have an interview after the submission and talk about the solution - how you talk at the code, design choices, possible optimizations and issues should weigh heavier than the quality of the submission itself.
So I don't think anyone using services doing it for them has any success with it. And if they do, you may have dodged a bullet.
Yes, that definitely happens. In fact, from what I've seen, it's the norm: companies who give out homework projects only bother spending any time with you if you completed them to their satisfaction.
Otherwise you just get a meaningless auto-generated that they're rejecting you.
> So I don't think anyone using services doing it for them has any success with it.
Why not?
If I'm a mediocre programmer in the US, I can go online and buy a solution with a full, detailed verbal explanation attached. Then I'll take maybe 40 minutes to read through the solution and explanation, and voila, I'm ready for any followup session you may have.
So they'll have you do one before you even come onsite. Then if you pass, they'll run you through the same whiteboard loop we all know and love. Even if your take-home performance was fantastic, they'll still reject you if your onsite performance was anything short of.
In practice, I take jobs that don't require demonstrating anything technical in the interview.
But a take-home test would only add information, not take anything away. So there is no reason in principle someone couldn't use it to hire.
A candidate like that will have 10+ companies trying to recruit them. Out of these, 9 will require 1-2 hours of remote screens at most for an onsite. The 10th will require a 5-10 hour take-home project for the same onsite.
Which one will the strong candidate skip?
There's no contract nor payment for these and I write them from scratch, so it's still my IP.
tl;dr: I feel you completely and will never waste my time like this again.
Come on. It's not completely unheard of to be given confusing or impossible requirements while working for some companies and while in those situations I would probably just communicate back to the Business Analysts what's broken about the ask, you aren't given that opportunity in a take-home with a new company. Presumably a bunch of other people have been given the task and successfully completed it, so I would probably try to power through it.
I'm sorry, but that's an asshole thing to do to someone. In an interview setting it's more okay, because you have a chance to mention it after a few minutes and talk it out and get feedback on it or they can stop you if it's clear you're not pointing that out and you waste a bunch of time, but with a take-home you won't get that quick feedback, even if you send them an email, and can waste hours and hours on that thing.
Interviewers reading this, don't do this, please. It would leave a very bad impression on me and probably a bunch of other people about your company.
...how about we give you a project that more closely reflects problems we solve here day-to-day that you can solve in the comfort of your home on your own hardware in whatever IDE makes you comfortable?
Engineers: That's absurd! I've drawn a line in the sand!
But, software engineering interviews are pretty rough. It is a uniquely miserable experience. Since I'm a temp worker I have a lot of them. They don't get easier.
I don't know of anyone who likes high pressure time crunch interrogations except maybe the ones getting to do the interrogating part.
A take home test isn't so bad. But, ya if you asked me today if I'd be willing to do one like that again I'd say probably not unless I was desperate. The reality is, it is a lot of effort to put in for it to all come crashing down with a rejection anyway. When you're applying at dozens of companies if they're all requiring that level of time and mental commitment from you, you can't swing it really and it's stressful. Interviewing and being under financial pressure at the same time is already a stressful position to be in, let alone being mentally and emotionally flogged for it.
All the jobs I was ever hired onto the hiring manager just had a good gut feeling about me and the tests never mattered. And, they were happy with my work.
The ones that never hire me usually rely on all the employees to vote on you, if even one says no you're out. Or, the ones that are very fixated on your GPA or whether you can invert a binary tree on the whiteboard while they yell at you "Wrong!" along the way.
At one, the resume didn't matter! When through the interviews, got the offer, and then "oh, can you send me your resume, so we have it on file?"
As someone who has been through this early career I agree with the sentiment. But being mid-career it is kind of fun to go through one now, the catch being that you shouldn't give a sh/*t if you get through or not. Think of it more as two-way courting instead of a one-way interview, not all of them pan out.
Something that "reflects problems we solve here day-to-day" is missing one feature: you're not paid for it. I have a pretty good job right now, but it's not like I'd keep doing it if they decided to stop paying me.
I had an interview a few years ago that I thought did something cool: you spent half a day working with people, and they paid you for your time (around $40/hour, if I recall), and made an offer based on how well you worked with people. I ended up not getting the job, but at least I didn't feel used or like I wasted me time in the process.
Me: How about we do something that more closely reflects problems you solve there day-to-day there and during the day? Your interviewer being present throughout means you'll get to see how I work and you'll be able to provide clarifications or answer questions as well. As a convenient side-effect, we'll both know that the exercise is a sufficiently valuable use of everyone's time to give up the equivalent amount of salary over it.
"How about you, a highly trained professional, do 5-10+ hours of unpaid work for us, that will be judged against desperate candidates who invested 25+ hours into them and/or bought a solution online, so you will get nothing for hours of work, except a meaningless rejection notice by some overworked programmer who didn't even read much of your code since he had 50 candidates' solutions dumped on his plate?"
Personally, I've hired many great candidates, and got several great jobs, via whiteboard interviews. It's not perfect but it's far superior to takehomes, which are completely one-sided, very hard to get right (by the recruiting company), and often are boring.
Whiteboard interviews and (even better) pair-programming with an interviewer are reciprocal and allow me as a candidate to learn about the company rather than just do some work for them for free. Often the tasks are more interesting too.
In my opinion, it is the takehome tasks that are "doomed". Their premise is flawed: "we don't want to invest an hour of our engineers' time in talking you worthless candidate, so we'll instead have you spend several hours coding up a task that our engineers will never take the time to check properly, since their time is more valuable."
In my opinion, the future is platforms interviewing.io, where candidates are vetted and top candidates get the minimal investment of an hour of the company's engineer time to pair-program, which is far superior and more reliable way of screening.
How about we also give you this other project?
If it were 5 hours on-site versus 5 hours at home, that would be different than 5 hours on-site plus 10 hours at home.
However, that set of people is outnumbered by the set of people who applied to Google or Facebook and got rejected.
Probably don't want to go through this process again, unless there's some amazingly good opportunities there.
I went to the interview, did a sit-down back-and-forth in-person discussion about my past work and resume; the usual. No whiteboard.
Then I was told I had to do a practical exam; they basically gave me a laptop (MBP with MAMP - this was for a PHP/MySQL position), sat me in a cube with the rest of their development team, and gave me a test to work thru. First half was just question-and-answer (multiple choice and t/f - maybe some written too IIRC). The second was to code up a simple CRUD app using PHP and MySQL.
They said I could use any resource I wanted - google, etc. This was about 3pm, and I had until 5pm (quittin' time) to finish. During this time I was working in the area I would be normally working, among potential future colleagues, with all the office distractions normally found at this place.
Oh - I forgot to mention where: Fender, in their marketing department.
So - randomly - people would start to play amplified bass guitars, or a drum solo out of nowhere. At one point, someone rang a cowbell and stuck a triangle. Fortunately, there wasn't any artists being introduced, but that was a possibility too. Various people on the team came up and introduced themselves. Distraction galore!
But that was how it was going to be - so that's how they tested you.
5pm rolled around, and they came by asked how I was doing - I let them know things were coming along well, but that if I had another 15 minutes I could get things more solid. They said, sure - take your time, we sometimes stay late - no problem. So I finished, and gave it back to them.
Went into a separate f2f with the director of the department and some of the team, chatting about my experience and such. The SWE supervisor and lead SWE came over, shut the door - and basically made me an offer right there.
I worked there for a couple of years before they did a downsizing and I had to "move on". But during that time, I found that this kind of test was actually very, very difficult for more than a few people. Some didn't even do anything - they just sat there. One guy sat for a few minutes, then got up and left, never to be seen again. Some would complete the first half of the practical, but completely blank on the second half. All of this, despite having a completely set up system in front of them, plus ability to reference the entire internet. If they didn't know PHP or MySQL, it wasn't for a lack of resources that they couldn't or didn't write anything...
Then again - I've heard of people failing to write fizzbuzz, so...
But overall - that was one of the top interviews I've ever had for an SWE position. I guess it was partly the practical testing, but also putting you "on the floor" to see how you'd fair in that environment (and yes, the whole instrument playing thing was a daily thing). I guess some people couldn't handle it. Others might've just not wanted to do the practicals feeling it was beneath them or something? And still others might have actually just "frozen up" for one reason or another - or really couldn't do what they claimed.
For me, it was one of the easiest and relaxing interviews I had ever done, and I would do again - and would love to give a similar interview to someone, if I were in that position.
1. A phone conversation with a manager to see if we were on the same page.
2. A one-hour on-site coding test, writing a fairly simple class using very strict TDD (i think we did this in the evening, after my day at my previous job had finished).
3. A whole day on site, spending each half of the day pairing with someone on something. In the morning we wrote a little demo app for using Redis on Cloud Foundry (in Ruby, which i didn't know!), and in the afternoon it was some made-up webapp (using Spring, which i also didn't know). Both things either were or were very like real development (on the Spring thing, we spent half an hour setting up the environment, fifteen minutes coding, then two hours debugging and reading Spring docs to find out why it wasn't working, so exactly like real Spring development).
The first two stages were pretty undemanding, which was good for me. The final stage took a lot of time and energy, but i got to spend the whole day with two current employees, both of whom really impressed me (shouts out to Chris B and Nick S), and that was really good for me.
In general, i really like to be given a stringent interview, because it indicates i'll be working with high-calibre people who have passed it. But it does have to be effective in its stringency, not just unpleasant. Amazon Instant Video's hiring process was the latter (and not just because it took a whole day with no break for lunch).
I'd be happy to interview people that way. When you can't talk enough about your project, nor about stuff they've worked on before, you can still engage them in a discussion of a hypothetical but real-world problem (not CS101 problem sets), and go from there.
If you know your own technical stuff, IME, it's pretty easy to tell from talking with someone that they can fizzbuzz, without a hazing gauntlet of whiteboard exercises that give little sense of anything (other than that the company thinks hazing experienced people is a good idea, and that the person will submit to hazing).
Review code.
It would be at least a few pages, in the language they would be working in, with a few bugs that you know of in advance. Depending on what the code does and the particular bugs you introduce you can get a pretty good idea of the candidates' comprehension of the things you need. (Bonus points if they find bugs you didn't introduce.)
One problem with this is that both the code itself and the evaluation of the candidates review have to be done by sharp folk who know what they are about. Otherwise GIGO.
"One of my best interview experiences was exactly that. The interviewer handed me a printed out class and said "tell me what this does, any mistakes you see, and any little improvements that you think could be made."
I got the job and later confirmed it was a class that was actually in the software I would be working on, not some toy class he came up with.
He also asked me a handful of technical questions and had me go over some source code I brought in, but that was about it. At one point he even said, "I know enough to know you can do the job, but now I'm curious what you really know." And asked me some no pressure really low level questions. When I said I didn't know the answer to something he'd spend a couple of minutes explaining the concept to me.
I walked out of that interview having learned a few new things about my field. A couple of them have stuck with me over a decade later.
And the guy was a grizzled veteran who was the lead on some massive mainstream video games I pretty much guarantee you've heard of. He used to code them in Assembly, even. He's probably the most knowledgeable programmer I've ever met.
I've never had anyone try going over actual code with me since, in probably a couple dozen interviews, and I don't understand why. It seemed very effective."
Honestly I would take home assignment over any technical interview. Even if it will cost me 3x time. At least then I will be working in a relaxed state and will do my best, compared to an interview where anything can go wrong.
Homework feels really unnecessary too. If the interviewer would like to see code I have written I have a USB stick full of old university assignments and some public projects on Github.
I did a coding test once that was organised, met the requirements of the test spec, was documented, included tests and in identified and solved an inconsistency in the test spec.
The feedback from the employer "they didn't like your code" - right there is the point at which I realised that coding tests are, without a quantifiable, systematic assessment method, complete garbage.
I just hate the technical beat stick that is the whiteboard test, feels like going up in front of the class in data structures and algorithms classes
You could say, why not giving Sudokus, or chess boards challenges to candidates? It's not totally bad either, it's just further from the context
All this is not incompatible with knowing other engineering fields, networking, security, data structures, code patterns, which are indeed more useful in day-to-day work.
Students are often selected the same way in exams, which won't really matter in their future job/life if you look at their content. But the process matters, school teaches you to learn mostly, like those small logical/algorithmic challenges trains you to think, visualize, reflect on data and more