I've tried getting interviews at Google and the only time their recruters have contacted me back were from internal referrals (none of those worked out in the end as they wanted me to move to SF, and I wanted to work from Canada).
How would one go about getting your foot in the door at such companies (I've had similar experiences with both Microsoft & Facebook as well) ?
Often employees will be open to referring you even if you don't know them well. Use social media, Facebook groups, Quora, etc. Recruiters want to be found.
Have an active github.
Make yourself Google-able. Have a website where you list projects. Screenshots help make things feel more "real", especially if it's a recruiter/sourcer who might not understand some of the technical details.
Hackathons and conferences also. They're swarming with recruiters.
For people not in college anymore, those are great advices.
Having been on the recruited and recruiting side for one of the big 5, I'd add an extra advice: Don't panic if you don't have any of the things mentioned above, especially when still young in College.
I didn't have an active github, nor a website, nor screenshots of my side projects. If you're genuinely passionate when you talk about it (no matter how small you think this is), the recruiter will notice this.
One of the most important advice that was given in this article for me is: "Show initiative even at the risk of failure"
-> You did X in a class project : It won't matter so much because you had to do it to pass your class. If it's a group project it will matter even less because there's no way for the recruiter to know if you did 90%/50%/5% of the job.
-> You did a small Android app or a personal command line tool for you, but you didn't publish it at all on github, and it's a private thing (a tool to help your grandma do X remotely), it's fine, but like the Amazon interviewee who started her game company: Mention this on your resume and to the recruiter!!
I've also seen people downplay achievements because they thought it looked lame compared to what is produced by Google/Amazon/FB/Microsoft ... but those products have dozens if not hundreds of engineers behind them, PM, UX teams etc. Of course your project will be lame compared to it. Still put it on your resume.
If you're going at a top CS school, having all of these doesn't matter as much, the recruiters usually know that the projects/classes you took are not trivial (via Alumni giving feedback on those), if you go to an average one, you need to show your passion in one way or another, working on a small side project you genuinely enjoy is a great way of showing this.
Finally, if you have very little time for side projects because you're busy working multiple side jobs to be able to pay your tuition and rent, mention it somehow when talking with a recruiter. You certainly don't want the recruiter to perceive you as what the authors describe "mentally lazy" when in fact the reason you're not doing that much in the side-project side is because you're working your ass off to simply be able to graduate. You can also use some of those side jobs experiences to show the recruiter you have some applied leadership/teamwork skills.
This github/side project thing is at the far end of diminishing marginal returns. It of course wouldn't hurt but most people I've met at these companies (where I also worked for a while) do not have side projects or public code.
So if your goal is to get the interview, go through referrals. Don't expect anything from side projects.
another quick question: in general, would it be considered ok to find recruters (and maybe employees from the same team that's hiring) on LinkedIn and contact them?
The biggest problem I have with these companies is that they feel like a cult.
And the people working there are doing a lot to keep that cult spirit alive.
All of these companies want you too feel and act like they are the single best entity in the entire universe and you should be honored that they even acknowledged you. This works great for young graduates that are full of dreams and hopes but it discourages experienced people from even attempting to go through the interview process.
Oh and the always responsive HR or whoever is in charge.
Getting back to you after 6 months.
Thanks but no thanks. I would rather make more money in a stupid corporate job than working on more aggressive ways of making people to click on ads.
I'd ballpark the average range for senior engineers at Google at being around 250-300k in the US. Staff engineers (one step above senior) will make significantly more than that, although not everyone reaches that level.
That depends, are you factoring cost of living into your calculations? 200-300k is sounds like a basic living wage in the bay area these days... What's the point earning that kind of cash if it's gonna be chewed up by housing and transportation?
How on earth can you think that 300k is a basic living wage in the bay. Even with insane housing prices, this is like 10x the national median wage. Knock out 1/3 for taxes and you are left with 200k per year. How are you spending the 190k per year on rent and transportation that would be needed to knock you down to typical post-expenses income levels?
> That puts 200k right around the mark for a livable wage
To begin with, when discussing wages most people are assuming for a single person taking care of his or herself. The wage needed to support a fully family including enough for a stay-at-home parent would be significantly higher.
Even with all of that a family of 4 with only a single adult working is estimated a living wage in SF area is deemed to be 70k.[1] Not 200k. I'm assuming you're meaning "livable wage" to be different than "living wage".
Even if we were to going with the businessinsider data, which says 90k for expenses for a family, you're claiming that a family would need to spend almost 10k in discretionary spending every month for a "livable wage"? Remember all required expenses have been paid for, this would just be for discretionary. If you've already paid for "housing, food, childcare, transportation, healthcare, taxes and other necessities, and taxes" and are still going through an additional 10k/month, you're beyond a "livable wage".
First of all, these are all gross income numbers and include no savings at all. If your estimated gross living wage is 70k, but you're making 200k, that means an extra 130k. About 55k of that goes towards taxes (see tax rates below), leaving 75k. That needs to cover all your savings (if you make 200k, you better be saving about 20-30k/year for retirement, plus you might want to have college savings and such if you have kids) plus discretionary spending. If you figure $10k/year for college savings for 2 kids (probably low!), that leaves 35-45k, or 3-4k/month in discretionary spending, which is still quite a bit, though nowhere close to the naive 10k/month you mention.
Looking at the sources of these various numbers, the $91k number in the busnessinsider article comes from http://www.epi.org/resources/budget/ which breaks down the annual costs as follows, annually, for the SF metro area:
* Housing: $23k (seems fairly low for a family of 4 in the Bay Area, but I guess it depends on how far out you live)
* Food: $9384 (quite reasonable)
* Child Care: $10815 (seems unrealistically low for 2 kids if both parents work; hard to tell if only one works, since I would expect that to be near-0 for a bare minimum).
* Transportation: $7300
* Health care: $12,453 (might be high depending on your employer)
* Other necessities: $15689
* Taxes: $12490
I assume that tax number is based in the actually cited $91k income, so to figure out how much more income we need for discretionary spending and savings we need to look at marginal tax rates. Looks like CA state income tax is 8 or 9.3% in the relevant income range (60k-500k for couples). Federal rates are 25-28% (75k-233k taxable income). Medicare is 1.45% until we get to 250k. Social Security is 6.2% until we hit the cap. So a marginal rate of 35% to 39% not counting Social Security.
Retirement savings should be ~15k here (15% of income is the common recommendation), and can often be done "pre-tax" (so only taxed by Social Security and Medicare); let's call this 16k gross income.
Are you saving for college for those two kids? Any other savings (e.g. for a house)? Any discretionary income at all? Multiply all those numbers by 1.6 or so to account for the taxes involved. Realistically I would expect this to come to another 20-30k easily for anything resembling a "middle class" lifestyle.
Looking at the numbers from your link, they have a much lower healthcare estimate (~6k/year), a somewhat higher housing estimate, correctly estimate childcare costs if only one parent works, but still have what look like ludicrously low numbers if both parents work. The "2 adults, both working, 2 children" number from your link is $83k and the difference between that and the $91k from the businessinsider article is all down to the health care number.
Anyway, the correct conclusion here is that in the Bay Area for a family of 4 you need an income in the 70-90k range (mostly depending on how much you pay for health insurance and whether you both work) just to barely scrape by, with no savings and no discretionary spending of any sort. At 150k you can probably do ok in terms of being able to save for retirement, college, maybe take some vacations every so often, etc. 200k should be reasonably comfortable if you have no pretensions to more than 2 bedrooms (notice that housing for 2 kids is claimed to cost the same as for one kid) and aren't trying to get into good school districts.
If you want good schools or more living space than a small 2-bedroom place way far from anywhere, the housing numbers cited above are almost certainly way too low...
Excellent detail - we've calculated the required income for the expensive end of common life people would have.
A family a 4, with only a single parent choosing to work, with fully funded retirement savings, while also saving for college, and possibly even saving for a house. Using the estimate above, that came to 150-200k.
So now, 65%+ of households have both parents working.[1] A single parent household is a luxury (without any judgement or whether it should or shouldn't be that way, it is). Assuming dual incomes in the above situation we would half our salary range. Taking the upper end, we'd get 100k as the required livable wage to match this household income.
Even more notably it's a calculation for a household income for a family a 4. Not a livable wage for a single-person which would be the other default expectation besides two parents working. Using the same two sources as previously we can see that the income required for a 1 Adult household is roughly 50% that of a family of 4 single breadwinner household. Take our 150-200k estimate multiple by this roughly ~50% factor, take the more generous end and again we end up with the salary in the range of 100k.
From this the most reasonable number for a livable wage would be ~100k.
> Taking the upper end, we'd get 100k as the required livable wage to match this household income
Unfortunately, not quite. If both parents make 100k, your tax burden is higher than if one of them makes 200k and one 0 (because of the social security cap). More importantly, your child care expenses are much higher. So are your food expenses, typically; this part is correctly reflected in the data table you linked to.
In practice, I would expect a two-earner household, both earning 100k, to have 10-30k/year more in expenses than a one-earner household; the range mostly depends on how much child care costs.
100k for a single person is certainly pretty comfortable, even in San Francisco. Much more comfortable than 200k for a family of 4, I suspect, because housing is so much simpler.
> I would expect a two-earner household, both earning 100k, to have 10-30k/year more in expenses
Ok, so 105k-115k / yr. I think we're reaching the point (in economic terms) where the marginal cost of corrections or adjustment exceeds the marginal value they provide.
The summary though is that 105-115k is not near the 200k originally mentioned and absurdly far from the 300k.
I make roughly 85% of that 200k figure (give-or-take, depending on stock option values), and my house cost $90 per square foot. If you're a good senior level engineer, you can make good money in far more than five or six companies in the world.
200k is mid-level at Google. Senior engineers make significantly more, and staff engineers make a LOT more.
Now if you want a big house, then most of the cities that Google is in are terrible for that. But there's more to life than having a big house, and the expensive metros have their own pros. It's pretty hard to beat Silicon Valley weather, and for what amounts to a giant suburb the restaurant scene is quite good (especially for Asian food).
Well sure. I'm not arguing otherwise. I'm just responding to a comment vaguely asserting that you can't make Google money in a boring corporate job. He wasn't arguing that the city was nicer. He just said, basically, "where else are you doing to get 200-300k". Well, if what you care about is how much of that you get to keep, lots of places.
200k is mid-level at Google. Senior engineers make significantly more, and staff engineers make a LOT more.
Now if you want a big house, then most of the cities that Google is in are terrible for that. But there's more to life than having a big house, and the expensive metros have their own pros. It's pretty hard to beat Silicon Valley weather, and for what amounts to a giant suburb the restaurant scene is quite good (especially for Asian food).
All you're doing is repeating the cult's mantra there. They'd better pay that much if they're going to depend so heavily on dehumanizing you.
It does take a lot of preparation for big-company clueless interviewing processes though. Next we'll see guides for how to ace your kid's application to the top-tier coding preschool.
All this sucking up to big companies with questionable morals and procedures, for the money, is so... 80s. So yuppie, so Reagan-era. Reachin' for the top! Go for it! Eye of the tiger! Pressure, pushing down on you, pushing down on me, no man ask for.
I've worked at Amazon and Google and neither felt terribly cult-ish to me. They do each have their own culture, but that's no different from any other kind of grouping: nation-state, people who go to events for a certain hobby, etc.
I would love to know instead what companies and recruiters have learned from interviewing, hiring, and monitoring the performance of candidates at these companies. What productive trends have you seen in how these companies adjust their hiring process?
It's very difficult to correlate interview performance and job performance. No one is willing to hire at random to really get the data. Companies only look at who got hired.
I've seen some companies decide "well, whiteboard interviews are flawed" and just scrap the whole thing and replace it with something else... that's much more flawed. They're really not being honest with themselves about what the flaws exactly were in whiteboard interviews and if those are fixable, or what the flaws are in their new process.
There's a lot that can be done to make whiteboard coding/algorithms interviews less flawed.
- Be clear with candidates and interviewers about expectations. What knowledge is okay to assume? What knowledge is not?
- Clear guidelines on what makes a good question. A lot of companies ask very simple or well-known questions but think they're emulating what Google does. They are not.
- Good interview training. Most companies don't have interview training, beyond maybe basic legal stuff. Even the companies that do have training typically have crappy training.
- Interviewers should candidates with certain things that might not be obvious. For example, writing examples of input and output on the whiteboard is helpful. Don't rush into coding.
There is A LOT that can be done to make whiteboard interviews more effective.
You can also replace them with some other things, but understand that those will come with their own flaws. Scrapping whiteboard interviews is not going to deliver you some perfect system.
CTCI is a great book. Is it acceptable to ask an interviewer for a different question? For example, if you're really bad at chess algorithms and you get the N-Queens question?
(Context: In chess, a queen can move along a row, column, or diagonal to attack. The N Queens problem is to place N queens on an NxN chessboard such that no two queens can attack each other.)
No one does poorly here because they are "bad at chess algorithms." They might do poorly because they think they're bad at chess algorithms. But this is not a "chess algorithm." It's an algorithm, with a chess skin.
But, sure, if you're bad at chess algorithms, I'll give you this problem: Given an N by N boolean matrix (all falses), set N cells to true such that no two rows, columns, or diagonals have two trues.
Same question, but with a boolean skin. Now will you be able to tackle it?
So, advice: Don't assume you're bad at __ type of algorithm question. For the most part, this isn't true. At most, there's a tiny bit of knowledge to tackle it (so you're not bad at it; you just don't know something). Nearly every time I hear someone say they're bad at some type of question, it's actually just an insecurity. They aren't even missing any knowledge.
The only partial exception here is recursion/dynamic programming, which does have its own little approach.
At a higher level: Can you ask for a new question if you're bad at that type?
You could, but it's risky. If I ask you a question that really involves pointers and you don't understand them, then okay. But realize that this might be a deal breaker for me. I might need that knowledge, or I might be concerned about the tendency to give up.
You're probably better off just voicing something like: "To be honest, I haven't worked much with pointers. I'm happy to give it a shot though, unless you want to move onto a different question."
Do you think it makes sense to ask interviewees to solve backtracking problems, given the fact that its mostly about already knowing the solution?
Someone who's never come in contact with backtracking won't be able to solve n queens "in time", unless they pull a mathematics stunt, but those who do know backtracking won't struggle much.
On a higher level - how much of an "already seen the algorithm" crapshot are tech interviews?
And given the fact that a lot of the problems in your book are trivial in higher level languages (reversing a string is only hard if you somehow don't know how to do pointers) - what's your opinion on trying to use Java on a whiteboard in 2017? Has python officially become synonymous with pseudocode?
It feels like the answer to both your question and mine is to do more problems. I have no issues with backtracking but your complaint mirrors my own with respect to dynamic programming.
That is really, really not true. An unintelligent engineer will never be able to practice enough to do well on a well-conducted interview at Google. (They could in rare cases do well if all their interviewers pick common problem.)
If it's really, really not true then why do you write
>will never be able to practice enough to do well on a well-conducted interview
Or in other words a true scotsman interview.
It sounds like you can't bring yourself to write "on an average interview at Google" because you wish they wouldn't pick common interview problems - but they do.
Based on what you just wrote, I'd certainly work through books of "interview questions". Because your phrasing just proved it works, even though ideologically that is not what candidates "should" do.
No. "On an average interview at google" is not really the same thing. It's not getting at the same point.
Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then that interview process is broken.
If you have a company with a bad implementation of whiteboard coding interviews (for example, who just pull questions out of Cracking the Coding Interview), then it's absolutely true that a bad candidate could pass this process.
This doesn't mean that whiteboard interviews are broken. It means that this company's implementation of whiteboard interviews is broken. There is a difference.
For Google specifically, their implementation is decent, but not ideal. A bad candidate would have low odds of passing an average interview at Google, but those odds are not as low as I'd like.
uh, you're willfully ignoring that we don't care about the company's side of things. In this thread where someone wrote:
>> On a higher level - how much of an "already seen the algorithm" crapshot are tech interviews?
>I'd say it is mostly that. Just do thirty to fifty leetcode medium problems (some of them on pen and paper), and you're good to go.
Nobody in this thread cares if the reason this advice actually (in actual practice) often works is that the interview process is broken.
nobody cares if the reason we get the job is because we exploited a flaw, and we "shouldn't have" done it that way. Basically, where you just wrote,
>Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then that interview process is broken.
you should have written:
>Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then I have to admit, if you strictly want to increase your chances of getting a job, then you can do so by preparing -- but I grit my teeth while saying that, because that interview process is broken.
that would have been honest and matches the reason others had for the above thread! anyway, thanks for the responses.
This question may be a bit too late, but I am curious.
I didn't get through at Google. However, I only asked for 3 weeks to prepare, and I have outside obligations (kids, coaching, that sort of thing). I can easily traverse a binary tree, print all permutations of a set, do DFS and BFS. But I'm not super sharp, especially at a whiteboard.
My review was "not bad, good analysis, but didn't make enough coding progress".
Maybe they were being nice. As I said in another comment, I'm not allowed to know what my scores or reviews were.
FTR, I was a math major, though I did take basic CS algorithms and data structures. So I have a background, but probably further to go than a typical CS major.
I know it will vary by individual, but how many hours, over what period of time, would you say counts as "enough practice" where you might start considering that you probably aren't going to be able to practice enough to do this. Could you ballpark it?
I'd prefer to avoid "unintelligent", but you know, a point at which you'd say, this probably isn't for you, might be time to get some new goals?
I really hope its not. I don't expect any of the people I interview to have ever seen my question or even the particular algorithmic solution before. Good candidates should be able to come up with efficient algorithms to transform and manipulate common data structures on the spot.
When I do interview training, I strongly discourage memoization / dynamic programming question. It's not that you have to have practiced this to do well. It's that you can have an okay candidate who practices it and then does well.
A properly conducted technical interview will not be about having seen the algorithm before. Some companies screw it up though. But then, these same companies also screw up other parts of the interview process.
I disagree that using certain languages makes a lot of problems trivial. Maybe it makes certain problem trivial, but those tend to be easier ones anyway. They're already trivial. And even if your language has a function that performs the exact thing being asked, the interviewer can easily ask you to go implement that thing.
Java is fine for a whiteboard. So is python. So are most languages.
No, python has not become synonymous with pseudocode. I'm not sure what you mean by that.
I have a question about the dynamic programming bit. These questions do not seem intuitive unless its one of the simple ones like, add or multiply the two previous values in the memo (fibonacci) or just check the min or max of the value and modify regarding that. Those patterns can sometimes be easy to spot.
The more difficult ones feel like you absolutely have to have seen the problem before because there is complex relationship between the sub problems that are used to solve by induction/bottom up. How is someone supposed to solve these? Throwing out tons of guesses at the start feels like it'll still end up with me getting shown out before lunch time.
Recursion and memoization is easy but dynamic programming doesn't really feel as natural. Ways to get better? Just do more?
> Recursion and memoization is easy but dynamic programming doesn't really feel as natural. Ways to get better? Just do more?
Once you have the recursive solution, the DP solution should be fairly easy. Draw out the recursion tree for an example (or do it more generally), convert it to a DAG by combining redundant nodes, and then do a topological sort. That topological sort is the order in which you need to solve the subproblems to get a DP solution.
If you want to flip a memoization problem into bottom-up dynamic programming:
1. Make sure you really understand the memoization approach
2. Look at the base cases. What are the very last things the recursive approach does?
3. Build up the next case from the base case.
4. Repeat
I actually solved this problem in my first year of learning programming; mind you I didn't have a computer, only (limited)access to it at college. The only thing I knew was recursion. I thought hard for 3 days, discovered (he hee..) back-tracking and found the solution. For some reason, I cannot solve interview puzzles, and tried and failed to get into Google and FB.
If you do not get an offer even though you did well on technical questions(verified your answers online). Should you reapply? Why? Personally I think it's the sign of cultural fit issue so they unlikely will like you just because you reapply.
When people don't get an offer after a technical interview, it's typically about technical skills. Even if they think they did well because they verified their answers online. Often, although their answers were eventually correct, it took them too long to get there which the interviewer perceived as weak problem solving skills.
>Often, although their answers were eventually correct, it took them too long to get there which the interviewer perceived as weak problem solving skills.
Does CtCI publish those times for its problems? How would I know I'm too slow to solve a kind of problem before interviewing?
No, I don't publish times. The times are misleading. Interviewers often give hints, which can dramatically change the amount of time. And there can be random factors that change the amount of time.
Has any link ever been determined between interview performance and on-the-job performance?
I've made the clichéd mistake mentioned in the article of wearing a suit to an interview where it wasn't appropriate, and felt the air being sucked out of the room. It strikes me as silly, but what do I know. Do these hiring practices result in a good cohort of employees?
they aren't necessarily trying to get the best employees but instead trying to avoid bad or mediocre ones. This means they sacrifice rejecting some smart people in order to gaurantee they don't have terrible people.
Google kinda-sorta did a study there, but the "study" was fundamentally flawed. They didn't look at people who didn't get hired (duh). So the most they could say is that once people passed the (very high, risk averse) bar, how much better they did beyond that didn't have a strong enough correlation with job performance to come out in a small-ish number of data points.
To me, it's pretty clear that it basically works. It's not perfect, but yeah it works. Virtually every top tech company has been formed with these practices. So unless you're going to argue that they could have hired random engineers, then it must be effective. (Note: this doesn't mean that it's perfect or that there's nothing better or that you can't improve it. It just means that it is reasonably effective.)
I [author here] have done a lot of interviews with companies going through acquisitions, and that's offered me the ability to talk with their managers about their job performance. My assessment is generally accurate. There are certain things it can't evaluate, of course, like work ethic. Work ethic is really important, but there's really no way to interview for that.
True but it should be possible to compare interview performance to job. I presume that interview performances for successful hires aren't uniform, and that on-the-job performances aren't uniform either. If these interview practices make sense then presumably there should be a clear correlation between the two.
That's not necessarily the case. People who are in shape have longer lifespans than those who aren't (probably true; let's assume it is). But if you look at just people who complete a marathon in under 3 hours, you might not see a correlation between their speed and their eventual lifespan. At some point, people might be "good enough" where beyond that is rapidly diminishing returns.
To complicate this:
1. The range in interview performance of those who get hired is really quite narrow. So you're looking at essentially the difference between a 3.1 and a 3.3. You'll need a lot of data to try to find a correlation there.
2. The interview score is only a partial measure of interview performance. If you and I experience an identical interview, I might call that a 3.1 and you might call that a 3.3. The hiring committee will hopefully understand that a 3.1 from me and a 3.3 from you are equally good (because they see our past scores). This makes the data on interview performance harder to correlate, and you'll need even more data. (Or you'll need to properly take this into account in the study, by looking at a score relative to that interviewer's past scores, rather than the score itself.)
3. The above point, repeated for job performance.
4. When a candidate with a lower score gets hired, they're more likely to have something that offset these lower scores. The data set of people who got hired is {people who interviewed well and had good or bad resumes} + {people who interviewed just okay but had great resumes}. When you correlate interview performance of those who got hired, the lower scores in there are skewed towards people with great resumes (or referrals or something else). That factor could totally eliminate any correlation between interview scores and job performance.
While you sort of have a point there, there's much more diversity in how people dress at top tech companies than in, say, finance or consulting.
It seems you're upset that an interviewer might be biased against someone for wearing a suit. That's fair; an interviewer shouldn't be biased against a candidate for that. And most wouldn't be -- consciously. But subconscious bias can still exist. It's best to skip the suit, particularly for a very technical role.
While maybe it shouldn't be a strong signal, showing up for a tech interview in a suit is kinda like showing up for a finance interview in jeans and a hoodie. It indicates that you're either unfamiliar with the norms in the industry, or deliberately violating them, and neither one is a great first impression.
The typical interview rule, IMO, is a good one, at least for men (women unfortunately are under very different pressures): dress one (and only one) notch nicer than the people that will be interviewing you.
There are some rare situations where dressing down compared to your interviewer makes sense, but a normal dev looking for a normal job is not one of them.
Dressing abnormally does indeed show that you're either unfamiliar with the norms in the industry or you're deliberately violating them. Agreed.
In finance, it's sort of reasonable to be concerned about someone showing up in jeans and a hoodie. A good part of that job is being able be "corporate" and adhere to norms. If you can't do that in an interview, how will you treat clients?
In a coding interview for a tech company, adherence to social norms is not particularly valued. Weirdness is tolerated, if not celebrated. So it's fairly different.
Tech companies would be far more accepting of a candidate in a suit than a finance firm would be of a candidate in jeans and hoodie. That said... it's still probably best to just avoid the risk of subconscious (or even unfair conscious) bias.
Unless you're interviewing at companies where everyone is expected to at least interview in a suit. I don't know what other areas are like but in Boston it seems like a 60/40 chance as to whether they want you to be dressed casual/business casual for the interview or want you in a full suit. its not just non technical people who expect the suit at time either. I've met at least a handful of engineers here who have said they straight up wont hire anyone who doesn't wear a suit to the interview.
You're being downvoted because you're phrasing is a bit harsh, but I think you have a valid point.
It's so hard as an interviewer not to have a higher score for candidates who look, act, and dress like me. I've caught myself doing it before and I hate it.
And the worst part is that most people (myself included) have a very hard time even recognizing they are doing it, which means it actually is good advice. Play on the interviewers unconscious biases. Ugh.
Sure- you don't need a CS degree but you need to know everything taught in a CS degree regardless of it's applicability to real life software engineering.
You should also buy a bunch of books and section off at least a month to study. Especially if you don't have that CS degree. Hope you don't have other obligations like a family or a demanding job.
You should also spend a lot of time doing coding competitions even if you find them boring because you got into engineering to build cool stuff- not solve abstract academic CS puzzles.
You should also practice coding in environments you would never use in real life - like a whiteboard. Every single software engineer on the planet codes in an IDE - but no, you will use a whiteboard. You haven't written by hand since like 2012 so your writing looks like a child's scrawl - better work on that too.
1. No, you don't need to know everything. You should know what a binary search tree. You probably don't need to know the specifics of how to balance one. (If a company is requiring this knowledge, then they're doing whiteboard interviews poorly [or they have a very specific skillset needed]. That's not a problem with whiteboard interviews. That's a problem with that company or interviewer. It takes maybe a week for a smart developer without a CS degree to learn what they need to know for interviews.
2. Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
3. Sure, you could do coding competitions if you want. That'll replace existing interview studying; you wrote your comment as though that's in addition to your previous studying.
4. Yes, true, you don't code on a whiteboard. But this, in and of itself, doesn't matter. What matters is whether whiteboards are more predictive and whether they create a better candidate experience. So, forget whether it's "real world" or not. Whiteboards have their pros and cons. The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.
5. You do not need to work on your handwriting. I've done A LOT of interviews and this has almost never been an issue.
>> You should know what a binary search tree. You probably don't need to know the specifics of how to balance one.
Fine. I know this. Will this give me a job? No
>> Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
No. I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.
If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?
>> The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.
I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?
The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.
Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?
I think we just have to accept that the skills needed to pass an interview are a largely different set than the skills needed to excel at the job. One set of skills is not more important than the other, because you need both of them to get and keep the job. We need to practice both.
>> >> You should know what a binary search tree. You probably don't need to know the specifics of how to balance one.
>> Fine. I know this. Will this give me a job? No
No, it's not about knowledge. The knowledge is needed. But the real focus of the interview is to tackle a new question.
>> I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.
The ability to see a hard problem, develop your own solution, and translate that into code is a pretty important skill. "Skills" aren't all about knowledge.
>> If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?
It's a lot more complex than that. It's not just about filtering out bullshitters. It's also about identifying people who might be great but are unable to talk about it. There's also lots of people who are great but haven't been in situations where they've been able to tackle interesting challenges (either because of their inexperience, industry, etc). Or maybe they've done interesting things but in a related field that's hard to understand.
But, yes, absolutely train interviewers. (Interviewing isn't really a role that you hire for.) But even a very well trained interviewer might not be very effective in a behavioral interview.
>> I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?
There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.
>> The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.
The interviewer is comparing the candidate to other candidates on the same question. So if a candidate is getting rejected because they failed to be perfect, this means that other candidates are actually getting to perfect solutions. Or you just have a crappy interviewer who rejects everyone.
>> Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?
If that's the case? Nothing. But that's not the case.
Thank you for being so patient with some of the more ornery commenters in this thread. It's a topic that cuts close to the bone for many HN users, that frequently brings up feelings of rejection (e.g. elitism of 'top' companies) and frustration (e.g. irrationality of interview processes).
>> No, it's not about knowledge. The knowledge is needed. But the real focus of the interview is to tackle a new question.
>>There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.
This would be great if interviewers actually knew that this is what they are testing for.
At this point, interviewers simply do not care about candidates and make a binary Yes/No decision based on "Did the candidate solve the problem perfectly in 15 mins".
I'm not opposed to coding interviews. But I am opposed to the way it is implemented right now.
The current way would be great if I get to reverse interview the interviewer and prove that I know (knowledge plus problem solving ability) better than the interviewer.
If the interviewer accepts this duel, it's fair game. Otherwise, the current process simply selects the people who can pattern match.
Yeah, there are interviewers who run crappy interviews. This is not a flaw with the process though. If you replace it with any other process, you're going to find interviewers who are bad at evaluating those too.
For example, on behavioral questions, I've seen many interviewers who think "In our company, we handle disagreements by doing X. So I'm going to see if, when the candidate hit a disagreement, they handled it by doing this."
But that's not right. Different companies have different cultures. What's good in one company is bad in another. This is essentially the behavioral question equivalent of the scenario you're raising for technical interviews -- people looking for particular answers, and not considering the context.
Both situations definitely happen, and are very common. But that doesn't make the process flawed. It generally means that the company hasn't adequately trained interviewers.
The process is flawed in that it ignores the ability of interviewers to rate a candidate.
The solution to this is not getting rid of coding interviews because (coding sure is important). But the overweight emphasis on coding as a binary decision-making is ridiculous.
There are actual engineers doing actual real coding work. They don't necessarily spend entire days solving puzzles. So, they won't do as well in whiteboard puzzle test as a person who slacked off at their last job/college and just practiced whiteboard coding problems.
But the current interview process would rather select the slacker practicing problems than the effective person who didn't practice problems.
Why?
Because the process is overweight on whiteboard coding accuracy and the interviewers who are incapable of identifying a person's ability outside of that overweighted process.
This is a flawed process. And improvements in the process is impeded by flawed interviewers.
I'm glad you asked. I have been following your blogs and posts about interviews and this is the first time I've seen you ask this. This is positive.
I do not have a bulletproof solution but I have implemented processes in my current company that filters in better (measured by performance at work) people than the average prolific whiteboard coder.
1. First, we need to establish the goal of the interview. What is our goal? The goal is find passionate candidates who will get the job done or are bring something that makes our team better. Golden State Warriors will not succeed with a team full of Stephen Curry clones. Note that the goal is not to reduce false positives but to hire the best we can get.
2. With that established, we need to interview candidates and see if they are worth it.
a. If they are a generic software engineer/just out of college/did not get a chance to do something interesting yet, what else besides coding (verified by 2 rounds of non-perfect whiteboarding) did the candidate do? We need to identify if they have the potential to do more than just toy coding
b. If they have been doing something interesting at previous jobs/school they obviously did not have time to practice coding. What were they doing? Is it relevant? Is it it interesting?
With point b, I know you get paranoid about bullshitters. I agree. Candidate b can be a bullshit artist. But, if the interviewer is capable enough, the bullshittery will not go anywhere.
In the light of above, I designed the following:
1. 2 coding rounds. It's ok to miss edge cases as long as you can fix them when prompted with decisive hints. The coding problems are also relevant problems from our own codebases and not just some random "skyline problem" ----Note that the emphasis is "OK on missing edge cases". We just want to know if you can solve the problem and translate it to code
2. System design round: Yes even for college grads. For college grads ask something trivial like how would you design a simple diff or a simple OS/compiler. For experienced folks, ask something relevant to their background. This works really well because it shows which college grad really likes exploring the world of "practical" computer science. This also helps remove bullshitters to a large extent. Linkedin has a great round on this and I really enjoyed that round because it got me engaged with the team and they got to explore sides of me that even I didn't know existed
3. "What is special about you round"...We want to know what makes you special (to compensate for lack of toy coding). Here is where a lot of really good engineers would shine. Some will give examples of a compiler they wrote, or how they applied machine learning to an unconventional area. There are no right answers and it quickly becomes a technical discussion. This round shows how insightful the candidate is. Also, this has to be a full round.....not just a 5 min chit chat thing. The reason for it to be a full round is that bullshitters run out of stuff to talk about whereas the person who was engaged in such projects will go far.
Granted some engineers aren't good communicators but whiteboarding is a bad proxy for hiding bad communication skills since a lot of engineering IS good communication. This same candidate will get hired, write design docs, give presentations, explain concepts, mentor interns, interview other candidates. This person better be a good communicator.
Is this process perfect? No. But neither is the current overweighted whiteboarding
So what does this process do? It tried to "select" the better candidate as opposed to trying to "reject" everyone in the name of reducing false positives.
Not perfect but gets me the candidates who are not just whiteboarders.
I am in the bay area and I'll be happy to walk through this process in great detail if necessary.
Oh, no, I've asked people their suggestions all the time. I typically do in these discussions. In fact, I have elsewhere in this comment section.
I'm sort of confused by your suggestions. You've decided a pretty typical interview process. What do you think is different here?
The biggest difference I can see is that you're saying that the coding is "a relevant problem from our codebase and now just some random skyline problem." And you're specifically calling out that it's okay to miss codebases.
Let's discuss these more.
1. It shouldn't really matter whether it's from your codebase, right? If you have two problems, ProblemA from the codebase (which is worse at identifying relevant skills) and ProblemB (which is a toy problem but better at identifying relevant skills), you should go with ProblemB, right? So really the question is to find a problem that is a good predictor of whatever you care about. If I really want to hire people who are smart, is giving them a challenging problem a good predictor? I think so.
2. The edge case thing. You seem very fixated on this. Why? What interviewers should be thinking about is the signal that the candidate sends. Certainly, in many cases, missing an edge case is just a "I haven't written in certain details yet." In some cases, missing an edge case (particularly after a conversation) might actually signal a much bigger issue. For example, imagine someone didn't write in a check to see if an array is sorted. No big deal, probably. But if there's a repeated issue with this and they can't write this code correctly, then we have an issue. Saying that edge cases don't matter is too broad. They might, and they might not. I encourage interviewers to focus on the signal and not on some binary rule.
>1. No, you don't need to know everything... It takes maybe a week for a smart developer without a CS degree to learn what they need to know for interviews.
I just went through a job search and I didn't find this true. I think the bar on algorithms has raised significantly with all of the prep material (your book included) out there. I also found a few companies sent over "things to study" lists that were quite comprehensive and included topics that you might not even see in undergrad.
>I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
I would estimate a reasonable amount of prep time prior to interviewing is around 100 hours. This is about how long it will take to brush up on and practice data structures, algorithms, DP, design questions, etc.
This is about 2 months of practicing an average of 2 hours a day, meaning you're still working a day job. I figure this could be reduced to 1 month of prep if you quit your job and have all day to yourself.
Usually these are people who are coming from companies with similar interview styles and are passively prepped by being an interviewer (for algos/coding) and designing web-scale systems (for system design).
See when I read that strip interview doc I don't worry at all. Thats what I do everyday as a software developer. I would love to demonstrate my ability in interviews like this.
These Cracking the Code interview type interviews are not even close to what we do all day everyday.
1. Obviously you have a lot of experience with this but I just can't see how a week is anywhere near accurate judging from my personal experience, people I've talked to, and the numerous online stories of people's personal experience.
2. To be honest, I kinda expect the world's smartest organization to find some better way to evaluate talent. Google says they want diversity but requiring a month of studying isn't going to allow for that. Their process throws false negatives on a lot of people that could otherwise be great engineers (which they largely admit to).
3. They're different skillsets. One is about knowledge, the other is about memorizing question patterns and practicing implementation speed.
4. Does Google have data here? My experience is a lot of candidates shut down at a whiteboard because it's not what they're used to. They're afraid to experiment or erase or use longer clearer variable names because of the overhead of writing by hand and dealing with space limitations. It's a lot more messy to modify and extend code you've already written.
5. No, it's really bad. It's like this weird mix of cursive and small caps. I'm not sure what is going on.
1. I'm not saying a week for preparation overall. I'm saying a week to learn the core algorithms and data structures. Frankly, that's probably much less than a week to learn.
2. Yes, it does throw false negatives. Lots of people get hired with zero prep though. I generally say about a month of prep is where you hit pretty low returns from more time. But that doesn't mean that you need a month, or that a month will be enough for you.
3. I'm not sure I see your point here. If coding interview prep is a different skillset from coding competitions, then you wouldn't do coding competitions in order to do interview preps.
4. Some candidates do shut down because it's not what they're used to. A well-trained interviewer can mitigate this to some extent (not always). Some people will also shut down at a computer. Interviews will always be an artificial environment and people can freeze for different reasons. At a whiteboard, it's often a bit easier to unfreeze candidates and create a happier atmosphere.
5. That's fine. As long as I can read it, it's fine. And if I can't read it and you can explain to me what you're writing, that will also work.
bad handwriting is not an issue for interviewer, but it could seriously ruin thought process for interviewee. It ruins it for me, for sure. You know, good code "looks" good, and that feeling is a major part of programmer intuition for me.
I do not go home and code after work. I have plenty of hobbies that all pertain to the outdoors (I call it balance). This quote stuck out:
"I once worked with a candidate at Google for a product manager role. During the interview, he started talking about how he kept chickens at his house. He was extremely passionate about building a door for the coop that opened and closed automatically to control the chickens’ movements."
I have 30 tomato plants in my backyard and was planning some automation around watering. It doesn't always have to include a raspberry pi even though I may go that route to monitor soil temp in the hot months. Filling up a 5 gallon bucket with water and compost and the right size holes can slowly drip feed the plants.
For chickens specifically, you need to open and close the door to their coop every day. and make sure that they're all accounted for, or else they get eaten.
This is good general, high-level advice and following it might slightly increase your chances, but I'm more and more convinced every day (and every jumped-through hoop) that there is no formula--and that random chance plays the biggest role in how far you get through any given interview funnel. The solution, if that's true, is just to make it a probability problem. Get as many interviews as you can, and as long as your chance of getting the job is greater than zero, you'll eventually get one. In tech, especially at these big companies, it's an employer's market, and they have the luxury of being able to be as picky as they want: There's always a line out the door of qualified candidates.
The advice about not giving up on the question is a key point here.
In a previous startup I would ask a single question that would take 30+ minutes to work through with no expectations that the candidate would ever get the answer. All I cared about was if you could break down the issues and move from one set to the next in trying to narrow down the issue. I cared deeply about how you thought, not what you knew.
As people will ask the question had to do with networking and knowning the answer required you to have seen the issue before and have had delved deep enough into the Ethernet HW specs to get the answer.
Narrowing down the question just required you to think and be able to problem sovle. At the end most people never figured it out but those that spent the time attacking the problem and asking question are the ones that got job offers.
"I would ask a single question that would take 30+ minutes to work through with no expectations that the candidate would ever get the answer"
The major problem with this approach (unless you are very vocal about the process up front, but still even then I see it as problematic) is it flies in the face of most education at all levels, at least as practiced in the USA. This is fundamentally more an issue with the way we scale teaching up to so many students than the way interviews are handled, but it still ends up being a huge problem.
People are expected to spend 20-ish-plus-something years learning and being tested in a pass/fail know-it-or-you-don't manner, and then are thrown into an interview environment that is entirely different.
It isn't surprising that a lot of people who would be perfectly capable of doing the work get completely hung up on the fact that they are wading through the muck of topics they don't fully understand in this environment that otherwise feels so very much like a school test of knowledge... especially when you throw in the types of social awkwardness and fear of public failure that are relatively common among engineers.
So I was always very upfront. I told the candidate you will not get the answer. This is so I can see how you attack a problem. All the candiates had a background in the subject matter. The question was not on subject matter not tied to the job.
Overall we found that the way we did this interview was very successful. At one point we lowered the bar and hired a bunch of people and they were all gone in six months. On the other hand those that were successful with the question and problem solving did very well and are still there. Some of those people I hired even work with me at my new startup.
Also I do not understand your comment about schools. Yes in K-12 there is a lot of pass / fail but when you go to collage for an STEM major there is a huge amount of critical thinking and problem solving involved. The interview was designed to test that, not if you memorized how to do a binary tree search.
> I told the candidate you will not get the answer.
I've had an interview like this. I said thanks for your time and walked out of the room.
I'm not interested in being told problems are unsolvable, I'd really prefer to work in a culture where there's a growth mindset. Even if a problem is unsolvable for me right this second, would it not be within the realm of possibility that with more experience or with a couple other folks on a team, I could solve it?
The problem is not unsolvable. I though I said that. Sorry if that was unclear. It is just caused by something that is pretty far down the stack which you would never know unless you have seen the problem. Once again this was to see if the canidate could think and problem solve. How far down could they get? Could the test for the obvious answers? If they got stuck and I gave them a hint could they go further? I would look at the background on the resume and say "so you worked with X before right? So you know how X has this behavior? This is similar."
There was no arrogance here. The job REQUIRED critical thinking and problem solving. Resumes are generally bullshit and unless you have a major public project were this quality is shown how do you test for it?
The first person hired that had to answer this question and go through this was me! I did not get the answer. I got real close. I am really glad I did. That job changed my life. The quality of the people I got to work with was a complete eye opener and made me want to do better and frankly made me a better person.
The company is now worth ~11B or so and everyone made money and was well cared for.
The question isn't posed as unsolvable. It's posed as being unreasonable to solve fully in a short interview setting. If someone gives you a challenge and says "I don't expect you to solve this in an hour" and your reaction is to walk out, then you're unprofessional and likely the interviewer will be happy you left.
"but when you go to collage for an STEM major there is a huge amount of critical thinking and problem solving involved"
I went to college for Computer Science and while there was definitely a larger emphasis on critical thinking and problem solving than in high school and grade school, testing (which is the closest to a school equivalent of a job interview) still ultimately boiled down to you either knew the material or you didn't. Testing was virtually never a conversational process during which I could ask the test-giver questions to get at a deeper problem, so ultimately very different than these problems where the "tester" (interviewer) doesn't actually expect you to solve them during the "test" (interview).
My comment has nothing to do with the experience of LEARNING in USA schools, but rather TESTING in USA schools (including the vast majority of colleges).
FWIW, as an older experienced software developer, I now excel at the sorts of interviews you are describing, but when I was younger and just out of school I was terrible at them. Not because I was a bad developer/engineer then, but because of anxiety brought about by being asked about things outside of my element, because I just wasn't used to that having never been exposed to it. Based on lots of conversations with other developers, I don't believe I am alone in having this difficult adjustment period with software interviews when first starting out as a professional.
It is entirely possible your process works fine for you because it is one of those ones that is unlikely to produce false positives, which is what you really want to avoid, but I'm sure there are also some great developers just out of school that totally bomb your interviews while they would do great at the actual job.
If you can hire enough developers to allow those false negatives to exist, then that's no skin off your back but your anecdote about having to lower standards suggests that maybe passing on some of those false negatives was detrimental at least in the short term.
So the I had a conversation once with the VP of Engineering about this. He an amazing brilliant and kind man. I basically asked if we could maybe hire top 2% vs the goal of top 1%. His answer was pretty interesting (and I am paraphrasing). "I can ID the top 1%, I do not know how to tell the deference between the reset (i.e. is this person top 2% or top 10%)."
We did have this issue of hiring quickly and as I said at one point we lowered the standards. No one from that group made it. We did try. The cost of those hiring mistakes on productivity and just time spent was not worth it even if it meant we passed over someone that was a good candidate.
To be fair it was not all cut and dry. Personal recommendation for someone you worked with in the past could override any interview issues. There was atleast one time where a candidate was rejected by some of the interviews but got hired anyway because of this and it worked out.
When I started the team was around 30 and made of veterans of various other companies. We all hand family and the goal was to get it done right and go home. Age ranges were around 30-40 for the core team at this point. We recruited heavily from collages at BS/MS level and within a year the the number of people in their 20s was more then the older folks, so we must have been able to do something right for the new kids in their interviews.
The one thing I have learned is hiring is so much an art as well as a scince. Google has published tons of data on their hiring practices and success rates. Most of it is pretty interesting as the things you think matter turn out not to be the key to those that were successful.
I wonder how much diversity initiatives impact evaluations, if at all?
I know Google is establishing pipelines with HBCUs like Howard and FB is looking into something similar I’ve heard. I also noticed a lot of Google engineers teaching MOOCs happen to be women, which is a great way to be inclusive to people looking to learn.
Just how far does that go in the actual interview evaluation (ie: Do feedback reports have any identifying information, are behavioral or design questions examined for cultural bias etc)
And? By your logic, if a lawyer tells you how to handle some unfair legal situation, that lawyer is doing something wrong. Because they're profiting from something that's unfair.
When I talk to a company about how to create a hiring process, I tell them that it's unfair to expect side projects from candidates.
>> And? By your logic, if a lawyer tells you how to handle some unfair legal situation, that lawyer is doing something wrong. Because they're profiting from something that's unfair.
Just putting things into perspective for OP. It's like Trump campaign has ties to Russia. Doesn't incriminate him automatically but I would be wary of vested interests. Just an additional data point
It's frankly not true, though. I don't bother with side projects beyond having a simple personal website. I've worked at startups that have been acquired as well as the largest/most reputable employers in the valley.
Personal projects are there to help compensate for an otherwise lacking resume. If your resume is on point, you don't need them.
Nine-to-five is absolutely enough if you are a diligent and thoughtful engineer.
You probably should have read what it was saying then: that the companies hope to see you've done something outside of work, like participated in meetups, or worked on a side project.
For starters, thank you for hanging out on this thread and answering questions.
Where it comes to taking a realistic look at what it takes to be hired at google and other top tech companies, I think you give great practical advice. I've recommended your book and blog to a lot of people.
Here's a summary of something I've posted elsewhere on HN, and I'd be interested in your angle.
It goes, essentially, like this. Technical interviews are, for all practical purposes, difficult oral whiteboard exams. Lots of professions and academic programs involve some kind of exam
Here's my objection to it - software tech interviews are essentially exams taken with little to none of the courtesies often extended to exam takers in other areas. I'm hand waiving a little here, but I do think that there is a kind of bill of rights that evolved between people who give important entrance or exit exams, and the people who take them.
For example, in the world of law, you (generally) must take the LSAT, and a lot is riding on this test. However, you get to see and review old tests. You have assurances that the test will be consistent across other candidates. You get a copy of the test you have taken, which will eventually become available to the public. You eventually get to know what your score is, and why.
On your way out, you take the bar exam. People study a very long time for this, and many consider it to be one of the more stressful events of their life. But again, there is information about who is grading it, what it contains, what your score is, and why. And lastly, it leads to a permanent, lasting credential respected in the industry.
This is just one example, but they often come up in other areas. My opinion is that one of the reasons we have technical exam/interviews is that our industry lacks a common and respected credential and exam.
I'm not necessarily arguing that there should be one, but keep in mind how stressful and how much prep these exams do take. An interview at google can take place over several days, and may involve relatively grueling technical tests, all at a whiteboard, where people worry about freezing up.
You don't get to know who is testing you, what their credentials are, and whether they are doing this consistently and fairly. You aren't even allowed to discuss the questions - you're typically under a nondisclosure agreement! You don't get to know how you did - my understanding is that google has numerical scores and screen captures of what I wrote on the whiteboard, but I'm not allowed to see them or know what they are. I understand much of this is motivated by fear or lawsuits, but all we really get, after waiting, is a "we've decided not to pursue your application further at this point". And if you do pass, it's known only to that company - if you interview elsewhere, you can't take these results with you. You have to start over again.
I really do think it's just too opaque, and the secrecy is just too conducive to abuse and bias. And keep in mind, this is in an industry/job category that both complains about a shortage of candidates and is not impressive where it comes to diversity in many job categories. Could the opacity and capriciousness of exams be playing a role in this?
Again, I'm not against exams like the ones google does at the whiteboard, I see the value. But I think that the conditions under which we take these exams are a problem.
If you're willing to offer your thoughts and take on this, I'd be interested in hearing them. Thanks!
I'm not really suggesting much here, more identifying a problem. I suppose my question is, what are some of those bad interview processes, especially around the coding interview, and how would you address them?
To an extent, you have addressed this - in a blog post a while back, you reviewed some of the other options available to a compnay trying to hire, and none are especially compelling or necessarily preferable to a technical exam style interview.
Here's the thing, I'm not just focused on clear abuses - bad question, bullying attitudes. I think even when "well administered", the customary technical exam interview itself may in fact be harming the industry pretty seriously. I think it's sufficiently onerous, intimidating, and secretive that it may be driving away talent. And I wouldn't be at all surprised that it tends to drive away people don't feel they fit the mould of the standard Silicon Valley developer at an even higher rate.
I'm starting to think this is one of those tragedy of the commons situations where each individual firm is acting in its best interest by giving candidates rigorous technical interviews - but the aggregate result ends up deterring people form applying for new jobs or perhaps entering the field at all. Imagine if you had to re-take the bar exam every time you changed law firms - and the only people who would know if you passed was the firm evaluating your candidacy! In many ways, that's what we have in software.
I really do think this drives a lot of potentially good people from our field. You mentioned in a blog post that taking time off to do take-home projects may deter women and others with family obligations from applying - but surely that's true as well for taking off 100 hours to study for a tech exam, especially one you might not pass (and if you don't, will never know why). People might be more willing to do this if there was a better process that ensured a fair hearing, a less secretive, more transparent approach to technical exam interviews.
So, I'm asking, maybe we should step back for a moment and ask how we, as people who buy into and use technical exams, may be collectively contributing to a real malaise in our industry, one that creates a problem for everyone, but perhaps an unusually severe one for the very people we're saying we are trying to get into this field.
I certainly don't think that there's an easy answer here, but perhaps our industry really does need some outside regulation, or a professional association that can administer exams in a way that is more transparent, or at least some set of standards. For instance, perhaps non-disclosures that might mask discrimination should be illegal or unenforceable? Perhaps companies and others should be obligated to share scores and reviews of candidate coding exams?
I can see big problems there as well. But I really do think the situation, as it stands, is pretty bad.
How similar are the Odds for a BS of CS graduate in 2017 in the United States to work in Silicon Valley versus the NCAA Division I Basketball Player to make it in the NBA?
Point being, this is extremely helpful batch of advice for a surprisingly narrow field. Take out the SV / Top 1% Tech Culture nuances and there's really not a lot to work with other than "Put your best foot forward, be flexible to learn new things if asked, and and to keep trying.
But, something like this just stings to me:
>Either way, it’s very common for a candidate to get rejected and then get an offer a year or two later.
It's extremely uncommon for the common person to get anywhere near that kind of follow up. Well, maybe for half the expected salary, but no, this doesn't happen to anybody I know outside of SV or Top 1% C-Suite management. The advice within should be constrained to its applicable universe, with cautions that viewing the process in other fields through this lens could actually be detrimental.
Quantifying assumes that the "raters" remain the same forever.
That's not true though. Raters change. Counter intuitively, as the rater gets more and more familiar with the problem, the less tolerance they have for mistakes.
I don't know where you got the idea that quantifying something fixes it for all time. Numbers can change. Measures of inter-rater reliability can be applied to multiple ratings over time by the same person (why not?)
OP here. I do absolutely realize that candidates getting rejected and then getting an offer later is... well, not quite the "definition", but it's a sign of a flaws in a process. However, you don't scrap the buggy program if there's not a better replacement; you fix the program. And you don't scrap this interview process if the alternatives are even worse.
What an absurd chain of logic. Interviews are conducted by human beings, of course their reproducibility sucks. It's not like non-whiteboarding interviews are going to be great at that either.
You could replace them by software, ala "here's a coding problem with a set of unit and integration tests to validate against, go nuts" but that would have its own set of issues.
I think the tide is turning against this type of interview. The industry is starting to notice how ineffective and stupid it is. The big tech companies don't care and will continue doing it because success hides all failures. When there's an endless flood of new college graduate applicants and near-monopoly positions bringing in billions of revenue, nobody cares if the hiring process is bad. They will care someday if those conditions change, but that's in the distant future. Most companies, especially small companies, are not in this position and should have a different process. It's becoming a competitive advantage, especially if a company is looking for senior engineers, to not have this interview process. For example, this list of companies was in a HN story a while ago: https://github.com/poteto/hiring-without-whiteboards The trend is growing and it's a good change for the industry.
> The big tech companies don't care and will continue doing it because success hides all failures.
Cynically, big companies might be selecting for a modicum of technical expertise along with demonstrating capacity to put up with some amount of abusive processes.
They have no impetus to change and think they have found an approach that works reasonable well when given absurd amounts of candidates. The problem is the ridiculous false negative rate.
I struggle to think of a better system. Using experience is unfair and misleading (I worked on awesome stuff... that I can tell you nothing about, I was totally the lead on this software, etc...). Take home stuff is fine with me but a lot of people have problems with it. Come in for a week type things are incredibly applicant hostile.
Sit down with me and lets try and fix this bug seems like a more reasonable solution to me. Dev environment has the most popular ide with their devs, the source for whatever, internet access, and a bug report. Person is there to assist.
I'd say homeworks are good. In a mix. In fact the process should be a mix. It's hard to get it right.
You can find out if they copied the homework. There should always be a follow up talk about it. They should be able to explain the details.
Ask them to add some functionality on spot. They have working code they should be familiar with and you can work with that. You want to test their approach more than anything.
Bug fixing is hard to prepare right. If you have a clear cut position, that would eliminate part of bullet one.
I'd argue that they should not be asked to find the bug. Only to fix it.
It does not have to be the same bug. I'd argue for a pool they can pick from.
You should be testing how they approach the problem and how they go about solving it. Heck you can debug and outline a fix for something you can't really code yourself, I know I did.
Pairing with someone from the team they'd work in could also provide some insight into the dynamic.
I agree there is no cookie cutter way for homework and bugs. That's why they need to be in a mix, well prepared and supervised.
> You can find out if they copied the homework. There should always be a follow up talk about it. They should be able to explain the details.
That doesn't prove anything. They could've hired a senior developer to help them and explain the concepts in detail.
> Ask them to add some functionality on spot. They have working code they should be familiar with and you can work with that. You want to test their approach more than anything.
The more 'real-world' this is, the more it's going to be biased towards those that have experience in a particular stack or with a particular type of development. Which isn't necessarily bad (it could even be good!), but it may be if you want a more agnostic interviewing process.
- The more 'real-world' this is, the more it's going to be biased towards those that have experience in a particular stack or with a particular type of development. Which isn't necessarily bad (it could even be good!), but it may be if you want a more agnostic interviewing process.
I fail to see how that particular point can ever be a negative for the hiring company. If you need specific skills, you dictate the environment. If you want the candidate to bring his own skills, just tell him to pick his favourite tools.
What I always prefer is a take-home programming assignment, and then present the results to a group of developers. You need to be able to explain what you did and answer questions about it.
Yes, the only people/entities who benefit from this type of interview process are the interviewers themselves - They make the interview process needlessly complicated and pointless and then they sell books giving people specific instructions on how to pass those interviews.
They create the problem and then they sell the solution as a book or as an online self-help service.
All the interview questions do is demonstrate that you've bought the right books and spent hundreds of hours practising pointless coding problems because you have nothing better to do with your time.
Maybe big companies are only interested in hiring sheep-like engineers who have no side projects to maintain and no sense of pragmatism... But then why do such companies (e.g. Google) spend hundreds of millions of dollars each year on acquiring startups full of ambitious and pragmatic (wolf-like) people (people who probably would not pass the technical tests).
With that strategy, over time, big companies just end up with a few pragmatic wolves at the top and a large number of sheep at the bottom. It's not natural and they're missing out on more balanced candidates who are neither sheep nor wolf but are excellent engineers nonetheless.
Whiteboard interviews are more complicated? The alternatives that people usually put forward are either "code a homework assignment for a few days", "pair program with a full timer for a few hours", or "look closely at the applicant's open source contributions".
All of those sound complicated as hell. Perhaps they are more effective, but the whiteboard interview is obviously less complex. And the side projects approach is just laughable considering the number of excellent engineers I know who don't have time or interest to code on their off hours.
A good whiteboard question can tell you a bit about the applicant's design sense, knowledge of key data structures and algorithms, and ability to integrate new ideas. It also has the nice benefit of quickly weeding out the huge number of applicants who can't write DFS on a tree.
The tide is definitely turning. The last two interviews I've been to were nice casual affairs where me and another engineer had a technical discussion on how I'd approach various problems they were having (such as deployment woes, wonky network topology etc). The best part is that I feel like I learned something as well, rather than just being shown my failure because I didn't get the interviewer's "trick" to how to solve his convoluted non-practical coding puzzle.
Both of these companies I can gladly say are some of the best places I've ever worked with bright, motivated engineers and a great culture. I think part of it is that if you know your stuff, you'll be easily able to suss out who is bullshitting in a discussion / debate and who isn't. If you just recite problems from Cracking the Coding Interview, you have pretty limited insight into the candidate's abilities.
> If you just recite problems from Cracking the Coding Interview, you have pretty limited insight into the candidate's abilities.
I agree with this. I tell companies to not ask questions from Cracking the Coding Interview. But that doesn't mean those style of interviews are fundamentally broken. It's just stupid to ask questions that candidates are likely to know.
Indeed, I don't write off whiteboard interviewing altogether - if you know exactly what you are hiring for, it can be very helpful to have e.g. a systems engineer write out some C code or go through tree problems. The issue comes in that a lot of people aren't entirely sure what they are hiring for, just that they need a warm body who can code to some arbitrary ability. That's why I always make sure to ask the interviewer what the burning problems are and what kind of action they are looking for on Day One.
Whiteboard interviews don't have to be ineffective. But empirical evidence suggests that companies greatly overestimate their ability to conduct them properly.
Mostly talking about previous experience and especially any projects they were persoanally interested in talking about.
I find there is only one thing you need to ensure a new hire can handle to avoid the major fakers. Give them a simple problem that requires a for loop. Usually it takes up like 5 minutes of the interview. I started asking it when I realized that the weakest people on my team would get stuck talking about problems that were simple iterations, they'd spend days trying to avoid a simple 'for foo in bar' coding solution. These were people who could program is the weirdest thing too. Fizzbuzz has become sort of a joke meme over the years but I've found Steve Yegge's core idea to be true, and you can see the deer in headlights with even the simplest code problem. So now it's all I do. Whiteboard coding is unnatural enough as it is, there's no reason to haze when all you care about is will they get hung up on stupid trivial things. Most code is basic CRUD there is no reason to ask about b-trees or tris, let alone implement them.
If I feel the need to raise the bar even higher, I may include a simple pointer based question, since I've come to notice that indirection another concept that weaker coworkers struggle with. But the problem here, is that pointers just aren't relevant to the majority of what devs do these days, and modern languages do well to hide their usage which means fewer candidates will even have worked in a language like C. So you're likely to get a lot more false negatives. And interviews are stupidly expensive so you really want to avoid false negatives.
For reference I work for a company with less than 1000 employees, serving web traffic that needs to handle 2000req/s peak 1000req/s sustained. There is little we do that doesn't fall under basic CRUD. Our biggest tech challenge is cache invalidation, followed by 3rd party API timeouts.
For the most successful jobs I've had, they just looked at my resume and asked some general questions about prior work.
I've also had several positions where I passed whiteboard hazings. Those didn't go as well. My theory is that selecting strongly for code jam types may correlate with not selecting carefully for people who work well together.
> For the most successful jobs I've had, they just looked at my resume and asked some general questions about prior work.
That's great for people who are good at talking. And certainly communication skills are important, but a strong bullshitter with mediocre programming abilities can probably pass such a test with flying colors.
I'm terrible at talking, and certainly at bullshitting. It happened that I was an excellent candidate for the role, but perhaps they might have hired a bullshit artist instead.
But I don't think so. I think I can often smell bullshit just by its sound, and I think a good manager could do even better.
Beyond that, you can be great at the whiteboard and still full of shit.
Most of it is good advice for fresh graduates who want to get into big companies. Shouldn't we be telling these young people to aim higher?.
If you derive your sense of worth from a bigger brand and not your own efforts and caliber, you might be complacent for a while then life happens and you will regret not accomplishing anything meaningful in life. Before preparing for your interview and seeking validation for your self worth from big companies, try out internships at startups and get exposed to disruptive ideas. Do not play by somebody's rules. That leads to stagnation. A small plant does not get enough sunlight when it is shadowed by a giant tree. The sunlight you need is exposure to problems you can solve.
Google and Amazon are the new IBM and AOL. They were once the disruptors, now ready to be disrupted by newer generations. Every other day we read about how internet is broken and finance is broken, try starting there.
If you decide to get a job at big company, its not all that bad, just do not take it so seriously.
Startups aren't that special, and I lost a lot of income by doing small startups first as an employee for 4/5 years.
Startups even in successful cases are just break even working at AFGAM. Usually you'll quickly be throwing away $100k/yr in income instead. And that $60k/yr your not putting in your stock brokerage account you could of used for whatever bootstrapped digital business or financial independence base you could of done instead. Over 8 years the difference can mean a literal million dollars in the bank.
Working at AFGAM won't make you lose your soul, and you'll learn a lot of stuff that you wouldn't learn in a startup. If you work at both types, you'll quickly realize that working at a SV startup and working at Twitter or FB is not that much different.
A lot of startups have fun, but writing bad disposable code is the standard operating procedure. Mentorship is usually non-existent. Wanting to write something quality is usually difficult to get approved because you might die tomorrow. Long hours are very typical.
Ships are safe at harbor but thats not what they are made for. I guess the world needs people who want to work for big companies and people who want to startup. The grass is always greener on the other side, so its not the worst advice to get exposed to wide range of problems and people who are solving them in the industry.
There's room for both. In the bay area, it's not uncommon for people to kind of cycle through, sometimes being at big companies, then they go off to be adventuresome at a startup, then either it gets acquired by a big company or it dies (and they go back to the stability of a big company) or it's successful enough to where it's not a startup anymore.
What do you think about take home projects/homework interviews?
I've noticed that they have become much more prevalent among smaller companies as a replacement for phone screens.
As a candidate, interviewing is already a time consuming hoop jumping process, so if the project will take more than 3-4 hours, I probably won't be interested but most companies I have applied to are keeping these projects short enough.
2. It allows a company to give a bunch of "interviews" (assignments) even for candidates they aren't serious about. It can waste a lot of candidate time.
3. There is so much cheating on these that you really can't use it for evaluation. It can really only be a screening tool (which makes 4+ hour assignments really unfair).
4. It primarily evaluates current skillset, not how good they'll be with a bit of training. It's the latter that you care about.
5. They're generally given with little thought about what the company really values. It's typically "uh, gee, we do iOS, give them an iOS app." These projects generally aren't very good at narrowing in on very specific things, like problem solving skills.
6. You don't get a lot of context about why the candidate did things a certain way, and how they would have done things different with some guidance.
7. The results vary dramatically based on how much time a candidate spent on something.
#2 and #5 are issues with the company/hiring manager.
#1, #4 and #7 are issues with traditional interview as well.
#4 It's incredibly difficult to accurately assess potential during an interview.
#1 and #7: This holds true for white-board interviewing. Most successful candidates (including myself) invested time to practice coding interview questions. I did over 150 questions on LeetCode and it dramatically increased my interview skills. But I guess the upside is that these skills are transferable across companies who do algo/ds interviews so my prep time isn't "wasted".
I think #3 is the only thing thats inherently an issue with the project/assignment interview.
#2 is not a trend we want to start in industry. With phone screens, both sides make an investment. With a pre-screened assignment, there is very little cost to the company. This gives them a much stronger incentive to throw everything on the wall and see what sticks. Or even feel that they're doing candidates a favor by 'letting' them take the 4hr assignment. Doing a phone screen aligns incentives.
You think you hate whiteboard interviews? Imagine having to to 15+ different take home assignments for different companies before you even get a chance to talk to a person.
I agree, it's bad for candidates but companies might like it. If everyone gives out take home assignments, it would be harder for candidates to negotiate by lining up multiple offers.
#1 -- Not as true about whiteboard interviews. Homework interviews generally come in addition to in person interviews, so it's a bigger commitment.
#2 -- yes, this is about the hiring manager. But as the candidate, you don't really know what's going on. So it really is specific to the homework interviews.
#3 -- This is a really, really big deal. You're trying to assess people based on homework... that might actually have been done by their buddy.
#4 -- Less true about coding/algorithm interviews. These are focused more on intelligence/problem solving, which is getting more at potential.
#5 -- Absolutely this is about the hiring manager. But it's also very difficult for a homework project to focus on one things (like problem solving skills). The cheating + lack of discussion makes this hard.
#7 -- Time spent makes a MUCH bigger difference for projects than for whiteboard interviews. A 1 hour project vs 20 hours will look very different. A bad candidate with 200 hours of prep will still be worse than a good candidate with 0 prep.
When I've had them, I liked doing them. Of course, I was usually in-between employers, so I had plenty of time on my hands and no reason to not give it a shot. I could see where it would be a problem if you aren't in that situation, and just looking for a new employer (and depending on your family/home life responsibilities as well). I like such projects (or on-site development tasks) over a whiteboard interview any day.
> "One Google candidate asked his lunch interviewers where all the “hot chicks” were"
It's shocking that people are this dumb, but sometimes brains just misfire in stressful interviews. Happens with the interviewer sometimes too. I was once asked, when an interviewer noticed my wedding ring, if I planned to have any kids soon and what my wife's career was, which is a definite no-no. I assumed he was joking, but he was honestly asking the question. The other interviewer stopped the interview and basically kicked him out of the room.
do you think he was asking in terms of a casual "get to know you" kind of question? or do you think it was more like he was trying to no-hire people with families?
it might not be, and it's an inappropriate question for a job interview, but it's not that weird to ask about in casual circumstances.
have you spent time around any new parents, recently? they're basically mono-maniacally focused on kids and it's nearly the only thing they ever talk about. I can easily imagine that the guy asking the question might be a new parent himself and it was just where his mind went to automatically while trying to do smalltalk.
or maybe not. maybe it is exactly what it seems to be. could go either way.
When I had a Google interview I was told (or it was at least heavily implied) that I wouldn't be evaluated by the people I was having lunch with. I didn't really believe that, and looks like I was right not to.
The thing about lunch interviews is mostly true. By default, the lunch interviewer does not actually evaluate you, they do not normally submit any feedback. But, if you say something that's considered way out of line/offensive, then yeah I'd expect the lunch interviewer to likely email the recruiter.
We've adopted a new process for interviewing software engineers. It has three interview sessions held on subsequent dates:
1. Candidate learns about the company and the tech during a set of 2-4 interviews with our engineers. We don't tell them to learn as much as they can because we want to see how inquisitive they are. Interviewers are coached to be fairly open and talk about our products and tech. If there are no red flags, the candidate receives a take home coding assignment. We give them 6 to choose from. There is no time limit, but we are interested in how long they take and which of the assignments they choose. The assignments are all relative to the type of code we develop.
2. Candidate returns with their code. We're looking for only 50-100 lines, as we don't want to ask for a lot of their time. They start the interview by hosting a code review of their code in front of 3-4 engineers. We expect them to be able to explain their coding choices and answer questions that show that the code is more or less their own authorship and that they know what they are talking about. After the code review, they will have 1-1 interviews.
3. If it goes well to this point, we have them back for more introductions to other company management to see if anyone notices anything concerning. We might take them to lunch.
With luck, we get 2 or 3 candidates through this process and then meet to discuss which gets the offer.
It's a new process and seems to be going Ok so far, but I'm curious if anyone else has tried this type if approach and if so, what was the outcome?
EDIT: Interviews are 2 hours each step. 1-1 interviews typically 1/2 hour. The code review is usually 1/2 - 1 hour, we don't put a time on it.
- First day is 4 interviews and a take-home project.
- Second day is an interview in front of a panel followed by an indeterminate amount of further interviews.
- Third day is a "culture fit" interview.
3 separate days of interviews and a take-home assignment? What professional has time for that?!
EDIT: I guess it depends on the length of the interviews. Is the total interview time for a candidate more like one all-day interview split into multiple days, or is it substantially more than that? If it's the former, why not just do it all in one day so that candidates don't have to use up a half week of vacation?
So far the people who we've called have been ok with this process. I'm pretty flexible because I don't want to impose too much, so we have shortened it to two a couple of times.
On the other hand, we need people who will stick around for a few years. Our tech is very domain specific and there are few candidates that can drop in and get to work. We have to spend a lot of time bringing them up to speed on the code base. So we don't want to pull the trigger too soon. And we definitely don't want to bring people in for subsequent interviews if it's obviously not going to work.
Regarding cultural fit, the question I ask the team is "You're going to see this person every work day for years. Do you think you'd be Ok with that?" I think it's reasonable to assess that as part of the process.
EDIT: Interviews are 2 hours each step. 1-1 interviews typically 1/2 hour. The code review is usually 1/2 - 1 hour, we don't put a time on it.
I'm curious, do you actually have people staying for years? If you have to be so cautious about working with someone, that may reflect more on you all/management and not the people coming in. All the same, this process is really deep and long, you guys better pay well.
Oh OK. You are offering a legitimate long-term career, so this makes more sense. It's worth putting up with an extra interview or two when you won't have to worry about them for 10+ years. :)
It's much more ridiculous when a web dev or phone app shop has hoops upon hoops to jump through.
You're going to get a lot of people dropping out of that funnel. If you only get 2 to 3 people who also have offers from 3 to 4 other places this kill your ability to hire. plus thats a huge time investment for the interviewer as well.
I'd interview at your company if you made your take-home assignment your first step and combined the interviews with engineers and management into a single day.
Here's our process:
1. quick phone-screen by recruiter. They ask some high-level job skills questions and inform the candidate of our process
2. take-home assignment. Should take only a few hours. We look at both general programming and software design skills, but the assignment also includes some data structures / algorithms stuff and thread-safety. Note that:
a. The assignment is open-book (no need to memorize computer science trivia)
b. The algorithms and data structures are actually used by us in production (it is relevant to the work being done)
3. Once the homework is complete, the hiring panel reviews the homework assignment and resume. I completely review the homework before even touching their resume, so that my appraisal of the code is as unbiased as possible.
4. The candidate is brought in for the interviews (assuming the panel approves). We do four 1-hour, 1-on-1 interviews (1 manager, 2 developers, 1 tester). The style is very conversational, and we want to make sure candidate can speak intelligently to different aspects of software engineering. The only white-boarding I may ask a candidate to do is to manually show the algorithm that they implemented as a part of the homework.
Three days of interviewing sounds like a good way to pre-filter out strong candidates who are already gainfully employed enough to not put up with this crap.
I can tell you as a candidate from a top university I never stick with interviews that are this involved! If it works for you that's good to hear but to me 1 day is enough.
Yeah - those are good; I've asked variants of those and others when I've interviewed. It's interesting; you sometimes get these incredulous looks (may be a red flag), other times they look surprised but launch into describing their process, failures, places to improve, etc. Sometimes this discussion can also be a means for me as a candidate to decide if I want to work there (ie - is it a broken environment where I might become the fall guy or it will cause me heartache in the long term). Sometimes it leads to more interesting tech discussion and such.
Some companies want to attract people who spend 40 hours a week on their job and nothing more and that's OK. Other might be out of reach unless you spend many more hours learning every week and that's OK too.
Whereas people are spending their free time with their kids or learning or watching television it's up to them.
Maybe his/her point is that we should not encourage the idea of having side projects as normal and expected. For many MANY of us (probably the vast majority), this is only our profession. We should treat it as such. Even top lawyers who love their work bill their clients ruthlessly.
No, it is related to diversity. I agree that it's unfair to expect all candidates to work on a side project. Some companies expect this out of all candidates (which is bad). Others see it as a plus (probably okay).
right? these are the same companies that are now implementing absurd Orwellian policies like "implicit bias training" and "external diversity auditing" because they're not even willing to confront the true cause of their lack of diversity: the working conditions imposed by management make the job almost impossible for anyone but young men.
I asked someone in the medical profession how he got his recent job. He said he called up someone at the hospital and asked if there was a position available. Got an interview the next day, and started the next monday.
On the other hand, I asked a friend in the bay area about an engineering position he had applied for. 13 rounds, and he was finally told no.
I would be very surprised if senior leadership candidates tolerate this kind of nonsense. Someone who could be senior management at the big 5 will not put up with 6-10 (sometimes more) rounds of relentless grilling.
This kind of process is only reserved for us lowly engineers. Right from the get go, the power dynamics are clearly established. And this in a field where it is supposedly "hard" to hire people.
My little sister is a travel nurse and whenever she needs a new assignment it goes like this:
1) Talks to recruiter about positions heybhave open, she selects the ones she wants.
2) They send her resume to these hospitals and if they are interested they let the recruiter know.
3) She has a phone interview that lasts 30min-1 hour.
4) If they like her she has an offer within a week, though sometimes within a day.
5) She moves to her new assignment.
She literally deals with life and death and it's a whole hell of a lot easier than any programming/tech interview. She also gets paid very well (her stints around the Bay Area were ~100k, which is awesome for mid-20s).
Tech is steadily losing all of their "alternative ways of doing things." They are just as entrenched in their processes as any other industry.
In Amsterdam, applying for a programming position usually means a phone call from a recruiter, a single talk, and then starting as soon as possible. A second round is rare.
A mixture of some good observations... with some regretabble bits of fanboy-ism thrown in. For example:
Don’t. Give. Up. Ever.
What they’re testing is your ability—and your willingness—to tackle problems that seem completely baffling. They are looking for you to keep working toward a better and better answer. If you’re ever about to give up remember this: The interview process is designed to challenge you, and even the best candidates feel “stumped.” The question is this: What do you do next?
Just when we thought the fad had finally crested on this patently useless, and more to the point, gratuitously toxic style of interviewing... here's another puff piece to keep the cargo cult going for another 12 months or so. As if the interviewers (themselves most likely just a couple of year out of college) could even hope to simulate the pressures of genuine, real-life, high-pressure technical or operational dilemmas through the device of a made-up puzzle problem or two.
Especially since, about 97 percent of the time, the quality and caliber of grit that brings the most value in business environments isn't of the "solve this made up problem, which I know the answer to (but am hiding from you so I can, you know, watch you think on your feet) -- or hit the road, Jack" variety. But rather of the "no one has any idea where this problem came from yet, let alone whether it's even really solvable" sort. And for which the most effective response, in the vast majority of cases, is decidedly not to go down the next immediately visible rabbit hole and fight the problem to the death... but rather to find a way to avoid going down said rabbit holes in the first place. (Typically by revisiting the stated requirements, and finding what the true customer need is, for example).
And if people think they can simulate these kinds of dilemmas -- let alone "test for" a candidate's ability to tackle them, in an interview context... frankly, they're kidding themselves. Which, being as the true "purpose" of these questions seems to be to buttress the egos of those asking... is apparently perfectly fine, in some quarters.
Do you truly believe that all those top tech companies have decided on this process to boost the egos of the interviewers? Are you saying this because you have actual evidence? Have you talked to interviewers who admitted this was their personal motivation?
I assure you that the purpose of these questions is to identify good engineers. You might think it fails to do so, but that is the purpose.
Are you saying this because you have actual evidence?
The place where evidence is needed is in support of the hypothesis that these questions actually work (in favor of achieving their stated ends). That is, the question to ask should be:
"Do any of these companies asking these questions have actual evidence that they actually work?"
I grant that many people who ask (overdrawn, larger-than-life) questions like these probably believe that they help identify good engineers. Whether these questions actually achieve that goal is an entirely different matter.
Someone attending an interview in a suit? Heaven forfend!
Seriously though, what's the "jeans and a nice top or shirt" "culture" about? It's obviously not "we're so chill" if someone's ostracised for wearing what they want and it happens to be 'too smart'.
One thing i've come to believe is that interviews are broken not because of some cultural relic or clueless managers or unintentionally evil bureaucracies and processes but because of the engineers themselves. I've been a part of interview panels where during the voting I think "well shit, I wouldn't pass this either".
I also came to believe that the questions I historically asked were also mostly pointless. They were either cookie cutter things from googling "how do ininterview" or things i thought up myself to go on an ego trip of seeing how well a person matched my specific thought process.
The process was heavily biased towards the type of person who would memorize "cracking the coding interview" who would often be people who would overcomplicate things and rapidly create terrible abstractions because the only thing they knew was making things DRY. Then there's the positive feedback loop that comes with this and the resulting codebase of increasing complexity from everyone trying to prove that they're smart.
It's ridiculous. Though i suspect it's intended. The other trait this process filters for is heads down order taking and the specific type of discipline necessary to memorize a dictionary. Also, the long hours of someone trying to prove their worth. A benefit in large companies.
'I've been a part of interview panels where during the voting I think "well shit, I wouldn't pass this either".'
Been in the exact same position. Couldn't agree more with this sentiment.
224 comments
[ 4.5 ms ] story [ 271 ms ] threadHow would one go about getting your foot in the door at such companies (I've had similar experiences with both Microsoft & Facebook as well) ?
Often employees will be open to referring you even if you don't know them well. Use social media, Facebook groups, Quora, etc. Recruiters want to be found.
Have an active github.
Make yourself Google-able. Have a website where you list projects. Screenshots help make things feel more "real", especially if it's a recruiter/sourcer who might not understand some of the technical details.
Hackathons and conferences also. They're swarming with recruiters.
Having been on the recruited and recruiting side for one of the big 5, I'd add an extra advice: Don't panic if you don't have any of the things mentioned above, especially when still young in College.
I didn't have an active github, nor a website, nor screenshots of my side projects. If you're genuinely passionate when you talk about it (no matter how small you think this is), the recruiter will notice this.
One of the most important advice that was given in this article for me is: "Show initiative even at the risk of failure"
-> You did X in a class project : It won't matter so much because you had to do it to pass your class. If it's a group project it will matter even less because there's no way for the recruiter to know if you did 90%/50%/5% of the job.
-> You did a small Android app or a personal command line tool for you, but you didn't publish it at all on github, and it's a private thing (a tool to help your grandma do X remotely), it's fine, but like the Amazon interviewee who started her game company: Mention this on your resume and to the recruiter!!
I've also seen people downplay achievements because they thought it looked lame compared to what is produced by Google/Amazon/FB/Microsoft ... but those products have dozens if not hundreds of engineers behind them, PM, UX teams etc. Of course your project will be lame compared to it. Still put it on your resume.
If you're going at a top CS school, having all of these doesn't matter as much, the recruiters usually know that the projects/classes you took are not trivial (via Alumni giving feedback on those), if you go to an average one, you need to show your passion in one way or another, working on a small side project you genuinely enjoy is a great way of showing this.
Finally, if you have very little time for side projects because you're busy working multiple side jobs to be able to pay your tuition and rent, mention it somehow when talking with a recruiter. You certainly don't want the recruiter to perceive you as what the authors describe "mentally lazy" when in fact the reason you're not doing that much in the side-project side is because you're working your ass off to simply be able to graduate. You can also use some of those side jobs experiences to show the recruiter you have some applied leadership/teamwork skills.
So if your goal is to get the interview, go through referrals. Don't expect anything from side projects.
Oh and the always responsive HR or whoever is in charge. Getting back to you after 6 months. Thanks but no thanks. I would rather make more money in a stupid corporate job than working on more aggressive ways of making people to click on ads.
Could you name a few "stupid corporate jobs" that pay better than Google or Facebook (200k - 300k for a senior engineer) ?
http://www.businessinsider.com/how-expensive-is-san-francisc...
You're already at 91,000 for basic expenses.
That puts 200k right around the mark for a livable wage and 300k pretty comfortable (and also at the high end of a senior dev wage by a fair margin).
To begin with, when discussing wages most people are assuming for a single person taking care of his or herself. The wage needed to support a fully family including enough for a stay-at-home parent would be significantly higher.
Even with all of that a family of 4 with only a single adult working is estimated a living wage in SF area is deemed to be 70k.[1] Not 200k. I'm assuming you're meaning "livable wage" to be different than "living wage".
Even if we were to going with the businessinsider data, which says 90k for expenses for a family, you're claiming that a family would need to spend almost 10k in discretionary spending every month for a "livable wage"? Remember all required expenses have been paid for, this would just be for discretionary. If you've already paid for "housing, food, childcare, transportation, healthcare, taxes and other necessities, and taxes" and are still going through an additional 10k/month, you're beyond a "livable wage".
[1]. http://livingwage.mit.edu/metros/41860
Looking at the sources of these various numbers, the $91k number in the busnessinsider article comes from http://www.epi.org/resources/budget/ which breaks down the annual costs as follows, annually, for the SF metro area:
* Housing: $23k (seems fairly low for a family of 4 in the Bay Area, but I guess it depends on how far out you live)
* Food: $9384 (quite reasonable)
* Child Care: $10815 (seems unrealistically low for 2 kids if both parents work; hard to tell if only one works, since I would expect that to be near-0 for a bare minimum).
* Transportation: $7300
* Health care: $12,453 (might be high depending on your employer)
* Other necessities: $15689
* Taxes: $12490
I assume that tax number is based in the actually cited $91k income, so to figure out how much more income we need for discretionary spending and savings we need to look at marginal tax rates. Looks like CA state income tax is 8 or 9.3% in the relevant income range (60k-500k for couples). Federal rates are 25-28% (75k-233k taxable income). Medicare is 1.45% until we get to 250k. Social Security is 6.2% until we hit the cap. So a marginal rate of 35% to 39% not counting Social Security.
Retirement savings should be ~15k here (15% of income is the common recommendation), and can often be done "pre-tax" (so only taxed by Social Security and Medicare); let's call this 16k gross income.
Are you saving for college for those two kids? Any other savings (e.g. for a house)? Any discretionary income at all? Multiply all those numbers by 1.6 or so to account for the taxes involved. Realistically I would expect this to come to another 20-30k easily for anything resembling a "middle class" lifestyle.
Looking at the numbers from your link, they have a much lower healthcare estimate (~6k/year), a somewhat higher housing estimate, correctly estimate childcare costs if only one parent works, but still have what look like ludicrously low numbers if both parents work. The "2 adults, both working, 2 children" number from your link is $83k and the difference between that and the $91k from the businessinsider article is all down to the health care number.
Anyway, the correct conclusion here is that in the Bay Area for a family of 4 you need an income in the 70-90k range (mostly depending on how much you pay for health insurance and whether you both work) just to barely scrape by, with no savings and no discretionary spending of any sort. At 150k you can probably do ok in terms of being able to save for retirement, college, maybe take some vacations every so often, etc. 200k should be reasonably comfortable if you have no pretensions to more than 2 bedrooms (notice that housing for 2 kids is claimed to cost the same as for one kid) and aren't trying to get into good school districts.
If you want good schools or more living space than a small 2-bedroom place way far from anywhere, the housing numbers cited above are almost certainly way too low...
A family a 4, with only a single parent choosing to work, with fully funded retirement savings, while also saving for college, and possibly even saving for a house. Using the estimate above, that came to 150-200k.
So now, 65%+ of households have both parents working.[1] A single parent household is a luxury (without any judgement or whether it should or shouldn't be that way, it is). Assuming dual incomes in the above situation we would half our salary range. Taking the upper end, we'd get 100k as the required livable wage to match this household income.
Even more notably it's a calculation for a household income for a family a 4. Not a livable wage for a single-person which would be the other default expectation besides two parents working. Using the same two sources as previously we can see that the income required for a 1 Adult household is roughly 50% that of a family of 4 single breadwinner household. Take our 150-200k estimate multiple by this roughly ~50% factor, take the more generous end and again we end up with the salary in the range of 100k.
From this the most reasonable number for a livable wage would be ~100k.
1. https://www.washingtonpost.com/news/wonk/wp/2016/12/22/the-d...
Unfortunately, not quite. If both parents make 100k, your tax burden is higher than if one of them makes 200k and one 0 (because of the social security cap). More importantly, your child care expenses are much higher. So are your food expenses, typically; this part is correctly reflected in the data table you linked to.
In practice, I would expect a two-earner household, both earning 100k, to have 10-30k/year more in expenses than a one-earner household; the range mostly depends on how much child care costs.
100k for a single person is certainly pretty comfortable, even in San Francisco. Much more comfortable than 200k for a family of 4, I suspect, because housing is so much simpler.
Ok, so 105k-115k / yr. I think we're reaching the point (in economic terms) where the marginal cost of corrections or adjustment exceeds the marginal value they provide.
The summary though is that 105-115k is not near the 200k originally mentioned and absurdly far from the 300k.
Now if you want a big house, then most of the cities that Google is in are terrible for that. But there's more to life than having a big house, and the expensive metros have their own pros. It's pretty hard to beat Silicon Valley weather, and for what amounts to a giant suburb the restaurant scene is quite good (especially for Asian food).
Now if you want a big house, then most of the cities that Google is in are terrible for that. But there's more to life than having a big house, and the expensive metros have their own pros. It's pretty hard to beat Silicon Valley weather, and for what amounts to a giant suburb the restaurant scene is quite good (especially for Asian food).
It does take a lot of preparation for big-company clueless interviewing processes though. Next we'll see guides for how to ace your kid's application to the top-tier coding preschool.
All this sucking up to big companies with questionable morals and procedures, for the money, is so... 80s. So yuppie, so Reagan-era. Reachin' for the top! Go for it! Eye of the tiger! Pressure, pushing down on you, pushing down on me, no man ask for.
I've seen some companies decide "well, whiteboard interviews are flawed" and just scrap the whole thing and replace it with something else... that's much more flawed. They're really not being honest with themselves about what the flaws exactly were in whiteboard interviews and if those are fixable, or what the flaws are in their new process.
There's a lot that can be done to make whiteboard coding/algorithms interviews less flawed. - Be clear with candidates and interviewers about expectations. What knowledge is okay to assume? What knowledge is not? - Clear guidelines on what makes a good question. A lot of companies ask very simple or well-known questions but think they're emulating what Google does. They are not. - Good interview training. Most companies don't have interview training, beyond maybe basic legal stuff. Even the companies that do have training typically have crappy training. - Interviewers should candidates with certain things that might not be obvious. For example, writing examples of input and output on the whiteboard is helpful. Don't rush into coding.
There is A LOT that can be done to make whiteboard interviews more effective.
You can also replace them with some other things, but understand that those will come with their own flaws. Scrapping whiteboard interviews is not going to deliver you some perfect system.
Does your book have more anecdotes of candidate "fails"?
Would you be willing to share some more stories?
(Context: In chess, a queen can move along a row, column, or diagonal to attack. The N Queens problem is to place N queens on an NxN chessboard such that no two queens can attack each other.)
No one does poorly here because they are "bad at chess algorithms." They might do poorly because they think they're bad at chess algorithms. But this is not a "chess algorithm." It's an algorithm, with a chess skin.
But, sure, if you're bad at chess algorithms, I'll give you this problem: Given an N by N boolean matrix (all falses), set N cells to true such that no two rows, columns, or diagonals have two trues.
Same question, but with a boolean skin. Now will you be able to tackle it?
So, advice: Don't assume you're bad at __ type of algorithm question. For the most part, this isn't true. At most, there's a tiny bit of knowledge to tackle it (so you're not bad at it; you just don't know something). Nearly every time I hear someone say they're bad at some type of question, it's actually just an insecurity. They aren't even missing any knowledge.
The only partial exception here is recursion/dynamic programming, which does have its own little approach.
At a higher level: Can you ask for a new question if you're bad at that type?
You could, but it's risky. If I ask you a question that really involves pointers and you don't understand them, then okay. But realize that this might be a deal breaker for me. I might need that knowledge, or I might be concerned about the tendency to give up.
You're probably better off just voicing something like: "To be honest, I haven't worked much with pointers. I'm happy to give it a shot though, unless you want to move onto a different question."
Someone who's never come in contact with backtracking won't be able to solve n queens "in time", unless they pull a mathematics stunt, but those who do know backtracking won't struggle much.
On a higher level - how much of an "already seen the algorithm" crapshot are tech interviews?
And given the fact that a lot of the problems in your book are trivial in higher level languages (reversing a string is only hard if you somehow don't know how to do pointers) - what's your opinion on trying to use Java on a whiteboard in 2017? Has python officially become synonymous with pseudocode?
Back to leetcode I guess.
I'd say it is mostly that. Just do thirty to fifty leetcode medium problems (some of them on pen and paper), and you're good to go.
>will never be able to practice enough to do well on a well-conducted interview
Or in other words a true scotsman interview.
It sounds like you can't bring yourself to write "on an average interview at Google" because you wish they wouldn't pick common interview problems - but they do.
Based on what you just wrote, I'd certainly work through books of "interview questions". Because your phrasing just proved it works, even though ideologically that is not what candidates "should" do.
Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then that interview process is broken.
If you have a company with a bad implementation of whiteboard coding interviews (for example, who just pull questions out of Cracking the Coding Interview), then it's absolutely true that a bad candidate could pass this process.
This doesn't mean that whiteboard interviews are broken. It means that this company's implementation of whiteboard interviews is broken. There is a difference.
For Google specifically, their implementation is decent, but not ideal. A bad candidate would have low odds of passing an average interview at Google, but those odds are not as low as I'd like.
>> On a higher level - how much of an "already seen the algorithm" crapshot are tech interviews?
>I'd say it is mostly that. Just do thirty to fifty leetcode medium problems (some of them on pen and paper), and you're good to go.
Nobody in this thread cares if the reason this advice actually (in actual practice) often works is that the interview process is broken.
nobody cares if the reason we get the job is because we exploited a flaw, and we "shouldn't have" done it that way. Basically, where you just wrote,
>Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then that interview process is broken.
you should have written:
>Taking a step back: If preparation can give a bad candidate a good shot at passing the interview, then I have to admit, if you strictly want to increase your chances of getting a job, then you can do so by preparing -- but I grit my teeth while saying that, because that interview process is broken.
that would have been honest and matches the reason others had for the above thread! anyway, thanks for the responses.
I didn't get through at Google. However, I only asked for 3 weeks to prepare, and I have outside obligations (kids, coaching, that sort of thing). I can easily traverse a binary tree, print all permutations of a set, do DFS and BFS. But I'm not super sharp, especially at a whiteboard.
My review was "not bad, good analysis, but didn't make enough coding progress".
Maybe they were being nice. As I said in another comment, I'm not allowed to know what my scores or reviews were.
FTR, I was a math major, though I did take basic CS algorithms and data structures. So I have a background, but probably further to go than a typical CS major.
I know it will vary by individual, but how many hours, over what period of time, would you say counts as "enough practice" where you might start considering that you probably aren't going to be able to practice enough to do this. Could you ballpark it?
I'd prefer to avoid "unintelligent", but you know, a point at which you'd say, this probably isn't for you, might be time to get some new goals?
A properly conducted technical interview will not be about having seen the algorithm before. Some companies screw it up though. But then, these same companies also screw up other parts of the interview process.
I disagree that using certain languages makes a lot of problems trivial. Maybe it makes certain problem trivial, but those tend to be easier ones anyway. They're already trivial. And even if your language has a function that performs the exact thing being asked, the interviewer can easily ask you to go implement that thing.
Java is fine for a whiteboard. So is python. So are most languages.
No, python has not become synonymous with pseudocode. I'm not sure what you mean by that.
The more difficult ones feel like you absolutely have to have seen the problem before because there is complex relationship between the sub problems that are used to solve by induction/bottom up. How is someone supposed to solve these? Throwing out tons of guesses at the start feels like it'll still end up with me getting shown out before lunch time.
Recursion and memoization is easy but dynamic programming doesn't really feel as natural. Ways to get better? Just do more?
Once you have the recursive solution, the DP solution should be fairly easy. Draw out the recursion tree for an example (or do it more generally), convert it to a DAG by combining redundant nodes, and then do a topological sort. That topological sort is the order in which you need to solve the subproblems to get a DP solution.
If you want to flip a memoization problem into bottom-up dynamic programming: 1. Make sure you really understand the memoization approach 2. Look at the base cases. What are the very last things the recursive approach does? 3. Build up the next case from the base case. 4. Repeat
Generally, no. Not unless you want to look like a whiny stick-in-the-mud.
Does CtCI publish those times for its problems? How would I know I'm too slow to solve a kind of problem before interviewing?
Edit: The question is based on this article I read a few months ago: http://blog.interviewing.io/lessons-from-3000-technical-inte... The bar chart in that page is very interesting. HN discussion here: https://news.ycombinator.com/item?id=13272840
But I didn't see much there that should surprise anybody other than several versions of "Don't panic", including:
"Don't panic -- everybody else looks imperfect too."
and
"Don't panic -- you're being judged on how well you perform under pressure."
I've made the clichéd mistake mentioned in the article of wearing a suit to an interview where it wasn't appropriate, and felt the air being sucked out of the room. It strikes me as silly, but what do I know. Do these hiring practices result in a good cohort of employees?
To me, it's pretty clear that it basically works. It's not perfect, but yeah it works. Virtually every top tech company has been formed with these practices. So unless you're going to argue that they could have hired random engineers, then it must be effective. (Note: this doesn't mean that it's perfect or that there's nothing better or that you can't improve it. It just means that it is reasonably effective.)
I [author here] have done a lot of interviews with companies going through acquisitions, and that's offered me the ability to talk with their managers about their job performance. My assessment is generally accurate. There are certain things it can't evaluate, of course, like work ethic. Work ethic is really important, but there's really no way to interview for that.
Just to be clear, does "top tech company" mean simply "large tech company?
To complicate this: 1. The range in interview performance of those who get hired is really quite narrow. So you're looking at essentially the difference between a 3.1 and a 3.3. You'll need a lot of data to try to find a correlation there.
2. The interview score is only a partial measure of interview performance. If you and I experience an identical interview, I might call that a 3.1 and you might call that a 3.3. The hiring committee will hopefully understand that a 3.1 from me and a 3.3 from you are equally good (because they see our past scores). This makes the data on interview performance harder to correlate, and you'll need even more data. (Or you'll need to properly take this into account in the study, by looking at a score relative to that interviewer's past scores, rather than the score itself.)
3. The above point, repeated for job performance.
4. When a candidate with a lower score gets hired, they're more likely to have something that offset these lower scores. The data set of people who got hired is {people who interviewed well and had good or bad resumes} + {people who interviewed just okay but had great resumes}. When you correlate interview performance of those who got hired, the lower scores in there are skewed towards people with great resumes (or referrals or something else). That factor could totally eliminate any correlation between interview scores and job performance.
In this video, a Google hiring committee were given their own anonymized hiring feedback and wouldn't even hire themselves.
It seems you're upset that an interviewer might be biased against someone for wearing a suit. That's fair; an interviewer shouldn't be biased against a candidate for that. And most wouldn't be -- consciously. But subconscious bias can still exist. It's best to skip the suit, particularly for a very technical role.
The typical interview rule, IMO, is a good one, at least for men (women unfortunately are under very different pressures): dress one (and only one) notch nicer than the people that will be interviewing you.
There are some rare situations where dressing down compared to your interviewer makes sense, but a normal dev looking for a normal job is not one of them.
Dressing abnormally does indeed show that you're either unfamiliar with the norms in the industry or you're deliberately violating them. Agreed.
In finance, it's sort of reasonable to be concerned about someone showing up in jeans and a hoodie. A good part of that job is being able be "corporate" and adhere to norms. If you can't do that in an interview, how will you treat clients?
In a coding interview for a tech company, adherence to social norms is not particularly valued. Weirdness is tolerated, if not celebrated. So it's fairly different.
Tech companies would be far more accepting of a candidate in a suit than a finance firm would be of a candidate in jeans and hoodie. That said... it's still probably best to just avoid the risk of subconscious (or even unfair conscious) bias.
It's so hard as an interviewer not to have a higher score for candidates who look, act, and dress like me. I've caught myself doing it before and I hate it.
And the worst part is that most people (myself included) have a very hard time even recognizing they are doing it, which means it actually is good advice. Play on the interviewers unconscious biases. Ugh.
You should also buy a bunch of books and section off at least a month to study. Especially if you don't have that CS degree. Hope you don't have other obligations like a family or a demanding job.
You should also spend a lot of time doing coding competitions even if you find them boring because you got into engineering to build cool stuff- not solve abstract academic CS puzzles.
You should also practice coding in environments you would never use in real life - like a whiteboard. Every single software engineer on the planet codes in an IDE - but no, you will use a whiteboard. You haven't written by hand since like 2012 so your writing looks like a child's scrawl - better work on that too.
Good luck!
2. Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
3. Sure, you could do coding competitions if you want. That'll replace existing interview studying; you wrote your comment as though that's in addition to your previous studying.
4. Yes, true, you don't code on a whiteboard. But this, in and of itself, doesn't matter. What matters is whether whiteboards are more predictive and whether they create a better candidate experience. So, forget whether it's "real world" or not. Whiteboards have their pros and cons. The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.
5. You do not need to work on your handwriting. I've done A LOT of interviews and this has almost never been an issue.
Fine. I know this. Will this give me a job? No
>> Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
No. I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.
If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?
>> The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.
I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?
The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.
Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?
No, it's not about knowledge. The knowledge is needed. But the real focus of the interview is to tackle a new question.
>> I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.
The ability to see a hard problem, develop your own solution, and translate that into code is a pretty important skill. "Skills" aren't all about knowledge.
>> If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?
It's a lot more complex than that. It's not just about filtering out bullshitters. It's also about identifying people who might be great but are unable to talk about it. There's also lots of people who are great but haven't been in situations where they've been able to tackle interesting challenges (either because of their inexperience, industry, etc). Or maybe they've done interesting things but in a related field that's hard to understand.
But, yes, absolutely train interviewers. (Interviewing isn't really a role that you hire for.) But even a very well trained interviewer might not be very effective in a behavioral interview.
>> I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?
There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.
>> The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.
The interviewer is comparing the candidate to other candidates on the same question. So if a candidate is getting rejected because they failed to be perfect, this means that other candidates are actually getting to perfect solutions. Or you just have a crappy interviewer who rejects everyone.
>> Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?
If that's the case? Nothing. But that's not the case.
>>There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.
This would be great if interviewers actually knew that this is what they are testing for.
At this point, interviewers simply do not care about candidates and make a binary Yes/No decision based on "Did the candidate solve the problem perfectly in 15 mins".
I'm not opposed to coding interviews. But I am opposed to the way it is implemented right now.
The current way would be great if I get to reverse interview the interviewer and prove that I know (knowledge plus problem solving ability) better than the interviewer.
If the interviewer accepts this duel, it's fair game. Otherwise, the current process simply selects the people who can pattern match.
For example, on behavioral questions, I've seen many interviewers who think "In our company, we handle disagreements by doing X. So I'm going to see if, when the candidate hit a disagreement, they handled it by doing this."
But that's not right. Different companies have different cultures. What's good in one company is bad in another. This is essentially the behavioral question equivalent of the scenario you're raising for technical interviews -- people looking for particular answers, and not considering the context.
Both situations definitely happen, and are very common. But that doesn't make the process flawed. It generally means that the company hasn't adequately trained interviewers.
The solution to this is not getting rid of coding interviews because (coding sure is important). But the overweight emphasis on coding as a binary decision-making is ridiculous.
There are actual engineers doing actual real coding work. They don't necessarily spend entire days solving puzzles. So, they won't do as well in whiteboard puzzle test as a person who slacked off at their last job/college and just practiced whiteboard coding problems.
But the current interview process would rather select the slacker practicing problems than the effective person who didn't practice problems.
Why?
Because the process is overweight on whiteboard coding accuracy and the interviewers who are incapable of identifying a person's ability outside of that overweighted process.
This is a flawed process. And improvements in the process is impeded by flawed interviewers.
I do not have a bulletproof solution but I have implemented processes in my current company that filters in better (measured by performance at work) people than the average prolific whiteboard coder.
1. First, we need to establish the goal of the interview. What is our goal? The goal is find passionate candidates who will get the job done or are bring something that makes our team better. Golden State Warriors will not succeed with a team full of Stephen Curry clones. Note that the goal is not to reduce false positives but to hire the best we can get.
2. With that established, we need to interview candidates and see if they are worth it.
a. If they are a generic software engineer/just out of college/did not get a chance to do something interesting yet, what else besides coding (verified by 2 rounds of non-perfect whiteboarding) did the candidate do? We need to identify if they have the potential to do more than just toy coding
b. If they have been doing something interesting at previous jobs/school they obviously did not have time to practice coding. What were they doing? Is it relevant? Is it it interesting?
With point b, I know you get paranoid about bullshitters. I agree. Candidate b can be a bullshit artist. But, if the interviewer is capable enough, the bullshittery will not go anywhere.
In the light of above, I designed the following:
1. 2 coding rounds. It's ok to miss edge cases as long as you can fix them when prompted with decisive hints. The coding problems are also relevant problems from our own codebases and not just some random "skyline problem" ----Note that the emphasis is "OK on missing edge cases". We just want to know if you can solve the problem and translate it to code
2. System design round: Yes even for college grads. For college grads ask something trivial like how would you design a simple diff or a simple OS/compiler. For experienced folks, ask something relevant to their background. This works really well because it shows which college grad really likes exploring the world of "practical" computer science. This also helps remove bullshitters to a large extent. Linkedin has a great round on this and I really enjoyed that round because it got me engaged with the team and they got to explore sides of me that even I didn't know existed
3. "What is special about you round"...We want to know what makes you special (to compensate for lack of toy coding). Here is where a lot of really good engineers would shine. Some will give examples of a compiler they wrote, or how they applied machine learning to an unconventional area. There are no right answers and it quickly becomes a technical discussion. This round shows how insightful the candidate is. Also, this has to be a full round.....not just a 5 min chit chat thing. The reason for it to be a full round is that bullshitters run out of stuff to talk about whereas the person who was engaged in such projects will go far.
Granted some engineers aren't good communicators but whiteboarding is a bad proxy for hiding bad communication skills since a lot of engineering IS good communication. This same candidate will get hired, write design docs, give presentations, explain concepts, mentor interns, interview other candidates. This person better be a good communicator.
Is this process perfect? No. But neither is the current overweighted whiteboarding
So what does this process do? It tried to "select" the better candidate as opposed to trying to "reject" everyone in the name of reducing false positives.
Not perfect but gets me the candidates who are not just whiteboarders.
I am in the bay area and I'll be happy to walk through this process in great detail if necessary.
I'm sort of confused by your suggestions. You've decided a pretty typical interview process. What do you think is different here?
The biggest difference I can see is that you're saying that the coding is "a relevant problem from our codebase and now just some random skyline problem." And you're specifically calling out that it's okay to miss codebases.
Let's discuss these more. 1. It shouldn't really matter whether it's from your codebase, right? If you have two problems, ProblemA from the codebase (which is worse at identifying relevant skills) and ProblemB (which is a toy problem but better at identifying relevant skills), you should go with ProblemB, right? So really the question is to find a problem that is a good predictor of whatever you care about. If I really want to hire people who are smart, is giving them a challenging problem a good predictor? I think so.
2. The edge case thing. You seem very fixated on this. Why? What interviewers should be thinking about is the signal that the candidate sends. Certainly, in many cases, missing an edge case is just a "I haven't written in certain details yet." In some cases, missing an edge case (particularly after a conversation) might actually signal a much bigger issue. For example, imagine someone didn't write in a check to see if an array is sorted. No big deal, probably. But if there's a repeated issue with this and they can't write this code correctly, then we have an issue. Saying that edge cases don't matter is too broad. They might, and they might not. I encourage interviewers to focus on the signal and not on some binary rule.
I just went through a job search and I didn't find this true. I think the bar on algorithms has raised significantly with all of the prep material (your book included) out there. I also found a few companies sent over "things to study" lists that were quite comprehensive and included topics that you might not even see in undergrad.
>I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
I think this is a false dichotomy. I had two friends that just interviewed at Stripe and loved their process: https://stripe.com/jobs/engineering-onsite.pdf
This is about 2 months of practicing an average of 2 hours a day, meaning you're still working a day job. I figure this could be reduced to 1 month of prep if you quit your job and have all day to yourself.
These Cracking the Code interview type interviews are not even close to what we do all day everyday.
1. Obviously you have a lot of experience with this but I just can't see how a week is anywhere near accurate judging from my personal experience, people I've talked to, and the numerous online stories of people's personal experience.
2. To be honest, I kinda expect the world's smartest organization to find some better way to evaluate talent. Google says they want diversity but requiring a month of studying isn't going to allow for that. Their process throws false negatives on a lot of people that could otherwise be great engineers (which they largely admit to).
3. They're different skillsets. One is about knowledge, the other is about memorizing question patterns and practicing implementation speed.
4. Does Google have data here? My experience is a lot of candidates shut down at a whiteboard because it's not what they're used to. They're afraid to experiment or erase or use longer clearer variable names because of the overhead of writing by hand and dealing with space limitations. It's a lot more messy to modify and extend code you've already written.
5. No, it's really bad. It's like this weird mix of cursive and small caps. I'm not sure what is going on.
2. Yes, it does throw false negatives. Lots of people get hired with zero prep though. I generally say about a month of prep is where you hit pretty low returns from more time. But that doesn't mean that you need a month, or that a month will be enough for you.
3. I'm not sure I see your point here. If coding interview prep is a different skillset from coding competitions, then you wouldn't do coding competitions in order to do interview preps.
4. Some candidates do shut down because it's not what they're used to. A well-trained interviewer can mitigate this to some extent (not always). Some people will also shut down at a computer. Interviews will always be an artificial environment and people can freeze for different reasons. At a whiteboard, it's often a bit easier to unfreeze candidates and create a happier atmosphere.
5. That's fine. As long as I can read it, it's fine. And if I can't read it and you can explain to me what you're writing, that will also work.
I want to work for a company that doesn't start with the assumption that I'm a liar, yes.
Going further, it'd be nice to have a company that basically knew what they were looking for, and knew how to evaluate that in a casual conversation.
"I once worked with a candidate at Google for a product manager role. During the interview, he started talking about how he kept chickens at his house. He was extremely passionate about building a door for the coop that opened and closed automatically to control the chickens’ movements."
I have 30 tomato plants in my backyard and was planning some automation around watering. It doesn't always have to include a raspberry pi even though I may go that route to monitor soil temp in the hot months. Filling up a 5 gallon bucket with water and compost and the right size holes can slowly drip feed the plants.
If you have a system that tells you if there are chickens missing from the coop, you can save some time by only going there if needed.
In a previous startup I would ask a single question that would take 30+ minutes to work through with no expectations that the candidate would ever get the answer. All I cared about was if you could break down the issues and move from one set to the next in trying to narrow down the issue. I cared deeply about how you thought, not what you knew.
As people will ask the question had to do with networking and knowning the answer required you to have seen the issue before and have had delved deep enough into the Ethernet HW specs to get the answer.
Narrowing down the question just required you to think and be able to problem sovle. At the end most people never figured it out but those that spent the time attacking the problem and asking question are the ones that got job offers.
The major problem with this approach (unless you are very vocal about the process up front, but still even then I see it as problematic) is it flies in the face of most education at all levels, at least as practiced in the USA. This is fundamentally more an issue with the way we scale teaching up to so many students than the way interviews are handled, but it still ends up being a huge problem.
People are expected to spend 20-ish-plus-something years learning and being tested in a pass/fail know-it-or-you-don't manner, and then are thrown into an interview environment that is entirely different.
It isn't surprising that a lot of people who would be perfectly capable of doing the work get completely hung up on the fact that they are wading through the muck of topics they don't fully understand in this environment that otherwise feels so very much like a school test of knowledge... especially when you throw in the types of social awkwardness and fear of public failure that are relatively common among engineers.
Overall we found that the way we did this interview was very successful. At one point we lowered the bar and hired a bunch of people and they were all gone in six months. On the other hand those that were successful with the question and problem solving did very well and are still there. Some of those people I hired even work with me at my new startup.
Also I do not understand your comment about schools. Yes in K-12 there is a lot of pass / fail but when you go to collage for an STEM major there is a huge amount of critical thinking and problem solving involved. The interview was designed to test that, not if you memorized how to do a binary tree search.
I've had an interview like this. I said thanks for your time and walked out of the room.
I'm not interested in being told problems are unsolvable, I'd really prefer to work in a culture where there's a growth mindset. Even if a problem is unsolvable for me right this second, would it not be within the realm of possibility that with more experience or with a couple other folks on a team, I could solve it?
There was no arrogance here. The job REQUIRED critical thinking and problem solving. Resumes are generally bullshit and unless you have a major public project were this quality is shown how do you test for it?
The first person hired that had to answer this question and go through this was me! I did not get the answer. I got real close. I am really glad I did. That job changed my life. The quality of the people I got to work with was a complete eye opener and made me want to do better and frankly made me a better person.
The company is now worth ~11B or so and everyone made money and was well cared for.
I went to college for Computer Science and while there was definitely a larger emphasis on critical thinking and problem solving than in high school and grade school, testing (which is the closest to a school equivalent of a job interview) still ultimately boiled down to you either knew the material or you didn't. Testing was virtually never a conversational process during which I could ask the test-giver questions to get at a deeper problem, so ultimately very different than these problems where the "tester" (interviewer) doesn't actually expect you to solve them during the "test" (interview).
My comment has nothing to do with the experience of LEARNING in USA schools, but rather TESTING in USA schools (including the vast majority of colleges).
FWIW, as an older experienced software developer, I now excel at the sorts of interviews you are describing, but when I was younger and just out of school I was terrible at them. Not because I was a bad developer/engineer then, but because of anxiety brought about by being asked about things outside of my element, because I just wasn't used to that having never been exposed to it. Based on lots of conversations with other developers, I don't believe I am alone in having this difficult adjustment period with software interviews when first starting out as a professional.
It is entirely possible your process works fine for you because it is one of those ones that is unlikely to produce false positives, which is what you really want to avoid, but I'm sure there are also some great developers just out of school that totally bomb your interviews while they would do great at the actual job.
If you can hire enough developers to allow those false negatives to exist, then that's no skin off your back but your anecdote about having to lower standards suggests that maybe passing on some of those false negatives was detrimental at least in the short term.
We did have this issue of hiring quickly and as I said at one point we lowered the standards. No one from that group made it. We did try. The cost of those hiring mistakes on productivity and just time spent was not worth it even if it meant we passed over someone that was a good candidate.
To be fair it was not all cut and dry. Personal recommendation for someone you worked with in the past could override any interview issues. There was atleast one time where a candidate was rejected by some of the interviews but got hired anyway because of this and it worked out.
When I started the team was around 30 and made of veterans of various other companies. We all hand family and the goal was to get it done right and go home. Age ranges were around 30-40 for the core team at this point. We recruited heavily from collages at BS/MS level and within a year the the number of people in their 20s was more then the older folks, so we must have been able to do something right for the new kids in their interviews.
The one thing I have learned is hiring is so much an art as well as a scince. Google has published tons of data on their hiring practices and success rates. Most of it is pretty interesting as the things you think matter turn out not to be the key to those that were successful.
I stopped reading at this point.
How arrogant to expect you to gift your unpaid free time for the privilege of working there.
When I talk to a company about how to create a hiring process, I tell them that it's unfair to expect side projects from candidates.
Just putting things into perspective for OP. It's like Trump campaign has ties to Russia. Doesn't incriminate him automatically but I would be wary of vested interests. Just an additional data point
Personal projects are there to help compensate for an otherwise lacking resume. If your resume is on point, you don't need them.
Nine-to-five is absolutely enough if you are a diligent and thoughtful engineer.
I agree that companies shouldn't demand side projects. However, if you're having trouble getting noticed, it can help.
Where it comes to taking a realistic look at what it takes to be hired at google and other top tech companies, I think you give great practical advice. I've recommended your book and blog to a lot of people.
Here's a summary of something I've posted elsewhere on HN, and I'd be interested in your angle.
It goes, essentially, like this. Technical interviews are, for all practical purposes, difficult oral whiteboard exams. Lots of professions and academic programs involve some kind of exam
Here's my objection to it - software tech interviews are essentially exams taken with little to none of the courtesies often extended to exam takers in other areas. I'm hand waiving a little here, but I do think that there is a kind of bill of rights that evolved between people who give important entrance or exit exams, and the people who take them.
For example, in the world of law, you (generally) must take the LSAT, and a lot is riding on this test. However, you get to see and review old tests. You have assurances that the test will be consistent across other candidates. You get a copy of the test you have taken, which will eventually become available to the public. You eventually get to know what your score is, and why.
On your way out, you take the bar exam. People study a very long time for this, and many consider it to be one of the more stressful events of their life. But again, there is information about who is grading it, what it contains, what your score is, and why. And lastly, it leads to a permanent, lasting credential respected in the industry.
This is just one example, but they often come up in other areas. My opinion is that one of the reasons we have technical exam/interviews is that our industry lacks a common and respected credential and exam.
I'm not necessarily arguing that there should be one, but keep in mind how stressful and how much prep these exams do take. An interview at google can take place over several days, and may involve relatively grueling technical tests, all at a whiteboard, where people worry about freezing up.
You don't get to know who is testing you, what their credentials are, and whether they are doing this consistently and fairly. You aren't even allowed to discuss the questions - you're typically under a nondisclosure agreement! You don't get to know how you did - my understanding is that google has numerical scores and screen captures of what I wrote on the whiteboard, but I'm not allowed to see them or know what they are. I understand much of this is motivated by fear or lawsuits, but all we really get, after waiting, is a "we've decided not to pursue your application further at this point". And if you do pass, it's known only to that company - if you interview elsewhere, you can't take these results with you. You have to start over again.
I really do think it's just too opaque, and the secrecy is just too conducive to abuse and bias. And keep in mind, this is in an industry/job category that both complains about a shortage of candidates and is not impressive where it comes to diversity in many job categories. Could the opacity and capriciousness of exams be playing a role in this?
Again, I'm not against exams like the ones google does at the whiteboard, I see the value. But I think that the conditions under which we take these exams are a problem.
If you're willing to offer your thoughts and take on this, I'd be interested in hearing them. Thanks!
I do agree that lots of companies create bad interview processes -- for coding interviews and other interviews. Those issues should be addressed.
To an extent, you have addressed this - in a blog post a while back, you reviewed some of the other options available to a compnay trying to hire, and none are especially compelling or necessarily preferable to a technical exam style interview.
Here's the thing, I'm not just focused on clear abuses - bad question, bullying attitudes. I think even when "well administered", the customary technical exam interview itself may in fact be harming the industry pretty seriously. I think it's sufficiently onerous, intimidating, and secretive that it may be driving away talent. And I wouldn't be at all surprised that it tends to drive away people don't feel they fit the mould of the standard Silicon Valley developer at an even higher rate.
I'm starting to think this is one of those tragedy of the commons situations where each individual firm is acting in its best interest by giving candidates rigorous technical interviews - but the aggregate result ends up deterring people form applying for new jobs or perhaps entering the field at all. Imagine if you had to re-take the bar exam every time you changed law firms - and the only people who would know if you passed was the firm evaluating your candidacy! In many ways, that's what we have in software.
I really do think this drives a lot of potentially good people from our field. You mentioned in a blog post that taking time off to do take-home projects may deter women and others with family obligations from applying - but surely that's true as well for taking off 100 hours to study for a tech exam, especially one you might not pass (and if you don't, will never know why). People might be more willing to do this if there was a better process that ensured a fair hearing, a less secretive, more transparent approach to technical exam interviews.
So, I'm asking, maybe we should step back for a moment and ask how we, as people who buy into and use technical exams, may be collectively contributing to a real malaise in our industry, one that creates a problem for everyone, but perhaps an unusually severe one for the very people we're saying we are trying to get into this field.
I certainly don't think that there's an easy answer here, but perhaps our industry really does need some outside regulation, or a professional association that can administer exams in a way that is more transparent, or at least some set of standards. For instance, perhaps non-disclosures that might mask discrimination should be illegal or unenforceable? Perhaps companies and others should be obligated to share scores and reviews of candidate coding exams?
I can see big problems there as well. But I really do think the situation, as it stands, is pretty bad.
Point being, this is extremely helpful batch of advice for a surprisingly narrow field. Take out the SV / Top 1% Tech Culture nuances and there's really not a lot to work with other than "Put your best foot forward, be flexible to learn new things if asked, and and to keep trying.
But, something like this just stings to me:
>Either way, it’s very common for a candidate to get rejected and then get an offer a year or two later.
It's extremely uncommon for the common person to get anywhere near that kind of follow up. Well, maybe for half the expected salary, but no, this doesn't happen to anybody I know outside of SV or Top 1% C-Suite management. The advice within should be constrained to its applicable universe, with cautions that viewing the process in other fields through this lens could actually be detrimental.
What OP implies is that the interview process is out of candidate's control and there are many variables in play.
What the OP doesn't realize is that this is the definition of a shitty process.
What would you call a piece of software that cannot reliably produce the same output most of the time?
vs
What would you call an interview process that cannot hire the same people most of the time?
https://en.wikipedia.org/wiki/Inter-rater_reliability
That's not true though. Raters change. Counter intuitively, as the rater gets more and more familiar with the problem, the less tolerance they have for mistakes.
You could replace them by software, ala "here's a coding problem with a set of unit and integration tests to validate against, go nuts" but that would have its own set of issues.
Cynically, big companies might be selecting for a modicum of technical expertise along with demonstrating capacity to put up with some amount of abusive processes.
They have no impetus to change and think they have found an approach that works reasonable well when given absurd amounts of candidates. The problem is the ridiculous false negative rate.
I struggle to think of a better system. Using experience is unfair and misleading (I worked on awesome stuff... that I can tell you nothing about, I was totally the lead on this software, etc...). Take home stuff is fine with me but a lot of people have problems with it. Come in for a week type things are incredibly applicant hostile.
Sit down with me and lets try and fix this bug seems like a more reasonable solution to me. Dev environment has the most popular ide with their devs, the source for whatever, internet access, and a bug report. Person is there to assist.
Talking about prior experience? Flawed for the reasons you mentioned.
Take home stuff? Flawed -- cheating and other issues (I discuss this in another comment).
Work with us for a week? Doesn't scale. Unfair to candidates. Lots of issues here.
Sit down and help me fix this bug? Flawed:
1. Huge bias based on whether they know this particular skill set -- the right tools, etc.
2. Pretty arbitrary as to whether they find that particular bug.
3. It can't be a real bug. It has to be a toy project in order to get a consistent evaluation.
You can find out if they copied the homework. There should always be a follow up talk about it. They should be able to explain the details.
Ask them to add some functionality on spot. They have working code they should be familiar with and you can work with that. You want to test their approach more than anything.
Bug fixing is hard to prepare right. If you have a clear cut position, that would eliminate part of bullet one.
I'd argue that they should not be asked to find the bug. Only to fix it.
It does not have to be the same bug. I'd argue for a pool they can pick from.
You should be testing how they approach the problem and how they go about solving it. Heck you can debug and outline a fix for something you can't really code yourself, I know I did.
Pairing with someone from the team they'd work in could also provide some insight into the dynamic.
I agree there is no cookie cutter way for homework and bugs. That's why they need to be in a mix, well prepared and supervised.
That doesn't prove anything. They could've hired a senior developer to help them and explain the concepts in detail.
> Ask them to add some functionality on spot. They have working code they should be familiar with and you can work with that. You want to test their approach more than anything.
The more 'real-world' this is, the more it's going to be biased towards those that have experience in a particular stack or with a particular type of development. Which isn't necessarily bad (it could even be good!), but it may be if you want a more agnostic interviewing process.
I fail to see how that particular point can ever be a negative for the hiring company. If you need specific skills, you dictate the environment. If you want the candidate to bring his own skills, just tell him to pick his favourite tools.
There is a world of difference between "created something" and "can explain it in detail."
They create the problem and then they sell the solution as a book or as an online self-help service.
All the interview questions do is demonstrate that you've bought the right books and spent hundreds of hours practising pointless coding problems because you have nothing better to do with your time.
Maybe big companies are only interested in hiring sheep-like engineers who have no side projects to maintain and no sense of pragmatism... But then why do such companies (e.g. Google) spend hundreds of millions of dollars each year on acquiring startups full of ambitious and pragmatic (wolf-like) people (people who probably would not pass the technical tests).
With that strategy, over time, big companies just end up with a few pragmatic wolves at the top and a large number of sheep at the bottom. It's not natural and they're missing out on more balanced candidates who are neither sheep nor wolf but are excellent engineers nonetheless.
But the article said that they are interested in people with side projects.
All of those sound complicated as hell. Perhaps they are more effective, but the whiteboard interview is obviously less complex. And the side projects approach is just laughable considering the number of excellent engineers I know who don't have time or interest to code on their off hours.
A good whiteboard question can tell you a bit about the applicant's design sense, knowledge of key data structures and algorithms, and ability to integrate new ideas. It also has the nice benefit of quickly weeding out the huge number of applicants who can't write DFS on a tree.
Both of these companies I can gladly say are some of the best places I've ever worked with bright, motivated engineers and a great culture. I think part of it is that if you know your stuff, you'll be easily able to suss out who is bullshitting in a discussion / debate and who isn't. If you just recite problems from Cracking the Coding Interview, you have pretty limited insight into the candidate's abilities.
I agree with this. I tell companies to not ask questions from Cracking the Coding Interview. But that doesn't mean those style of interviews are fundamentally broken. It's just stupid to ask questions that candidates are likely to know.
I assure that the big tech companies absolutely do care, but they struggle to find something better.
I find there is only one thing you need to ensure a new hire can handle to avoid the major fakers. Give them a simple problem that requires a for loop. Usually it takes up like 5 minutes of the interview. I started asking it when I realized that the weakest people on my team would get stuck talking about problems that were simple iterations, they'd spend days trying to avoid a simple 'for foo in bar' coding solution. These were people who could program is the weirdest thing too. Fizzbuzz has become sort of a joke meme over the years but I've found Steve Yegge's core idea to be true, and you can see the deer in headlights with even the simplest code problem. So now it's all I do. Whiteboard coding is unnatural enough as it is, there's no reason to haze when all you care about is will they get hung up on stupid trivial things. Most code is basic CRUD there is no reason to ask about b-trees or tris, let alone implement them.
If I feel the need to raise the bar even higher, I may include a simple pointer based question, since I've come to notice that indirection another concept that weaker coworkers struggle with. But the problem here, is that pointers just aren't relevant to the majority of what devs do these days, and modern languages do well to hide their usage which means fewer candidates will even have worked in a language like C. So you're likely to get a lot more false negatives. And interviews are stupidly expensive so you really want to avoid false negatives.
For reference I work for a company with less than 1000 employees, serving web traffic that needs to handle 2000req/s peak 1000req/s sustained. There is little we do that doesn't fall under basic CRUD. Our biggest tech challenge is cache invalidation, followed by 3rd party API timeouts.
I've also had several positions where I passed whiteboard hazings. Those didn't go as well. My theory is that selecting strongly for code jam types may correlate with not selecting carefully for people who work well together.
That's great for people who are good at talking. And certainly communication skills are important, but a strong bullshitter with mediocre programming abilities can probably pass such a test with flying colors.
But I don't think so. I think I can often smell bullshit just by its sound, and I think a good manager could do even better.
Beyond that, you can be great at the whiteboard and still full of shit.
If you derive your sense of worth from a bigger brand and not your own efforts and caliber, you might be complacent for a while then life happens and you will regret not accomplishing anything meaningful in life. Before preparing for your interview and seeking validation for your self worth from big companies, try out internships at startups and get exposed to disruptive ideas. Do not play by somebody's rules. That leads to stagnation. A small plant does not get enough sunlight when it is shadowed by a giant tree. The sunlight you need is exposure to problems you can solve.
Google and Amazon are the new IBM and AOL. They were once the disruptors, now ready to be disrupted by newer generations. Every other day we read about how internet is broken and finance is broken, try starting there.
If you decide to get a job at big company, its not all that bad, just do not take it so seriously.
Startups even in successful cases are just break even working at AFGAM. Usually you'll quickly be throwing away $100k/yr in income instead. And that $60k/yr your not putting in your stock brokerage account you could of used for whatever bootstrapped digital business or financial independence base you could of done instead. Over 8 years the difference can mean a literal million dollars in the bank.
Working at AFGAM won't make you lose your soul, and you'll learn a lot of stuff that you wouldn't learn in a startup. If you work at both types, you'll quickly realize that working at a SV startup and working at Twitter or FB is not that much different.
A lot of startups have fun, but writing bad disposable code is the standard operating procedure. Mentorship is usually non-existent. Wanting to write something quality is usually difficult to get approved because you might die tomorrow. Long hours are very typical.
I would also suggesting looking at this article: https://deardesignstudent.com/8-reasons-to-turn-down-that-st... , #4 is especially relevant.
1. The time commitment issue.
2. It allows a company to give a bunch of "interviews" (assignments) even for candidates they aren't serious about. It can waste a lot of candidate time.
3. There is so much cheating on these that you really can't use it for evaluation. It can really only be a screening tool (which makes 4+ hour assignments really unfair).
4. It primarily evaluates current skillset, not how good they'll be with a bit of training. It's the latter that you care about.
5. They're generally given with little thought about what the company really values. It's typically "uh, gee, we do iOS, give them an iOS app." These projects generally aren't very good at narrowing in on very specific things, like problem solving skills.
6. You don't get a lot of context about why the candidate did things a certain way, and how they would have done things different with some guidance. 7. The results vary dramatically based on how much time a candidate spent on something.
... among other issues.
#1, #4 and #7 are issues with traditional interview as well.
#4 It's incredibly difficult to accurately assess potential during an interview.
#1 and #7: This holds true for white-board interviewing. Most successful candidates (including myself) invested time to practice coding interview questions. I did over 150 questions on LeetCode and it dramatically increased my interview skills. But I guess the upside is that these skills are transferable across companies who do algo/ds interviews so my prep time isn't "wasted".
I think #3 is the only thing thats inherently an issue with the project/assignment interview.
You think you hate whiteboard interviews? Imagine having to to 15+ different take home assignments for different companies before you even get a chance to talk to a person.
#2 -- yes, this is about the hiring manager. But as the candidate, you don't really know what's going on. So it really is specific to the homework interviews.
#3 -- This is a really, really big deal. You're trying to assess people based on homework... that might actually have been done by their buddy.
#4 -- Less true about coding/algorithm interviews. These are focused more on intelligence/problem solving, which is getting more at potential.
#5 -- Absolutely this is about the hiring manager. But it's also very difficult for a homework project to focus on one things (like problem solving skills). The cheating + lack of discussion makes this hard.
#7 -- Time spent makes a MUCH bigger difference for projects than for whiteboard interviews. A 1 hour project vs 20 hours will look very different. A bad candidate with 200 hours of prep will still be worse than a good candidate with 0 prep.
It's shocking that people are this dumb, but sometimes brains just misfire in stressful interviews. Happens with the interviewer sometimes too. I was once asked, when an interviewer noticed my wedding ring, if I planned to have any kids soon and what my wife's career was, which is a definite no-no. I assumed he was joking, but he was honestly asking the question. The other interviewer stopped the interview and basically kicked him out of the room.
have you spent time around any new parents, recently? they're basically mono-maniacally focused on kids and it's nearly the only thing they ever talk about. I can easily imagine that the guy asking the question might be a new parent himself and it was just where his mind went to automatically while trying to do smalltalk.
or maybe not. maybe it is exactly what it seems to be. could go either way.
But the other interviewer still handled it appropriately.
The thing about lunch interviews is mostly true. By default, the lunch interviewer does not actually evaluate you, they do not normally submit any feedback. But, if you say something that's considered way out of line/offensive, then yeah I'd expect the lunch interviewer to likely email the recruiter.
1. Candidate learns about the company and the tech during a set of 2-4 interviews with our engineers. We don't tell them to learn as much as they can because we want to see how inquisitive they are. Interviewers are coached to be fairly open and talk about our products and tech. If there are no red flags, the candidate receives a take home coding assignment. We give them 6 to choose from. There is no time limit, but we are interested in how long they take and which of the assignments they choose. The assignments are all relative to the type of code we develop.
2. Candidate returns with their code. We're looking for only 50-100 lines, as we don't want to ask for a lot of their time. They start the interview by hosting a code review of their code in front of 3-4 engineers. We expect them to be able to explain their coding choices and answer questions that show that the code is more or less their own authorship and that they know what they are talking about. After the code review, they will have 1-1 interviews.
3. If it goes well to this point, we have them back for more introductions to other company management to see if anyone notices anything concerning. We might take them to lunch.
With luck, we get 2 or 3 candidates through this process and then meet to discuss which gets the offer.
It's a new process and seems to be going Ok so far, but I'm curious if anyone else has tried this type if approach and if so, what was the outcome?
EDIT: Interviews are 2 hours each step. 1-1 interviews typically 1/2 hour. The code review is usually 1/2 - 1 hour, we don't put a time on it.
This is a good idea that I've never seen anywhere.
- First day is 4 interviews and a take-home project.
- Second day is an interview in front of a panel followed by an indeterminate amount of further interviews.
- Third day is a "culture fit" interview.
3 separate days of interviews and a take-home assignment? What professional has time for that?!
EDIT: I guess it depends on the length of the interviews. Is the total interview time for a candidate more like one all-day interview split into multiple days, or is it substantially more than that? If it's the former, why not just do it all in one day so that candidates don't have to use up a half week of vacation?
On the other hand, we need people who will stick around for a few years. Our tech is very domain specific and there are few candidates that can drop in and get to work. We have to spend a lot of time bringing them up to speed on the code base. So we don't want to pull the trigger too soon. And we definitely don't want to bring people in for subsequent interviews if it's obviously not going to work.
Regarding cultural fit, the question I ask the team is "You're going to see this person every work day for years. Do you think you'd be Ok with that?" I think it's reasonable to assess that as part of the process.
EDIT: Interviews are 2 hours each step. 1-1 interviews typically 1/2 hour. The code review is usually 1/2 - 1 hour, we don't put a time on it.
It's much more ridiculous when a web dev or phone app shop has hoops upon hoops to jump through.
I went through some of these. I had just come back from a 6 month trip and was effectively 'unemployed'. I was fine with it then as well.
No way that I would have/make that time currently being a full time employee.
Just keep in mind you may be selecting for a particular type of candidate with this strategy.
2. take-home assignment. Should take only a few hours. We look at both general programming and software design skills, but the assignment also includes some data structures / algorithms stuff and thread-safety. Note that:
3. Once the homework is complete, the hiring panel reviews the homework assignment and resume. I completely review the homework before even touching their resume, so that my appraisal of the code is as unbiased as possible.4. The candidate is brought in for the interviews (assuming the panel approves). We do four 1-hour, 1-on-1 interviews (1 manager, 2 developers, 1 tester). The style is very conversational, and we want to make sure candidate can speak intelligently to different aspects of software engineering. The only white-boarding I may ask a candidate to do is to manually show the algorithm that they implemented as a part of the homework.
* Version control - what kind do you use? Do you use it?
* Deploy process - how do you deploy? How often? Who can and can't deploy?
* Security - who has access to mission critical systems? how do you secure your apps and network assets? What do you use for secrets management?
* Problems - what is the burning technical problem you're having right now? If you needed something fixed immediately, what would it be?
Some companies want to attract people who spend 40 hours a week on their job and nothing more and that's OK. Other might be out of reach unless you spend many more hours learning every week and that's OK too.
Whereas people are spending their free time with their kids or learning or watching television it's up to them.
> Whereas people are spending their free time with their kids or learning or watching television it's up to them.
If you're a single parent then you don't necessarily get a choice about what to do with your "free" time.
On the other hand, I asked a friend in the bay area about an engineering position he had applied for. 13 rounds, and he was finally told no.
This kind of process is only reserved for us lowly engineers. Right from the get go, the power dynamics are clearly established. And this in a field where it is supposedly "hard" to hire people.
1) Talks to recruiter about positions heybhave open, she selects the ones she wants. 2) They send her resume to these hospitals and if they are interested they let the recruiter know. 3) She has a phone interview that lasts 30min-1 hour. 4) If they like her she has an offer within a week, though sometimes within a day. 5) She moves to her new assignment.
She literally deals with life and death and it's a whole hell of a lot easier than any programming/tech interview. She also gets paid very well (her stints around the Bay Area were ~100k, which is awesome for mid-20s).
Tech is steadily losing all of their "alternative ways of doing things." They are just as entrenched in their processes as any other industry.
Don’t. Give. Up. Ever.
What they’re testing is your ability—and your willingness—to tackle problems that seem completely baffling. They are looking for you to keep working toward a better and better answer. If you’re ever about to give up remember this: The interview process is designed to challenge you, and even the best candidates feel “stumped.” The question is this: What do you do next?
Just when we thought the fad had finally crested on this patently useless, and more to the point, gratuitously toxic style of interviewing... here's another puff piece to keep the cargo cult going for another 12 months or so. As if the interviewers (themselves most likely just a couple of year out of college) could even hope to simulate the pressures of genuine, real-life, high-pressure technical or operational dilemmas through the device of a made-up puzzle problem or two.
Especially since, about 97 percent of the time, the quality and caliber of grit that brings the most value in business environments isn't of the "solve this made up problem, which I know the answer to (but am hiding from you so I can, you know, watch you think on your feet) -- or hit the road, Jack" variety. But rather of the "no one has any idea where this problem came from yet, let alone whether it's even really solvable" sort. And for which the most effective response, in the vast majority of cases, is decidedly not to go down the next immediately visible rabbit hole and fight the problem to the death... but rather to find a way to avoid going down said rabbit holes in the first place. (Typically by revisiting the stated requirements, and finding what the true customer need is, for example).
And if people think they can simulate these kinds of dilemmas -- let alone "test for" a candidate's ability to tackle them, in an interview context... frankly, they're kidding themselves. Which, being as the true "purpose" of these questions seems to be to buttress the egos of those asking... is apparently perfectly fine, in some quarters.
I assure you that the purpose of these questions is to identify good engineers. You might think it fails to do so, but that is the purpose.
The place where evidence is needed is in support of the hypothesis that these questions actually work (in favor of achieving their stated ends). That is, the question to ask should be:
"Do any of these companies asking these questions have actual evidence that they actually work?"
I grant that many people who ask (overdrawn, larger-than-life) questions like these probably believe that they help identify good engineers. Whether these questions actually achieve that goal is an entirely different matter.
Seriously though, what's the "jeans and a nice top or shirt" "culture" about? It's obviously not "we're so chill" if someone's ostracised for wearing what they want and it happens to be 'too smart'.
I also came to believe that the questions I historically asked were also mostly pointless. They were either cookie cutter things from googling "how do ininterview" or things i thought up myself to go on an ego trip of seeing how well a person matched my specific thought process.
The process was heavily biased towards the type of person who would memorize "cracking the coding interview" who would often be people who would overcomplicate things and rapidly create terrible abstractions because the only thing they knew was making things DRY. Then there's the positive feedback loop that comes with this and the resulting codebase of increasing complexity from everyone trying to prove that they're smart.
It's ridiculous. Though i suspect it's intended. The other trait this process filters for is heads down order taking and the specific type of discipline necessary to memorize a dictionary. Also, the long hours of someone trying to prove their worth. A benefit in large companies.