I am going to kill myself (exaggeration). I have maybe one of the best cv's (I am not saying that, it is feedback from industry) for last year student focused on low-level programming. But I cannot bypass recruiters. Why? because none of them even heard Boost.Beast or anything on my cv. Or any C++ thing I have done as project or my publication. They literally do bunch of find/search on the pdf. And because of that I am unable to get any internship, particularly now which there is no meet up or in person networking.
P.S. I used to have extremely successful in person networking before COVID19.
This feedback is well intentioned, so please don't take it as an attack.
As much as we'd rather it not be the case, even for programmers writing skills are incredibly important. This comment is contains a few grammatical errors and is quite hard to read - I checked your comment history and many others suffer from the same problem. As some who hires, if I read a job application with this quality of writing, I'd probably dismiss it.
If you're having trouble finding a job, it may be helpful to have someone read over any written material you submit alongside your CV (cover letters etc) just to make sure it's clear and easy to understand.
I really apricate your feedback. Just one thing you should take into consideration, that is, as ESL I am not going to put that much effort on writing comments on HN to check for every grammatical error. I simply don't have that much time and energy.
I do understand what you say. But that is not the case sadly.
Completely valid point and shows my shortcoming. I hope I will improve it. But with COVID19 and danger of going homeless, it has become hard for me to manage it all. But you are completely right.
>Just one thing you should take into consideration, that is, as ESL I am not going to put that much effort on writing comments on HN to check for every grammatical error. I simply don't have that much time and energy.
Yeah, I had a hunch that might have been the case, but didn't want to make too many assumptions. I can't string a single sentence together in another language, so I can only guess at what a pain it must be to deal with on a day-to-day basis!
If you're willing to associate your username with your personal identity, you could link to your projects/CV in your profile. Other commenters may be able to identify what recruiters are looking for and why your resume was dismissed.
Also, to the point of the post, do you consider whiteboard interviews to be good or bad for your situation? With covid it's harder for companies to find which applicants should be interviewed in the first place, and short questions are helpful for filtering out people who can't code at all.
About the leetcode thing, yes, I used to be pretty code. If you send me a link right now, I might not be able to do it, since I am living under huge stress right now, but with 1 week or 2 training I can get 80-90% in the coding challenge easily.
I'm not in a position to help get you an interview at the moment, so I'd feel uncomfortable asking for your resume. Participating on Hacker News and GitHub is an avenue for networking. Without linking to information about yourself you may not be able to take full advantage of that.
I'm sorry to hear about your stress. The worst part about the recruitment process is that it doesn't put folks in a typical development situation. I hope you're about to find something!
Consider applying to companies directly rather than going via recruiters. Recruiters often just look at past job experience, but hiring managers will look at your actual skillset.
Maybe it makes sense to pay someone (perhaps a fellow HNer[1]?) to review your resume and suggest some changes?
Japanese is my third language, so when I was shipping around my Japanese resume in Japan, I had an experienced native speaker review and revise. As a non-native speaker, there's a lot of nuance in writing that you just don't understand.
Your resume is often your first impression, so you want it to glow as much as possible. Honest, but glowingly so. :)
You are discovering that most of the value in software... is not in software.
Nothing on your resume will ever cause anyone to say, “We simply have to hire this guy.” On the contrary, the slightest typo, grammatical error, or stray bit-flipping particle will mean that your resume is instantly discarded.
When they do this to your resume, they are doing this to a representation of you: they are metaphorically throwing you in the trash like worthless garbage.
Are you offended by this? You should be.
So don’t use a resume.
Figure out some other way to get on the phone with the people you need to talk to.
You are no longer a tech wiz. You are in sales now. Your job is “selling yourself”.
To a large degree, these look like no-name companies who probably have to resort to this kind of thing to make up for worse pay and benefits as compared to FAANG.
"oh no, this 'no-name company' is _only_ going to pay me $100k+ / year while I live in a low-cost of living area, boo hoo hoo"
Scrolling through there are a lot of high-quality companies (Auth0, ASOS, Blue Bottle, Basecamp, Buffer, Canonical, CircleCI... and more and more).
Just because Google or Microsoft aren't on the list doesn't mean these are backwoods operations. Those companies don't need to bring in new candidates by being on these lists.
In other words the companies that 99% of the world's developers work for. What they are "resorting" to is a hiring process that finds good developers at a reasonable cost. "We can't pay you what FAANG pays but we don't put you through a shitty and lengthy interview hazing" is a pretty good proposition to the vast majority of us.
Most companies don't even have spare billions of dollars to waste playing these games. Your normal companies, with normal budgets, have normal projects, which need people who can get work done.
If you want some one who can build web sites using React JS you are better off hiring people who know and have built things using React JS. There is little logic in hiring people who can do some obscure fashion-of-the-week algorithms, assuming if they can do this they can learn React is just plain wrong. A person does what they are good at doing, if some one is good at interviewing their incentives are in changing jobs often to find the next company that can offer a raise. Not getting your work done.
Also this whole thing that one must know these algorithms because they might once in a life time face a need for Dijkstra's sometime during midnight at a place where they wouldn't find an internet connection, is unrealistic. C'mon. Get real. Companies are full of situations where people are digging with shovels and spoons, because people don't have the skills to use and build tools and system to save thousands of man hours of manual effort. Code bases are full of tech debt. Deployment problems because tests aren't written, or there is just no CI infrastructure. Lack of skills and productivity is a far more realistic and commonly occurring problem than these once in a decade algorithm needs. On top of this comes the need for proactive problem identification and solving(a.k.a innovation).
The fact that FAANGs have to acquihire or acquire companies to grow shows they are not hiring the right people either.
As of today if you are hiring for devs. You must look for turn-key projects executed, expertise in one main programming language/stack, ability to script quickly, skills to build tooling/monitoring, ability to produce deployable code, code maintenance, writing test cases etc etc.
I see Ableton on that list, is probably one of the most used DAW applications used in music production, everyone who's in audio production, engineering know these guys. I'd take a job with this company any time instead of working for a Google/Facebook as another code monkey
If I were a large company with money I would simply ask all qualified applicants if they would be willing to work on a small part of a real project as a contractor with the possibility of getting hired. Applicants get paid for their time, productivity is achieved, jobs might be offered, everybody is happy.
I have a small child and a full time job. It's really not very feasible for me to make time to do a "real project", paid or not. Especially if I'm interviewing with multiple companies.
Yea, I didn't consider that. My thinking was that the hiring process is such a time suck that the applicant should at least paid for those 'take home tests' and 'in person interviews' everywhere in that list ... but yea I'll go back to the drawing board.
It seems that you and a lot of people look at hiring as a zero sum game, because there can only be one standard. If it doesn't include someone, but changing it to include them would exclude you, then tough cookies. That's a very depressing outlook.
I prefer take home projects as opposed to whiteboarding, because my personal situation better allows for it - no wife or children, preference for building things, no NDA at my current job, and so forth. For me specifically I would prefer take home projects followed by a debriefing where I explain my design decisions over solving whiteboard problems. But that's just me. Why can't there be multiple accepted ways of interviewing instead of forcing everyone through a whiteboard gate, if whiteboarding isn't the best way for you to prove your competency?
At least we have lists of companies with different hiring processes as in TFA, so there is a small amount of choice. But we're still dependent on the goodwill of companies to make their hiring processes more tolerable.
Sure - That would neatly eliminate majority of qualified candidates :-)
Very few people who are successfully employed would take a risk / have a luxury to forsake their certain employment for an extremely short term contract (with all the registration, taxation, regulation etc headache this implies!) and extremely limited chance of gainful employment.
(it's perspectives like these that make me realize how much of a grouchy old man I am and have always been - always being a buzzkill by thinking things through next 3 steps! :P)
True, true. In my poor defense I was imagining all candidates as identical versions of myself in a bubble and never considered taxes or anybody else's lifestyle.
>with all the registration, taxation, regulation etc headache this implies!
It's not if you are in the US.
You will get a 1099 from where you worked, you write it on your 1040 and include the 1099 with your taxes if I recall from when I was an independent contractor for a while.
It takes maybe like 10 minutes extra and 1099-ing is incredibly common nowadays (unfortunately) so not a super weird case.
And it's not regulation. You don't need to worry about anything besides getting the 1099 like you would your W2s and appending it.
What if you get a 1099 from another state? Multiple states? Don't you need to file taxes with those states now? (Genuinely curious, I am not a CPA so I really don't know).
What if you don't have any other 1099s (so you have to pay self-employment tax just because of that one interview, at least I had once when my moving expenses were paid with a 1099).
Not even getting into possible illegal work for the candidates on work visas.
My city considers 1099s as businesses due to some shady contractors in the past. So my local government would try to collect a chunk of any contractor earnings (with that chunk getting huge if you didn't register with them first).
>with all the registration, taxation, regulation etc headache this implies!
>It's not if you are in the US.
Its also not if you are in Australia. I'd suggest that the number of developers and designers who at least occasionally take on paid side gigs or who have revenue generating side projects (even if it barely covers hosting etc) is about 1/3.
You are making a point that many others have made, and it makes me unhappy to see it yet again.
What you are saying in effect, is that hiring regimes can't be (or ought not to be) designed to include a group that is (commonly) discriminated against, if that discriminates against anyone, particularly the majority.
Implicitly, I see you defining the optimum as where everyone uses the same criteria that are as inclusive as possible, and just shrugging and saying "we're doing our best" when some are left out in the cold completely.
But isn't there an alternative vision where the optimum is employers using different ("diverse") sets of criteria that individually discriminate but create a society where there's a place for everyone?
Hmmm; I feel you are making some interesting and valid points, and I would like to engage in a conversation; but I am (genuinely) having trouble articulating/parsing what they are. I don't know if you are trying to be diplomatic if the message may not be politically correct, or are the key points very generalized and lean more toward implication/insinuation; or whether it's a philosophy-textbook-style language and I just lack experience in it to parse generically-sounding words in a specific-context-meaning.
My best effort:
* Who/what is the group you feel is a "commonly discriminated group", that would benefit from "contract first" hiring practices?
* If a company uses a practice that "discriminates against a majority", is that a practically beneficial funnel/sieve/practice for the company? Alternatively, if we are discussing companies which are making a stand / moral decision through hiring practices, I feel that's a very different discussion than what I interpreted original poster to posit as "best hiring practice for everybody in all situations / everybody should do THIS".
* I feel your last sentence is proposing each employer using different criteria in hope that cumulatively they'd create an overlapping and comprehensive venn diagram. My response to that notion would be a) I don't trust either free markets or regulation to make that venn diagram complete and b) that kind of stratification of the market sounds horrible... no matter what, I can only go to specific companies who have specific hiring practices that happen to match me? (I fully understand the irony in that this may in fact be the underlying unspoken reality of the market! my point is that it should be something we move away from, not explicitly design as a goal).
But again, I may be misinterpreting your argument and inadvertently taking discussion in the wrong direction...
>trying to be diplomatic if the message may not be politically correct
Not exactly; whether my message is politically correct depends on how you generalize it and what groups you apply it to.
You could use it selectively in other contexts and demonstrate either a left or right wing allegiance, but right here and now, I'm talking about people suited to whiteboarding vs. take home tests as the groups.
>that kind of stratification of the market sounds horrible
It's what we've got though, and always have. In your words, "the underlying...reality". Of course we agree things can be improved somehow. I think hiring should be more inclusive, and I think most people would at least give lip service to that.
But if different people are always going to be different, then a hypothetical convergence on a single "fair" standard means telling some fraction of the population they have no place in society at all.
Eliminating whiteboarding, strictly speaking, seems reasonable to object to. It would be unfairly discriminatory. But wanting that to be the only standard also is an extremist position.
If you want to draw a more general opinion out of what I wrote, one of the things on my mind is the general scourge of optimization by tech people who assume the solution to anything is to find the one best answer based on some metric.
That only works if the workload is low enough to do with a full time job, potentially out of state.
It is a good idea, but nobody is going to be quitting their current job to be a contractor with a job MAYBE attainable. It remains me a lot of retail seasonal employees who work their butts off to MAYBE get a full time position.
I'd also want there to be a clear decision-point (i.e. it isn't a way to get you to be an indefinite contractor, they make a hire/no hire decision within some set timeframe or at the end of the current work).
I don't think quitting is necessary to work on an unrelated side project. Many of the companies on the list ask you to do a either a take home project, test, or tele-coding - real work that doesn't guarantee a job - how does getting paid for your work suddenly make it different?
It would increase the barrier to entry, reduce the pool of potential candidates, and increase the load on HR and hiring managers. Most candidates already have a job and may not want to start contracting for multiple companies in addition to their current job. This would also exclude candidates on a work visa like H1-B.
This proposal comes up in every interviewing discussion on HN. Apart from the usual rebuttals (you are generally hiring only from unemployed people pool this way), here is a fresh one - this opens your company's codebase for reading and writing (even supervised but, still) to anyone. If you already open-source your code it's not a problem but otherwise you don't want random people looking into your code due to: patents, trade secrets, possible exploits to name a few reasons.
All large companies already have large contractor workforces that work on the non-critical products, I don't really see how that's different from employing highly qualified candidates with probably better resumes and referrals than those who are employed at 3rd party contractor companies.
Contractors are ran through background checks and bound by various NDAs and the literal contracts. Not to mention that the mentioned 3d parties are financially liable for the damages their contractors may cause.
No one enjoys whiteboard interviews but it's not clear that these companies have better alternatives. Many of them do take home projects which are time consuming. I'd prefer the employer look at my GitHub profile as there's plenty of code there already. Furthermore, I'm not a fan of "interviews" in general, I find that more information is transmitted by eating lunch with the team and talking like normal people.
Which is why the interviewer should ask specific and thoughtful questions about the project, so they can gauge the candidate's level of understanding. This concern could also be addressed by introducing a trial period.
At least they’re honest about it. Unless you’ve got some kind of contract in place (or live somewhere where this is prohibited by law) your current employer could fire you in 2 weeks if they didn’t like your performance, too. They could do it this afternoon even.
> Why would I leave a stable well paying job
Well, why are you doing that in the first place? Maybe the new opportunity is worth it or the current one is so bad that it's worth it.
I have heard of someone trying and pass off repos forked into their github account as their own work. A quick google or even just looking at the commit history in their fork made it obvious it wasnt true.
Manipulating one or more git repos to claim it as yours then ensuring your lie is not uncovered by googling would be technically challenging enough that those who could do probably don't need to.
Have the interviewee submit repos in which they're the only contributor? But, uh, with a take home project, you _really_ don't know that they wrote the code, either...
Do a take-home project but don't make them write from scratch. Give them a mostly working starter application.
Sprinkle in some bugs. Start with easy spelling errors and go all the way to concurrency problems.
Ask them to add a few simple features and fix any bugs they find. Bonus points if they find bugs you didn't add intentionally. Good unit tests are also a bonus.
Give them a couple days or something. Don't count off if they don't get everything done. It's better to do a few things correctly that fix everything with hacks.
Invite them to email questions/suggestions to the hiring team. This person might be on your team so it's nice to find out early if they have good ideas or they hound you with stack overflow questions all day.
I've never had this kind of interview but it sounds nice to me. It's more realistic to real daily work. You can cheat by having someone else do the work but that's always a possibility.
This sounds sadistic to me. I've spent a long time on dumb typos, which become a lot harder to spot when you are stressed and on a deadline. Chances are if you are applying for work, you are probably stressed about not having a job.
You've completely missed my point. It's not about being perfect. I want to work with people who leave code better than they found it but I can't expect perfection. I don't care if they can't find all the typos. I'm sure most software has typos.
Whiteboardimg a solution to n=np or building full applications is crazy and not realistic. Most interviews do that yet most work is fixing bugs and adding mundane features. Why can't interviews focus on the actual skills needed?
I think discussing the projects would be a better solution. The interviewer looks over his projects and asks questions about them, the applicant explains his thought process behind the decisions made.
Code often doesn't stand alone, there's going to be a commit history, there's often going to be comments and issues on other projects.
Ask questions about the code:
- why did you write this in this way
- explain how this works
- if you needed to add X into this, how would you do it
- are there other ways to achieve what you did here, what are the pros/cons of this?
- what would you do differently if you were to re-write this?
- are there any bugs or common issues, how do you go about diagnosing and debugging them? Could you write the code differently to be more resiliant to those issues?
Anyone who wrote that code (assuming it was relatively recently) is going to be able to talk about it sensibly, and be able to dig into the code and show how certain things are done.
I had a former coworker re-write a library we had open sourced as their own. They had used the original version at work so knew it well, knew the internals by asking lots of questions while working for performance reasons. They renamed things and changed pointers around to their preference. They did this right before they left and had it on github as their work. They could meet all of your questions. They didn't really write the code, but it was MIT, what can you do.
It was very obviously there as bait for people like you and it was removed as soon as they took a new job. It was never used in production in its "released" form.
I had no problem with the person's abilities to code, but for reasons like this I probably wouldn't work with them again, or hire them as a eng lead or higher.
I don't see a solution for this. What you're describing is someone who's clearly good enough to take a project and rewrite it, and still be able to talk in detail about the code.
I'd bet they could also pass a whiteboard, pair-programming or take-home test too.
That they are willing to lie about it and pass off other's work as their own isn't something that you're going to identify in your typical code interview. Using interrogation tactics to try and weed out the frauds is going to be counter-productive, by driving off all the good candidates.
I don't disagree. You might see some of the issues with this candidate on what they choose to highlight as their career successes, I've definitely caught dishonest people, or people I wouldn't work with via soft questions.
But I'm not sure, I've never interviewed this person.
One question is, if someone is doing elaborate deceptive work like this, gets hired and has no problems performing the job, does it actually matter? Some people mostly or entirely code at work, so this looks like a legitimate way to get a "good" project in GitHub that he knew well. It sucks that it's deceptive (which could be a major red flag), but it's also clear enough that he understood a large codebase well enough to explain it and modify and/or refactor it, which is honestly the majority of a SWE job. A lot of people are concerned about the "purity" of how someone solves an interview challenge, and outside of potential ethical red flags, people that "cheat" the system are often putting in a lot of work, and are as likely as anyone else to be capable of doing the work.
No if they had no problems performing on the job then no it's not an issue. Maybe they learned and this provided a good reset for them. Maybe they didn't and similar issues will crop up.
I put the anecdote up as a real-world example of one way the GGP's github reference can be faked.
You would have to dig really hard into their knowledge of the system to catch them, which you wouldn't have time to do.
For instance, I believe the issues would show up in a benchmark.
I would assume the best, then in a week when it is clear that they faked their way in you can just lay them off. If they faked their way in but are a great member of the team going forward and contributing to projects, do you really care as a manager?
Here's the thing though. It's a lot more effort to look through projects on GitHub and evaluate them. And even if you can, you then have to figure out how to cross compare that and set a consistent bar vs people who you interview because they don't have a GitHub presence.
People want hiring processes to be somewhat consistent, and in that world it's really kind of a non-starter to replace tech screens with GitHub profile reviews.
yup. and GitHub's useless for value adds like management, marketing, recruiting, or product skills.
for that matter, GitHub is useless if the projects you contribute to are on BitBucket or GitLab. that contributions graph has a lot of assumptions baked in.
There is not that much code I wrote three, four years ago that I’m still proud of. I figured that would get better with age but it’s only by degrees, and at any rate it may indicate that I’m getting soft rather than better.
So anything on github that I might want you to look at? I’d have to be on a three year replacement cycle, permanently, which is a hell of a lot more work than you might estimate that a reasonable github presence would entail.
It's so backward to see companies shift to these abstracted whiteboard or homework assignments as some sort of standard, considering standardized testing is now falling out of favor in higher education as it is rarely indicative of actual potential success. It strikes me as lazy, just following a trend unrooted in any empirical data.
Maybe the solution is for companies to hire more thorough managers who are capable of vetting a github profile to see if the candidates skills are relevant to the position, not turn the interview process into a performance.
I think it is also hard to compare different candidates this way, even assuming they both have a similar level of commitment on GitHub. A take home exercise at least allows them to compare the same thing
Recurse Center has a pretty nice model where they give you a part of a program (small) to bring ahead of time (or you can pick any program you've written) and the interview is going through the code and adding a feature or two.
That kind of interview mitigates the amount of trivia and preparation-gaming while also ensuring that the applicant can solve problems as a team and is a competent programmer. It is cheatable in that one can memorize implementations of some features, though it seems somewhat difficult and not more cheatable than just memorizing a bunch of CS trivia.
The Recurse Center is a retreat that you attend with other like-minded programmers. I recommend it if you are interested deepening your experience as a programmer, by working on personal projects as part of a helpful community. There is a selection process as mentioned above, but it isn't onerous.
I like whiteboards, if there is an interesting problem.
Going through my GitHub account however isn't good. It's full of quick one shot things, experiments and so on, very little production ready code and not representative of anything.
We place the candidate in a room with a large whiteboard for 15 minutes alone with the lights off. I then walk in with a 6 piece dresser [1] from IKEA (obviously keeping the lights out). I explain to the prospect that this particular dresser is missing one piece - BUT we need the dresser assembled then disassembled and placed back into the box [without anyone seeing]. More than HALF the candidates become violent and demand to see my github account - some even suggest I have a mental issue. I am often able to calm the candidate down - this way we can get into the REAL debate: KOPPANG vs MALMs. Any candidate caught assembling the IKEA dresser is quickly dismissed out of spite. This technique is from Boz at facebook - it is how all* senior principle engineers are screened at FB and the reason everyone codes PHP (but swears they do not).
HR has complained that my techinques are "ridiculous" or borderline illegal - but so far my successful candidates perform, tend to have a solid grasp of typescript and quickly move into senior management.
Note that upper management is also interviewed in such a fashion, except with night vision cameras and the entire board crammed into a tiny security cupboard near the elevators. Any candidate caught trying to hide the existence of the IKEA dresser entirely is immediately hired once they have successfully gaslighted the interviewer.
I'd much rather do a take-home project at my own pace funded by the company, seeing as how my personal GitHub is a barren wasteland (well...it was before I started working for an OSS company).
In the future, when looking at github becomes a standard hiring practice, companies will start going full closed source so that their talent can’t jump ship.
My former employer, a Fortune 500 company, wouldn't let me volunteer for a globally recognized charity because the boilerplate agreement the charity required volunteers sign, stated that IP created as a volunteer belonged to them, and my employer considered that everything within or outside work hours was theirs. Not that I was volunteering as a programmer anyway.
My current employer doesn't have explicit rules that I'm aware of, but early on, I asked our HR person if I could work on open source outside work and they said something along the lines of "what's open source?" Which, I mean, on the one hand what do you expect from HR, but on the other hand, the entire agency is concerned with operating, maintaining, and developing a software system.
This is how I get over that hump. Not only do I not work on anything related to my day job, I mostly build things in my free time that aren't really useful to anyone but myself. Sure my company may be able to claim ownership of my Sean Connery themed programming language but will they? What on earth do they plan to do with it?
They are happy that you can't leave, because you have nothing to legally show to other interviewers and no other prospective project in your drawer. That's not a good position to be in.
I was under the impression that this kind of interpretation of employment contracts was more or less illegal (or, at the very least, unenforceable) in many states (I'm assuming you're US, programming in Fortune 500, so maybe not applicable?). Sort of fits into the same kind vein as certain non-compete clauses, where companies put them in, but they'd be tossed out pretty quickly in court should things actually reach that point.
If it's not on employer-owned equipment during core working hours, then I believe it's pretty difficult for an employer to enforce that kind of thing. I don't know how important to you working at that charity was or participating in open-source now is to you, but if you're willing to take a harder line of telling rather than asking, you might find that when push comes to shove, there might not be much they can do without drastically escalating the situation (not always an organization's favorite thing to do, even if they can technically win, which they might not be able to).
Obvious disclaimer about being a non-expert on this sort of thing, but really just wanted to point out that the first answer in this situation may not actually be the final one.
My recollection is there was a compulsory agreement saying you would not do anything of a nature related to your work, or if you did, allow the company to decide if they wanted to claim it.
Also, running side businesses was not allowed.
Of course, the normal thing to do was just ignore the rules, because most likely nobody would notice or care. I explicitly asked for permission because I wanted to see what would happen.
I would love to see a push for more intellectual ownership over the code you contribute to your employer. I should be able to take any snippet I wrote and reuse it with impunity.
This might require engineers unionizing, but it would be a boon to the field. Patterns buried in closed source vaults would be disseminated and innovation would surge. It might usher in a golden age of software if everyone were able to freely share their intellectual ideas with their peers and not have to worry about risking their health insurance and home in the process.
> but it's not clear that these companies have better alternatives.
It obviously depends on your definition of "better". And it's yet to be seen whether whiteboarding is "better" than pulling candidates out of a hat. You'll note that lots of companies that do strict whiteboard-style interviews are also extremely comfortable firing employees. Would you be "better" just hiring anyone whose resume seemed applicable, that could talk like a normal person, and then firing them if it didn't work out?
> I find that more information is transmitted by eating lunch with the team and talking like normal people.
As an introverted and generally very socially anxious developer, this sounds like a dream compared to a standard interview, which has always felt to me more like an interrogation and generally very aggressive and hostile. I'm confident in my skills and I can communicate well in other situations, but the typical interview environment is cripplingly stressful to me. I would love to just get through a talkative lunch instead.
Well put. The one time i expressed interest but reluctance at leaving my previous job, I was invited on a coffee and chat with a couple lead engineers to help decide. We got along well, talked about technical things, swapped stories about problems solved. Afterwards, i got the impression going through the technical interview would be more of a formality.
I think part of the problem is not everyone is good at giving interviews. But most software engineers can talk about their work once you get them going.
FWIW, I think white boarding interviews are pretty fun.
Different things work for different people, personally white boarding is my favorite. I think the best would be to offer the candidate options. Let them come in for the culture fit portion of the interview, at the end present them a problem and let them choose if they prefer white boarding, pair programming, or take home and submit by the end of the day.
This seems to promote Google Docs > whiteboard, what? At least you can brainstorm or draw diagrams on a whiteboard.
IMHO the worst of these options is coderpad/hackerrank/etc. - it's robotic, discourages pseudocode, and perpetuates the idea that rote memorization is required to pass an interview.
Brainstorming with a candidate is great, when you can get them to let you in to their thought process. It easily can turn a technically incorrect answer into a strong answer.
Companies like google and facebook are paying 200k for entry level engineers. They aren't going to have any issues getting people to apply even with the whiteboard interviews.
Here we go again perpetuating the stereotype of the whiney privileged engineer put in a position of mild discomfort by the villainous abusive white board interview boogeyman. I don’t mind whiteboard interview exercises. They’re not great, but they often give a very good and reasonable signal that often has a high corollary to work performance. I think there’s bigger fish to fry in refining fair and objective interview processes, I’ll never understand the perpetual moaning around this aspect of technical interviews.
That whiteboard interviews are actually testing how you perform under stress rather than how good you are at your job.
Additionally, in my opinion, it measures how good the interviewer is. If you get a bad interviewer or even just one in a bad mood, just pack your bags because you aren't getting it.
Whiteboards are incredibly arbitrary in my opinion, if taken alone. If it is part of a complete assessment, ehhhh.
Interesting study and really good discussion on HN at the time, but the opposite of "[Whiteboard interviews] often give a very good and reasonable signal that often has a high corollary to work performance" would be something like "People who do well at whiteboard interviews tend to be worse performers at work." I don't think that's what the study said at all; rather that people do worse at whiteboard problems when they're being watched vs. not being watched.
Also people act as though software is a job without anxiety. There is plenty of anxiety in this job. Sometimes under pressure you really do have to perform or push back and its good to know if people can do it.
It doesn't have to be stressful. Whether or not you have a gun to your head is due to management. Deadlines are an abstract concept created by humans and can always be moved. The best companies keep them very elastic.
> Also people act as though software is a job without anxiety.
Not. All. Anxiety. Is. The. Same.
Ask a firefighter to give a speech in front of 1000 people. This is an entirely different kind of anxiety than running into a burning building. Ironically, it may be the former rather than the latter that makes the firefighter sweat.
Writing code is generally not a performance art. We're not stage performers. Interviews are an unusual situation that aren't anything like our daily jobs. Stress on the job is not the same as interview stress. Work emergencies are not the same as interviews.
We need to take into account human psychology. These factors are all well known, but the tech industry seems to act in deliberate ignorance of humanity.
Are you an engineer? If you are, then do you realize how dumb you are by not sticking up for your own profession? Do you think FAANG pays engineers 200k because they're oh so nice companies? Or is it because talented people are hard to find?
Yeah if a poor nerd makes some decent money, they need to abuse him with stupid whiteboard questions.
But no one is questioning the money CEO, executives,managers, lawyers, etc are getting.
You can confirm the same using a take-home assignment and a code review together with the candidate. Or even with an on-site pair programming. Neither of those require whiteboards.
I strongly believe there is some signal to shoving enough familiarity with at least one language to come up with mostly correct syntax.
I won't ever fault a candidate for failing to close a paren or being unsure of whether the size of a container is .length() or .size(), but if virtually every line a candidate writes is of questionable value, I'd say that means something.
I actually had the pleasure of interviewing at one of these companies before. They had a take-home project, which was to choose and implement and couple enhancements to a toy app. In the subsequent conversations, we discussed how I approached the problem, details of my design, technical tradeoffs, etc -- all the sorts of things you would expect a professional sw engineer to be able to do on the job. It was challenging but fair, and I was struck by how reasonable the process was. It left a wonderful impression on me.
Take-home projects are not a perfect solution to the interview problem. A major issue is that they require candidates to have free time outside of work to complete them. Personally, though, I'd much rather spend a few hours per application working on projects than countless hours prepping for stressful whiteboarding puzzles. As a developer who prefers to digest problems slowly, I look forward to the demise of on-the-spot whiteboarding interviews.
If you work at a company that subjects candidates to such high-stress algorithmic riddles, and you have the ability to make your process more sane, please consider following the lead of cos on this list.
I would rather spend 3-4 hours working on a take home test rather than writing on whiteboard.
Last year, I did take home test for 5 different companies and all 5 of them invited me to the next round. I yet to have a successful whiteboarding interview.
But did you get any of those jobs? I wouldn't call being invited to the next round successful if you don't get the job. In fact it's a loss, more lost time for the candidate.
Yeah seriously - "rounds" don't mean anything unless they publish numbers, and even then it's sketchy. You need to be able to verify that they're actually there to shrink the pool, not just provide the impression of it. Take-homes are free for the company to issue.
> A major issue is that they require candidates to have free time outside of work to complete them.
Just like onsite interviews. Any interview process will require candidates to spend time outside of work. The only way to fix that would be a true, universally recognized certification/diploma that would prove that you can write software, but I don't see that happening any time soon.
And unlike an onsite interview, I don't have to take time off during working hours to do them.
My mental model is that I'm willing to spend N hours in total for the job application process. I'm happy to spend some of those hours working on a take home problem as long as that is instead of and not in addition to spending those hours doing multiple rounds of interviews.
I hate take-home projects more than I hate whiteboard interviews.
I think the best approach would be giving the candidate a broken app and have they debug it live or in a take-home debug-and-fix or debug-and-improve exercise. Could also have some refactor assignment.
We do this at String and Key (and I adopted the process from my prior teams at Digitally Imported. I've posted jobs from both here on HN in the past).
Its a one-hour hands-on programing lab via screenshare - a pretty straightforward task of consuming some data from an API, processing it a little, and displaying it. We use Ruby for our work, and ask the same of our candidates in the lab. No handwritten syntax, exotic/tricky algorithms, or gotchas. Just a task to see how a candidate approaches a problem, deals with requirements, and implements them in our language of choice.
Happy to go into more detail if anyone is trying to implement something similar in their own hiring processes.
I'm interested in hearing more. I'm currently in the process of improving my interview approaches — currently a live coding session with a real-world problem involving an internal dataset.
It sounds like you're on the right track. I think the important parts are:
1) It should be a live exercise (at least mostly live). I'm less interested in the end result than how someone got there, and most people chose to narrate their thought process as they work through the exercise.
2) The candidate should be able to work in their own environment. Online assessment tools can be different enough to introduce friction in a timed interview scenario. I'm interested in seeing how fluid a candidate is with the tools they'll use every day.
3) It should have the same expectations as your day-to-day work. Did the editor have a linter configured, and did they use it? Were tests written for the parts of the lab that introduced nontrivial logic? Is there a README? Is it all one git commit, or did they habitually make commits as they worked?
4) The design of the lab should have a decision point - I'm currently using a lab which could be implemented successfully with either a relational database or some kind of cache store. The strongest candidates recognize there are multiple paths and can justify why they chose the one they did.
This works best for reasonably experienced devs who can actually be expected to create something from scratch from a set of requirements on a short time frame. For more junior people, I'd typically give a version of this lab that I completed myself and introduced a couple of bugs into. The task is to fix the bugs.
We are a remote company. I give take home projects too. Mostly I give them couple of days but mention they shouldn't work more then 4 hours on it.
Our test is actually quite simple and everyone that did any kind of development should have done these basic things.
But it tells me a log of the current state of the candidate. Most code samples on internet are not production ready code. Security and error handling are mostly not included. So this is where a candidate can shine... but I'm happy if you can string it together working even if it is by googling.
So you're saying spending 3 and a half hours on the assignment is fine. It's not.
Take-home assignments have little to no value for the candidate, they're only feasible if you are jobless or in college. The last thing I want to do in my spare time is to stay focusing on a bogus problem that may end up discarded.
Take-home projects are frustrating when they basically just say "Build an app that does $xyz", and we'll be in touch if we like it.
Obviously no one is going to submit an app that doesn't accomplish the primary task. I feel like you get judged more-so on how you decided to get the initial app up and running, which likely isn't something you'll need to do working there.
> Obviously no one is going to submit an app that doesn't accomplish the primary task.
I would have guessed the same thing before my company started doing homework interviews. But in fact, the ones that don't get to the finish line are sometimes the most interesting ones to review, and they aren't an automatic no-hire. (And we say so explicitly, if a bit more diplomatically, in our instructions.) If someone hands us clean, well-thought-out code and it's clear that they honored our time limit and just didn't quite get there before the clock ran out, we'll still talk to them.
I don't understand how take home interviews are being held up as a better solution. They have two major flaws:
They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.
There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.
As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
At Caltech, most exams were take home with a time limit of 2 hours. Some were "infinite time". The students hated the latter, because it means there was no end to the time you needed to spend on the exam, meanwhile there were other exams, etc., to take.
With a 2 hour exam, there's an end to it, and you know how much time it should take to do the problems. Much better.
It's even worse if you're being graded on a curve, as you would be in a job interview. You're competing against others who will spend infinite time on it.
Like you said, "there were other exams, etc., to take.". Presumably other competitors have other exams as well? You'll thus be competing against people with different time management skills.
My company does take-home interviews (or rather, we give candidates the option to do them; nearly all candidates choose take-home instead of a live coding session) and I don't think I agree with the "lowers the cost to zero" point. First of all, they are never the starting point; we only give a take-home problem to a candidate after we've had an initial discussion with them.
When we get a take-home submission, we essentially run it through our code review process, and as one of the engineers who does reviews, it usually takes me nearly as long to thoroughly examine a homework solution and write up my feedback for internal discussion as it would to do an in-person interview.
Being able to review a candidate's code on my own schedule rather than being yanked from the middle of some knotty problem to go do an interview at a fixed time is a big quality-of-life win for me. But the rate limit isn't much different than it would be with regular interviews.
All that said, it's totally possible our process is super atypical.
Totally agree. I've almost never completed a take home project (even though I liked the concept) because there were always other companies I liked that didn't demand as much of my time whose interview process moved faster and who gave me offers before I finished the take home interview process.
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable
I'm going totally nuts about this particular problem. I have written in as clear terms as possible that I really don't want the candidate to spend more than 2 hours, but I don't feel I am going to convince anybody - especially as this is a (mini)game programming test, and with games you can always add one or ten more things.
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable
> I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?
You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.
- If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you?
- Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like?
I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?
The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience.
Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know.
In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).
> but bias against other desirable candidate qualities such as experience
Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof?
There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on.
I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.
Apple did this in one of the sessions last time I interviewed there. They handed me a project in Xcode that had a number of bugs with varying degrees of difficulty — some caused crashes, others were UI glitches, etc. The assignment was to identify the problems and fix them. If you have more than a couple years’ experience the issues were pretty straightforward.
I really enjoyed this session, but I can see how it could be stressful to others.
We used to do this at an old job. We were a consultancy which specialised in a particular e-commerce framework, and were hiring people with experience in it.
So we set up a small project using this framework, and mildly wrecked it. Made some really basic mistakes, made some more subtle mistakes, added a bunch of dead code, wrote good tests for some of the broken stuff, wrote broken tests for some of the good stuff, etc.
Candidates came in, and we asked them to add some simple feature to the app, which we warned them may be broken. We had someone sort-of-pair with them as they work, to watch what they do and keep them on the rails to an extent. We then evaluated whether what the candidate did was sensible: did they run the tests, did they pay attention to the failures, did they find and fix the mistakes, did they get distracted by the dead code, etc. Candidates didn't have to do everything perfectly, but there is plenty of scope to go really wrong. I remember one case where the very first thing the candidate did was delete all the tests. Thanks for coming in, we'll let you know!
The company did quite a lot of rescue missions, where we would pick up a project that another consultancy had started and failed to deliver, and get it live. This interview wasn't just a trick, it was testing everyday skills!
> I think the best approach would be giving the candidate a broken app and have they debug it live
I had a company do this and it went really poorly for me. It was a python app and at the time I wasn't very expert in python. They told me to use my laptop and all I had for python was a vim environment. I knew enough python to do white-boarding problems. But debugging a complex app exercises a whole different set of skills I just hadn't developed in the language. I was also learning vim on the side. I looked like an incompetent amateur.
So there's a language and tooling issue. The bigger problem is that debugging an unfamiliar code base under time pressure is almost as niche as doing algorithm problems under time pressure. I've been on enough company wide on calls that I actually think I'm pretty good at debugging unfamiliar code under time pressure. But if I have someone who understands the code looking over my shoulder, they're usually at least trying to help.
It's still time pressure. It's still a somewhat contrived problem and code base. The interviewer is still comfortably omniscient. And it introduces some more complexity due to unfamiliarity of the language, tooling, code base or problem space.
Ultimately, I'm not sure it's any better than white-boarding.
A lot of these problems are solved with a take home question but then all the problems with take homes rise in their place.
Was this a timed take home project or something with a flexible timeline? That is, was it "You have to respond within 3 hours of receiving this email with your solution" or something more flexible?
I mostly understand why some companies do the x hours routine, and when I've done interviews this way I've historically performed well, but it just felt like it added unnecessary stress. For instance, anyone on the job doesn't have to worry about issues like, "What if I couldn't get the app up and running because I had the wrong version of X installed?" longer than maybe their first day on the job. And if you plan on hiring an engineer for a few years, one day is irrelevant. At best this felt like it favored contractors who were more experienced at jumping into new projects frequently.
I've had companies straight up pay me for doing a take home project as part of an interview. I think that's the ideal way as it gets rid of the potential for exploitation/excessive burden that can come from take home projects.
Or just offer both options. I certainly wouldn't eliminate the opportunity for unemployed people to get a little cash, just because of contract issues that gainfully employed people might have.
There are tax implications, for both employer and employee, to paying the candidate, at least in the UK. It would be quite a lot of headache for not a lot of money.
Donating to charity has tax implications for the company too, but they're much simpler.
I'm not a corporate accountant or anything, but I've been on both sides of this and it's very simple to pay people for outside work in the US.
And for that matter, giving real-but-unpaid work to do as part of an interview may be illegal. So paying for your time may be something companies are doing to protect themselves taxwise, not just for good will.
>This can add a lot of overhead, and some candidates may have jobs that forbid them from accepting outside work
How is this enforceable? Even in the case that someone finds out about it, what’s the company going to do? Sue the interviewing company for the rights to a few hours of work?
Well, if the employer finds out you violated your employment contract in this manner they could sue you, or perhaps even the other party that paid you. But IANAL, I'm just saying that contracts forbidding side jobs are not unheard of.
It’s almost certain there are no damages assuming you are just working on a take home project not something that they are going to actually use to compete with your employer.
Just because something is in the terms of your contract doesn’t actually mean you really need to worry about a lawsuit.
A company might fire you if they find out, but the chance that they fire you is to small to worry about.
And even if you don’t do take home work for pay, you don’t want your company finding out you are interviewing. Taking a token payment doesn’t change anything practically.
IANAL. I am just saying that some employment contracts forbid side work. I imagine many companies would prefer not to get in the middle of that, and so might be reluctant to pay people for work done in interviews.
Some people have contracts that forbid divulging any information or techniques they learned during their employment. This doesn't stop companies from asking questions about previous work projects, or asking them to solve problems on white boards despite that fact that some candidates are certainly violating the letter of their employment contracts to do so.
People have contracts with all kinds of stupid things in them. If they aren't practically enforceable, they can safely be ignored. Any company saying they don't pay for take home projects because of employment contracts is just using that as an excuse not to pay.
There are people out their with contracts that say they can't even interview at rival companies, how many companies out there refuse to conduct interviews because of this?
> choose and implement and couple enhancements to a toy app
I think this is a key piece why this kind of project is useful, because that's exactly what they'd do when they start in their coding role: jump into an unfamiliar code base and fix a bug, implement something tiny, make things better along the way.
When hiring for a React role, I went to find an open-source React app and simply asked the candidate to mod it a bit. We then discussed the why and how, but also how they liked the app and why.
We only gave this task to the final candidates, and the ones that had React experience were done in very little time, because typing wasn't the issue here, but reading unfamiliar code and understanding the mechanics of the app.
Writing a hello world app from scratch or coding fizz buzz doesn't tell me enough.
Take-home tests are fine in isolation, but if every company does that you'd pretty much be spending all your time doing take-home projects and nothing else.
Also many companies will insist on a take home project in addition to everything else, including a whiteboard interview. So you can pour time and effort into this project only to be eliminated further down the line (or just ghosted).
It's like they assume applicants are applying to one job at a time and have no other work or life commitments.
I dunno, it feels like the tech industry has forgotten how to interview candidates, as in "just chat with someone like a normal human being to see if they are a good fit for what you need", and like many other aspects of the profession have fallen for dumb processes, fads, cargo-culting, outsourcing to third parties, and dubious technical solutions in lieu of experience and social interaction. I learned this early in my career, and found I can pretty much size up someone's technical chops - assuming they're in my wheelhouse - and get a good idea of their professionalism - with a short, half hour phone/Zoom interview. Yes, I can see how this might lend itself to bias and prejudice, but it seems the solution there is to train interviewers to overcome biases and have more diversity (in all senses of the word) in your interviewing team.
Just look at their GitHub and if it looks good, dive in at the interview to make sure they actually wrote the code. Isolated take-home tests seem ridiculous to me if the candidate has public code available. You're filtering out anyone who likes to avoid duplicated effort.
If a person has some interesting Github activity (that they can back up) I would certainly discuss it with them. If I'm hiring for a Django developer and the candidate has contributed to the Django core or a related library, that would be a strong signal - but it should not be a requirement. A strong candidate is typically a mixture of good work history, side projects, hobbies and interests, personality and other things that indicate they would be a good fit for your team. How much and in what way depends on the job and the person. Maybe the startup company needs a person who does lots of side projects, builds stuff on their own etc because you need self-starters that don't require lots of hand holding and are ready to take the initiative to learn. On the other hand your 1000-strong engineering subdivision at big corp needs more heads-down 9-5ers who just produce reliable output. Maybe the things you ask a junior dev out of college are different to what you would ask a mid-career developer. There's no one size fits all approach that people seem to be looking for.
Sure, that's why I said "if" - if they don't have a public presence then give them a test.
But frankly I don't really buy that excuse. Those people have time for multiple coding tests but they don't have time for a (small) public project? Ehhh. Not convinced. If you can get away with no portfolio to begin with then you're not going to be inclined to accept coding assignments.
I don't think you should require GitHub, but if a candidate came to me with projects on GitHub I'd skip all technical aspects of the interview, since they have already demonstrated their technical ability.
I'll play devil's advocate and defend whiteboard interviews as compared to take home or online assessments.
Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [0].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [1].
Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can’t make any assumptions on what the candidates know. I’m not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
The advantage of leetcode is that you study once and then it applies for every job you interview for. Like democracy it's a horrible system except for all the others.
Take home project? There goes 4-12 hours per company that gives you one and sometimes more. Companies have no incentive to cut it down or not give it to even marginal candidates so you'll get a lot more of them than full in-person interviews.
Pair programming? That's basically white boarding with a better white board or, in the remote interview world, just regular white boarding. Sure the problem is more real world, in theory, but then you get issues of knowing the same frameworks, etc. So to do well you need to spend a lot of time beforehand studying their specific frameworks and code bases if they ask you to do an issue on their open source project.
Gotta disagree on pairing, as that gives you an actual computer. If it's your own environment, you're at least comfortable with that. You don't have to worry about penmanship and can get IDE suggestions. It's frustrating not being able to get your ideas down quickly enough like when writing code on a board.
I would argue the problem is it's not a one-time cost. You pay the cost almost every time you want to change jobs, because leetcode/algorithmic interview questions are so fundamentally different to what we actually do day-to-day as developers.
It is very easy to completely forget all about graph algorithms in, say, 3 - 5 years during which time you've been a productive, valuable member of a software development team that...you know...actually builds things customers care about.
Small price to pay to get more and more absurdly high paying jobs every 3-5 years.
Being a productive, valuable member of a software development team is teaching you things that will set you up for success in higher level roles. You won’t be taken for those roles on Leetcode interview performance alone.
>You won’t be taken for those roles on Leetcode interview performance alone.
I think the problem is that for some teams the entire process is some combination of leetcode-style "technical" interviews and then some "culture fit" interviews, without any chance to talk about actual dev work.
Large tech companies will have system design and project oriented behavioral interview which will somewhat cover actual dev work. Other than that it pays to build up your network during your career. You can basically bypass interviews at smaller companies with the right referral.
>because leetcode/algorithmic interview questions are so fundamentally different to what we actually do day-to-day as developers.
I mean, maybe hard or esoteric easy questions but I don't think that's true for basic things like 'find the first duplicate' or '-traverse- this graph'. When people rail on whiteboard coding, I think they mean obscure algorithmic questions and not fundamental data structure questions.
I don't expect someone to be able to do Dijkstra's algorithm (or even spell Dijkstra, did I do it right?) but everyone should be comfortable enough with a bit of help doing BFS or working with linked lists, hashmaps, no? Am I crazy here?
I don't think you're crazy, but if their work and your work are detached enough from BFS and linked lists, why use them as the assessment criteria?
It's not really that difficult to make quotidian software engineering tasks into interview questions. I've at least seen it done successfully before.
More anecdotes: a nontrivial number of the "algorithmic trick" interviews I've done were questions I knew the answer to beforehand – and often the interviewer didn't care when I told them ("I just want to see how you think" or "Just show me how you'd code it" or sometimes even "OK, just tell me the solution and we'll move on to the next part"). What happens to the people who don't have the same background? Are they really less competent software engineers because their algs class didn't cover making a queue from two stacks?
I've worked both as an electrical engineer, and as a SW engineer/developer.
"Find the first duplicate": In my actual SW roles, this was trivial: Use a hash/map/dictionary/set. However, this isn't a typical Leetcode question - often it will be with additional constraints (cannot use a set, need O(1) space, etc). I don't doubt there are domains where this is important, but I've never had to do this in my professional work.
"Traverse this graph": Probably had to do it a few times. Less than once a year.
BFS: Never for my job.
Linked lists: Never for my job.
Yes, it is good to know what a linked list is and know if your library's data structure is using one, but again: That's not your typical Leetcode question. You'll be expected to write a linked list to solve some problem.
I do get what you're saying: Stuff like linked lists and BFS are fundamental, even if most SW jobs almost never require them. It would be a red flag if a candidate didn't know the basics about them. It's another thing to ask them to code it on the fly.
Going back to my electrical engineering experience: We have to take and use a ton of calculus to get the degree - much more than a CS grad will take algorithms. Yet it's a common trope that most electrical engineers will never use calculus on the job (literally not hard to find one who had a whole career in EE without using calculus). If you're doing research or are at the forefront of a discipline - sure, you'll need calculus. Or if you're doing electromagnetics, etc. The theory is built on calculus, but you don't need to know it to apply the results of the theory.
Would it be a red flag if a candidate clearly didn't understand the basics of differentiation (i.e. rate of change) and integrals (i.e. summing quantities)? Perhaps.
But should I ask someone to do a tricky integral for an EE role that will not require the person to do any calculus? I would be criticized by my peers, and if I kept doing that I would be excluded from interviewing. In fact, even if I ask them to do non-tricky integrals I would be criticized.
Asking people to figure out a trick to solve a linked list problem is equivalent to asking EE candidates to do tricky integrals or differential equations.
I agree that take home projects and pair programming are nowhere near ideal, but I think the advantage you mention for the "leetcode" white boarding is one of the main disadvantages. If all you need to do is study these and memorize their solutions then what is it showing us of your actual ability.
I personally when I interview candidates prefer to get to know them and how they think before I even try to understand their ability to problem solve. Knowing the language or algorithms are all easy to learn things. Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.
>Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.
That's what leetcode used to test and then people optimized for it and now it tests memorization. At scale things work differently than in isolated cases. I've also found that most problem solving interviews really test how well I've seen the problem before and how well I can lie about seeing it before. Interview enough candidates and the good problem solvers will look dumb compared to the ones who simply saw it before and can act like they didn't.
Depends how the pair programming is done. Last time I did pairing I was told I could use any language I wanted to solve the problem regardless of whether or not the interviewer knew it. You just need an interviewer who is curious and asks questions about the language you're using to make sure they understand your approach.
My worst experience so far were pair-programming interviews. This requires so much training on the interviewee side of things that most companies just completely tank this one. They have some fixed expectations in their head and they have not a single clue how to objectively evaluate candidates.
Yup, leetcode at least is fairly easy to make interviewers competent enough at giving especially if you restrict the problems they can give. Although worst one me and my friends have gotten were system design. Friend got interrupted with questions every 30 seconds, told to just answer them directly and then failed for not elaborating deeply enough on things.
Same exact thing happened to me at Microsoft during the system design interview round. Interviewer wouldn't just let me explain and kept on arguing about a relatively minor issue and wouldn't let me proceed and in the end failed me because I didn't come up with a good design. Really the whole 45 minutes were like him repeatedly stopping me and coming back to that one issue.
It's always surprising to read people complaining about spending even 8 hours on a take-home problem in the comfort of their home, with no one looking over their shoulder, finished at their leisure. Meanwhile, anyone serious about applying to the same company wouldn't hesitate to take PTO and leave work to go on-site for a similar amount of time for an interview.
If you do the take-home problem, do you get to skip the interview? My understanding was that it might, at most, replace a 1-hour whiteboarding interview. The one time I did a take-home problem for an interview, I spent a whole weekend day on it (which felt hard, given that I was making my wife watch our child all day), and then I still had to spend a whole day on in-person interviews.
And then they didn't even ask me any questions about my take-home solution--it was just ignored. It felt like a pure waste. I'd much rather do whiteboarding.
The problem might be that you might need to interview for alot of positions and having a homework for each would add up.
The homework is no cost for the employer and you are not sure how serious about you they are when you do it (if it is in the start of the process) so I wouldn't bother if I had an alternatives to go for.
Except most companies give the take home as essentially a first line screen so the chance of passing is low (as they need to filter out 90+% of applicants with it). So instead of spending an hour per company I now need to spend 8. When you're talking to 10+ companies that adds up quickly. If you also have a family and a a day job it becomes untenable very quickly.
In the past we had some candidates complain about take-home problems. The primary complaint was that candidates weren't interested in dedicating 4-5 hours of their own time unless they felt that someone from the company was putting in an equal amount of their own time. Some candidates also had suspicions that we were trying to use them for free labor (Not true, we used the same toy problem for every candidate). One candidate even thought they should be able to bill us for their time spent applying (We passed).
Eventually I changed the strategy and offered candidates the choice of the take-home problem or an on-site whiteboard style interview where they solved a subset of the problem in pair-programming fashion with us. Zero people chose the on-site interview. Everyone chose the take-home problem.
No one complained any more after we reframed the take-home problem as the candidate's choice rather than the company's.
Aren't meaningless shenanigans generally associated with meaningless things?
Do you have any evidence that either strategy really matters? Does it boil down to, "We choose an interviewing strategy that selects for people who look like programmers."
I give people the choice of take-home, whiteboard, or code sample. Code sample can be any code they've written for a side-project or even a take-home they've done for another company. Obviously, they can't share actual code they've written for another company, but take-homes are generally fair game... And so that solves the O(N) problem for people who are applying to multiple companies.
I really don't understand the idea that take home is somehow more fair or accurate than whiteboards. I've judged and taken both whiteboards and take homes. Take homes are more annoying for the candidate, and in my experience much more subjective from the interviewer. In my experience take home is the absolute worst way to pick a candidate.
When I do a take home I have access to google, StackOverflow, etc. I can spend 15minutes on it, then pause to do something else (while I still sort of think about the problem), get back to it for 30 to 60 minutes, rinse and repeat. Which is exactly how I work in real life.
Take homes can actually be compiled and executed against a test suite for correctness.
Take homes tend to be more realistic exercises than balancing a f-ing tree or walking some idiotic graphs.
When I interview (and yes, there's whiteboard time), I tell the candidate up front and very clearly, "I will be your Google. I know we don't work in a vacuum"
Coding on a whiteboard in an interview is not "broken". What is "broken" however is making the problem the candidate needs to solve "hard". I can't stress this enough: a whiteboard coding test is nothing more than a negative filter. It's to filter out people who can't turn a simple idea into code as these people exist.
This is why FizzBuzz was such a simple problem.
But interviewers fall into the trap of thinking "this problem is too easy" and make it hard (eg figuring out a problem is g union-find on the spot and then implementing it) or, worse, they make it a crap shoot of whether you know the "trick" or not (eg reversing bits in O(log n) or the tortoise and the hare). This actually makes the test completely useless.
A negative filter is simply a filter whereby if you fail it, you almost certainly aren't a great hire but the reverse doesn't apply: there should be no A+ on a whiteboard coding test. It's straight pass/fail and an easy pass at that.
I realize some people have anxiety about doing this. Having helped or coached a bunch of people in the past about this, I can say that mock interviews and practice help the vast majority of these people. Some may be helped by doing this on a laptop in a shared doc instead, which is fine.
A few will struggle with the anxiety of that, even with coaching and practice. While I feel bad for those people, I do wonder if a reasonable-sized organization, which will generally require a certain level of communication skills and presentation ability, is the right place for you.
>which will generally require a certain level of communication skills and presentation ability, is the right place for you.
It's really not about presentation ability. It's "oh shit I have x minutes to do this and I have to get it right or I won't get the job, oh shit 5 minutes passed already and I haven't done anything" type of anxiety.
And you can't imagine that same person having this mental dialogue?
"Oh shit, I have to get up and talk about my project. What am I going to say? What if my director thinks I'm an idiot? What if they ask me a hard question? What if I can't answer the question? Will they cancel my project? Will they bring in someone to rescue me? Will I get fired?"
I agree with this. I work at a FAANG that does whiteboard coding interviews (opinions my own, etc etc).
When I interview people, I give pretty straightforward questions. My goal is not to run some algorithm guessing game. If I'm interviewing for a senior FE position, I make sure they know enough about async patterns in JavaScript that I feel confident that they could build an event handler without dropping promises on the floor.
A back end position? Can they use (not implement, just use) a hashmap efficiently and do they think about boundary conditions in input sets (you'd be surprised how many people ignore punctuation when breaking sentences into words - they just do a split(' ') and call it a day.
This is really basic stuff. I'm always able to (a) filter out people who just need more time learning to code (or are too nervous to perform well, which is still a problem with even these whiteboard activities) and (b) prompt a discussion about engineering practices.
My criterion is simple: do I think I could see this person submitting code to my project without me needing to constantly fix what they write? It isn't about memorizing algos or anything.
Aaalso: I didn't study leetcode, and managed to get my job just fine. I know there's a reputation of the process being just algos, and I'm sure a lot of the interviews are. But not all of them, even at the FAANG.
This is also a fair point. I usually do bad on white board coding. But I really enjoyed a white board interview that only tested my knowledge of javascript.
The problem didn't require 3 paragraphs of explanation and examples.
The interview involved a series of 1-2 sentence requirements that built on top of each other.
I interviewed a FAANG company and made to the whiteboard interview. The interviewer asked me to implement an algorithm to extract words from arbitrarily long string, which do not include any separators, using a dictionary, within 30 minutes and a code that compiles into $PROGRAMMING_LANGUAGE on the spot. Pseudo-code is not allowed.
Yeah, I tanked the interview but I really did not care if I pass when I got to that point.
I prefer the take home text. Less stress and I can really show what I know. Sure it's time consuming, but I always try to have fun with it. If I get a take home test, and it's writing some code, I know I have a shoe in the door.
I interviewed at one of these companies in the list. I'm nobody special, did the test, did the interviews which were more about doing the work and how it gets done and I how I think about it, and they made me a really good offer which I accepted.
Another tech interview thread. For the side that supports these types of interviews, I've never gotten a good answer to a simple question: in the year 2020, why are we expecting people to write compilable code on a whiteboard? It's just stupid at this point. Even a laptop that boots into some micro linux distro and has nothing but nano open would be better. Or just a fresh install windows laptop with nothing but notepad.exe open. Or a chromebook that is open to a HTML page that has nothing but a textarea element.
I'm not including a compiler or having build tools or an IDE. Just a basic simple text editing area that allows the basic functions of typing in text and editing it.
I'm against these leetcode interviews. But if you did nothing else but change this one thing, just stop expecting people to write code on a whiteboard or paper/pencil and allow them to write code the way it's actually done (a computing device with a keyboard and text edit area), that would be such a huge improvement. Writing code on a whiteboard or paper doesn't test anything. Think about how limited it is and how different it is (for example, you can't just press an up arrow and add a newline, have to find an eraser and start over).
It's yet another useless skill to learn just for interviews (writing compilable code on a whiteboard/paper) which also encourages rote memorization (because you have to get it right on the first try since editing the text is so painful and difficult).
I hope people start pushing back on this. The reaction should be, wow you care so little about your interview process that you can't get a $200 chromebook in here?
As somebody who has sadly had to make hundreds of candidates write on a whiteboard, here are problems I’ve run into getting the candidate a laptop:
* IT just fails: they’re unwilling to supply it, there aren’t enough, the current laptops on hand are junk, or the default password is wrong. These all happen with HR buy-in and funding.
* Recruiting just fails: they “have to move the candidate fast because of a competing offer,” they tried to work with IT but IT blocked them, they got laptops themself using their own budget but then ran out.
* I failed: I didn’t make time to prepare a non-whiteboard question, I didn’t tell my manager in each and every 1:1 that candidate experience sucks, I had three laptops and I forgot to bring my personal spare to the interview.
In general the core problem is there is zero incentive for hiring managers to do anything but shotgun candidate pools like they do today. There’s also no feedback loop to evaluators to help them improve. False negatives are completely tolerated despite there being a huge gap between CS jobs demand and CS graduate supply.
These are not good reasons and again shows that the company really doesn't care. IT gives a bad laptop? That's fine it doesn't have to be good, it just needs a text editor open. Get a USB drive with a linux distro and boot to it and open nano. Or bring your laptop and stick the USB drive linux distro in there and let them use it. Again there is zero excuse for a company to not have a laptop ready for an onsite interview.
All these companies expect you to write code using a site like hackerrank during the phone screen. And yet they can't have a chromebook open to the exact same site during the onsite? If that's too much, they can't even get a cheap laptop, stick in a USB drive linux distro, open nano and give that during the onsite?
I totally agree: the excuses are poor and companies don’t care. There isn’t much candidates can do. Hiring managers need to fundamentally change how they approach hiring.
Whiteboarding can fail. The pens might be dry. The room might be double-booked. The interviewer might be late or unavailable. Any interview setup can fail.
As somebody that gives both phone interviews with a shared typing environment and whiteboard interviews, there are some problems I can only use with a whiteboard, but no problems that I can only use with a typing environment. In particular, whiteboards are significantly better at letting candidates describe what they're thinking about the structure of a problem before (or instead of) actually writing correct syntax. When someone is sitting in front of a computer, they have a tendency to either jump straight into coding or try to describe their thoughts in a giant block of comments, neither of which is very productive.
Maybe it's just our phone interviews that allow us to focus on higher-level concepts in person, but I've really never found a candidate who did poorly on a whiteboard but seems like they would have done well on a computer. If anything, it's the inverse where candidates jump straight into code, to their own detriment, when placed in front of a keyboard.
The trouble with whiteboards are that some people think nonlinearly. If I'm writing an algorithm, I may write the "middle" first. On a whiteboard, this means I have to guess how much blank space I need to leave in various parts of my code so I have room to fill it later. I almost always guess wrong. On a computer, this is not a problem.
Now while you may be aware of this and not give the candidate a hard time for not leaving enough space, I read a Medium post once written by a Google employee with tips on doing well on the interview. He explicitly highlighted this problem, and literally encouraged candidates to learn to think and write linearly and not leave these kinds of blank spaces.
Yeah. From the interviewer's standpoint, there is no added benefit of having the candidate write code on a whiteboard, as opposed to a computer.
However, a whiteboard certainly can be easier to write anything that is not code on. Like for instance if the candidate chooses to walk the interviewer through a high-level approach, including a graph topology, before actually starting to code.
I know Google started letting candidates choose between laptop and whiteboard a few years back. I suspect others alos have.
> in the year 2020, why are we expecting people to write compilable code on a whiteboard
I think this is the bad assumption that's causing your confusion. Maybe this happens, but I think it's very uncommon.
The point of using whiteboards is to focus more on algorithms, data structures, and problem solving, not just spitting out code. If you make the applicant write actual code it limits the scope of the questions that can be asked.
So, it's actually beneficial that whiteboards aren't good for writing compilable code, because that's not what you want the applicant to do.
The issue with LeetCode style interview is that almost EVERYONE can solve those questions after studying a couple weeks/months.
The only thing that LeetCode questions predict is if the candidate has been training for LeetCode questions.
The fact that Google, Facebook etc still use it as a gatekeeping mechanism makes me believe they want to find cogs that will specifically spend hours studying for it. Making sure that those engineers will be ready to execute their mindless coding tasks at the company.
Right but in some ways this is what they need. Someone who can stare at computer science material, that is probably boring, for long periods of time. Someone who has a good memory for details, can remember routines. Can "solve" problems that have been solved before by applying theory. Someone who is willing to follow the rules and go through the process.
I think you made a logical somersault. Leetcode doesn't imply any of those, neither good memory for details or that you can "solve" problems that have been solved before by applying theory.
You missed the whole point of the OP's post it seems. The only thing that leetcode implies is that you can solve leetcode tasks.
Alot of people cant do it. If facebook is paying 450k to developers who can regurgitate leetcode I'm willing to bet theres some signal there. Its a pretty big assertion to claim the opposite
The reason companies like Google can and do it is because their funnel of incoming candidates is large enough that they can decide to choose the 'intersection' of many factors combined. Meaning they will filter down to people who trained for LeetCode questions AND they are also otherwise skilled.
Take home assignments can be automated the same way as leetcode tasks actually. So there must be something else to it.
I think people underestimate human nature in this case. It's a known thing that many interviewers just do the interviews for the sake of feeling superior than someone else.
It was another trend in the industry to ask stupid brainteasers like "Estimate how many gas stations there are in Manhattan" until Google banned the whole thing altogether. Google judged that these questions only serve to make the interviewer feel clever and self-satisfied. [0]
It's time to do the same with whiteboard interviews.
Identifying top tier tech talent is not that hard. Identifying yourself as a non top tier employer and setting your hiring bar appropriately - that is hard.
If you’re only hiring candidates who can pass Google interviews, what are you offering that Google doesn’t? (Hint: “we give you a 10 hour take home, then ghost you” is not a competitive advantage.)
What you really want to know is whether the take-home assignment really decides anything, or whether it's subjectively evaluated. Is there a rubric? Or is it the basis of an interview? A lot of take-home assignments are cargo culted; the people offering them don't have much faith in them, and still select candidates based on interviews and resume screens (or "pair programming" projects that are really just multi-hour or multi-day interviews).
I have a hard time believing that this many firms have take-home assignments that really matter, and aren't just another hoop candidates (or, maybe, disfavored candidates) are forced to jump through.
I’ve gotten to the point where it’s simpler than this. Firms get 16 hours to evaluate me. If the take home isn’t a huge component of that I know they’ve cargo-culted it. I won’t do it if they can’t promise that isn’t the case.
I say this as someone who was an extremely early adopter of take home tests and think they are the highest signal filter you can reasonably have in the US.
So many places have added take homes in the worst possible way without removing all the other nonsense. It’s as if orchestras started demanding high pressure auditions and then threw out the results if the person talked a good game in an interview.
Personally, the anxiety of whiteboard “coding as performance art” makes my brain freeze. I see it as a broken process. It certainly selects against me.
I think you can learn more useful stuff about a programmer by showing them a function prototype and asking how to black-box test it. Then show them complete code, and ask them to white-box test it. Test cases or a test plan, depending on scale. Then a discussion of test tools they have used.
The people who can produce quality code for your organization will show up pretty fast.
In higher education there has been a lot of walk back from tests like the GRE in most fields. It turns out, being good at the GRE only correlates to your ability to be good at taking the GRE, and doesn't correlate with actual graduate school performance or outcomes.
Performative interviews occupy a similar space. You spend all this time and effort studying for something at is ultimately irrelevant outside of the interview. Just a pure waste of brain cells and calories, with a few new gray hairs for you.
As somebody who has been interviewing recently, the format I've preferred is from TestDome. I've never used leetcode or others but I imagine they're similar. I had 4 coding problems for which I had to pass a set of test cases which were explained from a high level. The first 3 were fairly easy and had time limits of 10 minutes or so. The last one was more involved from a design and functionality perspective and had a limit of 45 minutes. Then, it was followed up by a conversation with a developer where we discussed the problems and approaches. The time commitment was finite and reasonable, and the opportunity to explain my code was appreciated.
One of the most telling questions I ask when interviewing is “what was your favorite or most complicated project you’ve worked on and how did you solve the problems you encountered?” I don’t need the interviewee to show me a perfect implementation of a red/black tree. Hell, I don’t remember it. But I want to know what they found challenging and how they went about solving it. Often times they will give me all the technical details I would have asked for, but in context. I don’t care that you know the exact name of that one function in PHP but if you mention it off hand in relevant context, I will place that much more trust in the fact that you actually used it instead of memorizing it.
Other questions I ask are things like do you work on your own projects and tell me a bit about the tech behind them, what’s your favorite pieces of open source or free software (I work with almost exclusively FOSS stuff though my own work is not FOSS), what do you do for fun, and if we were talking on the phone and I had never made tea, walk me through the process (this is a fun one). Then take them out to lunch if possible and watch how they treat service workers. I think this has so far worked well for everyone involved. At least as far as I know nobody I says yay to has been a bad hire.
Eh. I don’t care about downvotes. But yeah if people want to throw the same ten dozen puzzles at each other and then wonder why the people don’t work out, it’s definitely not time to try anything different :)
You're selecting for candidates that either have the kind of memory that lets them recollect those experiences on demand, or can fabulate it. Once word gets out about your interview process candidates will just prepare something ahead of time, as they do for the HR interview already.
What was the last interesting project you personally worked on and what were the challenges with it? What technology did you use to solve them?
I mean if you literally can’t remember anything of what you’ve worked on ever, no I don’t think you’d be a good hire. And also this part is surprisingly hard to bullshit. You can say “well I worked on an eCommerce site and it was hard because we had a lot of visitors” isn’t a full answer. If that’s as much as you can give me, no I don’t believe you contributed much to that project. Maybe I’m wrong and you are brilliant and just can’t remember, but it’s a risk of false negatives that I think most employers will find acceptable. But: “It was hard because we had a very complicated system for determining the final price of the whole cart because of discounts offered if you bought certain pairs of products together. I didn’t implement that system and we had plans to eventually rewrite it but in the meantime we started calculating the prices in our task system instead of during the web requests. It reduced web server load and made adding things to the cart feel faster.” Me: “OK that sounds neat but how did you prevent race conditions? Say while the first calculation was running the customer added a second item that triggered another calculation? What if the first task finished after the second?” Then: “Ah yes I recognized that it would be a problem when writing that code. At first I tried using a lock using Redis for the cart but that just made the tasks serial and the whole thing took longer. Plus if a task failed or was interrupted it could cause issues. So instead I just put the latest task ID onto the cart object in Postgres so I always knew which the latest task was. When a task finished and wanted to update the price first it would lock that cart with the Redis lock, then check if the ID still matched its own, then update and unlock.”
The above tells me how they think and what they think of. If I’m not getting enough technical detail out of them I will ask stuff like “what Redis primitive did you use to create the lock?” But usually the interviewee will go into lots of technical detail all on their own.
It's definitely important to prove you understand the core concepts required to be a successful developer, such as using efficient & scalable approaches to solve a problem, but writing quality code from scratch simply doesn't mix with high anxiety. And it's not representative of how programming work is done in the real world.
I think a good middle-ground is pair programming where the interviewer writes the code based on the pseudo-code provided by the interviewee. This way, the interviewee displays their thought process and communication skills without being bogged down by programming language nuances.
This is yet another tech interview discussion, so I'd like to highlight a subthread from the one earlier this month:
> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.
>> I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages.
>>> I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol.
Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.
Even as we debate the validity of the metrics measured by technical interviews, it's also worth considering the metrics used to measure engineer worth in the industry at large. This existentialist "fear and loathing in FAANG" discussion was quite illuminating, and a little sad.
There's no rule that says you can't apply for more than one role at a given FAANG. It's not like they blacklist you for not getting hired in one role. They know they have an intentionally high false negative rate, you get to try again.
> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.
Makes you question if this is the precise point of it.
470 comments
[ 2.7 ms ] story [ 289 ms ] threadP.S. I used to have extremely successful in person networking before COVID19.
As much as we'd rather it not be the case, even for programmers writing skills are incredibly important. This comment is contains a few grammatical errors and is quite hard to read - I checked your comment history and many others suffer from the same problem. As some who hires, if I read a job application with this quality of writing, I'd probably dismiss it.
If you're having trouble finding a job, it may be helpful to have someone read over any written material you submit alongside your CV (cover letters etc) just to make sure it's clear and easy to understand.
I do understand what you say. But that is not the case sadly.
Yeah, I had a hunch that might have been the case, but didn't want to make too many assumptions. I can't string a single sentence together in another language, so I can only guess at what a pain it must be to deal with on a day-to-day basis!
Also, to the point of the post, do you consider whiteboard interviews to be good or bad for your situation? With covid it's harder for companies to find which applicants should be interviewed in the first place, and short questions are helpful for filtering out people who can't code at all.
About the leetcode thing, yes, I used to be pretty code. If you send me a link right now, I might not be able to do it, since I am living under huge stress right now, but with 1 week or 2 training I can get 80-90% in the coding challenge easily.
I'm sorry to hear about your stress. The worst part about the recruitment process is that it doesn't put folks in a typical development situation. I hope you're about to find something!
Japanese is my third language, so when I was shipping around my Japanese resume in Japan, I had an experienced native speaker review and revise. As a non-native speaker, there's a lot of nuance in writing that you just don't understand.
Your resume is often your first impression, so you want it to glow as much as possible. Honest, but glowingly so. :)
Might help you in terms of landing better jobs.
[1] https://news.ycombinator.com/user?id=DoreenMichele
Nothing on your resume will ever cause anyone to say, “We simply have to hire this guy.” On the contrary, the slightest typo, grammatical error, or stray bit-flipping particle will mean that your resume is instantly discarded.
When they do this to your resume, they are doing this to a representation of you: they are metaphorically throwing you in the trash like worthless garbage.
Are you offended by this? You should be.
So don’t use a resume.
Figure out some other way to get on the phone with the people you need to talk to.
You are no longer a tech wiz. You are in sales now. Your job is “selling yourself”.
Congratulations.
Scrolling through there are a lot of high-quality companies (Auth0, ASOS, Blue Bottle, Basecamp, Buffer, Canonical, CircleCI... and more and more).
Just because Google or Microsoft aren't on the list doesn't mean these are backwoods operations. Those companies don't need to bring in new candidates by being on these lists.
And Elvie! (If you make it to ‘E’)
We’re hiring right now :) https://elvie.workable.com
If you want some one who can build web sites using React JS you are better off hiring people who know and have built things using React JS. There is little logic in hiring people who can do some obscure fashion-of-the-week algorithms, assuming if they can do this they can learn React is just plain wrong. A person does what they are good at doing, if some one is good at interviewing their incentives are in changing jobs often to find the next company that can offer a raise. Not getting your work done.
Also this whole thing that one must know these algorithms because they might once in a life time face a need for Dijkstra's sometime during midnight at a place where they wouldn't find an internet connection, is unrealistic. C'mon. Get real. Companies are full of situations where people are digging with shovels and spoons, because people don't have the skills to use and build tools and system to save thousands of man hours of manual effort. Code bases are full of tech debt. Deployment problems because tests aren't written, or there is just no CI infrastructure. Lack of skills and productivity is a far more realistic and commonly occurring problem than these once in a decade algorithm needs. On top of this comes the need for proactive problem identification and solving(a.k.a innovation).
The fact that FAANGs have to acquihire or acquire companies to grow shows they are not hiring the right people either.
As of today if you are hiring for devs. You must look for turn-key projects executed, expertise in one main programming language/stack, ability to script quickly, skills to build tooling/monitoring, ability to produce deployable code, code maintenance, writing test cases etc etc.
I prefer take home projects as opposed to whiteboarding, because my personal situation better allows for it - no wife or children, preference for building things, no NDA at my current job, and so forth. For me specifically I would prefer take home projects followed by a debriefing where I explain my design decisions over solving whiteboard problems. But that's just me. Why can't there be multiple accepted ways of interviewing instead of forcing everyone through a whiteboard gate, if whiteboarding isn't the best way for you to prove your competency?
At least we have lists of companies with different hiring processes as in TFA, so there is a small amount of choice. But we're still dependent on the goodwill of companies to make their hiring processes more tolerable.
Very few people who are successfully employed would take a risk / have a luxury to forsake their certain employment for an extremely short term contract (with all the registration, taxation, regulation etc headache this implies!) and extremely limited chance of gainful employment.
(it's perspectives like these that make me realize how much of a grouchy old man I am and have always been - always being a buzzkill by thinking things through next 3 steps! :P)
It's not if you are in the US.
You will get a 1099 from where you worked, you write it on your 1040 and include the 1099 with your taxes if I recall from when I was an independent contractor for a while.
It takes maybe like 10 minutes extra and 1099-ing is incredibly common nowadays (unfortunately) so not a super weird case.
And it's not regulation. You don't need to worry about anything besides getting the 1099 like you would your W2s and appending it.
What if you don't have any other 1099s (so you have to pay self-employment tax just because of that one interview, at least I had once when my moving expenses were paid with a 1099).
Not even getting into possible illegal work for the candidates on work visas.
>It's not if you are in the US.
Its also not if you are in Australia. I'd suggest that the number of developers and designers who at least occasionally take on paid side gigs or who have revenue generating side projects (even if it barely covers hosting etc) is about 1/3.
What you are saying in effect, is that hiring regimes can't be (or ought not to be) designed to include a group that is (commonly) discriminated against, if that discriminates against anyone, particularly the majority.
Implicitly, I see you defining the optimum as where everyone uses the same criteria that are as inclusive as possible, and just shrugging and saying "we're doing our best" when some are left out in the cold completely.
But isn't there an alternative vision where the optimum is employers using different ("diverse") sets of criteria that individually discriminate but create a society where there's a place for everyone?
My best effort:
* Who/what is the group you feel is a "commonly discriminated group", that would benefit from "contract first" hiring practices?
* If a company uses a practice that "discriminates against a majority", is that a practically beneficial funnel/sieve/practice for the company? Alternatively, if we are discussing companies which are making a stand / moral decision through hiring practices, I feel that's a very different discussion than what I interpreted original poster to posit as "best hiring practice for everybody in all situations / everybody should do THIS".
* I feel your last sentence is proposing each employer using different criteria in hope that cumulatively they'd create an overlapping and comprehensive venn diagram. My response to that notion would be a) I don't trust either free markets or regulation to make that venn diagram complete and b) that kind of stratification of the market sounds horrible... no matter what, I can only go to specific companies who have specific hiring practices that happen to match me? (I fully understand the irony in that this may in fact be the underlying unspoken reality of the market! my point is that it should be something we move away from, not explicitly design as a goal).
But again, I may be misinterpreting your argument and inadvertently taking discussion in the wrong direction...
Not exactly; whether my message is politically correct depends on how you generalize it and what groups you apply it to.
You could use it selectively in other contexts and demonstrate either a left or right wing allegiance, but right here and now, I'm talking about people suited to whiteboarding vs. take home tests as the groups.
>that kind of stratification of the market sounds horrible
It's what we've got though, and always have. In your words, "the underlying...reality". Of course we agree things can be improved somehow. I think hiring should be more inclusive, and I think most people would at least give lip service to that.
But if different people are always going to be different, then a hypothetical convergence on a single "fair" standard means telling some fraction of the population they have no place in society at all.
Eliminating whiteboarding, strictly speaking, seems reasonable to object to. It would be unfairly discriminatory. But wanting that to be the only standard also is an extremist position.
If you want to draw a more general opinion out of what I wrote, one of the things on my mind is the general scourge of optimization by tech people who assume the solution to anything is to find the one best answer based on some metric.
It is a good idea, but nobody is going to be quitting their current job to be a contractor with a job MAYBE attainable. It remains me a lot of retail seasonal employees who work their butts off to MAYBE get a full time position.
I'd also want there to be a clear decision-point (i.e. it isn't a way to get you to be an indefinite contractor, they make a hire/no hire decision within some set timeframe or at the end of the current work).
source: work at a place that does this
What’s your threat model here? People making up elaborate lies about code they’ve written?
Likely people making up elaborate lies about code they didn't write. It's trivial to push code of which you are not the author.
Why would I leave a stable well paying job for a job that basically says they don't trust me enough to not fire me in two weeks?
Manipulating one or more git repos to claim it as yours then ensuring your lie is not uncovered by googling would be technically challenging enough that those who could do probably don't need to.
Sprinkle in some bugs. Start with easy spelling errors and go all the way to concurrency problems.
Ask them to add a few simple features and fix any bugs they find. Bonus points if they find bugs you didn't add intentionally. Good unit tests are also a bonus.
Give them a couple days or something. Don't count off if they don't get everything done. It's better to do a few things correctly that fix everything with hacks.
Invite them to email questions/suggestions to the hiring team. This person might be on your team so it's nice to find out early if they have good ideas or they hound you with stack overflow questions all day.
I've never had this kind of interview but it sounds nice to me. It's more realistic to real daily work. You can cheat by having someone else do the work but that's always a possibility.
Whiteboardimg a solution to n=np or building full applications is crazy and not realistic. Most interviews do that yet most work is fixing bugs and adding mundane features. Why can't interviews focus on the actual skills needed?
Without looking at the commit history, given enough time this is a challenge. I often don't recognize code I wrote myself 6 months ago.
Ask questions about the code:
- why did you write this in this way
- explain how this works
- if you needed to add X into this, how would you do it
- are there other ways to achieve what you did here, what are the pros/cons of this?
- what would you do differently if you were to re-write this?
- are there any bugs or common issues, how do you go about diagnosing and debugging them? Could you write the code differently to be more resiliant to those issues?
Anyone who wrote that code (assuming it was relatively recently) is going to be able to talk about it sensibly, and be able to dig into the code and show how certain things are done.
It was very obviously there as bait for people like you and it was removed as soon as they took a new job. It was never used in production in its "released" form.
I had no problem with the person's abilities to code, but for reasons like this I probably wouldn't work with them again, or hire them as a eng lead or higher.
I'd bet they could also pass a whiteboard, pair-programming or take-home test too.
That they are willing to lie about it and pass off other's work as their own isn't something that you're going to identify in your typical code interview. Using interrogation tactics to try and weed out the frauds is going to be counter-productive, by driving off all the good candidates.
But I'm not sure, I've never interviewed this person.
I put the anecdote up as a real-world example of one way the GGP's github reference can be faked.
You would have to dig really hard into their knowledge of the system to catch them, which you wouldn't have time to do.
For instance, I believe the issues would show up in a benchmark.
People want hiring processes to be somewhat consistent, and in that world it's really kind of a non-starter to replace tech screens with GitHub profile reviews.
for that matter, GitHub is useless if the projects you contribute to are on BitBucket or GitLab. that contributions graph has a lot of assumptions baked in.
So anything on github that I might want you to look at? I’d have to be on a three year replacement cycle, permanently, which is a hell of a lot more work than you might estimate that a reasonable github presence would entail.
Maybe the solution is for companies to hire more thorough managers who are capable of vetting a github profile to see if the candidates skills are relevant to the position, not turn the interview process into a performance.
That kind of interview mitigates the amount of trivia and preparation-gaming while also ensuring that the applicant can solve problems as a team and is a competent programmer. It is cheatable in that one can memorize implementations of some features, though it seems somewhat difficult and not more cheatable than just memorizing a bunch of CS trivia.
Going through my GitHub account however isn't good. It's full of quick one shot things, experiments and so on, very little production ready code and not representative of anything.
We place the candidate in a room with a large whiteboard for 15 minutes alone with the lights off. I then walk in with a 6 piece dresser [1] from IKEA (obviously keeping the lights out). I explain to the prospect that this particular dresser is missing one piece - BUT we need the dresser assembled then disassembled and placed back into the box [without anyone seeing]. More than HALF the candidates become violent and demand to see my github account - some even suggest I have a mental issue. I am often able to calm the candidate down - this way we can get into the REAL debate: KOPPANG vs MALMs. Any candidate caught assembling the IKEA dresser is quickly dismissed out of spite. This technique is from Boz at facebook - it is how all* senior principle engineers are screened at FB and the reason everyone codes PHP (but swears they do not).
HR has complained that my techinques are "ridiculous" or borderline illegal - but so far my successful candidates perform, tend to have a solid grasp of typescript and quickly move into senior management.
https://www.ikea.com/us/en/p/koppang-6-drawer-dresser-black-...
My current employer doesn't have explicit rules that I'm aware of, but early on, I asked our HR person if I could work on open source outside work and they said something along the lines of "what's open source?" Which, I mean, on the one hand what do you expect from HR, but on the other hand, the entire agency is concerned with operating, maintaining, and developing a software system.
If it's not on employer-owned equipment during core working hours, then I believe it's pretty difficult for an employer to enforce that kind of thing. I don't know how important to you working at that charity was or participating in open-source now is to you, but if you're willing to take a harder line of telling rather than asking, you might find that when push comes to shove, there might not be much they can do without drastically escalating the situation (not always an organization's favorite thing to do, even if they can technically win, which they might not be able to).
Obvious disclaimer about being a non-expert on this sort of thing, but really just wanted to point out that the first answer in this situation may not actually be the final one.
And its not really the "employment contract" in Anglo Saxon legal systems its pretty much assumed.
Also, running side businesses was not allowed.
Of course, the normal thing to do was just ignore the rules, because most likely nobody would notice or care. I explicitly asked for permission because I wanted to see what would happen.
This might require engineers unionizing, but it would be a boon to the field. Patterns buried in closed source vaults would be disseminated and innovation would surge. It might usher in a golden age of software if everyone were able to freely share their intellectual ideas with their peers and not have to worry about risking their health insurance and home in the process.
It obviously depends on your definition of "better". And it's yet to be seen whether whiteboarding is "better" than pulling candidates out of a hat. You'll note that lots of companies that do strict whiteboard-style interviews are also extremely comfortable firing employees. Would you be "better" just hiring anyone whose resume seemed applicable, that could talk like a normal person, and then firing them if it didn't work out?
As an introverted and generally very socially anxious developer, this sounds like a dream compared to a standard interview, which has always felt to me more like an interrogation and generally very aggressive and hostile. I'm confident in my skills and I can communicate well in other situations, but the typical interview environment is cripplingly stressful to me. I would love to just get through a talkative lunch instead.
I think part of the problem is not everyone is good at giving interviews. But most software engineers can talk about their work once you get them going.
FWIW, I think white boarding interviews are pretty fun.
IMHO the worst of these options is coderpad/hackerrank/etc. - it's robotic, discourages pseudocode, and perpetuates the idea that rote memorization is required to pass an interview.
I imagine the 50th percentile get something more like 100-150k, which in the bay area, compares to perhaps 60-80k in the middle of the country.
Do you have any statistically significant data to back this claim?
That whiteboard interviews are actually testing how you perform under stress rather than how good you are at your job.
Additionally, in my opinion, it measures how good the interviewer is. If you get a bad interviewer or even just one in a bad mood, just pack your bags because you aren't getting it.
Whiteboards are incredibly arbitrary in my opinion, if taken alone. If it is part of a complete assessment, ehhhh.
Interesting study and really good discussion on HN at the time, but the opposite of "[Whiteboard interviews] often give a very good and reasonable signal that often has a high corollary to work performance" would be something like "People who do well at whiteboard interviews tend to be worse performers at work." I don't think that's what the study said at all; rather that people do worse at whiteboard problems when they're being watched vs. not being watched.
Not. All. Anxiety. Is. The. Same.
Ask a firefighter to give a speech in front of 1000 people. This is an entirely different kind of anxiety than running into a burning building. Ironically, it may be the former rather than the latter that makes the firefighter sweat.
Writing code is generally not a performance art. We're not stage performers. Interviews are an unusual situation that aren't anything like our daily jobs. Stress on the job is not the same as interview stress. Work emergencies are not the same as interviews.
We need to take into account human psychology. These factors are all well known, but the tech industry seems to act in deliberate ignorance of humanity.
One you are getting paid no matter what, the other you aren't and are 'performing' in the hopes of getting paid.
Assuming the study’s results are valid, that’s really all the justification we need to get rid of these tests.
if properly trained, interviewers ask questions that build without throwing the kitchen sink at you.
whiteboard interviews (when done holistically) assess for signal, not binary correctness
- do you have a solid framework to build a solution?
- are you considering multiple approaches / data structures / complexities
- referencing similar problems, vocabulary, situations to illustrate breadth of knowledge
sure, the stress component isn't ideal, but there are multiple relevant capabilities being assessed in a very short amount of time...
in other words it's a much better test of tacit knowledge than alternatives
if done right, whiteboarding should be almost the same as pair programming -- an iterative dialogue
WB interviews should be about problem solving not regurgitation of rote learned algorithms
I won't ever fault a candidate for failing to close a paren or being unsure of whether the size of a container is .length() or .size(), but if virtually every line a candidate writes is of questionable value, I'd say that means something.
I actually had the pleasure of interviewing at one of these companies before. They had a take-home project, which was to choose and implement and couple enhancements to a toy app. In the subsequent conversations, we discussed how I approached the problem, details of my design, technical tradeoffs, etc -- all the sorts of things you would expect a professional sw engineer to be able to do on the job. It was challenging but fair, and I was struck by how reasonable the process was. It left a wonderful impression on me.
Take-home projects are not a perfect solution to the interview problem. A major issue is that they require candidates to have free time outside of work to complete them. Personally, though, I'd much rather spend a few hours per application working on projects than countless hours prepping for stressful whiteboarding puzzles. As a developer who prefers to digest problems slowly, I look forward to the demise of on-the-spot whiteboarding interviews.
If you work at a company that subjects candidates to such high-stress algorithmic riddles, and you have the ability to make your process more sane, please consider following the lead of cos on this list.
Last year, I did take home test for 5 different companies and all 5 of them invited me to the next round. I yet to have a successful whiteboarding interview.
Just like onsite interviews. Any interview process will require candidates to spend time outside of work. The only way to fix that would be a true, universally recognized certification/diploma that would prove that you can write software, but I don't see that happening any time soon.
And unlike an onsite interview, I don't have to take time off during working hours to do them.
My mental model is that I'm willing to spend N hours in total for the job application process. I'm happy to spend some of those hours working on a take home problem as long as that is instead of and not in addition to spending those hours doing multiple rounds of interviews.
I think the best approach would be giving the candidate a broken app and have they debug it live or in a take-home debug-and-fix or debug-and-improve exercise. Could also have some refactor assignment.
Has any company tried something like this before?
https://www.quora.com/What-is-it-like-having-a-practical-int...
Its a one-hour hands-on programing lab via screenshare - a pretty straightforward task of consuming some data from an API, processing it a little, and displaying it. We use Ruby for our work, and ask the same of our candidates in the lab. No handwritten syntax, exotic/tricky algorithms, or gotchas. Just a task to see how a candidate approaches a problem, deals with requirements, and implements them in our language of choice.
Happy to go into more detail if anyone is trying to implement something similar in their own hiring processes.
1) It should be a live exercise (at least mostly live). I'm less interested in the end result than how someone got there, and most people chose to narrate their thought process as they work through the exercise.
2) The candidate should be able to work in their own environment. Online assessment tools can be different enough to introduce friction in a timed interview scenario. I'm interested in seeing how fluid a candidate is with the tools they'll use every day.
3) It should have the same expectations as your day-to-day work. Did the editor have a linter configured, and did they use it? Were tests written for the parts of the lab that introduced nontrivial logic? Is there a README? Is it all one git commit, or did they habitually make commits as they worked?
4) The design of the lab should have a decision point - I'm currently using a lab which could be implemented successfully with either a relational database or some kind of cache store. The strongest candidates recognize there are multiple paths and can justify why they chose the one they did.
This works best for reasonably experienced devs who can actually be expected to create something from scratch from a set of requirements on a short time frame. For more junior people, I'd typically give a version of this lab that I completed myself and introduced a couple of bugs into. The task is to fix the bugs.
Our test is actually quite simple and everyone that did any kind of development should have done these basic things.
But it tells me a log of the current state of the candidate. Most code samples on internet are not production ready code. Security and error handling are mostly not included. So this is where a candidate can shine... but I'm happy if you can string it together working even if it is by googling.
So you're saying spending 3 and a half hours on the assignment is fine. It's not.
Take-home assignments have little to no value for the candidate, they're only feasible if you are jobless or in college. The last thing I want to do in my spare time is to stay focusing on a bogus problem that may end up discarded.
Talking on-site or on the phone? Is that better spent time
Obviously no one is going to submit an app that doesn't accomplish the primary task. I feel like you get judged more-so on how you decided to get the initial app up and running, which likely isn't something you'll need to do working there.
I would have guessed the same thing before my company started doing homework interviews. But in fact, the ones that don't get to the finish line are sometimes the most interesting ones to review, and they aren't an automatic no-hire. (And we say so explicitly, if a bit more diplomatically, in our instructions.) If someone hands us clean, well-thought-out code and it's clear that they honored our time limit and just didn't quite get there before the clock ran out, we'll still talk to them.
They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.
There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.
As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
With a 2 hour exam, there's an end to it, and you know how much time it should take to do the problems. Much better.
It's even worse if you're being graded on a curve, as you would be in a job interview. You're competing against others who will spend infinite time on it.
When we get a take-home submission, we essentially run it through our code review process, and as one of the engineers who does reviews, it usually takes me nearly as long to thoroughly examine a homework solution and write up my feedback for internal discussion as it would to do an in-person interview.
Being able to review a candidate's code on my own schedule rather than being yanked from the middle of some knotty problem to go do an interview at a fixed time is a big quality-of-life win for me. But the rate limit isn't much different than it would be with regular interviews.
All that said, it's totally possible our process is super atypical.
I'm going totally nuts about this particular problem. I have written in as clear terms as possible that I really don't want the candidate to spend more than 2 hours, but I don't feel I am going to convince anybody - especially as this is a (mini)game programming test, and with games you can always add one or ten more things.
> I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?
- Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like?
I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?
Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know.
In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).
Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof?
There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on.
I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.
I really enjoyed this session, but I can see how it could be stressful to others.
So we set up a small project using this framework, and mildly wrecked it. Made some really basic mistakes, made some more subtle mistakes, added a bunch of dead code, wrote good tests for some of the broken stuff, wrote broken tests for some of the good stuff, etc.
Candidates came in, and we asked them to add some simple feature to the app, which we warned them may be broken. We had someone sort-of-pair with them as they work, to watch what they do and keep them on the rails to an extent. We then evaluated whether what the candidate did was sensible: did they run the tests, did they pay attention to the failures, did they find and fix the mistakes, did they get distracted by the dead code, etc. Candidates didn't have to do everything perfectly, but there is plenty of scope to go really wrong. I remember one case where the very first thing the candidate did was delete all the tests. Thanks for coming in, we'll let you know!
The company did quite a lot of rescue missions, where we would pick up a project that another consultancy had started and failed to deliver, and get it live. This interview wasn't just a trick, it was testing everyday skills!
I had a company do this and it went really poorly for me. It was a python app and at the time I wasn't very expert in python. They told me to use my laptop and all I had for python was a vim environment. I knew enough python to do white-boarding problems. But debugging a complex app exercises a whole different set of skills I just hadn't developed in the language. I was also learning vim on the side. I looked like an incompetent amateur.
So there's a language and tooling issue. The bigger problem is that debugging an unfamiliar code base under time pressure is almost as niche as doing algorithm problems under time pressure. I've been on enough company wide on calls that I actually think I'm pretty good at debugging unfamiliar code under time pressure. But if I have someone who understands the code looking over my shoulder, they're usually at least trying to help.
It's still time pressure. It's still a somewhat contrived problem and code base. The interviewer is still comfortably omniscient. And it introduces some more complexity due to unfamiliarity of the language, tooling, code base or problem space.
Ultimately, I'm not sure it's any better than white-boarding.
A lot of these problems are solved with a take home question but then all the problems with take homes rise in their place.
I mostly understand why some companies do the x hours routine, and when I've done interviews this way I've historically performed well, but it just felt like it added unnecessary stress. For instance, anyone on the job doesn't have to worry about issues like, "What if I couldn't get the app up and running because I had the wrong version of X installed?" longer than maybe their first day on the job. And if you plan on hiring an engineer for a few years, one day is irrelevant. At best this felt like it favored contractors who were more experienced at jumping into new projects frequently.
A good middle ground would be to have the company make a donation to the candidate's choice of nonprofit.
Now one worries, is this itself a test? Are they judging my desire to donate.
Then I'd be worried... are they watching to see whether I engage in hollow virtue-signalling?
It's kind of like at salary negotiation. You can impress them with how you really are there for your passion not materialism. Or you can get money.
Donating to charity has tax implications for the company too, but they're much simpler.
And for that matter, giving real-but-unpaid work to do as part of an interview may be illegal. So paying for your time may be something companies are doing to protect themselves taxwise, not just for good will.
How is this enforceable? Even in the case that someone finds out about it, what’s the company going to do? Sue the interviewing company for the rights to a few hours of work?
What are the damages?
As for suing you? Again what are the damages?
It’s almost certain there are no damages assuming you are just working on a take home project not something that they are going to actually use to compete with your employer.
Just because something is in the terms of your contract doesn’t actually mean you really need to worry about a lawsuit.
A company might fire you if they find out, but the chance that they fire you is to small to worry about.
And even if you don’t do take home work for pay, you don’t want your company finding out you are interviewing. Taking a token payment doesn’t change anything practically.
People have contracts with all kinds of stupid things in them. If they aren't practically enforceable, they can safely be ignored. Any company saying they don't pay for take home projects because of employment contracts is just using that as an excuse not to pay.
There are people out their with contracts that say they can't even interview at rival companies, how many companies out there refuse to conduct interviews because of this?
I think this is a key piece why this kind of project is useful, because that's exactly what they'd do when they start in their coding role: jump into an unfamiliar code base and fix a bug, implement something tiny, make things better along the way.
When hiring for a React role, I went to find an open-source React app and simply asked the candidate to mod it a bit. We then discussed the why and how, but also how they liked the app and why.
We only gave this task to the final candidates, and the ones that had React experience were done in very little time, because typing wasn't the issue here, but reading unfamiliar code and understanding the mechanics of the app.
Writing a hello world app from scratch or coding fizz buzz doesn't tell me enough.
Also many companies will insist on a take home project in addition to everything else, including a whiteboard interview. So you can pour time and effort into this project only to be eliminated further down the line (or just ghosted).
It's like they assume applicants are applying to one job at a time and have no other work or life commitments.
I dunno, it feels like the tech industry has forgotten how to interview candidates, as in "just chat with someone like a normal human being to see if they are a good fit for what you need", and like many other aspects of the profession have fallen for dumb processes, fads, cargo-culting, outsourcing to third parties, and dubious technical solutions in lieu of experience and social interaction. I learned this early in my career, and found I can pretty much size up someone's technical chops - assuming they're in my wheelhouse - and get a good idea of their professionalism - with a short, half hour phone/Zoom interview. Yes, I can see how this might lend itself to bias and prejudice, but it seems the solution there is to train interviewers to overcome biases and have more diversity (in all senses of the word) in your interviewing team.
Not everyone has the inclination or time to code outside of work. Many of these people are still very competent developers.
But frankly I don't really buy that excuse. Those people have time for multiple coding tests but they don't have time for a (small) public project? Ehhh. Not convinced. If you can get away with no portfolio to begin with then you're not going to be inclined to accept coding assignments.
Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.
I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [0].
For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [1].
Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can’t make any assumptions on what the candidates know. I’m not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.
[0] https://economictimes.indiatimes.com/tech/ites/95-engineers-...
[1] https://blog.codinghorror.com/why-cant-programmers-program/
Take home project? There goes 4-12 hours per company that gives you one and sometimes more. Companies have no incentive to cut it down or not give it to even marginal candidates so you'll get a lot more of them than full in-person interviews.
Pair programming? That's basically white boarding with a better white board or, in the remote interview world, just regular white boarding. Sure the problem is more real world, in theory, but then you get issues of knowing the same frameworks, etc. So to do well you need to spend a lot of time beforehand studying their specific frameworks and code bases if they ask you to do an issue on their open source project.
It is very easy to completely forget all about graph algorithms in, say, 3 - 5 years during which time you've been a productive, valuable member of a software development team that...you know...actually builds things customers care about.
Being a productive, valuable member of a software development team is teaching you things that will set you up for success in higher level roles. You won’t be taken for those roles on Leetcode interview performance alone.
I think the problem is that for some teams the entire process is some combination of leetcode-style "technical" interviews and then some "culture fit" interviews, without any chance to talk about actual dev work.
You’d be surprised.
I mean, maybe hard or esoteric easy questions but I don't think that's true for basic things like 'find the first duplicate' or '-traverse- this graph'. When people rail on whiteboard coding, I think they mean obscure algorithmic questions and not fundamental data structure questions.
I don't expect someone to be able to do Dijkstra's algorithm (or even spell Dijkstra, did I do it right?) but everyone should be comfortable enough with a bit of help doing BFS or working with linked lists, hashmaps, no? Am I crazy here?
It's not really that difficult to make quotidian software engineering tasks into interview questions. I've at least seen it done successfully before.
More anecdotes: a nontrivial number of the "algorithmic trick" interviews I've done were questions I knew the answer to beforehand – and often the interviewer didn't care when I told them ("I just want to see how you think" or "Just show me how you'd code it" or sometimes even "OK, just tell me the solution and we'll move on to the next part"). What happens to the people who don't have the same background? Are they really less competent software engineers because their algs class didn't cover making a queue from two stacks?
"Find the first duplicate": In my actual SW roles, this was trivial: Use a hash/map/dictionary/set. However, this isn't a typical Leetcode question - often it will be with additional constraints (cannot use a set, need O(1) space, etc). I don't doubt there are domains where this is important, but I've never had to do this in my professional work.
"Traverse this graph": Probably had to do it a few times. Less than once a year.
BFS: Never for my job.
Linked lists: Never for my job.
Yes, it is good to know what a linked list is and know if your library's data structure is using one, but again: That's not your typical Leetcode question. You'll be expected to write a linked list to solve some problem.
I do get what you're saying: Stuff like linked lists and BFS are fundamental, even if most SW jobs almost never require them. It would be a red flag if a candidate didn't know the basics about them. It's another thing to ask them to code it on the fly.
Going back to my electrical engineering experience: We have to take and use a ton of calculus to get the degree - much more than a CS grad will take algorithms. Yet it's a common trope that most electrical engineers will never use calculus on the job (literally not hard to find one who had a whole career in EE without using calculus). If you're doing research or are at the forefront of a discipline - sure, you'll need calculus. Or if you're doing electromagnetics, etc. The theory is built on calculus, but you don't need to know it to apply the results of the theory.
Would it be a red flag if a candidate clearly didn't understand the basics of differentiation (i.e. rate of change) and integrals (i.e. summing quantities)? Perhaps.
But should I ask someone to do a tricky integral for an EE role that will not require the person to do any calculus? I would be criticized by my peers, and if I kept doing that I would be excluded from interviewing. In fact, even if I ask them to do non-tricky integrals I would be criticized.
Asking people to figure out a trick to solve a linked list problem is equivalent to asking EE candidates to do tricky integrals or differential equations.
I personally when I interview candidates prefer to get to know them and how they think before I even try to understand their ability to problem solve. Knowing the language or algorithms are all easy to learn things. Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.
That's what leetcode used to test and then people optimized for it and now it tests memorization. At scale things work differently than in isolated cases. I've also found that most problem solving interviews really test how well I've seen the problem before and how well I can lie about seeing it before. Interview enough candidates and the good problem solvers will look dumb compared to the ones who simply saw it before and can act like they didn't.
If i'm really serious about joining a company and the many months/years to come with them, I can probably spare a day to show it.
It's always surprising to read people complaining about spending even 8 hours on a take-home problem in the comfort of their home, with no one looking over their shoulder, finished at their leisure. Meanwhile, anyone serious about applying to the same company wouldn't hesitate to take PTO and leave work to go on-site for a similar amount of time for an interview.
And then they didn't even ask me any questions about my take-home solution--it was just ignored. It felt like a pure waste. I'd much rather do whiteboarding.
The homework is no cost for the employer and you are not sure how serious about you they are when you do it (if it is in the start of the process) so I wouldn't bother if I had an alternatives to go for.
Eventually I changed the strategy and offered candidates the choice of the take-home problem or an on-site whiteboard style interview where they solved a subset of the problem in pair-programming fashion with us. Zero people chose the on-site interview. Everyone chose the take-home problem.
No one complained any more after we reframed the take-home problem as the candidate's choice rather than the company's.
> reframed
Aren't meaningless shenanigans generally associated with meaningless things?
Do you have any evidence that either strategy really matters? Does it boil down to, "We choose an interviewing strategy that selects for people who look like programmers."
Are you meaning they wanted your company to pay for the time used to solve the take-home problem?
If so, isn't it normally done that way in order to prevent companies potentially exploiting free labour?
Ironically, you passed on the smartest applicant of the bunch.
Do you offer candidates a remote pair-programming interview?
E.g. screen-sharing session over Skype/Zoom?
Take homes can actually be compiled and executed against a test suite for correctness.
Take homes tend to be more realistic exercises than balancing a f-ing tree or walking some idiotic graphs.
So I'm not sure how accurate this list is.
Coding on a whiteboard in an interview is not "broken". What is "broken" however is making the problem the candidate needs to solve "hard". I can't stress this enough: a whiteboard coding test is nothing more than a negative filter. It's to filter out people who can't turn a simple idea into code as these people exist.
This is why FizzBuzz was such a simple problem.
But interviewers fall into the trap of thinking "this problem is too easy" and make it hard (eg figuring out a problem is g union-find on the spot and then implementing it) or, worse, they make it a crap shoot of whether you know the "trick" or not (eg reversing bits in O(log n) or the tortoise and the hare). This actually makes the test completely useless.
A negative filter is simply a filter whereby if you fail it, you almost certainly aren't a great hire but the reverse doesn't apply: there should be no A+ on a whiteboard coding test. It's straight pass/fail and an easy pass at that.
I realize some people have anxiety about doing this. Having helped or coached a bunch of people in the past about this, I can say that mock interviews and practice help the vast majority of these people. Some may be helped by doing this on a laptop in a shared doc instead, which is fine.
A few will struggle with the anxiety of that, even with coaching and practice. While I feel bad for those people, I do wonder if a reasonable-sized organization, which will generally require a certain level of communication skills and presentation ability, is the right place for you.
It's really not about presentation ability. It's "oh shit I have x minutes to do this and I have to get it right or I won't get the job, oh shit 5 minutes passed already and I haven't done anything" type of anxiety.
"Oh shit, I have to get up and talk about my project. What am I going to say? What if my director thinks I'm an idiot? What if they ask me a hard question? What if I can't answer the question? Will they cancel my project? Will they bring in someone to rescue me? Will I get fired?"
When I interview people, I give pretty straightforward questions. My goal is not to run some algorithm guessing game. If I'm interviewing for a senior FE position, I make sure they know enough about async patterns in JavaScript that I feel confident that they could build an event handler without dropping promises on the floor.
A back end position? Can they use (not implement, just use) a hashmap efficiently and do they think about boundary conditions in input sets (you'd be surprised how many people ignore punctuation when breaking sentences into words - they just do a split(' ') and call it a day.
This is really basic stuff. I'm always able to (a) filter out people who just need more time learning to code (or are too nervous to perform well, which is still a problem with even these whiteboard activities) and (b) prompt a discussion about engineering practices.
My criterion is simple: do I think I could see this person submitting code to my project without me needing to constantly fix what they write? It isn't about memorizing algos or anything.
Aaalso: I didn't study leetcode, and managed to get my job just fine. I know there's a reputation of the process being just algos, and I'm sure a lot of the interviews are. But not all of them, even at the FAANG.
Id take a look :-)
Yeah, I tanked the interview but I really did not care if I pass when I got to that point.
how would you have liked that?
I interviewed at one of these companies in the list. I'm nobody special, did the test, did the interviews which were more about doing the work and how it gets done and I how I think about it, and they made me a really good offer which I accepted.
I'm not including a compiler or having build tools or an IDE. Just a basic simple text editing area that allows the basic functions of typing in text and editing it.
I'm against these leetcode interviews. But if you did nothing else but change this one thing, just stop expecting people to write code on a whiteboard or paper/pencil and allow them to write code the way it's actually done (a computing device with a keyboard and text edit area), that would be such a huge improvement. Writing code on a whiteboard or paper doesn't test anything. Think about how limited it is and how different it is (for example, you can't just press an up arrow and add a newline, have to find an eraser and start over).
It's yet another useless skill to learn just for interviews (writing compilable code on a whiteboard/paper) which also encourages rote memorization (because you have to get it right on the first try since editing the text is so painful and difficult).
I hope people start pushing back on this. The reaction should be, wow you care so little about your interview process that you can't get a $200 chromebook in here?
* IT just fails: they’re unwilling to supply it, there aren’t enough, the current laptops on hand are junk, or the default password is wrong. These all happen with HR buy-in and funding.
* Recruiting just fails: they “have to move the candidate fast because of a competing offer,” they tried to work with IT but IT blocked them, they got laptops themself using their own budget but then ran out.
* I failed: I didn’t make time to prepare a non-whiteboard question, I didn’t tell my manager in each and every 1:1 that candidate experience sucks, I had three laptops and I forgot to bring my personal spare to the interview.
In general the core problem is there is zero incentive for hiring managers to do anything but shotgun candidate pools like they do today. There’s also no feedback loop to evaluators to help them improve. False negatives are completely tolerated despite there being a huge gap between CS jobs demand and CS graduate supply.
All these companies expect you to write code using a site like hackerrank during the phone screen. And yet they can't have a chromebook open to the exact same site during the onsite? If that's too much, they can't even get a cheap laptop, stick in a USB drive linux distro, open nano and give that during the onsite?
Maybe it's just our phone interviews that allow us to focus on higher-level concepts in person, but I've really never found a candidate who did poorly on a whiteboard but seems like they would have done well on a computer. If anything, it's the inverse where candidates jump straight into code, to their own detriment, when placed in front of a keyboard.
Now while you may be aware of this and not give the candidate a hard time for not leaving enough space, I read a Medium post once written by a Google employee with tips on doing well on the interview. He explicitly highlighted this problem, and literally encouraged candidates to learn to think and write linearly and not leave these kinds of blank spaces.
However, a whiteboard certainly can be easier to write anything that is not code on. Like for instance if the candidate chooses to walk the interviewer through a high-level approach, including a graph topology, before actually starting to code.
I know Google started letting candidates choose between laptop and whiteboard a few years back. I suspect others alos have.
I think this is the bad assumption that's causing your confusion. Maybe this happens, but I think it's very uncommon.
The point of using whiteboards is to focus more on algorithms, data structures, and problem solving, not just spitting out code. If you make the applicant write actual code it limits the scope of the questions that can be asked.
So, it's actually beneficial that whiteboards aren't good for writing compilable code, because that's not what you want the applicant to do.
The only thing that LeetCode questions predict is if the candidate has been training for LeetCode questions.
The fact that Google, Facebook etc still use it as a gatekeeping mechanism makes me believe they want to find cogs that will specifically spend hours studying for it. Making sure that those engineers will be ready to execute their mindless coding tasks at the company.
You missed the whole point of the OP's post it seems. The only thing that leetcode implies is that you can solve leetcode tasks.
I think people underestimate human nature in this case. It's a known thing that many interviewers just do the interviews for the sake of feeling superior than someone else.
It was another trend in the industry to ask stupid brainteasers like "Estimate how many gas stations there are in Manhattan" until Google banned the whole thing altogether. Google judged that these questions only serve to make the interviewer feel clever and self-satisfied. [0]
It's time to do the same with whiteboard interviews.
[0] https://www.wired.com/2015/04/hire-like-google/
If you’re only hiring candidates who can pass Google interviews, what are you offering that Google doesn’t? (Hint: “we give you a 10 hour take home, then ghost you” is not a competitive advantage.)
I have a hard time believing that this many firms have take-home assignments that really matter, and aren't just another hoop candidates (or, maybe, disfavored candidates) are forced to jump through.
I say this as someone who was an extremely early adopter of take home tests and think they are the highest signal filter you can reasonably have in the US.
So many places have added take homes in the worst possible way without removing all the other nonsense. It’s as if orchestras started demanding high pressure auditions and then threw out the results if the person talked a good game in an interview.
I think you can learn more useful stuff about a programmer by showing them a function prototype and asking how to black-box test it. Then show them complete code, and ask them to white-box test it. Test cases or a test plan, depending on scale. Then a discussion of test tools they have used.
The people who can produce quality code for your organization will show up pretty fast.
Performative interviews occupy a similar space. You spend all this time and effort studying for something at is ultimately irrelevant outside of the interview. Just a pure waste of brain cells and calories, with a few new gray hairs for you.
Other questions I ask are things like do you work on your own projects and tell me a bit about the tech behind them, what’s your favorite pieces of open source or free software (I work with almost exclusively FOSS stuff though my own work is not FOSS), what do you do for fun, and if we were talking on the phone and I had never made tea, walk me through the process (this is a fun one). Then take them out to lunch if possible and watch how they treat service workers. I think this has so far worked well for everyone involved. At least as far as I know nobody I says yay to has been a bad hire.
> service workers ...
> watching how they treat people
They are not people though. But yes, agreed that we should treat them well
I mean if you literally can’t remember anything of what you’ve worked on ever, no I don’t think you’d be a good hire. And also this part is surprisingly hard to bullshit. You can say “well I worked on an eCommerce site and it was hard because we had a lot of visitors” isn’t a full answer. If that’s as much as you can give me, no I don’t believe you contributed much to that project. Maybe I’m wrong and you are brilliant and just can’t remember, but it’s a risk of false negatives that I think most employers will find acceptable. But: “It was hard because we had a very complicated system for determining the final price of the whole cart because of discounts offered if you bought certain pairs of products together. I didn’t implement that system and we had plans to eventually rewrite it but in the meantime we started calculating the prices in our task system instead of during the web requests. It reduced web server load and made adding things to the cart feel faster.” Me: “OK that sounds neat but how did you prevent race conditions? Say while the first calculation was running the customer added a second item that triggered another calculation? What if the first task finished after the second?” Then: “Ah yes I recognized that it would be a problem when writing that code. At first I tried using a lock using Redis for the cart but that just made the tasks serial and the whole thing took longer. Plus if a task failed or was interrupted it could cause issues. So instead I just put the latest task ID onto the cart object in Postgres so I always knew which the latest task was. When a task finished and wanted to update the price first it would lock that cart with the Redis lock, then check if the ID still matched its own, then update and unlock.”
The above tells me how they think and what they think of. If I’m not getting enough technical detail out of them I will ask stuff like “what Redis primitive did you use to create the lock?” But usually the interviewee will go into lots of technical detail all on their own.
I think a good middle-ground is pair programming where the interviewer writes the code based on the pseudo-code provided by the interviewee. This way, the interviewee displays their thought process and communication skills without being bogged down by programming language nuances.
> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.
>> I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages.
>>> I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol. Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.
https://news.ycombinator.com/item?id=23848717
Even as we debate the validity of the metrics measured by technical interviews, it's also worth considering the metrics used to measure engineer worth in the industry at large. This existentialist "fear and loathing in FAANG" discussion was quite illuminating, and a little sad.
Makes you question if this is the precise point of it.