One of the best interviews that I ever had was a phone interview where I just so happened to have a good session of meditation beforehand. I never realized how much how well I could do in an interview until I did one while feeling relaxed.
I think it’s pretty funny that HN constantly promotes articles about how shitty tech hiring is, and the threads are always a mix of workers agreeing and then managers often also agreeing but 100% confident that their hiring process works well and weeds out enormous numbers of faking bozos. It’s like, ok but almost none of you have useful actual data to back up your process and you can’t all be anywhere near as good as you think if hiring is, in fact, broken.
Same here. A guy was soaked in arrogance even though the test he submitted was average at best. "This looks good enough. when do I start?"
Even worse: I had an encounter with another guy who refused to make a test because his CV demonstrated everything we needed to know and asking him to solve a few problems was "incredibly disrespectful". He was also apparently going to start next Monday. Needless to say he didn't.
> the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding
Sure, no love for whiteboards from me. However, this does suggest an arbitrage opportunity for an employer who is willing to look differently at these candidates, and then profit handsomely, Moneyball-style.
There's a bunch of money just being left on the table and nobody is picking it up, really?
My two cents is that interviewing candidates can give a whole lot more insight into how to perform in an interview. Interview performance is a skill that is separate from the actual job, and gauging that performance is sub-optimal. But everything in this space is sub-optimal, it's all pick your poison.
Having capital or hiring authority doesn't correlate with willingness to try to actually hire value rather than assemble something that they think resembles a team.
What I don't get is why no one is attracted by the social brownie points they'd end up with when their moneyballed team inevitably turns out to be 70% female.
Many smaller companies do pick up the money, which is one of the major factors allowing them to succeed in a world where the big software companies toss around ridiculous amounts of compensation. The Googles and Facebooks of the world are structurally unable to do the arbitrage because many (probably most) of their interviewers don't particularly care about interviewing.
Huh, the only successful interviews I have are the ones where I'm burned out after a long, fruitless, frustrating job search. And so I walk in assuming the interview will be a waste of time and just not caring anymore.
A different way to view this is that being desperate makes you behave badly. For example, someone desperate for a job won't push back against bad decisions and will become a "Yes Man". Someone desperate for a date won't engage with the other person as a person.
That is a different way to view it. An incredibly negative seeming view of humans. Ill lay my cards down though, i have limited years of really varied experience (ups, ski resort, landscaper, metal grinder, fork lift driver, house painter, door to door vacuum sales, google pm, msft Analyst, trying to break into infosec.)
In my short list (ignoring years of time wasted travelling), i have only met 1 yes man. He came from money, and sales. Owned 4 large businesses, and would over promise, under deliver and over receive ALWAYS.
So what are your experiences like and from where?
I think no backbone can be a personality trait, sure. But it is NOT a trait of 'need' and action, in my experience.
Saying "Yes man" probably derailed my thesis since it's associated with the higher rungs of the corporate ladder. I thought that this would be uncontroversial.
I'd like to focus on Dating because I disagree with your offhand comment, even though it was in jest and is probably more cynical than you are in person.
> The more … apathetic ill be cherished. Sounds like dating ;)
Feeling bad makes your behavior suffer. Being angry makes you less nice to people. Being tired means you don't engage with people. Being scared or anxious makes you hide your vulnerabilities and close yourself off. I'm not saying that badly behaved people deserve to be ignored, I'm acknowledging that it is uncomfortable to interact with people who are hurting.
Being "driven and hungry" while dating makes you behave worse. It makes you over-eager to please the other person, and it flattens out your personality. Instead of focusing on making the person like you, you should be exploring whether you like each other. A date can be successful even if the two of you don't end up dating because you two would not make a good couple.
Some of this is like the Groucho Marx joke "I don't want to belong to any club that will have me". For companies, it's "I don't want to hire anyone that doesn't have a job". The thought being that if they were any good they'd have a job already.
This is one of the things I look for when I'm being interviewed. Does the company want me to be desperate?
Some companies will only hire those who are desperate and will do anything. I assume, hopefully correctly, that those are the companies not worth working for, so instead I use the interview opportunity to get better at interviewing. Because these companies want to take advantage of their employees, I do not feel bad wasting their time.
I interviewed at a few places for some really easy, fun-looking jobs while I was doing freelancing. I didn't care if I got the jobs, but they did look trivially easy compared to what I'd been doing earlier in my career. Surprisingly though, after making it through all the technical people and talking to the CEO in every case, I never got the job.
Later on I applied to a really awesome job at a FAANG-class megacorp, and the hiring manager's manager called me. He said "Look, you've gotten the highest marks I've ever seen for a candidate. Everyone loved you. But I have a hunch that you aren't going to be dedicated to this job. So before we move forward, I want your personal promise that you aren't planning to just hop ship in 6 months to start a company or something"
I was JUST starting to get nervous about finding a new job at this point. My consulting work had dried up, I was down to about 3 months' worth of cash, and I was thinking about selling my car to get a few more months' runway while I grind Leetcode in order to be able to pass the bar at Google.
So you can imagine how shocked I was to hear that this guy thought his awesome senior engineer FAANG job wasn't good enough for me. I promised to stick it out, got the job, and so far it's been my favorite job of all time (other than the glorious week where my crypto trading bot was working well and just printing free money..)
My previous startup I worked at had gone under, and I was thinking about starting my own. An old coworker of mine invited me to lunch. We had a couple of margaritas, and he asked if I wanted to see his office.
I went to check it out, and suddenly he asks if I can talk to some of his coworkers.
So I basically interviewed while drunk and not fully realizing I was being interviewed.
They offered me the job, and I have been working there for 8 years now.
Maybe the best way to be relaxed during a job interview is to not even realize it is happening. And having a couple of margaritas can't hurt.
That is funny as that is the way alot of my interviews go as well. I tend to have extreme anxiety on a job I really want and "pysch myself out" so to speak. I really dislike typical tech interviews as it is alot of pop quiz obscure answers under extremely tight timelines never seen on the job. I have had interviews where 3 interviewers are all demanding answers at the same time and it was horrible. I hope at some point in the future this is realized as a colossal fail in the industry and a new method is realized.
That's how non-FAANG goes. I work at an old, non-tech corp and the interview (just one) was basically a popularity contest with some light technical questions. Don't screw up and crack some jokes and you're in. Of course, that means our level of technical ability is low, and we have poor project outcomes because of it. But it's a nice place to work.
That can help a lot with anxiety. If you think it's pointless and you'll never get the job, you have a chance to just have fun with it and do what you can do.
My first time interviewing with Google (in 2008), the press was reporting that they were in a hiring freeze, I had no big names on my resume, so I figured it was basically hopeless and I'd just do the best I could and enjoy my trip to Silicon Valley. After my interview, I was like "Well, I did okay, but you really have to be perfect to work at Google and I wasn't perfect. I'm never coming back to California anyway, so I might as well do all the tourist stuff [I'd held an extra day to interview with Twitter, but they ended up not moving forwards with my candidacy] and just treat it as an all-expenses paid trip to SF." So I just went to Alcatraz and Fisherman's Wharf, treated myself to a nice dinner, and took the red-eye home to Boston. As I'm groggily waking up after a post-red-eye nap, the Google recruiter called me to say "Your interview feedback looks great. We'd like to fast-track your application on the assumption that the hiring committee will say yes."
Been living in California for 11 years now, and I just started back at Google after working there for 5.5 years, leaving, and doing startups for another 6. The most recent round of interviews was the same - I was super nervous for Lyft, Stripe, Coinbase, Netflix, etc. and ended up not getting the job, but with Google it was like "Hey, I worked there once, I know people who also know my interviewers, I used to give interviews like the one I'm sitting in now, what's the big deal?"
I also treated my Google interview as a free trip to the Bay Area, and used it to visit a friend and do touristy stuff. Wasn't expecting an offer, but got one and stayed there for many years.
I've read advice that you shouldn't schedule your interview on a Friday, but that's exactly what I did so that I could spend the following weekend in SF :)
Pretty much. If you leave and come back at the same level within 2 years they skip it, but I'd been gone for 6 years, and I was coming back 1-2 levels higher (part of the point of interviewing was to determine level). They want to determine if you a.) forgot how to code, which is entirely possible if you've been doing other stuff for a while and b.) have learned something new & valuable that warrants a promotion.
The panel was different, too - my first time around I had 2 algorithm, 2 UI (Javascript), and 1 system design interviews, while the second time was 2 algorithm, 1 behavioral/leadership, and 2 system design interviews. This is probably reflective of the differing role and higher level.
What’s frustrating to me is clearly you’ve spent 10+ years on the industry, worked at big cos like Google and startups, passed Google onsite twice, and still got rejected by all those companies. Which would be fine if industry execs weren’t constantly bemoaning about talent shortages and difficulty hiring engineers. They’ve established a cultural pattern of talking about how acceptable it is to reject highly qualified engineers, labeling them “false negatives”, then refuse to introspect on that when discussing why they find hiring challenging. (I’ve also spent 10+ years in industry, 3+ years as a Google SWE, and got zero feedback rejections from companies like Stripe and many many others in the last few years.)
I had a quick phone screen with a recruiter the other day. For unreleated reasons the night before I hadn't slept much. That morning I was clambering about in the woods and stumbled on a wasp nest and got about 15 stings all over my body. I took 3 Benadryl anti-histamines before I remembered that I had the interview that afternoon. I was a complete space cadet.
Somehow the recruiter was still interested.
Maybe this should be my strategy for every job application? :-)
I solved this horrible problem with legal anxiolytics. I crashed and burned every interview I had until I decided to drink kava[0] beforehand until I couldn't care less about whether I got the job. I've been working there for years now.
Like it or not, interviews are a game. Smart people will figure out how to play them after losing enough, though maybe some of us take a bit longer ...
How I got my internship that converted into a full time job and everything that followed from it. I literally gave the interview and went to the sports stadium. To this day I've been job hopping without interviews. Everyone who works with me knows my skills (and my drawbacks) and they're the ones who hire me or recommend highly of me to lead to a job. My next job may not be so lucky so I'm starting to dust off my "interview skills". It's frustrating. I'd rather be building something usual than solving these. But that's the nature of it today.
I got my first salary job this way. After that I did my typical analysis seeing if I couldn't learn from the situation and improve beyond it.
My conclusion was if I interview to gain interview experience, not to get the job, I do far better, and I gain interview experience. This is the local maxima I found from that, but there could be better.
+1. Either this, or I have an offer in hand and so I don't have to care anymore.
The role of anxiety in interviewing became very clear to me during a season when I failed all my interviews until I got my first (and a very good) offer. Suddenly I started passing all of them. It was night and day. I was very surprised having always had much longer and frustrating recruiting processes.
> For this study, researchers conducted technical interviews of 48 computer science undergraduates and graduate students. Half of the study participants were given a conventional technical interview, with an interviewer looking on. The other half of the participants were asked to solve their problem on a whiteboard in a private room. The private interviews did not require study participants to explain their solutions aloud, and had no interviewers looking over their shoulders.
> Researchers measured each study participant’s interview performance by assessing the accuracy and efficiency of each solution.
[...]
> “People who took the traditional interview performed half as well as people that were able to interview in private,”
I appreciate their effort to use science to support the narrative that tech interviews suck, because I do think they suck and need to change, but isn't N=48 a little small? Also, if it was only undergraduates or graduates, who's to say their performance is indicative of the rest of the tech workforce?
To acquire a larger N would mean recruiting more participants, which would require either compensation/extra credit or rely on people volunteering ~1 hour of their time. Likewise, collecting that data would require time and money. It is an time/resource issue with human research. You can require a small N and get some results or require a large N and not have enough funding to support it.
Also note, what is a "large N"? There is no set in stone amount and I've even seen reviewers say a sample size of 10,000 is too small.
I'm curious as to what makes a good size of N. To me, N=10 would be do small, N=48 seems a little under what is should be, and N=100 seems sufficient, but I don't have a real basis for this (in fact, it's probably because of the original count that had me settle on a min and max of 10 and 100). Maybe a better question is: What are factors that studies consider, besides the logistics you rightly pointed out, in determining a sufficient N?
I answered it in another response, but I think this is why we're seeing a rise in meta-analysis papers. Take a bunch of small N's, consolidate them, and analyze their trend. This analysis can also be strengthened by evaluating the effect size of the phenomena [1]. However, I would say using effect in meta-analysis is a very complex approach that limits the set of researchers that could conduct the analysis appropriately.
That depends on the size of the population being sampled from, the margin of error, and the confidence level.
For a huge effect like the one shown in the study, where one side performed 2x as well as the other, a sample size of 48 is more than large enough to say that the result is statistically significant. If there was as small effect, that wouldn't be the case.
Put it this way. You want to find whether people from California prefer Taco Bell or Pizza Hut, so you randomly sample 100 people. If all 100 people say Taco Bell, then you can be reasonably confident that more people from California prefer Taco Bell. Because if at least 51% of your population preferred Pizza Hut, the odds of not getting one of those people in your sample are minuscule (the odds of getting all Taco Bell people in your sample if 49% of the population prefers Taco Bell is 0.49^100).
If 51 prefer Taco Bell and 49 prefer Pizza Hut, your confidence level is too low to be useful--you need a larger sample size.
Of course I understand why the research has N=48, my question is whether it is adequate to fully support the claim in the title. I really don't know. I'm not a scientist or sociologist. My intuition tells me that it seems small, and using a population of college students intuitively feels like it wouldn't adequately capture the diversity of professional software developers, but maybe it does in this case.
It is; however, my next understanding is this is why we are seeing a rise in meta-analysis papers. It is difficult to acquire a sufficient N with a single study. As a result, you see authors consolidating all of the papers within a topic and discussing the trends seen in each.
However, research also has issues with reproducibility. No one will publish work that was simply a repeat of someone else work. Researchers have to twist or add something. It is a discussed issue, but no one in power has made attempts to rectify it.
Depends on the size of the effect you wish to measure, and the underlying variance. They got a massive effect, which to me, seems improbable, and I half suspect the randomization is at play.
I think it'd be pretty simple and cheap to replicate the things we care about in this study. All you really need is a whiteboard, some markers, a small room, a few interviewers and a candidate pool. The eyetracking getup really only gets in the way of that and seemed more like a problem in search of a solution.
The point of a whiteboard interview for me isn't about accuracy and efficiency. It's about seeing whether the person can code and can talk to me about coding.
Yes, of course. I rely on compilers, linters, and editor syntax highlighting when writing code, so it'd be pretty hypocritical for me to penalize people who can't access those things. People can sketch out a pretty messy pseudo-code implementation if they want, and then we'll just spend the rest of the interview cleaning it up, making it correct, extending it, talking about quirks of their implementation.
I was an undergraduate at NC State. I wonder how the study could be replicated at UNC and Duke. I'd wager they'd have a significantly higher success rate.
It was made the clearest to me when I interviewed a colleague who had left to another company and two years later was interviewing to return. I knew he knew everything and had the skills for the job because, well, I know him, I’d worked with him before. He’d done this very job previously. Yet he still performed terribly in the interview and (rightly or wrongly) I had to base my evaluation on my knowledge of him rather than his performance in the interview. Interviews suck so much for evaluating people. If you want a job and good career progression the best advice has to be prioritize making friends with lots of people.
Having multiple respectable people that can vouch for you is a much better performance predictor than a 1-day interview. Same reason that recommendation letters are more important than GRE tests for the ones seeking PhD studies.
It does suck that it’s like this in an industry where you have a lot of non-neurotypical people though. People who are very good and capable but on the spectrum, or shy and reclusive, who really can’t do the networking and conference scene thing, etc. Wish there was a better answer than to tell them they just have to be wired differently.
There are other avenues. I've been excited to hire somebody in the past pretty much on the strength of their detailed technical blogging. Kind of a mess in the interview, but I knew they could do the job coming in. (They were great.)
Let's not forget that the type of work in this industry and hours needed in the name of cost savings hinders many peoples' desire for professional development work outside of work.
I don't know about most people but lately, at the end of the day when I make enough progress to meet expectations, the last thing I want to do is anything related to work anymore, even if it's an investment in myself. At some point my life takes precedent over my career.
If delivery expectations are reasonable (i.e. time requirements), I don't mind networking outside of work. I've worked in environments with reasonable expectations and environments with unreasonable expectations. My observation has been that environments with reasonable expectations are in decline in the name of competitiveness and cost cutting measures, passing costs to employees.
This industry is becoming significantly less enjoyable to work in and growing to a toxic point of hypercompetiveness at every layer. It's not healthy for those working in the industry.
That’s really not a problem. If you think about it, everyone wants to help their network by default. And in startups, you care about outcomes. It doesn’t really matter if you’re shy or gregarious if you can make the crucial thing.
It’s a big company problem, not a startup problem.
Hiring primarily based on references almost guarantees bias. White engineers will tend to know more white engineers. Asian engineers will tend to know more Asian engineers.
Of course, you could have diverse acquaintances and friends, but most people are more inclined to be friends with people who share the same race and gender because they often share the same experiences. This puts a black female at a great disadvantage if her friends are mostly other black females because so few of them work in tech.
I feel in the tech industry, networking and knowing people only gets you so far - typically just to the whiteboard interview. Then you have to solve the leetcode problems just like everyone else. Especially so in larger companies.
FWIW, all of my friends who somehow managed to get into FAANG or other top tech companies swear that were they to go through the interview again, odds are they would fail. This despite all of them being great engineers.
I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.
Plus, stack on top of that, I've seen how the sausage is made. I've done 90+ interviews on the other side. At the end of the day, it comes down to how lucky you got in the loop. I've seen some people written off for some bullshit reasons. Similarly, I'VE declined on people who're likely far, far, far more skilled than I am because of the lowers/raises approach and the narrow slice you're tasked with reviewing in the 50min you're given.
idk if it’s just luck. i’ve had loops in college where i failed them. but then later in my career i had times where i got every company i interviewed at. Excessive preparation was the key difference. I do think luck plays a big part but i don’t think it’s the only factor, or even the biggest. (maybe 2nd biggest)
Luck is likely not far off from preparation in terms of effect though. Luck could be divided up into separate parts like: chance you get an interviewer who asks easy problems, chance you get a hostile interviewer, chance your interviewer is racist/sexist/tabs-ist/hates-your-face, chance your interviewer loves you for no apparent reason, chance your interview is going through a divorce, etc..
I've seen some of the best interviewers I know still fail at various companies. Including ones they were given offers to before. Luck might not be the #1 component but it sure feels like it's not a distant second.
There are some components to the interview you can directly affect, and some you can't.
By grinding leetcode, you raise the odds that you'll get a problem you've seen before, or at least similar enough that you recognize the way to solve it without having to waste time by starting from absolute scratch. I guess that's still luck, but this is one area you can increase the odds in your favor.
You can't change whether you get a hostile interviewer, etc. of course.
i agree but regarding your example about failing companies that you’ve previously had offers from: ive seen this as well. but it was a case of hubris where the person didn’t put in the work to prepare as hard the second time because of their previous results.
the shitty thing about interviewing is you have to prepare hard every time you approach it
While that might be the case for people you've known - I've seen it with people who prepared even more than the last time.
I've also seen the opposite - right? People don't prepare anywhere near as much as the last time and still get offers. I've seen folks get offers at Google and spent maybe a few hours preparing. Luck is just a very wild component.
Yeah, I'm 40+ but takes the time to prepare well for interviews and that has been the key for me. Gotten job offers from all the FAANG's without being very smart. Just go on leetcode and practice for a few months when needed.
This does very unfairly skew towards people like me without kids that can take the time to do this.
Being good at LC requires being smart. Most people won't be able to do "justify text" in 20 minutes without looking up an implementation (and potentially memorizing it).
for me atleast that also came with experience. ie that would have been impossible in my first job search. But repeating the process of searching for jobs a few times over a decade, each time i’m preparing for interviews i’m able to build somewhat on previous times to where problems like that are manageable.
Not really, most questions have answers and discussion threads - during an interview you can see people just writing up solutions flawlessly - they just memorised them. I have interviewed 100+ people.
I don't think regular coding questions, LC tell you anything meaningful in an interview.
I'm more experienced now, so refuse doing it and recruiters and interviewers get all funny about it. But please, please interview. I know I'm pretty privileged compared to others but I won't ever do a LC question anymore and tell that upfront to Google and the likes. I have canceled loops because of the inflexibilty of some companies in that regard.
Sometimes they tried sneaking it in last minute - also tells you much.
Can I ask why you refuse? For me it is more of a formality at this point. I know it won't tell a lot about me, except I cared enough to put in the hours to get this job, but still you have to do what you have to do. I'm pretty senior as well but don't really see why I should not do this when everyone else has to.
In general though the tech industry has a really hard time hiring senior people, perhaps for this reason. I was at Google for 10+ years and did 100+ interviews, HC etc and once you talked to the people in charge of the process they admitted that the process is really weak for hiring senior (L5+ in Google speak) candidates. They in many ways perform worse than new grads in coding interviews which is not really surprising.
So Google hardly hires anyone experience before you get to the director levels (hardly as a percentage of course).
The other well know secret is that if all of us would try to do an interview unprepared we would probably fail.
As someone who refuse myself I simply think it is a waste of my time. It isn't fun, it doesn't teach me anything I find interesting and it isn't a good way of filtering someone.
I would rather be with my girlfriend, play videogames after work hours, watch netflix, go see some friends.
I lost a team member to Amazon recently, he spent 6 months total (two 3 months stints) preparing for the interview. He bombed in the 1st try and passed in the second.
That's 6 months of your life studying after hours to get a job ... I really have better things to do!
He is there now and he says how his work is not as challenging as the one we were doing from a technical PoV and also how the culture is very toxic compared to ours.
Anyways... he is optimizing to retire early so props to him I guess.
I don't do whiteboard coding. When I thought it was meaningful, or might be meaningful, I did it even though I'm uniformly terrible at it. If I was great at it, I might do it even though I no longer think it's meaningful.
But I'm terrible at it, and I don't think it's meaningful, so I don't do it. Admittedly, it costs me some opportunities, but mainly at companies I'm not sure I want to work for, anyway.
At this stage I'm an old fart with a long resume, and more often than not I get hired by people who already know my work. I guess if I was as bad as my whiteboarding performance makes me look then people who know my work wouldn't want to hire me.
So I think this is partially why these questions actually have value. They cannot completely be faked, you need some level of competence even after months of leetcode.
I witnessed someone who couldn't code fizzbuzz end up at our company (OK it could have been an acquihire or something) then get fired, and then immediately hired at Amazon at a pretty good office. I always had a nagging doubt this person paid someone to stand in for the interviews. Thinking back, who actually cross checks this kind of stuff? Unless they know each other personally or take DNA/fingerprints it wouldn't be very hard to pull it off.
Even when you prepare for months, your preparation needs to be related to what you'll actually see in the interview to pay off. Because few companies actually tell people what to expect in the interview, even preparation involves luck.
Yep. "does this candidate raise the bar, or lower it". I.e., they have to be better than the average at the company already. As decided by you, from 50 minutes of interaction in a completely artificial, stressful situation.
I don't know what company they're referring to but at Amazon, the criteria for hiring someone is that interviewee would be better than 50% of the people currently doing that role at the company. They should 'raise the bar' in order to be hired.
how do you even gauge that? it sounds like you'd need to keep re-interviewing the current people there to make it relative and fair. which of course isnt going to happen so "raising" the bar doesnt make any sense whatsoever
I think that's part of the point. i.e. the usual line about wanting to avoid false positives (bad engineers putting on an act) even if it means sacrificing many false negatives (good engineers who suck at interviewing, or even just simply wasn't having a good day on the day of the interview).
Ironically the modern technical whiteboarding interview often does require the candidate to effectively act and put on a show,
Suppose programming ability is normally distributed with a mean of 100 and and SD of 15 (values cribbed from IQ). A team starts with 10 people drawn from that population.
More applicants are drawn from that population: if an applicant's ability is above the median of the current team, they are hired, replacing a randomly-selected member of the team; otherwise, we pass.
At the first timestep, half the applicants get hired right away. It takes a few hundred total failed interviews to make the 25th hire, and a few hundred thousand(!) for the 100th hire. These have very long tails too: the longest run (of 1000) took 800M total interviews to hire 100 people.
There's no way this rule is actually being applied in practice.
raising the bar would also mean that their pool of people who are merely good enough gets very small very fast and Amazon just doesn't have the reputation to attract the absolute best
Lowers/raises the bar. As part of interview feedback an interviewer is supposed to rate the interviewee as raises/meets/lowers.
Typically, just one lowers in an a loop of 6 interviews is sufficient to reject. On the other hand, a loop with all "meets" isn't a sufficient either. It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers" the candidate won't even make it to the debrief or worse the loop will be cut-short.
The bar-raiser (it's their job) are expected to specifically look for candidates who raise the bar. Raising the bar == better than 50% of "employees" (not job seekers) at their level. The rationale, when the bar-raiser program was created, was that the collective competency in a corporation should increase as more employees are inducted.
Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser, conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.
Companies tend to keep track of the onsite success to failure ratio and then if it's too low examine if the bar raising is too aggressive.
Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"
Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar raisers periodically met to discuss topics related to hiring and increasing interview bandwidth but we didn't evaluate the process as such.
That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through.
The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,).
And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.
My judgment of Amazon is tainted badly by a bad apple we couldn't fire soon enough - I had the unsavory role to collect evidence for failing the PIP that would inevitably happen. Week after person was let go, he was at Amazon in a pretty good division.
I keep wondering how they hired this person - never in a million years would they have passed a regular hiring chain even with fizzbuzz type questions. Wonder if they had someone stand in for the interviews.
It's often been mentioned that interviewing skills is different from actual work skills. Could be that he's an interviewing wiz but a poor day to day engineer?
Netflix is well known for firing people that don't pull their weight with far more ease than most companies. But a friend who works there tells me there's still people there that perform poorly but somehow managed to get both hired and stay on.
The general approach to dealing with low diversity in a team seems to be to increase the percentage of phone screens being allotted to diversity candidates (which on the other hand reduces the number of non-diversity applicants that get a phone screen).
And not actually lowering the bar during an interview / hiring committee meeting.
While I was there, increasing diversity wasn't a priority for them. Now that I think about it, I don't remember diversity even as a discussion topic let alone efforts to address it.
I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages. I've been "studying" Leetcode after work for a month or so but always seem to make stupid mistakes under the time pressure of the interview. I don't want to spend my entire career at one company, but it's clear I'll need to be very lucky and/or practice much harder on my own time if I want anything like a similar position/salary elsewhere.
This is something I find so disturbing, and I experience similar.
I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol.
Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.
If you want a good career, working on your interviewing skills is the most important thing. Networking and resume fluff are close seconds. Job skill is almost irrelevant. It's an ugly truth.
I think the answer is we need more innovation. We need a million small startups instead of a dozen tech giants. Software scales but starting up is a moonshot.
My personal career epiphany was moving from a tech giant to a late-stage, pre-IPO startup. The amount of freedom and range of challenges I dealt with were exhilarating. Did it pay as much? Not in the beginning, but I was given the room to grow.
At some rare companies people ask questions about real-world scaling which has been nice to leverage some actual knowledge but that's usually just a small part of the process and almost treated as a soft-skill which is kind of hilarious.
We just really want to hire a senior dev who can spit out highly optimized code that in the real-world would be handled by a stable open source library rather than people who have built stable systems.
My dream is to have some kind of "working for myself" career like the creators of the Apollo or Overcast apps. Not contracting with companies as clients, just working on one small great project for regular people to use and selling it yourself. It sounds fantastic.
Yeah, sometimes you may not be thinking straight, a have a little sleep hangover/anxiety Induced brain fog on the day of, and you can say bye bye to the job.
Stop lumping Apple into FAANG in this way. Facebook, Amazon, Netflix and Google all have centralized hiring. Apple does not and no two groups interview the same and there is tons of latitude for managers to make this lower toil. There is no cross-organization level setting or job descriptions for any position at Apple outside of an SVP business unit. I know from experience working at two other FAANGs that Apple does things way differently. Apple's biggest weaknesses and strengths come from their lack of centrality, but here it is to the benefit of the interviewee since they are given more individual care out at the periphery.
Amazon does not really have centralized hiring. Each team hires for itself, though there are bar raisers from outside the team designed to be a counterweight to desperate hiring managers.
I was this way too. I got into Amazon via acquisition so I never actually passed a real interview process (though there was a limited one as part of the acquisition).
I transferred teams within Amazon and then left for my current company. I do wonder if I'll ever be willing to put in the work to get a FAANG job again but at the moment I really don't care. I like where I am so I'll just cross that bridge when I get to it. If I want it badly enough at some point then I'll go through the steps. If I don't, I'll just find another job.
I don't understand this obsession with leetcode style questions specifically. I just got a job at a FAANG on an algorithm heavy team and all they asked me was linked lists, some basic graph questions, and some basic (and I mean really basic, high school non-calc level) vector math. I think just having a rock solid understanding of the basics will help much more than trying to grind questions and memorize patterns.
> I think just having a rock solid understanding of the basics will help much more than trying to grind questions and memorize patterns.
It isn’t about memorizing patterns, but about hopefully spotting the exact question.
A friend is currently interviewing for a FAANG and the recruiter flat out told him that his best chance of passing was to do a pile of previously seen questions for that FAANG and hopefully he gets it again in the interview.
If all the celestial bodies align and your luck is through the roof, you'll get exact problems you've solved before with the optimal solutions ingrained in your mind.
But second best is you'll get problems that are at least similar and you are familiar with the patterns, tricks, and gimmicks to go about tackling it, instead of having to try to think up a solution completely from scratch.
Horrible luck? It's a question you've seen and practiced but your brain freezes and you can't remember how you solved it. Or something you've seen and earmarked to practice, but couldn't get to it before the interview day. This has happened to me.
> It's a question you've seen and practiced but your brain freezes and you can't remember how you solved it. Or something you've seen and earmarked to practice, but couldn't get to it before the interview day. This has happened to me.
Same. I had an interview the day before I was scheduled to review binary trees. What was it on? Binary trees.
In 5th grade, I was in a poor school district. My parents had been pushing (with other parents) for a 'gifted program' for years. Finally, in November, a program was being introduced. I was given a test - IQ test perhaps, but an entrance test of some sort. I went home, dejected at the stuff I didn't know - was asking parents what the questions were - "what is an octave?" was one I remember not knowing at the time. I'd "got in" to the program, but... we suddenly moved over Christmas to a newer school district. It was richer by comparison, although still in the same county.
First week at the new school, I was taken aside and given a test for their gifted program. Exact same test, which ... I completely aced, because I'd figured out all the questions I hadn't known in November (from parents). It felt like cheating, and I told the teacher about it later. I don't know why they couldn't have just taken the test results from the previous school - same county, shouldn't have been an issue.
Anyway, if you keep studying known questions, it will increase your chances of passing in the future ;)
That's so disheartening. When I'm looking to work with somebody, I look for somebody that can solve problems (and admit when they don't immediately know the solution, which is extremely important imo) not just regurgitate a solution to a problem they know already. If this is actually what the recruiter said, this is a huge indictment of that company's hiring practices.
This practice seems common. I've had recruiters at several top tech companies (including two of the FAANGs) tell me this during initial chats.
I imagine recruiters actively want you to succeed and get hired because that benefits them too, so they do what they can to aid you in your chances. They'll tell you the official, pre-canned lines about "we're interested in what your thought processes are and how you think", but then they'll also tell you the truth and recommend you take some time to grind leetcode.
By comparison, in a recent FAANG interview for general dev, I was asked to reconstruct a BST from preorder and postorder traversals (Leetcode 889), and also to design a thread pool that is aware of dependencies among tasks. It sounds like the experience really varies!
I had a similar experience. I just accepted an offer for a senior engineering position at a FAANG, and only got a few basic coding questions. Out of eight hours of interviewing, I think they spent six hours on behavioral questions.
My understanding and experience is that one of the A's and the N stand apart from the other letters in having a less of a "one size fits all" approach.
Not a close friend, but do have a close associate who was VP level at Google. We were having coffee and he offered to refer me, but said I'd still have to go through the whole standard interview loop - even the phone screen in fact.
Depends on the company. I think the FAANGs basically all require jumping through the same hoops. Optimistically it prevents nepotism, but I think that's less of a worry in tech circles, and the times I've seen interviewers overridden because of connection between tech people, it's generally worked out. Of course, for non-tech (product, marketing and the like), it's been hilariously bad.
I was recently interviewed by a Google bet with a referral from a well-liked VP level employee for a job I was well qualified for (IMO) and didn't make it past the phone interview. /shrug I just blame it on Covid.
It's very different at smaller companies where the whole interview might be waived (or just no leetcode) if you worked with one of the technical leaders or founders before.
One of my ex-coworkers/now-friend is what I call a master networker. His network casts wide across the tech departments of a huge number of small hedge funds in NYC. If he wants a job, he can just call up any of his friends/ex-colleagues at these places and odds are several of them will simply just give him a job at their firm. If an opening doesn't exist, sometimes they'll literally create an opening for him.
The downside is that he has to be very un-picky on what the role entails whether it be working with some super unsexy legacy technology (Excel VBA! Legacy ASP.NET webforms!) or doing devops or even helpdesk duty too. Hedge funds being hedge funds, small as they are, they all pay pretty well, though not at the level of a top tech company.
It's a combination of these places being pretty small shops and having friends there in high places. Including, from what I hear, a billionaire hedge fund exec or two.
Eh, I got my job at a very early stage startup through a friend I met on Twitter. The "interview" was a chat with the (reasonably technical) CEO, where we talked about political philosophy for 45 minutes, then I said something about "oh, I should probably tell you about my experience and resume," so we went over that briefly. No coding challenge or anything. I assume my case is an extreme outlier, though.
But yeah. "Networking" by meeting groups of strangers for drinks probably wont get you a job. But making friends who happen to work in (entrepreneurial) tech might.
It s not, I ve been interviewed that way and interviewed others that way. Many times it led to horrible mistakes so now I give a simple project to do at home with no deadline.
If a guy can do well that kind of well rounded project, even by cheating, I want him and his network all the same.
Startups and smaller companies are a different story. One of my friends got a job at a startup by virtue of standing in line at a cafe holding a programming book, and the (non-technical) founder standing behind him hired him after a quick chat.
Bigger companies often don't let you through like that though. My friends are constantly offering to refer me to roles at their companies, but every one of them would still involve going through the whiteboard interview ritual. For now, I'm not taking them up on the offers because I'm not confident I'll pass the leetcode rounds.
I did use "FAANG and other top tech companies" as an example, but I'm seeing this practice expand out towards many other companies both in and outside the tech industry.
I myself work at a non-tech company (bank) and even with my team/department, all a referral will do is just get you to the leetcode interview and maybe a slight advantage if the final decision is between the referred candidate vs. another non-referred candidate with all other factors being equal.
Tech seems to be the only field that does these dumb types of interviews. Accounting, law, medicine etc seems to work well without these filters. Why are tech interviews so stupid?
The whiteboard standard already acts like a de facto license.
Algorithmic/data structure more theory-oriented problems are unrepresentative of "actual work" and are annoying, but what's more annoying is how the process is 1) opaque, and 2) repetitive. So you end up with candidates grinding through 80+ Leetcode problems to both master the skills necessary to ace these problems, as well as covering as many commonly-asked problems in a bid to game the system by preempting their interviewees questions. Goodhart's law abounds. This basically makes the process into an interview version of standardized testing. But- despite the standard prep process and a standard bank of questions, it can all easily fall apart because of subjective interviewer opinion, and so all the preparation is for naught. Candidates end up having to do the interview gauntlet for each company they apply at.
So standardize it. Put out clear, industry-standard rubrics for how one's performance is graded. Don't say "we only want to know how you think" and then ding candidates for not getting the right solution. You obviously care about the right answer as much as the thought process- don't be disingenuous about it. Maybe even standardize the level of difficulty of problems. And most of all:
Make this process a one-time thing.
Or rather, a once every five year thing. Outsource the ds/a interviewing section to a third party testing organization, like what Triplebyte is attempting to do, and have the actual interview be personalized to the company. Ask domain specific questions, relevant experience questions, system design, and culture fit questions during the company interview. And leave the DS/A portion be akin to the licensure tests that other engineering disciplines and STEM professions already have.
Software engineers may hate credentialism, but if it's a credential that you only need to take once or twice in a decade, then it's already far superior to the current system that forces candidates to undergo this each time they change jobs.
There's a difference between licensing and certification. A license is when the government says you need to meet some requirements to practice. A certification is when some entity, usually a board or industry group certifies that you are good enough at your job. We don't need licensing, but it would be great if we had broadly accepted certifications for software development. Even if you believe in FANG style interviews, ideally you should be able to pass one time and have it be accepted by every company. Once you have the certification, we could do interviews that focus on the type of things that individual teams and companies care about.
Not a license to practice, but how about some sort of exam I can pass that would let me never have to do the whiteboard dance ever again?
As it is, I'm basically studying for an unknown and nowhere near standard "certification", more or less, in the form of the technical interview, with only an inkling of what I might have to study for every single interview I go on.
I guarantee my career has been stunted because I delay my job search because I just don't want to go through that bullshit all over again, and stick with jobs long after I've stopped being passionate about them, despite there being a huge demand for software engineers.
Not everyone has to take the standardized exam, but if you do, you can show that to someone and they go "okay, I'm going to skip the technical pop quiz, let's move on to other things".
That's supposed to be what my degree was for, but apparently no one trusts that anymore, so why did I waste the time and money in the first place?
The other fields have various ways to filter out people.
* Accounting has CPA exams/certifications
* Law has the bar exams/law school
* Medicine has the USMLE/medical school
Tech prides itself on the fact that "anyone" can be a great tech worker regardless of how you get there - read some books, attend a bootcamp, go to college, or even just start banging out code on the keyboard until you gain experience
Unfortunately, that means that the companies have to have some filter
Did that. Doesn't seem to matter for shit in any interview I've had, except maybe to pass an initial HR filter. Waste of fucking money that was (I'm kind of joking, I did learn some valuable things in college).
Parent poster is pointing out that _in addition_ to your degree you must pass a professional exam as well. This is common in professional fields where there is a licensed title (e.g. M.D. or P.E.).
I know people with a computer science degree that couldn't write a function of production code if their life depended on it, and I know people without one that are gods on a keyboard.
I think a better alternative is some type of bar-like or CA-like test that anyone can take regardless of qualification, but which you must pass in order to be considered a competent software engineer.
Well, the same is true for lawyers. I know attorneys that have to look up everything in books. And I know attorneys that can cite basically any supreme court verdict from the last 10 years out of their head.
They instead use credentials. Where did you go to school, what was your gpa, where was your residency, what law firm did you work for, how likable are you.
Yes tech interviews could be much better but other industries hiring process is just as if not more broken than ours.
I had a guy ask me the number of hardware interrupts there were on a processor I used 4 years prior and what the exact part number was on an interview. Just incredibly stupid questions.
> I feel in the tech industry, networking and knowing people only gets you so far
I think it depends how influential your connection is.
At my previous FAANG job, my director was tapping his network to back-fill some positions and i heard through the grape vine that she waved parts of the interview in order expedite the process.
it could've been a rumor, but then again the person who joined was terrible to work with so i'm biased.
If you know the right people, they might tell you pretty much what Leetcode questions will be asked, who you might be interviewing with, what their personalities are like, etc. though. That is alot easier than going in blind (for most people I would assume).
The trick is knowing someone who works with your interviewers. If they talk you up to the interviewers ahead of time, it will bias them to be more forgiving, offer more help because "I'm sure it's just nerves", etc.
It's generally not even on purpose, just subconscience bias.
>FWIW, all of my friends who somehow managed to get into FAANG or other top tech companies swear that were they to go through the interview again, odds are they would fail. This despite all of them being great engineers.
Im fairly confident that's at least part of the dominance of this approach in big tech. If the tech companies can't collude with no-hire agreements (as they have in the past) to prevent developers from jumping ship to seek raises (i.e. introducing direct anticompetitive barriers) they can collude on an unspoken agreement to raising the barrier to entry, making the labor market less mobile and therefore, artificially decreasing competition. It would be pretty difficult at this point to make a case that the current hiring trends are intentionally colluded anticompetitive labor market practices but the end result is the same.
I say part of the reason because the mess that is the current hiring process is really a multifaceted win approach for big tech and loss approach for most of the labor force.
I suspect if you had a large movement of mid-senior level developers that wanted to shift positions at the same time and began refusing businesses participating in this practice, the industry might be forced to make reasonable changes.
The issue is that our industry has no labor organization and the probability of a critical mass of labor independently making such a fundamental shift (especially in the current economic client) is near zero.
Interesting theory, but does the game theory hold up? If high interviewing standards are only good for scaring employees into staying put, couldn't one of the FAANGs "defect" and lower the bar, gaining a hiring advantage over the others?
Probably not, at least in the sense being talked about.
After all, there is a glut of applicants to the top companies, combined with the idea of wanting to avoid false positives (bad engineer putting on an act) at any cost, even if it means losing out on many false negatives (good engineers who interview poorly).
Not sure if that is FAANG or US but nah; I recommend people and people recommend me to large corps and the interview is pleasant banter and nothing else. If I see a whiteboard I would walk out anyway; what is this, grade school? If my resume and references do not help then I have no interest working for you.
I'm sure there are still companies like that out there, but getting fewer and fewer. My current company (bank, non-tech) hired like that when I joined 5+ years ago. At most you would have gotten some language or framework trivia questions. Fast forward to today, and interviews are now all leetcode on whiteboard - just like a FAANG, albeit easier with lower standards. My direct manager and half my team is in London, so I know any UK candidates also get leetcoded the same way US candidates are.
Same thing with other banks and hedge funds - 5 years ago I think the only major bank that leetcoded me was Goldman, and even that was leetcode easy level. Today, every bank leetcodes candidates AFAIK.
Pretty much every company, both tech and non-tech, I've interviewed with the past 3 years has done leetcode interviews at various levels of difficulty and rigor. Interestingly enough, recently FAANG and a few other top tech companies seem be creating frontend specific interview tracks that are arguably more practical tests of JavaScript knowledge rather than pure leetcode problems.
Maybe time to look for my pension then! But no, I do not have the same experience; when I recommend someone (always seniors though; you sound like one), they do not have to do that. Another way to avoid (in recent experience) is to apply for positions as an agency (so a consultancy company which might be a one person company but looks like a pro outfit, website wise), work for 3 months (none of our people ever got interviewed: they trust us to have done that) and then apply for a job. Skip all the hiring bollocks as you already worked there and with the team etc. Again, not speaking for FAANGs.
Networking can in fact overturn results of the interview loop. In some cases, the result is known and the interview is arranged to support the case. You may outright fail some questions, and still get a green light from those interviewers.
Your advice goes directly against your example and I would say that most companies interview like that. Surely the advice should be to spend time on interview prep regardless of how good you are at the job.
Genuine programming skill has no correlation to the skill of regurgitating algorithms in a reality-show type setting. Actual development, of real products, isn't done like that. This is the reason why it's so perpetually controversial to put developers through this type of interviews which measure nothing relevant to the job.
> regurgitating algorithms in a reality-show type setting
You're not wrong, but this falls under the genus of "companies that suck at interviewing," and thereby hurt themselves (i.e. the company). In other words, it's not an inherent fault of the programming interview.
But there is more to it than meets the eye.
Some companies actually just are that incompetent.
But, to take a particularly successful and egregious offender: is Google actually just that incompetent? So that they systematically do hiring in a poor way?
No, of course not.
They actually don't want the best people.
They want the mediocre people who are also obedient. The kind of people who can be told to study an algorithms book, and will do it, and will regurgitate it, as a show of obedience, mostly, but also mediocre (but not great) intelligence---not great, because if they had great intelligence, they wouldn't waste their time on that kind of activity.
Such people can be treated as interchangeable cogs, which is why BigCo wants to filter for them.
> They want the mediocre people who are also obedient. The kind of people who can be
> told to study an algorithms book, and will do it, and will regurgitate it, as a
> show of obedience, mostly, but also mediocre (but not great) intelligence---
> not great, because if they had great intelligence, they wouldn't waste their
> time on that kind of activity.
Perhaps a bit cynical but there is truth to that. A huge company just needs cogs that can do the job and don't think outside the box much.
I interviewed someone that several of my coworkers had worked with before, and spoke very highly of. He even was my current manager's manager at their last job together.
He kinda blew the interview. He did okay, but not even close to the level of experience he had.
It was very obvious he was nervous, so we hired him anyway. No regrets.
At my company we interviewed two people. Both did fairly poorly in different ways. One we hired because he was a referral and came highly recommended despite a not great interview. He's been great.
The second guy was rejected but later hired as a contractor and he's worked out well as well and now we're looking at offering him a full-time position.
My conclusion is that our interview process isn't good at determining if a candidate would be a good employee.
My preference would be to hire someone as a contractor and give them a project then use that to determine their quality and fit. Unfortunately, HR tells me that's not feasible for reasons I don't understand.
My preference would be to hire someone as a contractor
When I was contracting it was common to be offered a job at the end of the contract. I always turned it down because I liked being a contractor. From talking to my peers at the time this was common.
If someone didn't want to be a contractor then they wouldn't accept your offer in the first place.
> In short, the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding.
In most of my programming jobs, explaining your work was most of the job and writing code was less important.
The whiteboard environment is a bit unnatural but I don’t want to hire someone who gets the right answer if they can’t explain it to me.
Most of the job: was that explaining while you write the code, under time pressure, or explaining your debugged code after you've tested it a bit and have confidence in it?
If I’m in the room, I will steer them away from dead ends and ask leading questions when they get stuck. The point of the interview is to give the interviewee as much of a chance as possible to demonstrate their skills.
I’ll say for sure, not everyone interviews this way. A lot of devs running interviews think of it as a “test” with pass/fail. I’d say the better interviewers see it as an opportunity to dig around and figure out how to get evidence of the candidate’s skills. That means being creative, that means responding to what the candidate says and finding opportunities to focus on their strengths.
The problem is that there's lack of transparency in the process. Candidates don't know the "real" rubric that interviewers use, so anxiety could make the situation far more antagonistic than it is. And to some point, it is inherently antagonistic; the interviewee is being judged, vulnerable to being rejected, by someone who may or may not be unfair. So the anxiety ratchets up. This can lead to all sorts of bad behaviors like candidates not asking questions for fear of revealing ignorance or looking stupid. And a lot of interviewers as you say, especially at the top firms, don't really care about "how a candidate thinks" at all; they want to see correctness, they want to see efficiency, they want to see whiteboard code that can compile.
I don't think this necessarily means the "answer question alone and present answer after" approach is the right solution to interviewing, but it does show that interviewers might need to take some steps to make the process less opaque and daunting to lower candidates' potential anxiety.
> And a lot of interviewers as you say, especially at the top firms, don't really care about "how a candidate thinks" at all;
From my direct knowledge of some “top firms”, if you only cared about correctness then you wouldn’t be able to fill out the interview feedback form.
Transparency may help, but in my experience, that’s what recruiters actually do and some of them are fairly good at it. The recruiter has a strong incentive to get you hired (they get bonuses for hiring people!) and will explain as much of the process as they can, if you are willing to listen.
That said, there seems to be a fair bit of turnover in tech recruiting.
And, as you said, every interviewer will be different. But I don’t see a way to fix that without eliminating technical interviews altogether. You are being judged, the interviewers are subjective, they do disagree about the rubric, and a bunch of money is on the line. It’s completely reasonable to be anxious. I don’t really see a way out.
> And, as you said, every interviewer will be different. But I don’t see a way to fix that without eliminating technical interviews altogether. You are being judged, the interviewers are subjective, they do disagree about the rubric, and a bunch of money is on the line. It’s completely reasonable to be anxious. I don’t really see a way out.
Perhaps what we should be working towards is making it more transparent- it's still ubiquitous for candidates to be rejected without feedback- and more objective, and improving the technical interview process further. I'm not arguing for throwing out the baby with the bathwater.
There is a world of difference in explaining your work to someone you've never met before on a problem you've never seen in an environment you've never worked in.
Versus explaining your code to your coworker whom you've worked with for the past X time who understands some of your eccentricities or idiosyncrasies and without a pass/fail judgment lingering over your head like a sword of Damocles.
I would definitely not want to work for you, I think there's something deeply ineffective when most of the job is explaining your code.
I say that as someone who cares a lot about writing easy to read code and has no problem explaining it. Also, I'm very bad at solving a problem while being required to think out loud.
The skill of explaining your work is a very important one in practically any creative problem solving field. This only increases as the technical complexity of the problems faced increases.
Sometimes this explaining happens in textual format, like code reviews and design specifications, but often it happens verbally. Absent an explanation, it's hard to build consensus among the parties involved (co-workers, users, management) that your solution (as manifested by the resulting code) is the right one, and for anything non-trivial, you surely can't expect others to just assume that it is the right one.
explanation is important but its unreasonable to expect someone to do it on the spot in a time-pressured environment. Often you have to go away and think about it alone in order to come up with a good way of explaining things to others.
> Also, I'm very bad at solving a problem while being required to think out loud.
There’s a simple strategy for this, which works fairly well in both interviews and meetings. You give people time to focus and think about the problem and solving it, and don’t force people into discussions right away. It’s not a panacea but in practice it is very effective.
The reason why this is important in meetings, as someone running a meeting, if you don’t give people time to think, it will always be the same one or two voices in the room. This is why brainstorming sessions suck. The same thing applies when you are choosing who will take responsibilities—the same few people always speak up first.
> That's not what is being asked. It's explaining your thought process while actively doing the work in an "unnatural" environment.
In short, that’s a limitation of the study.
The study is comparing “explain your thought process aloud as you solve the problem” against “solve the problem by yourself and don’t explain anything”. There’s a whole range of sensible middle options, like “solve the problem by yourself and explain it afterwards” or “solve the problem with the interviewer and then explain it”.
In one of my recent interviews at a mid size silicon valley company, I was given 4 hours to make a tetris like game.
One of the most enjoyable interviews ever with no whiteboard and no one around me to check on my progress. I was blank for first half an hour or so but made great progress by the end. Eventually got the offer.
I find it interesting that academia often points out that skills interviews have a soft-skills bias, but rarely do I see them consider that this may be a feature and not a bug.
> “Our study suggests that a lot of well-qualified job candidates are being eliminated because they’re not used to working on a whiteboard in front of an audience.”
This is a common daily task at any company I've ever written software at. People who can communicate their ideas effectively are better at their job.
Right? At least pre-covid, I would say I spent at least an hour (sometimes many hours) every working day "explaining things on a whiteboard in front of an audience". It's a huge part of my job.
(Post Covid, I do the same thing, but have to do it over chat)
IMO: "Do you have a clear understanding of your area of expertise, and can you communicate that understanding on to others?" is a far more useful predictor of success than "Can you find the right answer to the kind of problem that can be solved on a whiteboard in 45 minutes, or in 5 minutes by searching for an answer on stackoverflow?"
That said, I agree that generally interviews are terrible. But the (implicit) screening for soft skills isn't the terrible part.
I've had the opposite experience. In >10 years of software, I've never written code on a whiteboard outside of an interview. Systems layout, designs, code pathing, sure; but never real, compilable code.
I agree with the last point, but I don't think either of us can tie our anecdotes to it.
There is something to be said for the soft-skills evaluation being relative to the expectations of the job position, sure.
I'm saying: I find it odd that I rarely see these types of studies discussing the merits of that nuance. Especially when the idea of differentiating between soft and hard skills is not something that would be foreign to the average interviewer.
"Interviews are 100% for soft skills and technical evaluations are 100% for hard skills" seems to be a common assumption which I'd argue is false. Few tasks a developer does is purely a hard or a soft skill, there's always some overlap.
If you get anxiety during a job interview it is because you don't have enough experience, you have a massive ego or you were raised improperly. I've never met anyone who gets crippling anxiety from a job interview when they've done what they say they have done and they are humble enough to just admit when they don't know something.
Its really funny. I was interviewing at a large company recently and the way they do interviews is weird. Each team that is interested in you does their own on-site. The first one I bombed b/c I was so anxious but after I experienced it once, I aced all the other teams I interviewed with. Nerves definitely play a role, its cool to see that it is being quantified.
Hmm...but if everybody is stressed out, then it equalizes and you're testing for skills again, just skills under pressure.
And having skills retained under pressure is actually a valuable trait in this industry. And at least at desirable companies, the process is designed to minimize false positive, it doesn't care that much about false negatives. This is by design.
My guess would be that they filter out orders of magnitude more people with poor resume-writing skills or that didn't go to the right school.
Although I do agree that interviews that you can/must practice for are silly and there's a lot oof that around.
That experiment assumes that SWE job is exclusively about solving technical problems, so we can just "lock the candidate in a private room and let them solve it".
In reality human to human interactions, sometimes stressful, are part of the job. Customer & PM requirements, technical debates, giving feedback, explaining your technical solutions, etc. are common.
Maybe part of the problem is the education that does not prepare people to reality of the job.
Note: I do realize that SWE interviews suck, and I'm not trying to defend them. Just making comment on the study.
Agreed. When I'm conducting interviews, the actual code produced is less than half of what I'm looking for.
I want to assess if the candidate can communicate their thoughts effectively, if they can identify alternative approaches and justify their decisions, how they receive and incorporate feedback.
Locking someone in a room to write on a whiteboard is not a coding interview.
I think a hybrid model (create solution in private, then present it and answer questions) might be better. That way you can assess the person's communication without looking over their shoulder.
Hopping aboard your train of thought here... I've never been asked to whiteboard anything truly heinous, and I absolutely despise interviews that are set up as a gauntlet to be ran.
But if I'm interviewing you for a senior SQL engineer position, and you can't talk me through whiteboarding a simple query with one join and one aggregate function... you're not senior material. I'm sorry. We need people that can communicate and explain themselves, and talk through their own line of reasoning, where I work.
Again, I'm not defending SWE interviews... but of course people are better at expressing their skillset when they're in a controlled environment. The trouble is, the workplace isn't a controlled environment. Life isn't a controlled environment.
I don't want to work with brilliant programmers that have zero social skills. If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?
Well, I can communicate and explain things reasonably well to a room full of people. I ran a side business doing that for about seven or eight years.
And I can ship products. I've done that a bunch of times--in several cases I've written most or all of the code in the shipping product.
I can't interview all that well, though, and I'm absolutely terrible at whiteboard coding, or pretty much any kind of problem-solving with people watching me. Except pair programming. Pair programming with a colleague that I know well is no problem.
I have two obstacles. One is anxiety. I score really high on trait neuroticism. It's one of the so-called "Big Five" personality traits. One of the things it means is that I have a lot of self doubt regardless of cause or circumstance, and a lot of anxiety to go with it. As for "working on myself a little bit," the Big Five are extremely resistant to change. Basically, the only things we know of that affect them much are psychedelic drugs and catastrophic trauma. Working on myself a little isn't going to reduce my trait neuroticism much.
I've been through quite a few tech interviews, including some familiar big names. I've gotten offers. But that's never been because of my performance in coding or solving brain teasers, at which my anxiety has been uniformly awful, and so has my performance.
The second obstacle I have is that I cannot combine certain classes of cognitive activity. For example, I cannot navigate and carry on a conversation at the same time. I can drive just fine--safely and competently. I just can't navigate if involved in a conversation, and will predictably get lost. It's predictable enough that when my daughter was a teenager she exploited it for laughs. "Let's see where we end up!"
Another thing I cannot do is solve logic and programming problems while being watched or while carrying on conversations with strangers. The interaction with the strangers forcibly occupies 100% of my attention.
I don't know if I'm representative of a vanishingly small fraction of the population. Maybe so. But maybe there are others more or less like me.
Doesn't mean you need to change your hiring preferences. Probably does mean you won't hire me. Oh well. I seem to be making do.
If there are others like me reading, and wondering what to do about it, one thing that has worked well for me is to do good work for people and cultivate good working relationships with them. I have more often than not been hired by someone who already knew my work, or on the recommendation of someone who knew my work.
I'm sorry to hear that. Every situation is different. If I were in your shoes I'd try to be open about it with recruiters and figure out / propose some alternative that work better for you - whatever that might be. Also keeping good portfolio of Open Source projects, blogposts etc. might help convince potential employers that you're worth considering and that your 'condition' (for a lack of a better word) is real and does not affect your work performance. Keeping good reputation and connections (which you already mentioned) is also very important.
From my experience companies are very flexible and open minded - mostly they care about just not hiring someone that they will have to fire later.
No need for sorrow. Everyone is dealt some advantages and some disadvantages. We work with what we have. I seem to have done all right. My resume is long and I have good stories to tell. I can't complain.
I was responding to another poster who said that "If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?"
I mean, maybe. Maybe sometimes. Then again, maybe not.
Maybe "working on themselves a little" isn't necessarily the answer. There isn't a known way to "work on myself" to affect my trait neuroticism in any significant way, for example.
Maybe it's not necessarily about "courage". I mean, courage is doing something that frightens you because it needs to be done. I know what that's like. Technical interviews don't even rate. But that doesn't solve the brainteaser on the whiteboard when a stranger's watching me.
In general - working on yourself is usually somewhat possible. For many people public speaking etc. is something they have to practice and they do eventually learn. In some situations though it might not be. All individual cases are different.
So I wouldn't be too hard on myself, but also I wouldn't give up. From what I read, you might have already tried, so oh well, maybe you're just stuck with it.
I like "We work with what we have". That's something I deeply agree with.
That's a good point. But for a lot of people it's not even a problem to do that in front of your co-workers who you work with on a daily basis. Even when I had to work with customers, I had zero problems talking to them, getting requirements, and figuring things out.
If I had to land that customer for the business, I would surely botch it. That's why I didn't go into sales.
I think that the difference is you have much higher stakes in an interview. If you make a mistake with a co-worker or customer, you can usually just go back and fix it. If you make a mistake in your interview, it can cost you the job and it's not so easy to go back and fix.
How right you are particularly when a role has any tie to a production support situation. Not only do you need to solve the problem, but you need to do so when other people are weighing on with their opinions on what might be wrong, escalating when it's not solved fast enough, demanding answers you might not have or asking you to defend the ones you do, particularly if they answers are ones no one wants to hear.
This is the genesis of the infamous behavioral interviews where people ask about conflict resolution skills, how and when you'd escalate problems you can't solve, name a time when you had a problem and how you solved it, etc.
Yes, but real-life situations are rarely as high stakes as interviews, on the individual level.
Sure - you can find yourself in very stressful situations at work, but I have never in my life been in a situation where you're given 45 mins (or hours, for that mater) to solve some problem, or you get fired.
Simply but, there's an incredible discrepancy between the stakes at play, between interviews and work.
To put it a bit more extreme - but to hammer down the point: Imagine working on some problem, with no help whatsoever, and that in front of you, there's some guy pointing a gun against you.
Just imagine how stressful that must be, and how much of your focus is distracted on reading that guy.
Same goes for interviews. For a lot of candidates, that interview is their way out of poverty or lower-class living. And the judge and executioner of your future, is sitting in the same room as you. It's a very stressful situation - probably one of the most stressful situations in your life.
I think that's one of a big problem for candidates to fix - that mindset is wrong and should be changed. An interview should not be a "high stakes situation". It should be more of a casual event - trying to get each other and see if there's any fit. An interview that didn't go well is effectively the same as not having an interview at all. Worst case is losing some time and even then it's often a possibility to learn something.
Taking it all too personally is a big mistake. Sometimes even candidates that did really well don't get a job offer for weird reasons, etc.
It's very similar to dating. Desperation ruins everything.
I think that might be a reason why a lot of people report getting a job after they stopped carrying anymore.
Engineers tend to over-focus on correctness. I’ve run hiring process in the past where over a year we interviewed around 50 engineers. Feedback from our own engineers involved in the process would often be like “Seems like a nice guy. But his solution to the challenge - one he’d never seen before - had a bug in it! And the solution was just O(2^n) and failed to notice the O(n^2) solution. Even though I glared at him a lot and kept distracting him with hints” ... I’m exaggerating for effect of course but some form of that would happen often
It is. However, many times people just have not been shown 'the trick' before. For example the thing that basically started this coding interview style, FizBuzz. Unless you know the modulus trick you will fail it. If however I can show someone the modulus trick in the interview and they 'get it' right away I have no problem with that person. It shows they are willing to learn. If however you go try to go with a hint like 'is there a better way?' you are just going to frustrate them and fluster them.
In fact I usually try to shy away from coding a quiz that has 'a trick'. I usually want to see you do indeed have a decent grasp on the language, can properly decompose a problem, you can finish in front of me and quickly, and most importantly ask questions to clarify. I also try to stay away from things that require a particular framework. As those come and go and usually people can learn new ones fairly quickly with some good examples. I also ask if they have read any of the 'coding interviewing' books and what they thought. Usually if they can understand those books they are fine on complexity issues.
This can be said of all interviews, not just tech. It’s hard for great many people to put themselves out there. I mean interview itself is a performance, isn’t it?
Having said that communication and interpersonal skills are big part of most jobs even in tech. We hardly develop software in isolation these days.
I don’t know about other industries, but in Tech there are some really inexperienced interviewers out there, who’re impulsive and lacks discipline in conducting objective and meaningful assessment.
I hire for industrial IT, and being able to handle the anxiety of a high-pressure situation with grace (e.g. mill down, losing tens of thousands of dollars a minute) is an important factor. No one expects miracles, but you need to be able to keep your cool and work the problem, not waste time fighting with your own rising anxiety.
Seeing how they deal with a stressful situation is an integral part of the interview in my opinion, and I imagine it is a feature for any IT role where that is the case. Standard dev roles, or other roles where time-based stresses aren't as prevalent, may be different.
Only yesterday I had a candidate bow out 10 minutes into the whiteboarding exercise, which is probably best for both the individual and the company if that kind of situation is untenable for them.
I do great in a real production fire and stay cool and collected. Put me in a room with a stranger who’s judging my every move and my mind just goes blank
Stress when interviewing is similar, but not the same as on the job stress.
One major factor, with a job: you're still getting paid. I think the economic anxiety one faces is pretty stark, if you fail a job interview you're going to be unemployed until you can pass the next one.
Yes, interview stress is completely different. You have to commit a pretty egregious offense on the job to get fired on the spot, but literally anything you do during a job interview can cause you to lose the job.
Also, performing in front of strangers is completely different from performing in front of people you already know (co-workers).
It's extremely common for people to be nervous when meeting strangers but fine among associates. This is such a blatantly obvious empirical fact, it mystifies me how the tech industry can act as if this phenomenon does not or should not exist.
And often you won't have any idea what caused you to lose the job in a particular interview. There's generally no feedback at all so it can be very difficult to try to improve.
Yeah, devs that work for me also work in a critical environment such as yours.
As soon as an issue is raised by our clients, they immediately kick off a 40 minute long timer and put a physical whiteboard in front of us asking us to solve the whiteboard problem in front of them in under an hour. All the while the client is physically behind us commenting on our code and watching our every move as we paint the whiteboard with a solution. This is literally what happens to me and my engineers on the Job. All of our code happens on the whiteboard (we don't use editors or keyboard or mice!.)
The only thing we need to deliver to our client is a picture of our code on the whiteboard along with some trivial complexity analysis and our client pays us money for coding under pressure.
Our interviews just consists of me doing the exact same thing as what happens on the job and therefore our interviews are 100% identical to our real life coding problems.
When science tells someone that the interview process they've been conducting for years is essentially wrong people have to sort of remake reality and their own reasoning to justify the science rather than admit fault. Obviously, this isn't what's happening with me or you in this case.... the interview process at my company is 100% identical to real world scenarios so I'm good.
I've always been telling everyone I know since before this article that I interview for anxiety not useless technical skills; anxiety is what determines whether a developer is great or not.
Our company consults in a variety of software related things. For example in our interviews we ask our candidates why are manhole covers round?
We ask them this because, literally on the job our employees are suppose to program robots that make manhole covers. Daily problems we face include deciding whether we should program the robots to make square manhole covers or round manhole covers. Really hard stuff... and you really need to be able to figure out why manhole covers are round or how many golf balls can fit in a bus in order to do the job... If you can't figure this stuff out you're not qualified to work for my company.
Not to mention, as I said before, none of our programming happens using computers. We do it all on a whiteboard in under an hour while the client is over our shoulder watching.
It's fine if you believe this is true for your environment, but that's really quite rare.
I can see 'high pressure' particularly in Devops, but that's a different kind of problem solving.
The headline speaks perfectly to my intuition after many years in the trade, and that we are 'measuring confidence'.
Note that they assumed that interviews were set up to be fair - they are not - and that's actually ok. If a company has preferred candidates, it's perfectly within their right to do that (so long as 'preferred' doesn't just mean 'gender' or whatever).
It's been a while since doing this for me, but when I do it again, the 'on site' will be a page with some neat problems, some Q&A and then they take basically as long as they want to go through them, the interviews start when they are ready and you just go over 'what they did' and then interview for the rest of the issues.
'Having someone watch you solve a problem' is very specific social condition.
Honestly if you built a better bridge more would come over.
Looking at white boarding comfort and judging that the person can't handle stressful situations when they have a defined role and understand the stack are two different things.
I feel like everyone wants to make a quick judgement and pat themselves on the back.
Multiple discussions about this on HN today, that's good. Maybe it will lead to the pendulum swinging back a bit.
Interviewers need to do their due diligence. Coding tests can have their place when interviewing for certain kinds of positions, especially for new grads and entry level positions. In the end nothing can replace checking references and looking into a person's work experience.
There is no algorithms and data structure whiteboarding test that can differentiate between a junior and a senior developer. Being senior is a product of acquired wisdom, not just skill. And in fact senior devs can easily bomb out a whiteboarding test because we're further from university CS experience.
Your last paragraph sums what why everyone on HN is so upset about the interview process. That is, the process is tailored for new grads, and what is being tested is not at all related to the job. 5 years of schooling, and 10 years of experience is not enough if you can't implement depth first search using double linked lists, hashes - and ultimately graph theory - within 45 mins. Heck, new grads can't even do so.
Well, yes, if you looked at my comments you'd see I'm part of that "everybody" set.
But to moderate it a bit; in my interview training at Google it was clear: the interviewee doesn't have to get the answer correct. The point is to see how they think.
I mean, I think it's still insulting, but there is that.
My first "real" CS-related interview, which had technical questions, I had on the very same day that I had taken my final exam in our data structures and algorithms class. And because our DSA class was quite rigorous, I knew the material in and out.
I absolutely aced those questions, but did poorly on the systems design questions because, well, I simply did not have much experience designing or building complete systems/software.
Years later, it was the other way. I stumbled my way through the DSA questions, while acing the systems questions.
But with that said, I think also the order of your performance matters. If you walk in, and absolutely bomb the first topics, I think there's going to be a bias against you. Maybe they'll think you're a moron, or something like that.
I think the more accurate headline is likely to be "anxiety can negatively affect performance in interviews."
I would love to see what a control group of non-tech interviews looks like. Not sure what the equivalent "whiteboard while out of the room" is, but it's hardly just software devs who get nervous and can panic in interview situations.
Interesting. I usually do pretty well in interviews and I've always attributed that to my four years on the debate team in high school. This tells me it might be even more true than I previously thought. Even when I'm nervous as hell, when you put me in front of a judge (interviewer), I click into "performance mode" where I project confidence even if I know what I'm saying is BS, let alone just a bit unsure. This is how you win parliamentary debates when you've had five minutes to prepare for a topic you know little to nothing about—and I guess it helps in interviews too!
That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense (useful when you have 2 minutes to prepare a 3-5 minute speech on a topic you've never seen before). So when I get up in front of a whitebard, I just start talking confidently, and even if by the end haven't solved the problem quickly or efficiently or at all, it sounds like I've been thinking sensibly about it.
My takeaway, I guess—parents, make your kids do debate!
> That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense
You're effectively advocating for more bullshit. The problem is that this works in the first place - we shouldn't be rewarding it.
We can't expect everyone utterance to be rational and logical. We're all just human at the end of the day. We should expect job candidates to try their best to get the job.
What's the point of asking these sorts of questions? Typically to try to get a sense of how somebody thinks and what sort of rapport they have with the interviewer while problem solving, not that they get to the "correct" answer.
I think they have some utility because this is a common occurrence when dealing with clients.
He's saying "this makes you more likely to win at the game, perhaps you want to teach your kids this skill so they can win." He is not saying "the game is fair."
don't hate the player, hate the game. easier to tweak personal actions to address the world as it is than to rant against how the world doesn't work per your ideals. at least this guy is self aware enough to admit there is some amount of bullshit in what he says. I am more concerned with people who refuse to admit this, or worse, genuinely don't believe it.
You're right, unfortunately that's what it sounds like I'm saying. It's not really what I meant. The idea I was trying to get across is closer to "being able to communicate what I'm thinking about as I'm thinking about it, and verbalizing that as a coherent sentence rather than just fragments." Ironically, I wish I had communicated that better :)
I’m a person who finds out what I’m thinking as I speak. Iterating towards an idea with confidence in each turn.
Even if you end up in the vicinity of where you started, problem not actually solved, you demonstrate how you move through problem space.
Similar to another comment I read comparing interviews to athletic tryouts and competition, which tries to assess how a body moves through physical space.
Rhetoric was the most respected skill in Ancient Athens.[0] I think we would do well to stop thinking of rhetoric, debating, and public speaking as hobbies for some high school students. They are essential skills for wealth creation and advancement in society.
> That and debate gave me the ability to just start a sentence without knowing where it will end ... it sounds like I've been thinking sensibly about it.
I've always structured my interviews to weed that kind of thing out.
Comprehension. Ironically, I just started writing a blog post about it 10 minutes ago.
To summarize, I try and get the candidate to discuss tradeoffs between different approaches to common situations that the candidate should be familiar with.
For example:
Should GetUserById(int userId) return a null, or throw an exception, when there is no user with the ID? Explain the tradeoffs; or explain why each approach is better in different situations.
Should you deserialize (xml or JSON) to a dictionary or a strongly-typed object? What if you need to manipulate the document while preserving fields that you didn't know about at compile time?
Assuming you have the experience I'm screening for, you either know the tradeoffs or you don't.
will benefit greatly from being able to synthesize a bunch of information out of some simple base. Yea you need to know the technical details but being able to explain tradeoffs is more abstract and the OPs method will just lead to a better answer here.
Exactly, yes! It's about explaining what you're thinking about as you think about it. Silence is a killer in interviews in my experience (not always—you can be strategic—but for the most part), and I try to minimize dead air in any interview I'm in.
Why is it a bad thing? It's off the cuff, and likely less filtered than a prepared statement. If they end up somewhere sensible, then your interview process actually worked; you found someone who when confronted with a new problem, is able to think sensibly about it, even in an interview situation.
But OP said he ended up somewhere sensible. He -was- thinking about the problem at hand. As he spoke. He got there in a roundabout fashion, perhaps, and had to think quickly, but it's still -valid-. He's not just bullshitting the interviewer (which is very noticeable and easy to suss out, and never ends anywhere sensible).
Duplicating a comment I made above, the idea I wanted to communicate is closer to "being able to communicate what I'm thinking about as I'm thinking about it" — NOT just rambling my way to a BS answer.
> That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense
That sounds like a good skill to have in marketing, sales or politics. It sounds like a bad skill to have for engineering.
> project confidence even if I know what I'm saying is BS
One of my very worst bosses had this quality. Perversely, it was about the only thing I admired about him.
The guy could walk into a meeting of NASA flight engineers and engage in conversation without uttering a single fact that would betray his total lack of any knowledge/qualifications.
n.b. He was fired not long after I left the company.
To be clear: I don't advocate for doing that, and I don't try to BS my interviewers. That's why I said "let alone if I'm just a bit unsure"—if, during high school debate, I can confidently BS, then now, during an interview, I can confidently talk even if I'm nervous and not fully sure of myself.
An upvote is warranted for a life long gift to kids to be able to communicate confidently.
A downvote is warranted for giving a bad advice on gaming a system and giving both parties in an interview process a poor indication of what is expected in possibly years to come.
Thus, no vote given.
I think given what I read about debate teams -- on podcasts and here in HN, I am on the fence on suggesting that to my kids. There are certainly great benefits, but not too certain if those out weights the drawbacks.
I'm not from a country that really does "debate teams", but as I understand them a lot seems to be about "winning the argument", which doesn't strike me as especially great.
On the other hand, it probably makes people better at spotting a bad/fallacious arguments, and gain appreciation that complex topics tend to have multiple reasonable opinions on it – two things we desperately need more of.
True, "winning the argument" is one of the main drawbacks I fear. The main benefits for me is to foster (often quick) logical thinking and reasoning and the ability to articulate it. It is difficult to build this skill without a "debate team" setting. More so if your kids are more on the shyness scale.
I don't mean to give advice on gaming the system—I'm sorry if I came across that way. The point I was trying to make was much closer to your first line—that the current interviewing system rewards confidence and communication skills, and that debate helped me build those. Ironically, I wish I had communicated that better :)
Also, I agree that there are drawbacks to doing debate. For one, there's a certain "debate kid" mentality that I picked up and had to unlearn.
Thank you for clarifying and confirming my other fear of undesired debate mentality. FYI, it's a good idea to edit your HN profile. When you write good comments, people may check it.
I concur. I did BP for only 1 semester in college but the experience proved to be valuable down the line. It forces you to articulate yourself "on the fly", streaming out ideas as they are formed (from core principles / bullet points you have in your mind).
Interviewing is a skill. I once had a good job where I was happy and didn't move for 10 years. Big mistake - after that I bombed at interviews. After doing 10 interviews I'm now up to speed. I kinda feel like I need to do some interviews every year just to remind me how to behave and how to keep market ready.
I was in the same boat. Nearly 10 years experience at the same company that hired me after university. My first phone interview had me write a linked list and I froze. I had even practiced writing linked lists that week. The interviewer probably thought I lied about my resume.
I spent 4 months doing leetcode problems and reading books before I started doing interviews again. I would interview at places like Facebook that I had no interest in working at but I needed the experience. 7 months after I started I finally got an offer.
I don't want to go through that again but I agree that you probably need to do interviews every year inorder to not completely lose that skill.
1,179 comments
[ 2.9 ms ] story [ 533 ms ] threadOn my experience far too many companies are using processes that select for entirely the wrong criteria.
Even worse: I had an encounter with another guy who refused to make a test because his CV demonstrated everything we needed to know and asking him to solve a few problems was "incredibly disrespectful". He was also apparently going to start next Monday. Needless to say he didn't.
Sure, no love for whiteboards from me. However, this does suggest an arbitrage opportunity for an employer who is willing to look differently at these candidates, and then profit handsomely, Moneyball-style.
There's a bunch of money just being left on the table and nobody is picking it up, really?
My two cents is that interviewing candidates can give a whole lot more insight into how to perform in an interview. Interview performance is a skill that is separate from the actual job, and gauging that performance is sub-optimal. But everything in this space is sub-optimal, it's all pick your poison.
What I don't get is why no one is attracted by the social brownie points they'd end up with when their moneyballed team inevitably turns out to be 70% female.
That's when I get an offer.
Sounds like dating ;)
In my short list (ignoring years of time wasted travelling), i have only met 1 yes man. He came from money, and sales. Owned 4 large businesses, and would over promise, under deliver and over receive ALWAYS.
So what are your experiences like and from where?
I think no backbone can be a personality trait, sure. But it is NOT a trait of 'need' and action, in my experience.
I'd like to focus on Dating because I disagree with your offhand comment, even though it was in jest and is probably more cynical than you are in person.
> The more … apathetic ill be cherished. Sounds like dating ;)
Feeling bad makes your behavior suffer. Being angry makes you less nice to people. Being tired means you don't engage with people. Being scared or anxious makes you hide your vulnerabilities and close yourself off. I'm not saying that badly behaved people deserve to be ignored, I'm acknowledging that it is uncomfortable to interact with people who are hurting.
Being "driven and hungry" while dating makes you behave worse. It makes you over-eager to please the other person, and it flattens out your personality. Instead of focusing on making the person like you, you should be exploring whether you like each other. A date can be successful even if the two of you don't end up dating because you two would not make a good couple.
Some companies will only hire those who are desperate and will do anything. I assume, hopefully correctly, that those are the companies not worth working for, so instead I use the interview opportunity to get better at interviewing. Because these companies want to take advantage of their employees, I do not feel bad wasting their time.
I interviewed at a few places for some really easy, fun-looking jobs while I was doing freelancing. I didn't care if I got the jobs, but they did look trivially easy compared to what I'd been doing earlier in my career. Surprisingly though, after making it through all the technical people and talking to the CEO in every case, I never got the job.
Later on I applied to a really awesome job at a FAANG-class megacorp, and the hiring manager's manager called me. He said "Look, you've gotten the highest marks I've ever seen for a candidate. Everyone loved you. But I have a hunch that you aren't going to be dedicated to this job. So before we move forward, I want your personal promise that you aren't planning to just hop ship in 6 months to start a company or something"
I was JUST starting to get nervous about finding a new job at this point. My consulting work had dried up, I was down to about 3 months' worth of cash, and I was thinking about selling my car to get a few more months' runway while I grind Leetcode in order to be able to pass the bar at Google.
So you can imagine how shocked I was to hear that this guy thought his awesome senior engineer FAANG job wasn't good enough for me. I promised to stick it out, got the job, and so far it's been my favorite job of all time (other than the glorious week where my crypto trading bot was working well and just printing free money..)
Then my brilliant plan was spoiled.
My previous startup I worked at had gone under, and I was thinking about starting my own. An old coworker of mine invited me to lunch. We had a couple of margaritas, and he asked if I wanted to see his office.
I went to check it out, and suddenly he asks if I can talk to some of his coworkers.
So I basically interviewed while drunk and not fully realizing I was being interviewed.
They offered me the job, and I have been working there for 8 years now.
Maybe the best way to be relaxed during a job interview is to not even realize it is happening. And having a couple of margaritas can't hurt.
My first time interviewing with Google (in 2008), the press was reporting that they were in a hiring freeze, I had no big names on my resume, so I figured it was basically hopeless and I'd just do the best I could and enjoy my trip to Silicon Valley. After my interview, I was like "Well, I did okay, but you really have to be perfect to work at Google and I wasn't perfect. I'm never coming back to California anyway, so I might as well do all the tourist stuff [I'd held an extra day to interview with Twitter, but they ended up not moving forwards with my candidacy] and just treat it as an all-expenses paid trip to SF." So I just went to Alcatraz and Fisherman's Wharf, treated myself to a nice dinner, and took the red-eye home to Boston. As I'm groggily waking up after a post-red-eye nap, the Google recruiter called me to say "Your interview feedback looks great. We'd like to fast-track your application on the assumption that the hiring committee will say yes."
Been living in California for 11 years now, and I just started back at Google after working there for 5.5 years, leaving, and doing startups for another 6. The most recent round of interviews was the same - I was super nervous for Lyft, Stripe, Coinbase, Netflix, etc. and ended up not getting the job, but with Google it was like "Hey, I worked there once, I know people who also know my interviewers, I used to give interviews like the one I'm sitting in now, what's the big deal?"
I've read advice that you shouldn't schedule your interview on a Friday, but that's exactly what I did so that I could spend the following weekend in SF :)
lol this industry is hopeless
The panel was different, too - my first time around I had 2 algorithm, 2 UI (Javascript), and 1 system design interviews, while the second time was 2 algorithm, 1 behavioral/leadership, and 2 system design interviews. This is probably reflective of the differing role and higher level.
Somehow the recruiter was still interested.
Maybe this should be my strategy for every job application? :-)
Like it or not, interviews are a game. Smart people will figure out how to play them after losing enough, though maybe some of us take a bit longer ...
[0] https://en.wikipedia.org/wiki/Kava
(I'm older now, have hangovers less often, and find them much more tiring, so this might not apply any more)
My conclusion was if I interview to gain interview experience, not to get the job, I do far better, and I gain interview experience. This is the local maxima I found from that, but there could be better.
The role of anxiety in interviewing became very clear to me during a season when I failed all my interviews until I got my first (and a very good) offer. Suddenly I started passing all of them. It was night and day. I was very surprised having always had much longer and frustrating recruiting processes.
> For this study, researchers conducted technical interviews of 48 computer science undergraduates and graduate students. Half of the study participants were given a conventional technical interview, with an interviewer looking on. The other half of the participants were asked to solve their problem on a whiteboard in a private room. The private interviews did not require study participants to explain their solutions aloud, and had no interviewers looking over their shoulders.
> Researchers measured each study participant’s interview performance by assessing the accuracy and efficiency of each solution.
[...]
> “People who took the traditional interview performed half as well as people that were able to interview in private,”
To acquire a larger N would mean recruiting more participants, which would require either compensation/extra credit or rely on people volunteering ~1 hour of their time. Likewise, collecting that data would require time and money. It is an time/resource issue with human research. You can require a small N and get some results or require a large N and not have enough funding to support it.
Also note, what is a "large N"? There is no set in stone amount and I've even seen reviewers say a sample size of 10,000 is too small.
[1]https://en.wikipedia.org/wiki/Effect_size
That depends on the size of the population being sampled from, the margin of error, and the confidence level.
For a huge effect like the one shown in the study, where one side performed 2x as well as the other, a sample size of 48 is more than large enough to say that the result is statistically significant. If there was as small effect, that wouldn't be the case.
Put it this way. You want to find whether people from California prefer Taco Bell or Pizza Hut, so you randomly sample 100 people. If all 100 people say Taco Bell, then you can be reasonably confident that more people from California prefer Taco Bell. Because if at least 51% of your population preferred Pizza Hut, the odds of not getting one of those people in your sample are minuscule (the odds of getting all Taco Bell people in your sample if 49% of the population prefers Taco Bell is 0.49^100).
If 51 prefer Taco Bell and 49 prefer Pizza Hut, your confidence level is too low to be useful--you need a larger sample size.
However, research also has issues with reproducibility. No one will publish work that was simply a repeat of someone else work. Researchers have to twist or add something. It is a discussed issue, but no one in power has made attempts to rectify it.
Depends on the size of the effect you wish to measure, and the underlying variance. They got a massive effect, which to me, seems improbable, and I half suspect the randomization is at play.
I think it'd be pretty simple and cheap to replicate the things we care about in this study. All you really need is a whiteboard, some markers, a small room, a few interviewers and a candidate pool. The eyetracking getup really only gets in the way of that and seemed more like a problem in search of a solution.
Not when the effect is large. 48 people is fine when there is a 2x difference.
I don't know about most people but lately, at the end of the day when I make enough progress to meet expectations, the last thing I want to do is anything related to work anymore, even if it's an investment in myself. At some point my life takes precedent over my career.
If delivery expectations are reasonable (i.e. time requirements), I don't mind networking outside of work. I've worked in environments with reasonable expectations and environments with unreasonable expectations. My observation has been that environments with reasonable expectations are in decline in the name of competitiveness and cost cutting measures, passing costs to employees.
This industry is becoming significantly less enjoyable to work in and growing to a toxic point of hypercompetiveness at every layer. It's not healthy for those working in the industry.
It’s a big company problem, not a startup problem.
Who can? Conferences are so expansive!
This is very much not how the Silicon Valley I've been in for decades works!
This approach does discriminate against people who are looking for their first job. I think that's both unavoidable and fine.
FWIW, all of my friends who somehow managed to get into FAANG or other top tech companies swear that were they to go through the interview again, odds are they would fail. This despite all of them being great engineers.
Plus, stack on top of that, I've seen how the sausage is made. I've done 90+ interviews on the other side. At the end of the day, it comes down to how lucky you got in the loop. I've seen some people written off for some bullshit reasons. Similarly, I'VE declined on people who're likely far, far, far more skilled than I am because of the lowers/raises approach and the narrow slice you're tasked with reviewing in the 50min you're given.
I've seen some of the best interviewers I know still fail at various companies. Including ones they were given offers to before. Luck might not be the #1 component but it sure feels like it's not a distant second.
By grinding leetcode, you raise the odds that you'll get a problem you've seen before, or at least similar enough that you recognize the way to solve it without having to waste time by starting from absolute scratch. I guess that's still luck, but this is one area you can increase the odds in your favor.
You can't change whether you get a hostile interviewer, etc. of course.
the shitty thing about interviewing is you have to prepare hard every time you approach it
I've also seen the opposite - right? People don't prepare anywhere near as much as the last time and still get offers. I've seen folks get offers at Google and spent maybe a few hours preparing. Luck is just a very wild component.
This does very unfairly skew towards people like me without kids that can take the time to do this.
I don't think regular coding questions, LC tell you anything meaningful in an interview.
I'm more experienced now, so refuse doing it and recruiters and interviewers get all funny about it. But please, please interview. I know I'm pretty privileged compared to others but I won't ever do a LC question anymore and tell that upfront to Google and the likes. I have canceled loops because of the inflexibilty of some companies in that regard.
Sometimes they tried sneaking it in last minute - also tells you much.
In general though the tech industry has a really hard time hiring senior people, perhaps for this reason. I was at Google for 10+ years and did 100+ interviews, HC etc and once you talked to the people in charge of the process they admitted that the process is really weak for hiring senior (L5+ in Google speak) candidates. They in many ways perform worse than new grads in coding interviews which is not really surprising.
So Google hardly hires anyone experience before you get to the director levels (hardly as a percentage of course).
The other well know secret is that if all of us would try to do an interview unprepared we would probably fail.
I would rather be with my girlfriend, play videogames after work hours, watch netflix, go see some friends.
I lost a team member to Amazon recently, he spent 6 months total (two 3 months stints) preparing for the interview. He bombed in the 1st try and passed in the second.
That's 6 months of your life studying after hours to get a job ... I really have better things to do!
He is there now and he says how his work is not as challenging as the one we were doing from a technical PoV and also how the culture is very toxic compared to ours.
Anyways... he is optimizing to retire early so props to him I guess.
But I'm terrible at it, and I don't think it's meaningful, so I don't do it. Admittedly, it costs me some opportunities, but mainly at companies I'm not sure I want to work for, anyway.
At this stage I'm an old fart with a long resume, and more often than not I get hired by people who already know my work. I guess if I was as bad as my whiteboarding performance makes me look then people who know my work wouldn't want to hire me.
Ironically the modern technical whiteboarding interview often does require the candidate to effectively act and put on a show,
Suppose programming ability is normally distributed with a mean of 100 and and SD of 15 (values cribbed from IQ). A team starts with 10 people drawn from that population.
More applicants are drawn from that population: if an applicant's ability is above the median of the current team, they are hired, replacing a randomly-selected member of the team; otherwise, we pass.
At the first timestep, half the applicants get hired right away. It takes a few hundred total failed interviews to make the 25th hire, and a few hundred thousand(!) for the 100th hire. These have very long tails too: the longest run (of 1000) took 800M total interviews to hire 100 people.
There's no way this rule is actually being applied in practice.
Typically, just one lowers in an a loop of 6 interviews is sufficient to reject. On the other hand, a loop with all "meets" isn't a sufficient either. It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers" the candidate won't even make it to the debrief or worse the loop will be cut-short.
The bar-raiser (it's their job) are expected to specifically look for candidates who raise the bar. Raising the bar == better than 50% of "employees" (not job seekers) at their level. The rationale, when the bar-raiser program was created, was that the collective competency in a corporation should increase as more employees are inducted.
Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser, conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.
Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"
That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through.
The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,).
And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.
I keep wondering how they hired this person - never in a million years would they have passed a regular hiring chain even with fizzbuzz type questions. Wonder if they had someone stand in for the interviews.
Netflix is well known for firing people that don't pull their weight with far more ease than most companies. But a friend who works there tells me there's still people there that perform poorly but somehow managed to get both hired and stay on.
Which itself is a Bar-lowerer.
And not actually lowering the bar during an interview / hiring committee meeting.
I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol.
Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.
as long as 9 other candidates are willing to run the LC gauntlet, the 10th doesn't stand out.
We just really want to hire a senior dev who can spit out highly optimized code that in the real-world would be handled by a stable open source library rather than people who have built stable systems.
I transferred teams within Amazon and then left for my current company. I do wonder if I'll ever be willing to put in the work to get a FAANG job again but at the moment I really don't care. I like where I am so I'll just cross that bridge when I get to it. If I want it badly enough at some point then I'll go through the steps. If I don't, I'll just find another job.
It isn’t about memorizing patterns, but about hopefully spotting the exact question.
A friend is currently interviewing for a FAANG and the recruiter flat out told him that his best chance of passing was to do a pile of previously seen questions for that FAANG and hopefully he gets it again in the interview.
If all the celestial bodies align and your luck is through the roof, you'll get exact problems you've solved before with the optimal solutions ingrained in your mind.
But second best is you'll get problems that are at least similar and you are familiar with the patterns, tricks, and gimmicks to go about tackling it, instead of having to try to think up a solution completely from scratch.
Horrible luck? It's a question you've seen and practiced but your brain freezes and you can't remember how you solved it. Or something you've seen and earmarked to practice, but couldn't get to it before the interview day. This has happened to me.
Same. I had an interview the day before I was scheduled to review binary trees. What was it on? Binary trees.
First week at the new school, I was taken aside and given a test for their gifted program. Exact same test, which ... I completely aced, because I'd figured out all the questions I hadn't known in November (from parents). It felt like cheating, and I told the teacher about it later. I don't know why they couldn't have just taken the test results from the previous school - same county, shouldn't have been an issue.
Anyway, if you keep studying known questions, it will increase your chances of passing in the future ;)
I imagine recruiters actively want you to succeed and get hired because that benefits them too, so they do what they can to aid you in your chances. They'll tell you the official, pre-canned lines about "we're interested in what your thought processes are and how you think", but then they'll also tell you the truth and recommend you take some time to grind leetcode.
Maybe I just don't have friends at the >= VP level?
Not a close friend, but do have a close associate who was VP level at Google. We were having coffee and he offered to refer me, but said I'd still have to go through the whole standard interview loop - even the phone screen in fact.
The downside is that he has to be very un-picky on what the role entails whether it be working with some super unsexy legacy technology (Excel VBA! Legacy ASP.NET webforms!) or doing devops or even helpdesk duty too. Hedge funds being hedge funds, small as they are, they all pay pretty well, though not at the level of a top tech company.
It's a combination of these places being pretty small shops and having friends there in high places. Including, from what I hear, a billionaire hedge fund exec or two.
But yeah. "Networking" by meeting groups of strangers for drinks probably wont get you a job. But making friends who happen to work in (entrepreneurial) tech might.
If a guy can do well that kind of well rounded project, even by cheating, I want him and his network all the same.
Bigger companies often don't let you through like that though. My friends are constantly offering to refer me to roles at their companies, but every one of them would still involve going through the whiteboard interview ritual. For now, I'm not taking them up on the offers because I'm not confident I'll pass the leetcode rounds.
Another time, apparently saying the phrase "binary tree" was enough to clear the whiteboard and move along to the next question with a gold star.
Another time I was hired after doing zero programming exercises and just telling the company that I had a competing offer (which I did).
I've had some baffling rejections as well.
EDIT: Ahh, didn't see the FAANG qualifier.
I did use "FAANG and other top tech companies" as an example, but I'm seeing this practice expand out towards many other companies both in and outside the tech industry.
I myself work at a non-tech company (bank) and even with my team/department, all a referral will do is just get you to the leetcode interview and maybe a slight advantage if the final decision is between the referred candidate vs. another non-referred candidate with all other factors being equal.
Algorithmic/data structure more theory-oriented problems are unrepresentative of "actual work" and are annoying, but what's more annoying is how the process is 1) opaque, and 2) repetitive. So you end up with candidates grinding through 80+ Leetcode problems to both master the skills necessary to ace these problems, as well as covering as many commonly-asked problems in a bid to game the system by preempting their interviewees questions. Goodhart's law abounds. This basically makes the process into an interview version of standardized testing. But- despite the standard prep process and a standard bank of questions, it can all easily fall apart because of subjective interviewer opinion, and so all the preparation is for naught. Candidates end up having to do the interview gauntlet for each company they apply at.
So standardize it. Put out clear, industry-standard rubrics for how one's performance is graded. Don't say "we only want to know how you think" and then ding candidates for not getting the right solution. You obviously care about the right answer as much as the thought process- don't be disingenuous about it. Maybe even standardize the level of difficulty of problems. And most of all:
Make this process a one-time thing.
Or rather, a once every five year thing. Outsource the ds/a interviewing section to a third party testing organization, like what Triplebyte is attempting to do, and have the actual interview be personalized to the company. Ask domain specific questions, relevant experience questions, system design, and culture fit questions during the company interview. And leave the DS/A portion be akin to the licensure tests that other engineering disciplines and STEM professions already have.
Software engineers may hate credentialism, but if it's a credential that you only need to take once or twice in a decade, then it's already far superior to the current system that forces candidates to undergo this each time they change jobs.
As it is, I'm basically studying for an unknown and nowhere near standard "certification", more or less, in the form of the technical interview, with only an inkling of what I might have to study for every single interview I go on.
I guarantee my career has been stunted because I delay my job search because I just don't want to go through that bullshit all over again, and stick with jobs long after I've stopped being passionate about them, despite there being a huge demand for software engineers.
Not everyone has to take the standardized exam, but if you do, you can show that to someone and they go "okay, I'm going to skip the technical pop quiz, let's move on to other things".
That's supposed to be what my degree was for, but apparently no one trusts that anymore, so why did I waste the time and money in the first place?
* Accounting has CPA exams/certifications
* Law has the bar exams/law school
* Medicine has the USMLE/medical school
Tech prides itself on the fact that "anyone" can be a great tech worker regardless of how you get there - read some books, attend a bootcamp, go to college, or even just start banging out code on the keyboard until you gain experience
Unfortunately, that means that the companies have to have some filter
Btw there is a Software Engineering PE exam but I don't know anyone who has taken it: https://ncees.org/wp-content/uploads/2015/07/SWE-Apr-2013.pd...
If you take and pass it you can add the title PE to your name in the US.
I think a better alternative is some type of bar-like or CA-like test that anyone can take regardless of qualification, but which you must pass in order to be considered a competent software engineer.
Yes tech interviews could be much better but other industries hiring process is just as if not more broken than ours.
I think it depends how influential your connection is.
At my previous FAANG job, my director was tapping his network to back-fill some positions and i heard through the grape vine that she waved parts of the interview in order expedite the process.
it could've been a rumor, but then again the person who joined was terrible to work with so i'm biased.
It's generally not even on purpose, just subconscience bias.
Im fairly confident that's at least part of the dominance of this approach in big tech. If the tech companies can't collude with no-hire agreements (as they have in the past) to prevent developers from jumping ship to seek raises (i.e. introducing direct anticompetitive barriers) they can collude on an unspoken agreement to raising the barrier to entry, making the labor market less mobile and therefore, artificially decreasing competition. It would be pretty difficult at this point to make a case that the current hiring trends are intentionally colluded anticompetitive labor market practices but the end result is the same.
I say part of the reason because the mess that is the current hiring process is really a multifaceted win approach for big tech and loss approach for most of the labor force.
I suspect if you had a large movement of mid-senior level developers that wanted to shift positions at the same time and began refusing businesses participating in this practice, the industry might be forced to make reasonable changes.
The issue is that our industry has no labor organization and the probability of a critical mass of labor independently making such a fundamental shift (especially in the current economic client) is near zero.
After all, there is a glut of applicants to the top companies, combined with the idea of wanting to avoid false positives (bad engineer putting on an act) at any cost, even if it means losing out on many false negatives (good engineers who interview poorly).
Same thing with other banks and hedge funds - 5 years ago I think the only major bank that leetcoded me was Goldman, and even that was leetcode easy level. Today, every bank leetcodes candidates AFAIK.
Pretty much every company, both tech and non-tech, I've interviewed with the past 3 years has done leetcode interviews at various levels of difficulty and rigor. Interestingly enough, recently FAANG and a few other top tech companies seem be creating frontend specific interview tracks that are arguably more practical tests of JavaScript knowledge rather than pure leetcode problems.
Getting good at interviewing should also be a priority, and is doable.
Getting good at interviewing is largely about developing (a) genuine programming skill; and b) at least some people/communication skill.
When you have those skills, you have less anxiety about the interview, and can perform well.
You're not wrong, but this falls under the genus of "companies that suck at interviewing," and thereby hurt themselves (i.e. the company). In other words, it's not an inherent fault of the programming interview.
But there is more to it than meets the eye.
Some companies actually just are that incompetent.
But, to take a particularly successful and egregious offender: is Google actually just that incompetent? So that they systematically do hiring in a poor way?
No, of course not.
They actually don't want the best people.
They want the mediocre people who are also obedient. The kind of people who can be told to study an algorithms book, and will do it, and will regurgitate it, as a show of obedience, mostly, but also mediocre (but not great) intelligence---not great, because if they had great intelligence, they wouldn't waste their time on that kind of activity.
Such people can be treated as interchangeable cogs, which is why BigCo wants to filter for them.
Perhaps a bit cynical but there is truth to that. A huge company just needs cogs that can do the job and don't think outside the box much.
He kinda blew the interview. He did okay, but not even close to the level of experience he had.
It was very obvious he was nervous, so we hired him anyway. No regrets.
The second guy was rejected but later hired as a contractor and he's worked out well as well and now we're looking at offering him a full-time position.
My conclusion is that our interview process isn't good at determining if a candidate would be a good employee.
My preference would be to hire someone as a contractor and give them a project then use that to determine their quality and fit. Unfortunately, HR tells me that's not feasible for reasons I don't understand.
You are asking them to do some work.
When I was contracting it was common to be offered a job at the end of the contract. I always turned it down because I liked being a contractor. From talking to my peers at the time this was common.
If someone didn't want to be a contractor then they wouldn't accept your offer in the first place.
In most of my programming jobs, explaining your work was most of the job and writing code was less important.
The whiteboard environment is a bit unnatural but I don’t want to hire someone who gets the right answer if they can’t explain it to me.
I’ll say for sure, not everyone interviews this way. A lot of devs running interviews think of it as a “test” with pass/fail. I’d say the better interviewers see it as an opportunity to dig around and figure out how to get evidence of the candidate’s skills. That means being creative, that means responding to what the candidate says and finding opportunities to focus on their strengths.
I don't think this necessarily means the "answer question alone and present answer after" approach is the right solution to interviewing, but it does show that interviewers might need to take some steps to make the process less opaque and daunting to lower candidates' potential anxiety.
From my direct knowledge of some “top firms”, if you only cared about correctness then you wouldn’t be able to fill out the interview feedback form.
Transparency may help, but in my experience, that’s what recruiters actually do and some of them are fairly good at it. The recruiter has a strong incentive to get you hired (they get bonuses for hiring people!) and will explain as much of the process as they can, if you are willing to listen.
That said, there seems to be a fair bit of turnover in tech recruiting.
And, as you said, every interviewer will be different. But I don’t see a way to fix that without eliminating technical interviews altogether. You are being judged, the interviewers are subjective, they do disagree about the rubric, and a bunch of money is on the line. It’s completely reasonable to be anxious. I don’t really see a way out.
Perhaps what we should be working towards is making it more transparent- it's still ubiquitous for candidates to be rejected without feedback- and more objective, and improving the technical interview process further. I'm not arguing for throwing out the baby with the bathwater.
Versus explaining your code to your coworker whom you've worked with for the past X time who understands some of your eccentricities or idiosyncrasies and without a pass/fail judgment lingering over your head like a sword of Damocles.
I say that as someone who cares a lot about writing easy to read code and has no problem explaining it. Also, I'm very bad at solving a problem while being required to think out loud.
Sometimes this explaining happens in textual format, like code reviews and design specifications, but often it happens verbally. Absent an explanation, it's hard to build consensus among the parties involved (co-workers, users, management) that your solution (as manifested by the resulting code) is the right one, and for anything non-trivial, you surely can't expect others to just assume that it is the right one.
There’s a simple strategy for this, which works fairly well in both interviews and meetings. You give people time to focus and think about the problem and solving it, and don’t force people into discussions right away. It’s not a panacea but in practice it is very effective.
The reason why this is important in meetings, as someone running a meeting, if you don’t give people time to think, it will always be the same one or two voices in the room. This is why brainstorming sessions suck. The same thing applies when you are choosing who will take responsibilities—the same few people always speak up first.
That's not what is being asked. It's explaining your thought process while actively doing the work in an "unnatural" environment.
> The whiteboard environment is a bit unnatural
I don't know anyone that codes on a whiteboard.
> but I don’t want to hire someone who gets the right answer if they can’t explain it to me.
Again, that's not what is being asked. They are being asked to explain their work before they get an answer.
In short, that’s a limitation of the study.
The study is comparing “explain your thought process aloud as you solve the problem” against “solve the problem by yourself and don’t explain anything”. There’s a whole range of sensible middle options, like “solve the problem by yourself and explain it afterwards” or “solve the problem with the interviewer and then explain it”.
One of the most enjoyable interviews ever with no whiteboard and no one around me to check on my progress. I was blank for first half an hour or so but made great progress by the end. Eventually got the offer.
> “Our study suggests that a lot of well-qualified job candidates are being eliminated because they’re not used to working on a whiteboard in front of an audience.”
This is a common daily task at any company I've ever written software at. People who can communicate their ideas effectively are better at their job.
(Post Covid, I do the same thing, but have to do it over chat)
IMO: "Do you have a clear understanding of your area of expertise, and can you communicate that understanding on to others?" is a far more useful predictor of success than "Can you find the right answer to the kind of problem that can be solved on a whiteboard in 45 minutes, or in 5 minutes by searching for an answer on stackoverflow?"
That said, I agree that generally interviews are terrible. But the (implicit) screening for soft skills isn't the terrible part.
I agree with the last point, but I don't think either of us can tie our anecdotes to it.
I'm saying: I find it odd that I rarely see these types of studies discussing the merits of that nuance. Especially when the idea of differentiating between soft and hard skills is not something that would be foreign to the average interviewer.
"Interviews are 100% for soft skills and technical evaluations are 100% for hard skills" seems to be a common assumption which I'd argue is false. Few tasks a developer does is purely a hard or a soft skill, there's always some overlap.
People have lives outside of a job interview. Family, children, jobs, or otherwise. All of those add to anxiety and can creep into your interviews.
They're not just robots that can turn off their emotions when they walk in the door.
This doesnt' happen to me after I grinded leetcode for a couple of months.
you are anxious if you feel unprepared.
And having skills retained under pressure is actually a valuable trait in this industry. And at least at desirable companies, the process is designed to minimize false positive, it doesn't care that much about false negatives. This is by design.
My guess would be that they filter out orders of magnitude more people with poor resume-writing skills or that didn't go to the right school.
Although I do agree that interviews that you can/must practice for are silly and there's a lot oof that around.
In reality human to human interactions, sometimes stressful, are part of the job. Customer & PM requirements, technical debates, giving feedback, explaining your technical solutions, etc. are common.
Maybe part of the problem is the education that does not prepare people to reality of the job.
Note: I do realize that SWE interviews suck, and I'm not trying to defend them. Just making comment on the study.
I want to assess if the candidate can communicate their thoughts effectively, if they can identify alternative approaches and justify their decisions, how they receive and incorporate feedback.
Locking someone in a room to write on a whiteboard is not a coding interview.
But if I'm interviewing you for a senior SQL engineer position, and you can't talk me through whiteboarding a simple query with one join and one aggregate function... you're not senior material. I'm sorry. We need people that can communicate and explain themselves, and talk through their own line of reasoning, where I work.
Again, I'm not defending SWE interviews... but of course people are better at expressing their skillset when they're in a controlled environment. The trouble is, the workplace isn't a controlled environment. Life isn't a controlled environment.
I don't want to work with brilliant programmers that have zero social skills. If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?
And I can ship products. I've done that a bunch of times--in several cases I've written most or all of the code in the shipping product.
I can't interview all that well, though, and I'm absolutely terrible at whiteboard coding, or pretty much any kind of problem-solving with people watching me. Except pair programming. Pair programming with a colleague that I know well is no problem.
I have two obstacles. One is anxiety. I score really high on trait neuroticism. It's one of the so-called "Big Five" personality traits. One of the things it means is that I have a lot of self doubt regardless of cause or circumstance, and a lot of anxiety to go with it. As for "working on myself a little bit," the Big Five are extremely resistant to change. Basically, the only things we know of that affect them much are psychedelic drugs and catastrophic trauma. Working on myself a little isn't going to reduce my trait neuroticism much.
I've been through quite a few tech interviews, including some familiar big names. I've gotten offers. But that's never been because of my performance in coding or solving brain teasers, at which my anxiety has been uniformly awful, and so has my performance.
The second obstacle I have is that I cannot combine certain classes of cognitive activity. For example, I cannot navigate and carry on a conversation at the same time. I can drive just fine--safely and competently. I just can't navigate if involved in a conversation, and will predictably get lost. It's predictable enough that when my daughter was a teenager she exploited it for laughs. "Let's see where we end up!"
Another thing I cannot do is solve logic and programming problems while being watched or while carrying on conversations with strangers. The interaction with the strangers forcibly occupies 100% of my attention.
I don't know if I'm representative of a vanishingly small fraction of the population. Maybe so. But maybe there are others more or less like me.
Doesn't mean you need to change your hiring preferences. Probably does mean you won't hire me. Oh well. I seem to be making do.
If there are others like me reading, and wondering what to do about it, one thing that has worked well for me is to do good work for people and cultivate good working relationships with them. I have more often than not been hired by someone who already knew my work, or on the recommendation of someone who knew my work.
From my experience companies are very flexible and open minded - mostly they care about just not hiring someone that they will have to fire later.
I was responding to another poster who said that "If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?"
I mean, maybe. Maybe sometimes. Then again, maybe not.
Maybe "working on themselves a little" isn't necessarily the answer. There isn't a known way to "work on myself" to affect my trait neuroticism in any significant way, for example.
Maybe it's not necessarily about "courage". I mean, courage is doing something that frightens you because it needs to be done. I know what that's like. Technical interviews don't even rate. But that doesn't solve the brainteaser on the whiteboard when a stranger's watching me.
So I wouldn't be too hard on myself, but also I wouldn't give up. From what I read, you might have already tried, so oh well, maybe you're just stuck with it.
I like "We work with what we have". That's something I deeply agree with.
If I had to land that customer for the business, I would surely botch it. That's why I didn't go into sales.
I think that the difference is you have much higher stakes in an interview. If you make a mistake with a co-worker or customer, you can usually just go back and fix it. If you make a mistake in your interview, it can cost you the job and it's not so easy to go back and fix.
This is the genesis of the infamous behavioral interviews where people ask about conflict resolution skills, how and when you'd escalate problems you can't solve, name a time when you had a problem and how you solved it, etc.
Sure - you can find yourself in very stressful situations at work, but I have never in my life been in a situation where you're given 45 mins (or hours, for that mater) to solve some problem, or you get fired.
Simply but, there's an incredible discrepancy between the stakes at play, between interviews and work.
To put it a bit more extreme - but to hammer down the point: Imagine working on some problem, with no help whatsoever, and that in front of you, there's some guy pointing a gun against you.
Just imagine how stressful that must be, and how much of your focus is distracted on reading that guy.
Same goes for interviews. For a lot of candidates, that interview is their way out of poverty or lower-class living. And the judge and executioner of your future, is sitting in the same room as you. It's a very stressful situation - probably one of the most stressful situations in your life.
Taking it all too personally is a big mistake. Sometimes even candidates that did really well don't get a job offer for weird reasons, etc.
It's very similar to dating. Desperation ruins everything.
I think that might be a reason why a lot of people report getting a job after they stopped carrying anymore.
In fact I usually try to shy away from coding a quiz that has 'a trick'. I usually want to see you do indeed have a decent grasp on the language, can properly decompose a problem, you can finish in front of me and quickly, and most importantly ask questions to clarify. I also try to stay away from things that require a particular framework. As those come and go and usually people can learn new ones fairly quickly with some good examples. I also ask if they have read any of the 'coding interviewing' books and what they thought. Usually if they can understand those books they are fine on complexity issues.
Having said that communication and interpersonal skills are big part of most jobs even in tech. We hardly develop software in isolation these days.
I don’t know about other industries, but in Tech there are some really inexperienced interviewers out there, who’re impulsive and lacks discipline in conducting objective and meaningful assessment.
Seeing how they deal with a stressful situation is an integral part of the interview in my opinion, and I imagine it is a feature for any IT role where that is the case. Standard dev roles, or other roles where time-based stresses aren't as prevalent, may be different.
Only yesterday I had a candidate bow out 10 minutes into the whiteboarding exercise, which is probably best for both the individual and the company if that kind of situation is untenable for them.
But the reality is that the situations are very different and elicit very different responses.
One major factor, with a job: you're still getting paid. I think the economic anxiety one faces is pretty stark, if you fail a job interview you're going to be unemployed until you can pass the next one.
It's extremely common for people to be nervous when meeting strangers but fine among associates. This is such a blatantly obvious empirical fact, it mystifies me how the tech industry can act as if this phenomenon does not or should not exist.
As soon as an issue is raised by our clients, they immediately kick off a 40 minute long timer and put a physical whiteboard in front of us asking us to solve the whiteboard problem in front of them in under an hour. All the while the client is physically behind us commenting on our code and watching our every move as we paint the whiteboard with a solution. This is literally what happens to me and my engineers on the Job. All of our code happens on the whiteboard (we don't use editors or keyboard or mice!.)
The only thing we need to deliver to our client is a picture of our code on the whiteboard along with some trivial complexity analysis and our client pays us money for coding under pressure.
Our interviews just consists of me doing the exact same thing as what happens on the job and therefore our interviews are 100% identical to our real life coding problems.
When science tells someone that the interview process they've been conducting for years is essentially wrong people have to sort of remake reality and their own reasoning to justify the science rather than admit fault. Obviously, this isn't what's happening with me or you in this case.... the interview process at my company is 100% identical to real world scenarios so I'm good.
I've always been telling everyone I know since before this article that I interview for anxiety not useless technical skills; anxiety is what determines whether a developer is great or not.
We ask them this because, literally on the job our employees are suppose to program robots that make manhole covers. Daily problems we face include deciding whether we should program the robots to make square manhole covers or round manhole covers. Really hard stuff... and you really need to be able to figure out why manhole covers are round or how many golf balls can fit in a bus in order to do the job... If you can't figure this stuff out you're not qualified to work for my company.
Not to mention, as I said before, none of our programming happens using computers. We do it all on a whiteboard in under an hour while the client is over our shoulder watching.
I can see 'high pressure' particularly in Devops, but that's a different kind of problem solving.
The headline speaks perfectly to my intuition after many years in the trade, and that we are 'measuring confidence'.
Note that they assumed that interviews were set up to be fair - they are not - and that's actually ok. If a company has preferred candidates, it's perfectly within their right to do that (so long as 'preferred' doesn't just mean 'gender' or whatever).
It's been a while since doing this for me, but when I do it again, the 'on site' will be a page with some neat problems, some Q&A and then they take basically as long as they want to go through them, the interviews start when they are ready and you just go over 'what they did' and then interview for the rest of the issues.
'Having someone watch you solve a problem' is very specific social condition.
Looking at white boarding comfort and judging that the person can't handle stressful situations when they have a defined role and understand the stack are two different things.
I feel like everyone wants to make a quick judgement and pat themselves on the back.
People can learn quickly but no one believes it.
Interviewers need to do their due diligence. Coding tests can have their place when interviewing for certain kinds of positions, especially for new grads and entry level positions. In the end nothing can replace checking references and looking into a person's work experience.
There is no algorithms and data structure whiteboarding test that can differentiate between a junior and a senior developer. Being senior is a product of acquired wisdom, not just skill. And in fact senior devs can easily bomb out a whiteboarding test because we're further from university CS experience.
But to moderate it a bit; in my interview training at Google it was clear: the interviewee doesn't have to get the answer correct. The point is to see how they think.
I mean, I think it's still insulting, but there is that.
My first "real" CS-related interview, which had technical questions, I had on the very same day that I had taken my final exam in our data structures and algorithms class. And because our DSA class was quite rigorous, I knew the material in and out.
I absolutely aced those questions, but did poorly on the systems design questions because, well, I simply did not have much experience designing or building complete systems/software.
Years later, it was the other way. I stumbled my way through the DSA questions, while acing the systems questions.
But with that said, I think also the order of your performance matters. If you walk in, and absolutely bomb the first topics, I think there's going to be a bias against you. Maybe they'll think you're a moron, or something like that.
I would love to see what a control group of non-tech interviews looks like. Not sure what the equivalent "whiteboard while out of the room" is, but it's hardly just software devs who get nervous and can panic in interview situations.
As a really quick reaction it's worth noting that coding is probably less than 25% of my job as an engineer.
In others' experience is this also the case for you?
That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense (useful when you have 2 minutes to prepare a 3-5 minute speech on a topic you've never seen before). So when I get up in front of a whitebard, I just start talking confidently, and even if by the end haven't solved the problem quickly or efficiently or at all, it sounds like I've been thinking sensibly about it.
My takeaway, I guess—parents, make your kids do debate!
You're effectively advocating for more bullshit. The problem is that this works in the first place - we shouldn't be rewarding it.
/s
I think they have some utility because this is a common occurrence when dealing with clients.
He's encouraging parents to make their kids learn this skillset. He is absolutely advocating for it.
Similar to another comment I read comparing interviews to athletic tryouts and competition, which tries to assess how a body moves through physical space.
[0] https://liberalarts.online/trivium-and-quadrivium/
I've always structured my interviews to weed that kind of thing out.
To summarize, I try and get the candidate to discuss tradeoffs between different approaches to common situations that the candidate should be familiar with.
For example:
Should GetUserById(int userId) return a null, or throw an exception, when there is no user with the ID? Explain the tradeoffs; or explain why each approach is better in different situations.
Should you deserialize (xml or JSON) to a dictionary or a strongly-typed object? What if you need to manipulate the document while preserving fields that you didn't know about at compile time?
Assuming you have the experience I'm screening for, you either know the tradeoffs or you don't.
> Explain the tradeoffs
will benefit greatly from being able to synthesize a bunch of information out of some simple base. Yea you need to know the technical details but being able to explain tradeoffs is more abstract and the OPs method will just lead to a better answer here.
The right thing to do is to narrate what you're thinking about. It's very different than starting a sentence without knowing what you're going to say.
But OP said he ended up somewhere sensible. He -was- thinking about the problem at hand. As he spoke. He got there in a roundabout fashion, perhaps, and had to think quickly, but it's still -valid-. He's not just bullshitting the interviewer (which is very noticeable and easy to suss out, and never ends anywhere sensible).
That sounds like a good skill to have in marketing, sales or politics. It sounds like a bad skill to have for engineering.
One of my very worst bosses had this quality. Perversely, it was about the only thing I admired about him.
The guy could walk into a meeting of NASA flight engineers and engage in conversation without uttering a single fact that would betray his total lack of any knowledge/qualifications.
n.b. He was fired not long after I left the company.
A downvote is warranted for giving a bad advice on gaming a system and giving both parties in an interview process a poor indication of what is expected in possibly years to come.
Thus, no vote given.
I think given what I read about debate teams -- on podcasts and here in HN, I am on the fence on suggesting that to my kids. There are certainly great benefits, but not too certain if those out weights the drawbacks.
On the other hand, it probably makes people better at spotting a bad/fallacious arguments, and gain appreciation that complex topics tend to have multiple reasonable opinions on it – two things we desperately need more of.
Interestingly, I was an extremely shy kid growing up—less so now.
Also, I agree that there are drawbacks to doing debate. For one, there's a certain "debate kid" mentality that I picked up and had to unlearn.
What kind of edits do you have in mind? I somewhat-deliberately kept my profile vague, but I'm definitely open to suggestions on how to improve it.
I spent 4 months doing leetcode problems and reading books before I started doing interviews again. I would interview at places like Facebook that I had no interest in working at but I needed the experience. 7 months after I started I finally got an offer.
I don't want to go through that again but I agree that you probably need to do interviews every year inorder to not completely lose that skill.