Doubt it. Big companies use competition coding problems as a proxy for IQ, which is the strongest predictor of success at work and in life. Theres alot of data internally at FAANGs that the leetcode questions work.
Doubt it. Big companies using coding problems as a proxy for knowledge of coding, which turns out to not be a strong predictor of success at work and in life. There is a lot of data internally at FAANGs that being good at coding competitions correlates negatively with job performance.
The first two statements you make are incorrect. Well, I'll give you that you might find a big company who thinks of it this way (but they are confused).
There is a lot of data at FAANGS about hiring, but it will be structurally very hard to infer anything about broader activity from it, even if you had all the data.
Maybe it was a proxy for IQ back when leetcode type interview questions (and websites to help you prepare for them) were less popular. Now that they are so common, its just another specific skill that candidates can grind, so it measures preparation as much/more than IQ.
Having participated in competitive programming and comparing it to development work it feels to me comparing chess tactic puzzles to classical chess. If you're a good classical player you're probably reasonably good at puzzles, but the opposite is not necessarily true.
Competitive coding, despite superficially involving typing code into an editor, has almost nothing to do with working on large pieces of software. It's a lot of rote memorisation, learning algorithms, matching them onto very particular problems, and so on, it's more of a sport. Just like playing too much bullet chess can be bad for your classical chess I can honestly see how it gets into the way of collaborative work.
At lower levels like where I'm at, players are prone to mistakes and blunders, so having a good eye for tactics allows you to take advantage of those moments in the game as well as prevent yourself from getting into a bad situation.
But at elite levels, tactics have less importance (as he says in the video he estimates it drops to 50%) as every player at that level is extremely solid.
It's actually more subtle. Yes, chess is very tactical but the way you approach tactics in a puzzle is very different from how you mentally approach tactics in a game of chess.
If you already know that there is a tactic in the position your entire frame of reference changes. Which is actually why puzzle composition is treated very differently from actually playing, and a lot of famous composers are not particularly strong players.
This is why I feel it compares well to coding competitions. It looks so similar, but the mindset is very different. And only looking at tactics, just like only looking at coding as a game problem is I think why it may damage your performance at work.
The terminology associated with chess challenges [to use a neutral term] is unfortunate.
"Chess problem" is a term of art that refers to an artificial composed position with a unique solution that is constructed to both be a challenge to the solver and have aesthetic value. They often have constraints on the solution such as that White must deliver checkmate in two moves (three ply). This is what I assume you're referring to.
A position from an actual game (or that easily could have been) that demonstrates a tactic (or combination of them) is generally known as a "chess puzzle", largely because the term "chess problem" was already squatted on.
Somewhere in between the two is the "study", which is a constructed position, less artificial than a chess problem but still very carefully made to have a unique solution that walks a tightrope and generally requires absolutely exact calculation rather than working by general tactical principles.
I was very good at competitions, but terrible at rote memorization, including memorizing algorithms and matching them to particular problems. I'd just create the algorithms on the fly. E.g. I was presented with a maze solving problem, never had read about maze solving before, and just created my own version of it.
It's easy to make generalizations that minimize or downplay some of these things. But it's no more knowledge then the original study on too little data was.
Cool headline but not really data or a serious article. Obviously it’s a mix of factors, but some of the best I know are also very good at coding interviews, contests, etc.
Now imagine that the originator of the aforementioned "not really data" came to this very thread to admit that the proposed correlation disappeared once the dataset became more "huge" (c.f. Norvig's actual comment)
This is Berkson's Paradox. Even if coding competition performance correlates positively with job performance in the general population (which it certainly does, given that most people can't code), selecting for this attribute in the hiring process leads to a negative correlation among those hired.
Simple analogy. There is no correlation between height and salary across NBA players.[1]
The naive conclusion would be that height has nothing to do with basketball ability. The real answer is that markets are efficient and are already correcting one important feature against other predictors. Steph Curry wouldn't even be in the NBA if had the shooting ability of Gheorghe Mureșan.
Why doesn't the NBA just lower the height of baskets by a foot? They would get much better athletes, so surely the games would be more entertaining and the league would make more money, no?
Aren’t the advantages of height mostly relative to the opponent, ie when trying to block him or shoot over him? I think 7ft players are already too tall to be able to shoot from an optimum angle.
I don't think that this is the conclusion I would come to. Height is not something that can be changed, therefore it cannot be used as an adjustable variable to make that market efficient. You can't train to be taller like you can train at coding competitions.
I would say that height is an advantage up to a certain point in basketball, but tall people are not especially rare. Within the market of basketball players, you can find tall people who also have other skills, sometimes you find short people (Steph Curry) who have exceptional skills.
Also, Steph Curry is not really short, he just looks that way next to NBA centers and forwards, but he's actually 96th percentile in height amount American men.
A better example would be Muggsy Bogues who was a full 12" shorter than Steph Curry and he could dunk.
Muggsy re:the NBA has always stood out to me as an example of the failure of a supposedly efficient market due to a massive but unintuitive oversight. It would be one thing if he had just been a middling player, but even with the MASSIVE height disparity between him and the rest of the league, he proved to be a standout player, easily in the top 50% of players historically, and probably much higher. Clearly, there's a role for short men even at the upper echelons of the sport - not just as a curiosity, but as an effective value-add above an average replacement, in part because of his lack of stature. But you almost never see NBA players below 5'9". The players are tall. The coaches are tall. Surely being tall is generally necessary for success in the sport? But then, Muggsy.
Read between the lines. If all the players are tall, and all the coaches are tall, and the game has been played for more than a half century with that assumption... who knows how to train/coach a short player?
He also could jump almost a foot and a half higher off the ground than the average NBA player, who in turn can jump almost a foot higher than the average man of the same age.
What proportion of people out there can learn to jump so high, even with extensive training/practice?
Hard to know, considering that said extensive training/practice is not as well-known as other basketball-related training, and that the practice of fielding players who would benefit from it is discouraged.
That said, a quick search of "training to dunk 5'6"" on Youtube brings up a number of videos.
Steph Curry is 6'3", in the 98th percentile for height. Wouldn't quite call him short. And there are only ~2,800 7-footers in the world, many of which are in the NBA. So tall players - meaning over 7 feet - are extremely rare.
It should be mentioned that Steph Curry was drafted behind Hasheem Thabeet, Tyreke Evans, Ricky Rubio, and Jonny Flynn, among others.
Hiring is always a crapshoot. Pro sports teams spend a lot more time and money on talent evaluation than tech companies and still get it hilariously wrong all the time.
On the flip side, every NBA player who has won league MVP in the lottery era (>1985) was drafted in the first 15 picks. Steph Curry of course has won MVP twice and was drafted behind the 6 players you mention, but then he was drafted next out of every other eligible player. I'd argue that through this lens, NBA teams are really good at "hiring".
Some teams draft for current skill, others draft for absolutely maximum possible potential, others draft for some combination of both. Some teams are willing to risk a "bust" if there is the potential of ultra-elite league-best skill. And considering that no player who has reached the MVP level has fallen further than 15th, I'd say as a whole the NBA teams are doing very well.
> every NBA player who has won league MVP in the lottery era (>1985) was drafted in the first 15 picks.
I mentioned Steph Curry because the original commenter did, but in general it's very strange to focus on the MVP. That's a small sample and cherry-picking the results, only talking about the successes and overlooking all of the draft busts. There's only 1 MVP in the league every season, and some players have won it multiple times. It was won 5 times by Michael Jordan, who incidentally was drafted 3rd (behind Sam Bowie). Only 21 players have won NBA MVP during that period.
In any case, that MVP record doesn't hold in other sports. For example, NFL MVP Tom Brady was drafted behind 198 other players in 2000.
> NBA teams are really good at "hiring".
Some long-suffering Minnesota Timberwolves fans might say otherwise.
The more accurate conclusion is you get a vastly smaller population to choose from at extreme heights which largely offsets the slight height advantage, when looking at people who make the cut.
Like it efficiently chooses all hockeyplayers with birthdays in January to March? [0]
The real answer is people hire based on their biases and organisational restrictions more than they hire on objective metrics - and we have plenty of evidence for that.
Depending on how they define "winner at programming contests", this might narrow down the population to just a handful of "sport programmers". The same handful of guys win all the contests.
The statement might as well be "tourist has bad job performance". (https://en.wikipedia.org/wiki/Gennady_Korotkevich) And that isn't surprising given how much he has to train everyday to stay on top. He even turned down offers from Google/Facebook just to continue qualifying for the big annual competitions like Google Code Jam and Facebook Hacker Cup.
Just because I see some stronger-worded rebuttals in this thread, I want to point out that just because this is true (it is Berkson's Paradox), that does not mean it cannot be a valuable observation. As the author pointed out, for example, it might mean that this attribute is overweighted in hiring, which is something worth considering.
That write up is excellent and I found this paragraph particularly interesting.
> An interesting paper [1] claims a negative correlation between sales performance and management performance for sales people promoted into managers. The conclusion is that “firms prioritize current job performance in promotion decisions at the expense of other observable characteristics that better predict managerial performance”. While this paper isn't about hiring, it's the exact same theory here: the x-axis would be something like “expected future management ability” and the y-axis “sales performance”.
Can you define cheating? Does looking on Stack Overflow for solutions to real life job problems count as cheating?
(That was a partly rhetorical question. My point is that what is considered cheating in coding competitions is pretty much normal in real life jobs. Somebody who knows to look for solutions on Stack Overflow, I call them "resourceful", not cheater. Being resourceful is really important in real life jobs.)
In the linked video, Peter Norvig attaches a big caveat which is that this analysis was done on people who were hired and therefore met Google's various other hiring criteria. So the context isn't random people off the street, but rather that among people who are already (presumably) competent at what they do, winning coding competitions correlates inversely with job performance for some reason.
So I would even not just say "the context isn't random people off the street", but that "the context isn't random employed/employable programmers". It's people working at Google. Deducing anything from that with high certainty is hard. Not only it could be inversed in other companies, but even if not it is hard to apply statistics to individual candidates.
Every coding competition I’ve participated in required correctness as a first criteria for basic acceptance, and then speed as a secondary scoring criteria.
For Advent of Code, you lose a minute of time for every wrong answer.
Also in most how-to-leaderboard guides it's heavily noted that making mistakes (ie. bugs) is going to cost you the most times (anecdotally, I'd agree with this). Those who leaderboard have to write bug free code.
And just as a reference the difference between two performers is usually between 1 and 15s so one minute is a lot. On current days, if you are fast 1 minute may still get you leaderboard as the delta between number 1 and 100 is 10 min for day 1 it was 34s.
Given that companies regularly struggle to evaluate performance well or even accurately, an equally valid conclusion could be “Coding and playing politics to be well-ranked by peers are negatively correlated”. Now the question becomes is coding or playing politics what you value more organizationally.
I hate this kind of comment where people take synonyms and say they are the different things. Knowledge and wisdom are the same thing despite whatever bullshit definitions people apply to try to make them seem different.
Perhaps what you mean is knowledge in artificial crafted problems is not the same as knowledge in practical tasks that you would perform in a real job on a real world application.
The dictionary definition does not make any reference to this. At most you could say the difference is knowledge can be purely theoretical while wisdom is knowledge + practical experience.
But nothing about a coding challenge is purely theoretical. Its mostly experience with the specific set of problems that come up in these challenges which is a different set of problems that come up in a commercial situation.
They're not even close to being the same thing. The point here is that wisdom is more akin to judgment than mere possession of facts. For example, sufficient wisdom may give one the humility to reflect on the completeness of their understanding before submitting an inflammatory reply.
In my hiring for software engineers and architects I always consider coding/design skill to be merely a necessary condition - something you just briefly test for, mostly to detect b*llshit on the resume.
What I found is far more important are social skills.
Can the person work as member of a team? How do they respond to feedback? Or when something is hard? When they get stuck? How do they communicate a design? Rally a team around them? Deal with disputes? React to changing situations? Can they take the initiative or need to be told what to do? Etc, etc.
Together with actual coding/design skills - and with proper management - these are the necessary conditions. All my humble opinion, of course.
IMHO, part of the technical questions is to evaluate reactions. Perhaps I disagree with a solution. Or I do not give all the information needed to solve a problem so the person has to ask. Or I give hints on the way and see how receptive the person is. And so on.
After that it's just part of the conversation. Probing the resume. And asking some of these question flat out. I also discuss real practical problems/challenges we have at work, most of which require a team to work together.
We also (used to pre-covid) take the person for lunch and decidedly talk not about work at that time.
Of course it's different between an engineer fresh from college and a seasoned software engineer or architect.
In the end, I think interviewing is itself a skill one needs to train. Just having a person solve some brain-teasers doesn't cut it. I have _way_ more teams, co-founders, etc, fail due to social issue rather than technical skills.
And I am by no means presuming that I'm good at it, just that these are things I look for.
An important part I forgot to mention in the first post that the person is interviewing the company as much as we interview the person. So we leave a significant amount of time for asking questions about the company... and try to make a good impression.
There are actually a fair amount of scientific studies on good interview practices. The book "Becoming the Evidence-Based Manager" gives a nice summary of many of them.
I'll spoil a bit of the book though. Free form "coffee interviews" or lunch interviews are the worst possible types. They introduce a ton of bias into the process that's not related to job performance.
Dn't be sorry, I'm glad you explained that. I've been involved in some hiring and my workplace isn't equipped at all to deal with those questions properly...
I think you'll find a good sample of people who did poorly on their interviews but were still hired due to stellar references, corporate politics or nepotism.
Berkson's, right? A perfect interview process would result in a population where none of the people who are hired as a result of it have any attributes that could be correlated with job performance. i.e. all the information has been 'used up' by the selection filter. If you have a correlation, then you can improve the selection filter, so it can't be perfect.
This can have interesting outcomes. For instance, when Triplebyte published their blog post about which environments get the most hires⁰, it revealed the areas they haven't yet entirely accounted for in their quest to increase matching performance.
Hard to call that a "perfect interview process", because from individual candidate point of view, some with unusual characteristics could be unfairly disadvantaged (and other unfairly advantaged), while reaching your overall "neutral" distribution at the end. Short of being an omniscient interview process, I'm not sure this can absolutely be avoided, so in practice you have to be very careful with the kind of correlation presented here. Even if that ends up not being Berkson, but a property of all programmers.
I don't follow the reasoning. Even if you have a reliable prediction of performance, what prevents you from hiring some candidates who are exemplary and some who are just well-qualified? Or are we assuming that the best candidates would be given a higher-tier job for which they just meet the requirements?
It's a spherical cow conversation. The cows you hire can be placed in a smooth manifold of job difficulty and you have a predictable prediction of the job performance distribution.
Well yeah. Coding competitions are like an F1 race. High speed, high skill, high stress.
Building products and services is like driving an 18-wheeler cross country. Needs patience, dedication, long drawn effort, some degree of talent, team work, guidance, and what not.
Coding competitions are also like F1 races in that the problem space usually is very well-defined and very narrow. You'll only run in this track, you know pretty much exactly who your opponents are, you know exactly how much budget you have. The only things that are unpredictable are the weather on the day of the race, the accidents that may happen, a team member getting sick.
Building actual products and services is like driving an 18-wheeler in that the road is much more open and you don't know what other vehicles and/or reckless drivers you'll come across, the weather variation over a very long distance, traffic, road works and detours. The driver also needs to stay awake for much, much longer lengths of time.
Anecdotal, but I have noticed a similar correlation between achievements in Capture The Flag (CTF) competitions and performance in the cybersecurity industry. As some people have pointed out, this effect appears when observing just people within the field, not the general population.
Disclaimer: I don't have any data to back this up.
I've seen the opposite. Many of the very best security researchers are also top performers in CTFs (to give one example, Orange Tsai). It's hard for me to see how your correlation works unless "cybersecurity industry" is defined extremely narrowly.
Correlations don't have to "work" as they don't imply causality. Orange Tsai is indeed quite talented at both competitions and research (but it would prove as little as my own anecdotes).
If I'm biased, it might be because I defined the "cybersecurity industry" too broadly, not too narrowly: One can acquire certain skills from competing in CTFs/competitions, e.g. hard skills related to reverse engineering and vulnerability research... but I believe in most cases further skills are additionally needed to succeed in the industry, e.g. software engineering, and softer skills such as communication, planning and negotiation (useful for other jobs as well).
Overly optimizing skills to win CTFs while neglecting other matters can be harmful, like badly assigned character points in an RPG. :-)
Just like writing code on a whiteboard with people watching, it's almost like activities that have nothing to do with what you'd do on the actual job actually aren't a great predictor of job performance. haha.
Most coding competitions tend to assess a specific set of skills: puzzle solving ability; algorithmic knowledge; and being able to code fast. All of these skills are useful in "real life" programming.
However, since the code you write will be thrown away post-competition, your focus is on churning out solutions that "just work" — proper engineering practices and maintainability isn't relevant. All your code needs to do is to generate the correct outputs.
Does competing turn you into a strong coder? Absolutely. Does this equate to being a strong engineer? Nope. Software engineering isn't just about coding fast.
This is anecdotal, but from what I've seen (as a trainer and friend of several IOI medalists): some of them appreciate that coding != engineering and proceed to develop their engineering skills. Others don't and remain stuck at the "I'll come up with a fast solution" mindset.
Whether one or the other happens very much depends on the person, plus, I believe, whom they end up working with. After all, we've all heard about the "10x" programmer – and when your colleague or subordinate appears to code at 10x speed, you just might think twice about whether you're qualified to advise or guide them. That results in their keeping any bad habits they might have.
Voluntarily feature creep. Instead of writing simpler one-off projects, try to keep plugging at the same project for a longer time and expand the product in ways you couldn't have guessed at the beginning. You have to learn how to write maintainable code and plan ahead.
Coding is social, because most code needs to be maintained by more than one person (over it’s lifetime). And if the code is so brilliant that only few people can understand it, it becomes a maintenance headache over the longer haul.
The same is arguably true for many professions and walks of life.
If you can make your work serviceable by more people, it becomes less expensive to do so. And in many (not all) cases, that’s a superior life-time value of your work.
I can already see folks commenting out that this is the reason we should stop whiteboard interviews. But let's take a step back.
Whiteboard weeds out candidates that cannot code period. I wouldn't believe the legend of the "senior engineer" who couldn't FizzBuzz until I met him. I've personally conducted interview where a simply word count function (take a string, count the words) couldn't be implemented in 30 minutes. And the resume listed proficiency in several programming languages.
Now this study tells us that competitive programmers aren't that great among the candidates that were hired, not among the pool of candidates. That's very different.
I have very little frame of reference for other people's technical skills when they enter the profession since I'm entirely self-taught and still move dirt for a living. Thank you for sharing that story, it drove me to try my hand, it took less than 15 minutes.
134 comments
[ 2.8 ms ] story [ 203 ms ] threadThere is a lot of data at FAANGS about hiring, but it will be structurally very hard to infer anything about broader activity from it, even if you had all the data.
Competitive coding, despite superficially involving typing code into an editor, has almost nothing to do with working on large pieces of software. It's a lot of rote memorisation, learning algorithms, matching them onto very particular problems, and so on, it's more of a sport. Just like playing too much bullet chess can be bad for your classical chess I can honestly see how it gets into the way of collaborative work.
At lower levels like where I'm at, players are prone to mistakes and blunders, so having a good eye for tactics allows you to take advantage of those moments in the game as well as prevent yourself from getting into a bad situation.
But at elite levels, tactics have less importance (as he says in the video he estimates it drops to 50%) as every player at that level is extremely solid.
If you already know that there is a tactic in the position your entire frame of reference changes. Which is actually why puzzle composition is treated very differently from actually playing, and a lot of famous composers are not particularly strong players.
This is why I feel it compares well to coding competitions. It looks so similar, but the mindset is very different. And only looking at tactics, just like only looking at coding as a game problem is I think why it may damage your performance at work.
"Chess problem" is a term of art that refers to an artificial composed position with a unique solution that is constructed to both be a challenge to the solver and have aesthetic value. They often have constraints on the solution such as that White must deliver checkmate in two moves (three ply). This is what I assume you're referring to.
A position from an actual game (or that easily could have been) that demonstrates a tactic (or combination of them) is generally known as a "chess puzzle", largely because the term "chess problem" was already squatted on.
Somewhere in between the two is the "study", which is a constructed position, less artificial than a chess problem but still very carefully made to have a unique solution that walks a tightrope and generally requires absolutely exact calculation rather than working by general tactical principles.
From where people assert so confidently such nonsense? Chess is 90% tactics at the under 1800 Elo level or so. At the 2700+ level? No way.
It's easy to make generalizations that minimize or downplay some of these things. But it's no more knowledge then the original study on too little data was.
Alternatively, is it possible that this is an instance where your local experience doesn't generalize.
Great write-up by Erik Bernhardsson, CTO of Better, here: https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the....
The naive conclusion would be that height has nothing to do with basketball ability. The real answer is that markets are efficient and are already correcting one important feature against other predictors. Steph Curry wouldn't even be in the NBA if had the shooting ability of Gheorghe Mureșan.
[1] https://rpubs.com/msluggett/189114
I would say that height is an advantage up to a certain point in basketball, but tall people are not especially rare. Within the market of basketball players, you can find tall people who also have other skills, sometimes you find short people (Steph Curry) who have exceptional skills.
A better example would be Muggsy Bogues who was a full 12" shorter than Steph Curry and he could dunk.
Read between the lines. If all the players are tall, and all the coaches are tall, and the game has been played for more than a half century with that assumption... who knows how to train/coach a short player?
What proportion of people out there can learn to jump so high, even with extensive training/practice?
That said, a quick search of "training to dunk 5'6"" on Youtube brings up a number of videos.
Hiring is always a crapshoot. Pro sports teams spend a lot more time and money on talent evaluation than tech companies and still get it hilariously wrong all the time.
Some teams draft for current skill, others draft for absolutely maximum possible potential, others draft for some combination of both. Some teams are willing to risk a "bust" if there is the potential of ultra-elite league-best skill. And considering that no player who has reached the MVP level has fallen further than 15th, I'd say as a whole the NBA teams are doing very well.
I mentioned Steph Curry because the original commenter did, but in general it's very strange to focus on the MVP. That's a small sample and cherry-picking the results, only talking about the successes and overlooking all of the draft busts. There's only 1 MVP in the league every season, and some players have won it multiple times. It was won 5 times by Michael Jordan, who incidentally was drafted 3rd (behind Sam Bowie). Only 21 players have won NBA MVP during that period.
In any case, that MVP record doesn't hold in other sports. For example, NFL MVP Tom Brady was drafted behind 198 other players in 2000.
> NBA teams are really good at "hiring".
Some long-suffering Minnesota Timberwolves fans might say otherwise.
Ricky Rubio is famous for playing elite basketball since he was a scrawny 14 year old.
And Ricky Rubio has pretty much the same height as Steph Curry.
Like it efficiently chooses all hockeyplayers with birthdays in January to March? [0]
The real answer is people hire based on their biases and organisational restrictions more than they hire on objective metrics - and we have plenty of evidence for that.
0 - https://en.m.wikipedia.org/wiki/Relative_age_effect
The statement might as well be "tourist has bad job performance". (https://en.wikipedia.org/wiki/Gennady_Korotkevich) And that isn't surprising given how much he has to train everyday to stay on top. He even turned down offers from Google/Facebook just to continue qualifying for the big annual competitions like Google Code Jam and Facebook Hacker Cup.
For a more in-depth account on how the top people train, you can check out this guy's advice on how to get two gold medals in IOI: https://codeforces.com/blog/entry/69100 and his training schedule: https://codeforces.com/blog/entry/69100?#comment-535272
Or this guy, who won IOI this year: https://www.youtube.com/watch?v=V_Cc4Yk2xe4&feature=youtu.be...
Just because I see some stronger-worded rebuttals in this thread, I want to point out that just because this is true (it is Berkson's Paradox), that does not mean it cannot be a valuable observation. As the author pointed out, for example, it might mean that this attribute is overweighted in hiring, which is something worth considering.
> An interesting paper [1] claims a negative correlation between sales performance and management performance for sales people promoted into managers. The conclusion is that “firms prioritize current job performance in promotion decisions at the expense of other observable characteristics that better predict managerial performance”. While this paper isn't about hiring, it's the exact same theory here: the x-axis would be something like “expected future management ability” and the y-axis “sales performance”.
[1] https://www.nber.org/papers/w24343.pdf
(That was a partly rhetorical question. My point is that what is considered cheating in coding competitions is pretty much normal in real life jobs. Somebody who knows to look for solutions on Stack Overflow, I call them "resourceful", not cheater. Being resourceful is really important in real life jobs.)
The only benefit is you are better at interviewing and they are fun to do but the cons far outweigh the pros.
Every coding competition I’ve participated in required correctness as a first criteria for basic acceptance, and then speed as a secondary scoring criteria.
Perhaps what you mean is knowledge in artificial crafted problems is not the same as knowledge in practical tasks that you would perform in a real job on a real world application.
But nothing about a coding challenge is purely theoretical. Its mostly experience with the specific set of problems that come up in these challenges which is a different set of problems that come up in a commercial situation.
Sorry, just ribbing you a bit.
I believe the words do have subtle but distinct differences in meaning.
Agreed on coding challenges being a different set of problems then most commercial applications.
What I found is far more important are social skills. Can the person work as member of a team? How do they respond to feedback? Or when something is hard? When they get stuck? How do they communicate a design? Rally a team around them? Deal with disputes? React to changing situations? Can they take the initiative or need to be told what to do? Etc, etc.
Together with actual coding/design skills - and with proper management - these are the necessary conditions. All my humble opinion, of course.
After that it's just part of the conversation. Probing the resume. And asking some of these question flat out. I also discuss real practical problems/challenges we have at work, most of which require a team to work together.
We also (used to pre-covid) take the person for lunch and decidedly talk not about work at that time.
Of course it's different between an engineer fresh from college and a seasoned software engineer or architect.
In the end, I think interviewing is itself a skill one needs to train. Just having a person solve some brain-teasers doesn't cut it. I have _way_ more teams, co-founders, etc, fail due to social issue rather than technical skills.
And I am by no means presuming that I'm good at it, just that these are things I look for.
An important part I forgot to mention in the first post that the person is interviewing the company as much as we interview the person. So we leave a significant amount of time for asking questions about the company... and try to make a good impression.
(Sorry for the essay.)
I'll spoil a bit of the book though. Free form "coffee interviews" or lunch interviews are the worst possible types. They introduce a ton of bias into the process that's not related to job performance.
This can have interesting outcomes. For instance, when Triplebyte published their blog post about which environments get the most hires⁰, it revealed the areas they haven't yet entirely accounted for in their quest to increase matching performance.
0: https://triplebyte.com/blog/technical-interview-performance-...
Building products and services is like driving an 18-wheeler cross country. Needs patience, dedication, long drawn effort, some degree of talent, team work, guidance, and what not.
Coding competitions are also like F1 races in that the problem space usually is very well-defined and very narrow. You'll only run in this track, you know pretty much exactly who your opponents are, you know exactly how much budget you have. The only things that are unpredictable are the weather on the day of the race, the accidents that may happen, a team member getting sick.
Building actual products and services is like driving an 18-wheeler in that the road is much more open and you don't know what other vehicles and/or reckless drivers you'll come across, the weather variation over a very long distance, traffic, road works and detours. The driver also needs to stay awake for much, much longer lengths of time.
Disclaimer: I don't have any data to back this up.
If I'm biased, it might be because I defined the "cybersecurity industry" too broadly, not too narrowly: One can acquire certain skills from competing in CTFs/competitions, e.g. hard skills related to reverse engineering and vulnerability research... but I believe in most cases further skills are additionally needed to succeed in the industry, e.g. software engineering, and softer skills such as communication, planning and negotiation (useful for other jobs as well).
Overly optimizing skills to win CTFs while neglecting other matters can be harmful, like badly assigned character points in an RPG. :-)
Most coding competitions tend to assess a specific set of skills: puzzle solving ability; algorithmic knowledge; and being able to code fast. All of these skills are useful in "real life" programming.
However, since the code you write will be thrown away post-competition, your focus is on churning out solutions that "just work" — proper engineering practices and maintainability isn't relevant. All your code needs to do is to generate the correct outputs.
Does competing turn you into a strong coder? Absolutely. Does this equate to being a strong engineer? Nope. Software engineering isn't just about coding fast.
This is anecdotal, but from what I've seen (as a trainer and friend of several IOI medalists): some of them appreciate that coding != engineering and proceed to develop their engineering skills. Others don't and remain stuck at the "I'll come up with a fast solution" mindset.
Whether one or the other happens very much depends on the person, plus, I believe, whom they end up working with. After all, we've all heard about the "10x" programmer – and when your colleague or subordinate appears to code at 10x speed, you just might think twice about whether you're qualified to advise or guide them. That results in their keeping any bad habits they might have.
Is there a way to focus practice on that?
Program with people. Learn from them. Exchange ideas. Get code reviews and give them.
Look at program source code on github for inspiration.
And just keep writing programs.
The same is arguably true for many professions and walks of life.
If you can make your work serviceable by more people, it becomes less expensive to do so. And in many (not all) cases, that’s a superior life-time value of your work.
Whiteboard weeds out candidates that cannot code period. I wouldn't believe the legend of the "senior engineer" who couldn't FizzBuzz until I met him. I've personally conducted interview where a simply word count function (take a string, count the words) couldn't be implemented in 30 minutes. And the resume listed proficiency in several programming languages.
Now this study tells us that competitive programmers aren't that great among the candidates that were hired, not among the pool of candidates. That's very different.