After running over a hundred interviews within the last few years, this doesn't surprise me at all. Interviewing is a skill separate from school, and if you train for it, you can become good at it, regardless of your pedigree.
I actually think studying for the SATs is a better comparison - it's a test that doesn't exactly translate to real-world performance, but has huge bearing on prospects, and often the best advice for acing both (tech interviews and the SATs) is to simply do as many practice tests/problems as possible in the areas you're weakest in.
I was always under the impression that SAT scores correlate strongly with IQ, not with practice. I had almost a perfect score[0] on my SATs, and I only used about half the time available, and I did zero with a capital Z practice problems beforehand -- many other kids took practice tests every weekend and still performed abominably.
[0]: I only got two questions wrong on the entire test, both math problems. Since I was practically unstoppable at math back then, I always suspected that there were errors in grading.
For the record, while I agree that evidence suggests that SAT correlates strongly with IQ, your comment is a great case study in how SAT scores correlate poorly with tact
For the record, I practiced math the most, writing second and literature last and scored exactly in that order. I also tracked my progress before and after practice and I got dramatically better. I really don’t think the SAT measures anything useful other than how well someone can solve SAT problems.
I don't see how it would be possible that SAT scores would not correlate with practice. The fact alone that SAT correlates with IQ is proof, because IQ test scores themselves can be improved by practicing.
A large portion of the SAT is knowledge of English vocabulary which obviously has nothing to do with IQ.
The mathematics questions nearly all use the same basic rules and are presented in the same format, and can easily be learned and trained.
SAT prep courses are ubiquitous and expensive because they work. Do people without training do well? Sure. Do people with training still do poorly? Sure. But a good SAT question would be "does this mean that training doesn't help or that there isn't a correlation between training and success?" because the obvious answer is no.
>The many different kinds of IQ tests include a wide variety of item content. Some test items are visual, while many are verbal. Test items vary from being based on abstract-reasoning problems to concentrating on arithmetic, vocabulary, or general knowledge.
That's odd, when I learned about it a long time ago I was told it was intended to measure "inherent" intelligence, not the level of exposure/memorization of a massive set of (likely trivial) data points.
Vocabulary very clearly belongs on an achievement test, not an aptitude test (as I imagined IQ to be).
I find training for interviews to be funny. Passing an interview is not correlated to doing well in a job. The purpose of an interview is to determine if you will improve the team and whatever the team is doing, not prove to be a whiteboard genius. If you design the interview to get whiteboard geniuses that's all you will hire.
On the other hand, they discount self selection bias, and the bias of "qualifying".
Plus the charts don't even show up, probably due to load.
edit: And the bias of the interviewers who use their platform, who may not be able to attract top tier talent, and are happy to get even barely competent people... It's impossible to say since the charts don't show up and we know nothing about the methodology and motivations of the different actors.
Graphs fixed. Just added static image fallback for when Plotly runs into issues. Sorry about that, and thanks for calling it out!
Re qualification bias, we call it out explicitly in the article. Without the coding assessment before the real interview, the result may very well have been different.
Lastly, re the bar, our interviewers are coming from Google, Facebook, Dropbox, AWS, and so on.
There is huge selection bias. The talent that already perform well in technical interviews most likely spend less time practicing (and using interviewing.io)
The charts were moderately cool when one of them loaded the first time I opened the page, but they're not really superior to static images, especially when they don't load at all. And when they are elements that your whole article hinges on, well...
> What this means is that top-tier students are achieving the same results as those in no-name schools
This is an misleading conclusion that ignores a HUGE selection bias. I doubt top MIT CS students, for example, would feel the need to practice coding interviews on interviewing.io
I went to MIT and knew a good number of people who were confident in technical interviews. The problems in interviews were much easier than what I got in class, or even CTY CS classes in middle school.
While there's definitely good people at non-elite universities, selection bias is at play here too.
For example, in the senior year case, you're just looking at people who were concerned enough about interviews to go to interviewing.io during their senior year. Most people I knew at MIT had a full time offer they were happy with from their Junior year internship by their senior year.
You can actually see the impact of this in your graphs too, notice that the % of people with a 1 or a 2 in elite schools increases for senior year vs junior year. People who have been doing well in technical interviews and have good return offers won't spend as much time preparing.
When interviewing in tech jobs I had :
- build an asynchronous communication queue from scratch (which implies building semaphores)
- optimize the drawing of a sphere on a screen
the rest was more regular (I didn't apply to super elite jobs, and both the companies with these two hard tests failed, so it's no indication neither about them, neither about me)
It lasted two hours. Now the way it worked was that I was doing on my own but I had to constantly explain to a "peer" my line of reasoning. So I was not alone in front of a white paper :-) This was an extremely interesting wy to interview. I felt I had the opportunity to demonstrate my skills on a difficult problem (which I didn't solve). I didn't remember the details of Dijkstra's algorithm but I had the opportunity to show that I can approach a complex problem in a logical and efficient way (well, that day, that kind of problem; don't ask me to work on helicoidal tomography :-))
I would not have noticed that one chart in each set of 4 has a different scale, so things that are the same height aren't actually the same number. It certainly looks nice and tidy but it is indeed misleading.
Indeed. Being an MIT dad, I know a few course 6 undergrads. Bright kids get social axiety, too. And can brain-freeze at inappropriate times. And may be better in a quiet, thoughtful problem-solving mode than the kind of TV quiz show that modern interviewing has become.
I just don't get the modern interview process. Why are we looking for lint in human form? I'd rather see someone sketch some psuedo-code for an interesting problem with several acceptable solutions.
I think I learn more by asking a fuzzy, underspecified problem question and seeing what clarifying questions I get back. I want to know if the person can wring a spec out of a customer - coding it up with all the semicolons in the right place is the easy part.
>> Most people I knew at MIT had a full time offer they were happy with from their Junior year internship by their senior year.
This leads me to think that top-universities student are already "chosen/watched" by companies before the recruitment phase so both companies and student are much more confident they can work together.
So basically, top companies just recruit people the best way : by looking at students while they are still student. Which is fine, except the top universities are, I guess, the most expensive ones. But if that is true too, then that's another bias : companies have rich employees, so they hire people with the same set of personal values...
Admittedly, this is a bit of an exaggerated conclusion. It is definitely possible that some subset of top CS MIT students would get anxious about coding interviews and would go to interviewing.io to hone their skills.
However, the following types of top students are significantly less likely to practice on interviewing.io, for example:
- Seniors who have already done internships at top companies (Google, FB, Stripe, etc)
- People who excelled at programming contests and generally enjoy algorithmic challenges: IOI, TopCoder, Project Euler, etc.
- People who have built awesome open-source projects that got significant recognition. For example, Feross Aboukhadijeh was a Stanford student who built Youtube Instant and immediately received an internship offer from Youtube founder Chad Hurley [1]. I doubt people like him would need to practice on interviewing.io
- In general, students who are regarded as top n% of MIT/Stanford/CMU CS are inundated with recruiting pitches (I speak from experience as a CS graduate from one of these universities. Most of us constantly received an overwhelming number of e-mails from recruiters, many of them offering to get us to get fast-tracked in the interview process).
Granted, not every top CS student at MIT/Stanford/CMU has done competitive programming or built really cool stuff that's widely recognized. But a significant percentage of the top n% of students do have this kind of background (admittedly, this is somewhat of a circular definition of "top" student). That's what I mean by selection bias.
So? Ignoring that this is a single datapoint, it could be that interviewing.io is relatively unheard of across the board (which would say nothing about the relative frequency of top-tier vs. not-top-tier students using it), or less heard of outside the US.
You bring up an interesting point about selection bias. Did you consider the other angle that top-tier students at no-name schools overrepresent the data from those schools? As in, low-teir students, regardless of school, are unlikely to practice.
I think this is a bit specious, at least from anecdotal evidence. I know bunches of people at my school (Top 50) that have interviewed at companies that ask leetcode-style questions yet very few get offers or even make the 2nd round or onsite. If you look at LinkedIn you'll see the vast majority work at Cisco, IBM, SAS, Amazon etc and not Google, Jane Street, or Facebook.
The quality of the cohort and, to a lesser extent, the quality of their DS&A classes matters quite a bit.
As someone that has gone through it, no. It is possible to get intern and new grad full time offers by passing one online debugging test and one coding test (in rare cases a single one!).
Also the comp is a league below the others I mentioned.
*with that said, Amazon does work on a lot of really revolutionary stuff that I definitely wouldn’t be able to work on in 90% of other companies. Really enjoyed my internship there.
Charts don’t have the same scale. As the result, it’s hard to visually compare these - for example, for juniors, the shapes are similar for score 4, but top tier school students are above 20 mark, while the rest are below.
Interviewing is expensive, especially if you hire the wrong person, and if a given population tends to cluster around higher scores, that what you’d pick first.
Maybe what they've proven is that technical interviews are not adding much information to the hiring process. This pretty much matches with my experience that quality of interview is at best only very loosely correlated with actual ability as a software engineer. Much more relevant is direct experience working with them. School quality is a strong signal.
I'm leaning towards blaming this on the interviewing process.
Last year I spent some time interviewing job candidates and what I learned from this is that the recruitment process in IT, in general, is broken.
A typical candidate is first faced with some kind of task to weed out the lower 20% of applicants. After that comes the proper interview during which the candidate is asked various questions like what was their major or how is the abbreviation "SOLID" expanded.
In my experience, none of this correlates highly with future performance and this may be the reason why regardless of the school they attended those students performed roughly the same.
Time and time again the one thing that seperates the best from the rest is their ability to perform code reviews. I have yet to find somebody who's a poor developer, but a great reviewer.
Print out a page of code and review it with a candidate. Have them explain what it does, or how they might go about figuring that out, and what kinds of improvements they might make. Include some glaring bugs or code style problems if you want to ask those sorts of questions.
I feel like small fragments of code work well for refactoring questions, whole programs tend to have a bunch of boiler plate code, but going over a bigger program to find the real core features quickly is also a valuable skill.
I've done this with SQL too. Printed out a real statement from an application and ask about performance, optimization, what the application might be doing, etc.
Or load a 100,000k line application in their favorite editor and ask "okay, try to find the frobzing subsystem, I want to know how it frobz bazzes". (I haven't done this yet...)
We have a sample repository with a simple, but broken, web application. We ask our candidates to review the code as they would if a coworker had submitted it as a merge request.
it gives quite a bit of insight about what aspects they latch onto as issues of note. Do they catch the insidious, subtle issues or just the low hanging fruit? Do they understand what stdlib function arguments expect as input? Do they just pass your review through a linter and call it a day?
I would extrapolate this out one layer higher and say that communication is the most valuable skill. This expresses itself both verbally and in written form.
That's a good way of looking at it. Code is also a form of communication. By writing code, you're communicating the the computer, with other developers who will come and read it, and with your future self who may not remember what you were doing.
People have forgotten how to train their core social and psychological skills like resilience and confidence. Unfortunately, these are not things we are born with or can code our way out of. These are things we have to train very hard at similar to how martial artists or athletes approach their training. The thing most programmers and knowledge workers in general are missing is true belief in their own skills (you have been doing this how long? come on snap out of it) and the resilience that comes from rejection after rejection, and failure after failure. After a certain point in your experience you should come to understand that raw skills are not really the end all, be all. If you are not the guy then you are not the guy no matter how many degrees you have. Sometimes the guy with the PhD from MIT is not the right guy because he doesn't connect with the rest of the team, or the vision, or whatever, simple as that.
Also, if you're testing only for specific skill sets rather than aptitude and interest then you already failed as a hiring manager.
People who bother to practice for coding interviews (using the same website) and successfully land an interview do similarly well on coding interviews... That's a lot of selection bias.
> What this means is that top-tier students are achieving the same results as those in no-name schools.
When NYU and Arizona State are "Top 50" while Michigan State and Vanderbilt are classified as "no-name schools", I question the meaningfulness of this blog's baseline and what it means for an interview to be technical.
I think the bucketing of schools by overall rank is about as good as arbitrary. Why not rank by strength of CS program? If we use PayScale's "Computer Science Majors by Salary Potential"[1] as an indicator for interview success, it looks very much like the CS rankings.
That's not very far off from general prestige. Harvard is too low, but only UCSD, UW, and maybe CMU wouldn't be on the list if it were that instead. And flipping it around missing from that list are University of Chicago, Johns Hopkins, and maybe Northwestern. (Also Caltech but it's tiny.)
For Phd programs there are some real surprises, but for undergraduates there's not much in the way of major specific strength differences. That's largely because the strength of an undergraduate program is mostly a function of the strength of the matriculants not faculty.
> Why not rank by strength of CS program? If we use PayScale's...
CS alone is far from representative of tech industry in the aggregate.
Furthermore, PayScale's list clearly doesn't control for locality bias--e.g. consider Purdue #191, UW Madison #138, UM Ann Arbor #88, UT Austin #79 to name a few; or what Duke #10 while UNC Chapel Hill #67 even means?? How do top programs which feed industry around tech hubs like Austin and Research Triangle meaningfully compare to the Bay Area and NYC if unadjusted gross salary serves as the sole basis of value?
Also, those numbers are absurdly low in my experience. It claims that Purdue's starting is $64,000, but I don't know a single person who's graduated and started at less than $75k, and I know plenty more that have started well into six figures.
What is used in divvying up a self-selecting cohort of undergrads performing on technical interviews in 2018 are the US News & World report rankings for graduate schools composed in 2014 "based on a survey of academics at peer institutions" (https://www.usnews.com/best-graduate-schools/top-science-sch... … and I don't think the lack of that link in the article is an accident).
Having a selective grad school doesn't mean much if anything for the standards and teaching in that school's undergrad program, especially for state schools with giant undergrad populations and relatively small grad programs.
For example, Illinois' grad school is selective and highly thought of by professors and thus it's treated as an "elite" school and students of UCSD's undergrad CS program are classified as "top 15".
Regardless, my alternate hypothesis would be "students of a specific level of confidence in their ability to pass a technical interview use interview.io for a limited period of time in which it provides value to them … that population of students receives a certain distribution of scores".
This really makes no sense. Even if every school had identical curriculum and teachers, students who manage to get into an MIT or Stanford have to have work ethic and prior experience that places them well above the average CS student. The fact that these students have already been vetted by a respected school is one reason why I think companies prioritize them.
I am all for big data analysis disproving common conceptions, but this feels off.
> get into an MIT or Stanford have to have work ethic and prior experience that places them well above the average CS student.
You're comparing apples to oranges. The worst at MIT is not necessarily any better than the average at some state school. I've seen more than a few people with bachelor's degrees from UCSD and UCLA be below the class average at Cal Poly for their second bachelors or masters degree. If there's any metric you can rely on it's that their average is usually better than the average of another school.
One data point: I was at Georgia Tech for undergrad, now at Caltech for a CS PhD. Both are good schools, but Caltech is definitely considered more "elite". As a Caltech student, I definitely attract attention that I just didn't get at a Georgia Tech student.
There is one problem with conclusions, they assume that interview score is the same as real world success of a candidate. It is not so, interview result correlates with future success but there is no strict determinism.
So companies prefer students that successful in interview AND come from good school, rather than just rely on interview results. It allows them to maximize probablilty of the finding a good employee.
We all know that interviews are terrible. Wouldn't that be the problem? Then students from top-tier universities achieving the same interview score as student from lower-tier universities is meaningless. The interviews fail to capture what matters.
The best jobs I had and also jobs I performed the best I got through credentials and experience, not scribbling on a whiteboard.
If you went to top 10 school in the world, you worked hard (at least it used to be this way). I don't need to look at specifics of what you did but I know you needed grit, good work-ethic, personal time sacrifice, etc to get through. Unless you're super smart. In either case it adds to your personal brand.
I think that's what perpetuates the system. Given unlimited time I'd be happy to give everyone a shot. But time is precious so stick to what you know? I know how things are in the Uni I went to. If a CV landed on my desk form someone who did the same course, I'd put them on top of my list.
This sounds reasonable enough, but on the other side, so is the observation that often people who get into and get through top 10 schools are actually ones that did not work hard.
The point of using interviewing.io online technical interview as the first step of the process is that it makes your "I'd be happy to give everyone a shot" take far less time than alternatives. Still can't get unlimited time or perfect fairness, but it's better.
I went to a "top 10 school." I'd say that only maybe 20% of the students (in CS) were people that I would want to hire if I were running a company. A lot of top school students get accepted because a) mom and dad are rich, sent kid to feeder high school, pushed kid to do ECs/do tutoring/get help with essays b) they fit into some admissions "bucket" like {has inspirational story} or {huge volunteering effort} or {also amazing artist}.
The worst students were always the ones that went to feeder schools and had wealthy parents, which are incidentally the majority of students at top schools
Exactly. Those a great examples, and you could add the ones of students who are not interested in being engineers. Of course, technical interviews and really any other recruiting process, are not perfect, but it gets one closer.
I wonder if your experience taints your opinion here. I don't know because I barely went to a state college for a minute before I got the job I wanted and dropped out. It's possible if you spent the same time at an average university first you might find 50 or 80% of the "top 10" students to be worth hiring.
Replacing tech interviews (even as quirky as they are) with credentials and school rating is even worse as it weeds out the wast majority potentially prospect candidates that had no opportunity to attend top universities. Of course, graduating top university shows that a person is capable, but it doesn't automatically means that (s)he's better everyone else who graduated lower ranking schools.
I started wondering why CS interviews aren't more based around reputation and experience, like other jobs.
I've seen a lot of bad devs get hired into places with supposedly high bars, or devs being let go and then ending up at Google next. While some places often let go of good devs, I'm talking of cases where I believe it was justified.
> Indeed, statistical significance testing revealed no difference between students of any tier when it came to interview performance.
Ah, the classic confusion between not finding enough evidence to prove there is a difference, and finding enough evidence to prove there is no difference.
The juniors from elite schools in particular have fewer 1s and 2s and more 3s and 4s than the other juniors. Really, you found statistical evidence that they aren't from different distributions? I'd love to see it.
I would love to see more recruiting at state schools and community colleges etc from tech companies. There are a lot of hard working people who could be great to work with but didn't have the family background they needed to be noticed.
99 comments
[ 3.1 ms ] story [ 186 ms ] threadI actually think studying for the SATs is a better comparison - it's a test that doesn't exactly translate to real-world performance, but has huge bearing on prospects, and often the best advice for acing both (tech interviews and the SATs) is to simply do as many practice tests/problems as possible in the areas you're weakest in.
[0]: I only got two questions wrong on the entire test, both math problems. Since I was practically unstoppable at math back then, I always suspected that there were errors in grading.
Can you practice to get good at quickly solving basic algebra problems? Yes. Which means that you can practice to get better at the SAT.
The mathematics questions nearly all use the same basic rules and are presented in the same format, and can easily be learned and trained.
SAT prep courses are ubiquitous and expensive because they work. Do people without training do well? Sure. Do people with training still do poorly? Sure. But a good SAT question would be "does this mean that training doesn't help or that there isn't a correlation between training and success?" because the obvious answer is no.
https://en.wikipedia.org/wiki/Intelligence_quotient
>The many different kinds of IQ tests include a wide variety of item content. Some test items are visual, while many are verbal. Test items vary from being based on abstract-reasoning problems to concentrating on arithmetic, vocabulary, or general knowledge.
Vocabulary very clearly belongs on an achievement test, not an aptitude test (as I imagined IQ to be).
Plus the charts don't even show up, probably due to load.
edit: And the bias of the interviewers who use their platform, who may not be able to attract top tier talent, and are happy to get even barely competent people... It's impossible to say since the charts don't show up and we know nothing about the methodology and motivations of the different actors.
Re qualification bias, we call it out explicitly in the article. Without the coding assessment before the real interview, the result may very well have been different.
Lastly, re the bar, our interviewers are coming from Google, Facebook, Dropbox, AWS, and so on.
This is an misleading conclusion that ignores a HUGE selection bias. I doubt top MIT CS students, for example, would feel the need to practice coding interviews on interviewing.io
While there's definitely good people at non-elite universities, selection bias is at play here too.
For example, in the senior year case, you're just looking at people who were concerned enough about interviews to go to interviewing.io during their senior year. Most people I knew at MIT had a full time offer they were happy with from their Junior year internship by their senior year.
You can actually see the impact of this in your graphs too, notice that the % of people with a 1 or a 2 in elite schools increases for senior year vs junior year. People who have been doing well in technical interviews and have good return offers won't spend as much time preparing.
The hell? What sort of problems are you getting?
Were you doing edit distance and word break and shortest palindrome in middle school?
* Print pascal's triangle up to the n-th level, with proper spacing (so that it looks like a triangle).
* Make the game Go for 2 players. I got a sub problem of this during my Google interview (determine whether a piece is captured).
the rest was more regular (I didn't apply to super elite jobs, and both the companies with these two hard tests failed, so it's no indication neither about them, neither about me)
How long were these interviews? Seems like something I could do in a few days not a 2-hour interview!
And, please, when showing graphs for comparison purposes: make the Y axis scale identical, or the results are quite visually misleading.
What brings you to this conclusion?
I'm not saying technical interviews are good or bad. I just think it's a different skill from what a student generally learns.
I just don't get the modern interview process. Why are we looking for lint in human form? I'd rather see someone sketch some psuedo-code for an interesting problem with several acceptable solutions.
I think I learn more by asking a fuzzy, underspecified problem question and seeing what clarifying questions I get back. I want to know if the person can wring a spec out of a customer - coding it up with all the semicolons in the right place is the easy part.
>> Most people I knew at MIT had a full time offer they were happy with from their Junior year internship by their senior year.
This leads me to think that top-universities student are already "chosen/watched" by companies before the recruitment phase so both companies and student are much more confident they can work together.
So basically, top companies just recruit people the best way : by looking at students while they are still student. Which is fine, except the top universities are, I guess, the most expensive ones. But if that is true too, then that's another bias : companies have rich employees, so they hire people with the same set of personal values...
Not necessarily, of course, but it'd be pretty surprising if it didn't trend in that direction.
However, the following types of top students are significantly less likely to practice on interviewing.io, for example:
- Seniors who have already done internships at top companies (Google, FB, Stripe, etc)
- People who excelled at programming contests and generally enjoy algorithmic challenges: IOI, TopCoder, Project Euler, etc.
- People who have built awesome open-source projects that got significant recognition. For example, Feross Aboukhadijeh was a Stanford student who built Youtube Instant and immediately received an internship offer from Youtube founder Chad Hurley [1]. I doubt people like him would need to practice on interviewing.io
- In general, students who are regarded as top n% of MIT/Stanford/CMU CS are inundated with recruiting pitches (I speak from experience as a CS graduate from one of these universities. Most of us constantly received an overwhelming number of e-mails from recruiters, many of them offering to get us to get fast-tracked in the interview process).
Granted, not every top CS student at MIT/Stanford/CMU has done competitive programming or built really cool stuff that's widely recognized. But a significant percentage of the top n% of students do have this kind of background (admittedly, this is somewhat of a circular definition of "top" student). That's what I mean by selection bias.
[1] https://feross.org/youtube-instant-media-frenzy/
If anything, those students would be the ones working the hardest to study for their tech interviews.
I am not from MIT, but instead a top university in China. I was accepted into Google last year, and I never even heard of interviewing.io before.
The quality of the cohort and, to a lesser extent, the quality of their DS&A classes matters quite a bit.
Also the comp is a league below the others I mentioned.
Interviewing is expensive, especially if you hire the wrong person, and if a given population tends to cluster around higher scores, that what you’d pick first.
The best programmer I ever met was a musician with no college.
Last year I spent some time interviewing job candidates and what I learned from this is that the recruitment process in IT, in general, is broken.
A typical candidate is first faced with some kind of task to weed out the lower 20% of applicants. After that comes the proper interview during which the candidate is asked various questions like what was their major or how is the abbreviation "SOLID" expanded.
In my experience, none of this correlates highly with future performance and this may be the reason why regardless of the school they attended those students performed roughly the same.
Time and time again the one thing that seperates the best from the rest is their ability to perform code reviews. I have yet to find somebody who's a poor developer, but a great reviewer.
Print out a page of code and review it with a candidate. Have them explain what it does, or how they might go about figuring that out, and what kinds of improvements they might make. Include some glaring bugs or code style problems if you want to ask those sorts of questions.
I feel like small fragments of code work well for refactoring questions, whole programs tend to have a bunch of boiler plate code, but going over a bigger program to find the real core features quickly is also a valuable skill.
I've done this with SQL too. Printed out a real statement from an application and ask about performance, optimization, what the application might be doing, etc.
Or load a 100,000k line application in their favorite editor and ask "okay, try to find the frobzing subsystem, I want to know how it frobz bazzes". (I haven't done this yet...)
it gives quite a bit of insight about what aspects they latch onto as issues of note. Do they catch the insidious, subtle issues or just the low hanging fruit? Do they understand what stdlib function arguments expect as input? Do they just pass your review through a linter and call it a day?
Real programmers read far, far more code than they write.
Reading code, and writing readable code, are perhaps the two most valuable skills in the profession.
Also, if you're testing only for specific skill sets rather than aptitude and interest then you already failed as a hiring manager.
When NYU and Arizona State are "Top 50" while Michigan State and Vanderbilt are classified as "no-name schools", I question the meaningfulness of this blog's baseline and what it means for an interview to be technical.
[1] https://www.payscale.com/college-salary-report/best-schools-...
For Phd programs there are some real surprises, but for undergraduates there's not much in the way of major specific strength differences. That's largely because the strength of an undergraduate program is mostly a function of the strength of the matriculants not faculty.
CS alone is far from representative of tech industry in the aggregate.
Furthermore, PayScale's list clearly doesn't control for locality bias--e.g. consider Purdue #191, UW Madison #138, UM Ann Arbor #88, UT Austin #79 to name a few; or what Duke #10 while UNC Chapel Hill #67 even means?? How do top programs which feed industry around tech hubs like Austin and Research Triangle meaningfully compare to the Bay Area and NYC if unadjusted gross salary serves as the sole basis of value?
What is used in divvying up a self-selecting cohort of undergrads performing on technical interviews in 2018 are the US News & World report rankings for graduate schools composed in 2014 "based on a survey of academics at peer institutions" (https://www.usnews.com/best-graduate-schools/top-science-sch... … and I don't think the lack of that link in the article is an accident).
Having a selective grad school doesn't mean much if anything for the standards and teaching in that school's undergrad program, especially for state schools with giant undergrad populations and relatively small grad programs.
For example, Illinois' grad school is selective and highly thought of by professors and thus it's treated as an "elite" school and students of UCSD's undergrad CS program are classified as "top 15".
Regardless, my alternate hypothesis would be "students of a specific level of confidence in their ability to pass a technical interview use interview.io for a limited period of time in which it provides value to them … that population of students receives a certain distribution of scores".
I am all for big data analysis disproving common conceptions, but this feels off.
You're comparing apples to oranges. The worst at MIT is not necessarily any better than the average at some state school. I've seen more than a few people with bachelor's degrees from UCSD and UCLA be below the class average at Cal Poly for their second bachelors or masters degree. If there's any metric you can rely on it's that their average is usually better than the average of another school.
I wish companies like this didn't exist to enable these types of interviews.
The best jobs I had and also jobs I performed the best I got through credentials and experience, not scribbling on a whiteboard.
If you went to top 10 school in the world, you worked hard (at least it used to be this way). I don't need to look at specifics of what you did but I know you needed grit, good work-ethic, personal time sacrifice, etc to get through. Unless you're super smart. In either case it adds to your personal brand.
I think that's what perpetuates the system. Given unlimited time I'd be happy to give everyone a shot. But time is precious so stick to what you know? I know how things are in the Uni I went to. If a CV landed on my desk form someone who did the same course, I'd put them on top of my list.
The point of using interviewing.io online technical interview as the first step of the process is that it makes your "I'd be happy to give everyone a shot" take far less time than alternatives. Still can't get unlimited time or perfect fairness, but it's better.
The worst students were always the ones that went to feeder schools and had wealthy parents, which are incidentally the majority of students at top schools
I've seen a lot of bad devs get hired into places with supposedly high bars, or devs being let go and then ending up at Google next. While some places often let go of good devs, I'm talking of cases where I believe it was justified.
Ah, the classic confusion between not finding enough evidence to prove there is a difference, and finding enough evidence to prove there is no difference.
The juniors from elite schools in particular have fewer 1s and 2s and more 3s and 4s than the other juniors. Really, you found statistical evidence that they aren't from different distributions? I'd love to see it.