Interesting and surprising, especially the experience thing. I think I am a significantly better engineer than earlier in my career, so I assumed experience would count for a fair bit. Then again I have inherited projects from experienced guys who make crap high level architecture decisions and the code is way more difficult to work with than it ought to be.
But then this article seems to be measuring interview performance, not actual ability on the job. So is any of it actually relevant at all?
Hey, author here. You're absolutely right. We hope that with time and data, the interview process can get better at predicting on-the-job performance -- one of the things we're really interested in is seeing how predictive different interview questions can be. To do that, we're also working on collecting data around what happens after interviewing.io users find jobs.
Most of the things that make us better during our career have nothing to do with programming faster: If anything, they can make us program slower, and do worse in interviews.
For instance, an interview I took last year came with a premise that I found downright bonkers. It was based on code the company had in production, but it stopped being a good problem to look at years before. I was having trouble coming up with the right tradeoffs for the implementation because all my experience was telling me that the entire approach was misguided in the first place, so the problem should not be solved. I passed, but it was a far rougher performance than I would have liked.
There's also how being far from college makes the least important knowledge gained fade away, and the least important thing I studied was memorizing algorithms. I write new algorithms at work sometimes, and I implement off the shelf ones too, but I don't have to recall them off the top of my head. Nobody has to implement distributed consensus algorithms under time pressure, or write HyperLogLog from memory.
So ultimately, there's what easy to measure, and then there's what is valuable and important. We go with what is easy, and those are things that are taught in college. Understanding the right level of testing or designing a system for observability are far more valuable in the long run: It's crazy how much downtime in well known companies comes from people not learning those things in college. But since we are bad at measuring those things, and kids right out of school don't know them, we don't interview for that.
And sadly this is why we all end up hiring by network so much: We can't tell if someone is good in a day, and we can't really ask people to dedicate two weeks to work with us in a probationary period if they have real jobs, but we sure can recall quality former coworkers and ask them to join in.
Most pedagogy, over time, gravitates toward what's easy to measure, not what's valuable to learn. So we've been very lucky in hiring by network even though we kinda feel terrible about it. On the other hand we're hiring more women now too as a result, not least because women are now much more conscientious about managing their network of other female developers than men.
I guess it might be because with experience your expectations also rise. You may be easily qualified for starting programming position somewhere, but now, when you're more experienced, you'll aim higher, right?
I am conducting interviews at the moment. I would say experience gives you a much better chance of getting to interview but once you are there other factors can be more important, like motivation and communication abilities.
If someone works at a firm with bad processes / poor or no code review, this can actually make their experience a detriment as they continue to use and reinforce bad habits.
It's rather shocking how much effect Udacity/Coursera had on interview performance - more than graduating from a top school or being employed at a top company:
"...only 3 attributes emerged as statistically significant: top school, top company, and classes on Udacity/Coursera."
Interviewing is a specific skill in itself that most people have to develop independently, so studying other things independently should be a strong correlation.
It's not just 'impact' it's 'correlation' (or both).
Universities don't teach programming, so it's not surprising you don't get applied skills out of the box with a degree.
'Good courses' are focused directly on teaching specific programming skills.
I think they are a good idea because learning programming by 'osmosis' (as you go along) can result in 'not knowing' a lot of key things, even when they are right in front of your face the whole time.
Most importantly - people who are going to take the time to specifically learn new skills, are displaying the kind of conscientiousness that you want in your talent - learning the actual skills is another benefit of that.
It would be interesting to see if all 3 predictors of performance held up in a simultaneous regression. Most likely, the predictors are highly intercorrelated. If so, then only one or none would be significantly related to interview performance in a multiple regression analysis.
> It's rather shocking how much effect Udacity/Coursera had on interview performance
Having done Udacity and also taking a couple CS classes at Cornell, this doesn't surprise me at all. People who take Udacity classes are doing so voluntarily on their own time, so they are going to generally be smarter and of higher socioeconomic status.
If you look at people who take CS classes over the summer at college when they don't have to be and when there are no student loans, you're going to get a similar population.
I don't buy the high socioeconomic status bit. MOOCs are hugely popular in countries like India where education is extremely competitive. I help run an international science camp for late-teenage kids. More and more we're seeing applicants who list the MOOCs that they've taken, and most often they're not from 'developed' countries.
If anything the opposite is true. MOOCs have enabled vast swathes of economically challenged people to learn from high quality video material, whereas before they were limited to textbooks.
The key is that these kids are absolutely determined. And similarly if you really want that job at one of the Big Four, you might also consider soaking up as much prep material as you could.
Some schools, and now some bootcamps and online courses, include a separate unit on technical interviews, covering common types of problems and how to approach them and come off well.
So this is unsurprising. The thing people should really be thinking about is why, "how to pass a tech interview" is considered a separate and unrelated skill from "how to code", given how many people claim their interview processes are supposed to separate people who can code from people who can't.
(spoiler: it's because those interview processes don't accomplish their stated goal)
"I need to pass an interview where I'll be asked about specific technologies/algorithms/etc. I will take a free online course to memorize/rehearse that information before my interviews."
I am perplexed why anyone would think that interview performances has any interesting statistical relevance. Much more interesting would be how successful the candidate was after receiving a job at the company.
To get a job at a company you have to be given the opportunity to interview, and to succeed at the interview. If you only evaluate performance at the job level, you don't evaluate performance of anybody who failed the interview. This is one of the fundamental challenges in hiring - you have a biased sample when trying to determine how effective the hiring process is at screening candidates.
It is weird that so much effort is spent on interviewing and so much effort is spent on performance reviews. But AFAIK, hardly any organization tries to use performance evaluation of current employees to inform their hiring decisions.
Most hiring managers would hope that interview performance is at least partially correlated to subsequent job performance. And once experience is gained as an interviewer and team manager, it is not an unreasonable hope.
The problem is that experience isn't very useful without feedback. And managers and interviewers get so little feedback about how well they do as interviewers I would imagine they don't improve much.
1. From the perspective of a job seeker, the interview is what gets you the job, so it's in your interests as a job seeker to learn how to look better as a candidate based on data about how interviewers judge candidates.
2. The purpose of an interview is to be a predictor for job performance, so while this article doesn't address this part of the question, you want interviews to predict job performance, and how to make interviews do that is interesting.
Interviewing skill is one way you get a job. After you've been doing this for long enough, and have a reputation, you tend to just get jobs by referrals.
I decided in my last change of positions to try interviewing for jobs like most people without referrals, since I had several other referral positions open.
And I was horrible. I was ridiculously bad, and couldn't get a job at jobs that paid less, and were for less skill. It was quite hilarious.
Thankfully, all of the referral jobs were begging for me, and that was fine. There was no real loss, but it was a bit concerning that if I didn't have an extensive connection network that I would be in a significantly worse role.
Which parts of the interviews were you really bad at? How would you change the interviews you had so they don't reject qualified candidates?
I've been interviewing a lot of developers lately, and while I'm fairly happy with my process (I've managed to avoid bad hires so far), I always have to wonder if I'm turning down people I shouldn't.
"I hate anything that asks me to design on the spot. That's asking to demonstrate a skill rarely required on the job in a high-stress environment, where it is difficult for a candidate to accurately prove their abilities. I think it's fundamentally an unfair thing to request of a candidate." Scott Meyers
The technical question part. Because I work on so many different things at the edge of innovation, I have a wide array of experience for a short period of time, and then I move on.
When I interview for jobs, I open up to any of the possible things in my background, of which it will take me perhaps 2 weeks to get back up to speed. But I am not going to take that 2 weeks for each job I interview for.
I have trained my brain to learn things, and discard them keeping only the current tasks in memory. The rest I know how to obtain again.
This means that I get asked questions I know that I've done repeatedly in the past, but can only vaguely explain without a refresher.
To the interviewer, this means I am underqualified at best, and at worst, lying about my past skills.
A lovely example of this during the process was that I was asked a fairly basic SQL question that I failed to explain when I had just landed from my second flight of the day. I didn't get the presales role for working with the database. (non referral job.)
However, I did get offered a job working on development for that database, as well as product management for the database.
So I couldn't get a presales role, that I've done in my past, because I couldn't answer a fairly basic sql question. I knew exactly how to answer it, the venn diagram of sql joins, a half second google, but in an interview, I blanked on the answer.
That is indicative of my experience with non-referral positions. I've given training on sql joins a good 30 times in my life, and couldn't explain the join in an interview.
Just the way it goes for me, thankfully it's not actually impactful in my life.
The conclusion is a bit obvious: if you ask people algorithms questions, the people who do the best will be people who spent a lot of time rote learning algorithms material. This means schools which place a lot of emphasis on the theoretical core or people who've taken online algorithms courses.
I'd be much more interested to see performance on other questions e.g. Google's typical curveball for new grads is an architecture question like "How would you implement YouTube?".
These are not curveballs though. Nowadays it's just another category of interview questions - System Design. There are tons of resources to prepare for this type of questions just like with the Algorithms & Data Structures problems.
" interview performances has any interesting statistical relevance"
I'm perplexed as to how people don't think it has any relevance at all.
The entirety of every industry is based on 'interview performance' as being one fairly important factor of future success.
Interviews are obviously not perfect predictors, but they are pretty strongly correlated with outcomes.
Someone who does very well in a technical interview is more likely to be a 'good engineer' than someone who fails miserably - though I totally accept that is not always the case.
There's no doubt we could use a little more science to our approach, however.
I've honestly stopped working for places that interview based on whiteboards and brain teasers. I think it leads to a rather arbitrary selection of candidates and pretty terrible teams.
Very unsurprising for me. You are measuring your ability to solve algorithm puzzles. Most engineers don't actually do many algorithm puzzles in day-to-day work, especially the types of algorithms that interviews tend to focus on like sorting and dynamic programming. So "years of experience" is not measuring experience in what you're actually being tested on. On the other hand, you do exactly those types of things in many CS classes, and in Coursera classes, algorithms are exactly what you practice. So it makes sense it correlates.
Top company is a predictor for the obvious reason - it's selection bias for people who already passed those interviews at the company. You're not good at the interview because you worked at the company, you work for the company because you're good at the interview.
Master's degrees seem like largely for international students needing visas, career switchers, etc so not surprised they are not a strong predictor. And if anything the course material moves past the intro data structures stuff the whiteboard interviews tend to test.
The only huge surprise for me here is that Coursera is a stronger predictor than top company and top school. I would have predicted top company > top school > Coursera.
The post that I would be much more interested in is correlating performance reviews to interview performance. That gets suggested as a possible future post.
Coursera might top them because it's a good indicator that they're actively trying to round out what they know, which means they likely studied up on algorithms right before the interview.
Those riddles are second semester problems on CS, thus anybody that isn't an student spent at least the last 4 years not working on them. The same probably applies to top companies, people probably interviewed there more than 2 years ago.
But Coursera stuff that gets into a CV is almost certainly recent.
Totally agree with this. I'm reminded of when I interviewed with Amazon a few years back and it was clear that the questions were best suited to filtering down to the best graduates. In my mid 30's I was too rusty to satisfy the interviewer and my real world experience counted for nothing as the entire process was structured like a series of exam questions.
I think there's a point - I would guess at about 8+ years of experience - where a technical interview will stop being a reliable indicator of an engineer's ability. It then becomes more important to evaluate what the candidate has been doing in their career and how that experience can be taken advantage of within the team that they'd be joining. Technical ability can still be judged by asking to see some existing code if it's that essential.
Interesting bit on the MS degree. I followed the link, and I'm not quite as surprised that the correlation is poor, or even negative, given the way the data was collected and analyzed.
Absolutely agree that some MS degrees are pretty much less rigorous cash cows by now, that allow students to skip the fundamentals such as data structures, operating systems, and compilers.
However, many CS MS degrees actually do require this as a background, to the point where some programs have emerged to prepare non-CS majors for MS degrees, kind of like those post-bac premed programs. It's hard to believe that those MS degrees, which require a decent GPA in those core courses, along with high GRE scores (sorry, but we are talking about interviewing skill, which may be more related to exam taking ability than job performance), wouldn't result in a similar profile to people with CS degrees from top schools.
This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading.
That's an aside, though. The value may very well be in the prep for these degrees (ie., the post-bac CS coursework required for admissions to a reputable MS program). If you can get that through online courses (udacity or coursera) through genuinely rigorous self-study? Yeah, that might do it, for far less money. I've audited a few of them, and they're the real deal, that's the real coursework there.
I'm curious what would happen if they looked at, as separate categories, people who mastered out of a doctoral program, people who did the 5-year BS/MS, and people who obtained a doctoral degree as well as a master's.
Agreed completely. I would suspect that BS/MS and Ph.D. interviewees would be very distinct from a straight MS, but this was a missed opportunity to look at that, even if it is just interview performance.
That could be useful. The article mentioned rhat there was no distinction made between CS masters degrees and Info Systems degrees.
My guess is that pure MS would correlate positively from CS programs that require a background in more advanced math and algorithms. Info sys degrees typically don't.
I'm not knocking those degrees, keep in mind we're measuring performance on a technical interview, which often leans very heavily toward algorithms, trees, sometimes graphs, and occasionally numerical analysis or computing.
If the MS degree data includes a lot of programs that dont include this background, then this woukd not be a surprising result.
Nothing! The author explicitely states that the data and analysis was not granular enough tp make distinction between the different kinds of MS degrees.
This leaves open the possibility rhat some categories of MS degrees might correlate positively, even though MS degrees overall correlate negatively.
> Absolutely agree that some MS degrees are pretty much less rigorous cash cows by now, that allow students to skip the fundamentals such as data structures, operating systems, and compilers.
At what point do we not consider operating systems and compilers "fundamental"? What percentage of CS/programming jobs require deep knowledge in these arenas?
Absolutely I think they still should be considered fundamental.
By definition, these are the subjects that allow you to keep going when your abstractions leak[0].
To say a job "requires" them is pedantic, but what's the point of calling something a CS degree if it is not at least signaling an understanding of these fundamentals.
It should be optional or advanced level. If you are a web developer for example OS and compilers are totally not important to get along and have a healthy career.
I disagree, compilers are relevant to web development because of all the compiling and transpiling that a single source file goes through in large application development.
If you are interested in _writing_ compilers then sure. If I just want to use those compilers, then I don't really need to understand them just use them as specified.
Now you'll say that I'll make a mess if I don't know what the compiler does. That's what best practices are for if say I pick TypeScript. Also, optimization is the compilers responsibility.
from Latin fundamentum "a foundation, ground-work; support; beginning." Whether these subjects are fundamental is totally detached from whether they are useful in a certain percentage of industry jobs.
I agree that knowing fundamentals is not that essential for a large number of programming jobs. They're still fundamentals though.
I did general engineering in undergrad, and software engineering (not CS) for my master's, so I'm somewhat of an outsider to a "computer science" degree. But if you have 6 whole years to turn someone into a "computer scientist," I would expect that, in ADDITION to compilers and operating systems, they'd also have a healthy dose of physics, chemistry, and especially math. I mean, for crying out loud, the degree has the word "science" in it.
My freshman year physics class had us building computer models of the solar system from Kepler's laws, constructing vocoders by manipulating raw sine waves, and discovering the equations that governed op-amp circuits through experimentation and a little deep thinking.
At my last job, one of my first tasks was to take raw data from truck engines computers (a data point ever 300ms) and turn point measurements like "instantaneous velocity" into data like "how far as the truck traveled over the last week"? Trivial. By combining that with engine speed and fuel usage, you could also say things like "what gear is the truck in?" "Has the engine performance declined recently?" "What percentage of the time has the driver idled in the last month?" And roll all that data up into an efficient database, build a super-flexible API around it, and create a series of web apps to display the useful stuff that managers wanted.
Having a deep understanding of the physical relationships between measurements, understanding what the data can and can't tell you, being able to spot potential problems in a proposed project (for instance, something that relies on an impossibly accurate GPS in a subtle way) -- these are all things that computer scientists absolutely should be able to do.
Yes, they should understand their computers from the electrons on up -- operating system included -- but they should understand the world that the computers interact with, as well.
That's a really complicated topic, of course. CS is in an interesting spot, in terms of how people perceive it and what students expect from it.
On one extreme, people see CS as a kind of trade school housed in a research university, teaching only what is useful on the job. Some people see CS as professional degree, kind of like law - you must teach theory for students to understand the field, but in the end, you're producing practicing lawyers, not abstract thinkers. And then, at the other end of the distribution, you find people who see CS as an academic field, a branch of scholarship, where the question "what percentage of programmers need to understand compilers" would be akin to asking "what percentage of math majors who become actuaries use real analysis?".
I got a CS masters and even though it was a worthwhile experience to me but I never expected to necessarily make more or be chosen over another candidate purely based on education (assuming all other things being equal.
I felt like my knowledge definitely cured during the process with graduate level data structures, architecture, operating systems and networking. The 2nd pass on some of these areas (despite getting mostly A's from a generally-considered challenging but now well known program). I took the opportunity to craft my program of study more than I would have been able to as an undergrad (both due to lack of knowledge and how undergraduates are given a lot less leeway) to include electronics and automation courses. It also gave me the opportunity to work with and mentor other students which was rewarding and I think has benefited me in and out of the workplace.
I wouldn't argue for doing it purely for career reasons and its not something to do just because you've been in school so long you don't know how to do anything else, but I found it worthwhile and you just need to consider what you want out of it.
In my experience almost all candidates applying recently with an MS in CS have been H1B visa seekers. Presumably, many of them obtained the MS specifically because it increases the odds of winning the H1B lottery.
The Bay Area had several shady MSCS-granting institutions. Some of them have gotten in trouble with ICE.
Even at programs run by major public research universities, the same economic incentives apply.
It's always hard to tell the value-add of a degree apart from selection bias. This case is no different.
Um, did you actually look at the article? There was no data shown between interview performance and MS degree status. The graph you saw was interview performance vs. years of experience. This isn't surprising since a person with 20 years experience will generally interview for a much more challenging job than someone with 0 years.
Yes, I read the article. Under the section "Master’s degree & years of experience", the author writes:
"No surprises here. I’ve ranted quite a bit about the disutility of master’s degrees, so I won’t belabor the point."
And provides a link to another post the author has written. Look for ("disutility of masters degrees"). That page will contain more of the content we are discussing.
This is what I was referring to when I wrote:
"This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading."
I would agree that no data is provided here, the author simply states MS degrees are an indicator of poor technical performance "in my experience" and goes on to list reasons why this might be the case.
I'm sure there's a reasonable case to be made that an MS in CS isn't worth the time, effort, and money, even from a top school. But I don't think that grouping these degrees with MS degrees in Info Sys (among others) and then reporting experiences without that distinction provides much useful insight into the topic.
Um, did you actually look at the article? There was no data shown between interview performance and MS degree status. The graph you saw was interview performance vs. years of experience. This isn't surprising since a person with 20 years experience will generally interview for a much more challenging job than someone with 0 years.
Something I wonder is how the participants in these interviews were selected from the general population of job candidates. Painting with a broad brush, the best workers might not even be candidates, because they've already been hired. And the best candidates might be the least likely to seek coding interview practice.
I thought the most interesting finding was that completing Udacity or Coursera courses on programming/algorithms (for non-top school graduates) was highly predictable of strong interviewing performance.
I know some schools and bootcamps now have a separate "How to pass coding interviews" unit in their curriculum (since this is a separate skill unrelated to how to write good code on the job, sadly). If those courses include such a unit, it would explain the performance on stereotypical tech interviews.
I feel like things are operating according to the following pattern:
1. Go to college:
a. spend many semesters in lectures
all of which tangentially brush
upon the final exam based on the
whims of the lecturer.
b. cram for final exam last minute
panic to crunch memory according
to advice on content which was
brushed upon during lectures.
2. Interview for job:
a. cram for interview by going to
coursera to crunch memory according
to interview memes based on the
whims of the interviewer.
b. spend the rest of term of employment
exercising skills which tend to be
tangentially brushed upon during
both interview and schooling while
the majority of actual tasks are often
googled and stack-overflowed
into place based on arbitrary design
decisions and politicized stack
choices.
3. Results:
a. good interviewees have learned
appropriate memes to reassure interviewers.
b. good students have learned obligatory cruft
to reassure professors.
c. actual necessities are tangential to
many or most entry barriers.
completely agreed, which is why it seems like years of experience has no correlation to technical interview performance, despite the likelihood that there would be some correlation between years of experience and actual job performance--because memetic technical interviews and what the job actually entails are generally miles and miles apart.
On top of that, I would assume that the bar is much higher for candidates with years of experience, either because the more experienced candidates self-select for more senior jobs or because it sets up some expectations from the the interviewer ("he's been a tech lead for 12 years and didn't know about quick sort!").
Ah damn; and I had been doing so well mushing down my imposter complex :P
I've found the exact opposite, the longer I dev the more I look for things. I've found it entirely impractical to keep the esoterica of dozens of different languages, frameworks and tools in my head at once. What little memory I have is typically reserved for thinking about abstractions, arch/features, and core concepts, and I let myself lean heavily on lookup resources for things that don't generalize. Just today I'm writing in py+full web stack+cpp+bash+powershell+sql(both PG and ms), and using full suites of associated libs. I at the very least am entirely unable to keep all of that in my head at once, I'd be curious if you find this same problem with bringing in a wide breadth of components.
(My honest fear is that I've pushed too far to become a generalist and am sacrificing strength in depth because of this, although I am somewhat proud of the end to end results it lets me accomplish)
Yeah I suppose YMMV. I work mainly with JS on a video player which uses no frameworks. I still lean on MDN a lot and use SO for learning new things, but I've learned to rely on myself and my teammates to solve problems and where that isn't enough, to look through Git issues and documentation. I've developed my own standards of "goodness" which helps me come up with solutions. It also helps that SO does not have answers to 99% of my problems (how do I make HLS videos upswitch faster?).
I back you up on this. I'm about 10 years in my career, and I use SO more and more. Because, conceptually, I know what I want to do. Last night it was trying to use the asyncio library for Python 3 to use sockets. I know what I wanted to do: async, accept, listen, read and write to sockets. But how, _specifically_, do you do that with python 3 ? Don't know and dont care. Ask SO or find someone's blog howto and be done with it. For better or worse, I dont use books at all anymore. What I find is that they take the long way to go about things. The more I learn, the more I just want to see EXAMPLES. I dont even want to read a single word "about" topic X, I just want to see topic X in use.
I'm about ready to write my own manifesto about why man pages are worthless. When I do, I'm going to blog it and submit it here. I don't even know where to start about man pages. They are reference material, but totally useless at showing you how to use the tool. even though they tell you what EVERY dash-this and dash-that option means, you can STILL screw up by not ordering them properly, forgetting a required dash-this, or not formatting the arg properly (information that is either omitted completely or buried in thousands of words inside the man page, but could be explained concisely with one simple example). the corollary is also true: examples are so effective (in my opinion) that they could go ahead and omit the "dash-this and dash-that" from the man page, and I could likely infer what those options mean just by seeing the example, or an example with a one sentence comment saying "recursively scan directories" (and I see -r). the letters often don't correlate to an actionable-thing that the tool does, the man pages don't cover the most common, useful way that tool will be used, giving equal weight (or rather, equal ambiguity) to arcane, never-used options. They take 100 words to explain what a 10-word example could show. I honestly don't recall the last time I read a man page and actually found what I needed.
Working within a single language (Python), I've come to the point where I have sufficient knowledge to be able to answer many things about it without googling. However, my false positive rate is sky high because I know too many implementation details. It was unavoidable in working within a particular stack for ~5 consecutive years, but the false confidence it promotes can be counterproductive, as well as useful.
You're not alone. The more experience I get, the more I feel that specifics in a language are a commodity, but what you do with that language is what matters.
Those abstractions, identifying problems, talking with peers, deciding the next step, etc. The language is not the problem, you will learn it.
With time I got to know more core concepts but I don't know the syntax of whatever language the interviewer's company is using. It is nice to find people interviewing you who care more about what you're trying to say than the syntax.
Don't underestimate the knowledge gained from tangential brushes. Being able to Google and Stack-Overflow a question is dependent on knowing the vocabulary and keywords related to the problem. While tangential brushes don't give you full mastery of the domain, it gives you a bookmark on what is possible and how to find it again in the future. If you talk to newer programmers (new anything really), you will find they lack to knowledge to even properly express their problem, let alone search for answers. While more experience people may not know everything in detail, they have a better handle on various knowledge domains and where to look.
I agree with you on this. It's the difference between "I get a 500 error when I try to submit the blah form and here's the error in the log - it seems to crash in the foo method of the bar service when it receives a null value but it's not clear even from the debugger why it's getting null" vs "X doesn't work".
This exact situations happens to me far too often, especially when working with developers who refuse to learn the other side of the stack. Had lots of "this doesn't work" conversations with frontend devs who refuse to get their arms dirty with the database, and same from backend to frontend as well.
I majored in math & physics, not CS, but was aware of the CS coursework at the time, 30+ years ago. Back in the day, the exams were more or less designed around the problems in the textbook. You could set yourself up pretty well for the exams by working the problems over and over. (Mostly derivations in physics, with or without numerical results, or proofs in math). Doing so would commit the factual content (e.g., formulas, definitions) to memory and get you quick enough in the mechanics to have some spare time during the exam for dealing with the unexpected.
What you learned in class was, among other things, the style of communicating results, i.e., what a well written derivation looks like. I imagine that a CS course could be structured in this way, if a program in CS is analogous to a proof in math. Whether it's actually a good way of teaching is anybody's guess, but it's how things typically worked.
The students who thought that they were expected to mimic the "tricks" and opinions of the professor, were suspected of simply not grasping the material. On the other hand, though I grasped the formula for getting good grades, nobody ever told us that formula, and a student who never guessed it, would be faced with nothing but frustration and sorrow.
When I taught college math and engineering for a brief period, many years later, I told my students the trick, on the first day of class.
Of course I have no idea if this was a good preparation for the workforce, but I'm pretty sure students were expected to combine their textbook learning with experiences gained outside of the classroom, e.g., from research projects, internships, etc.
If the interviews are "structured", the correlation between interview performance and job performance is probably around .3 based on meta-analyses of research findings. If the interviews are "unstructured", the correlation will be within sampling error of zero.
Not the person who originally replied to you, but the HR class I took in college broke down the correlation of interview performance to job performance into a few finer-grained buckets than just structured and unstructured. Sorry, no sources here, as I've since gotten rid of that textbook.
Structured vs. unstructured are the fairly general categories, but that correlation number is accurate. The best predictor I can recall (~0.6 IIRC?) is a "work sample test". You can Google that and find all kinds of HR resources about it.
That is HR jargon for, in this case, writing code to solve a real problem under normal conditions for that company. That is, on an actual computer, not on a whiteboard, and not being constantly scrutinized throughout the process.
Top school is also serving as a proxy for wealth and privilege. Good grades are also a proxy for wealth. Kids who have rich parents get private tutoring.
The privilege to value education and to spend time studying, yes. Almost all resources can be found online nowadays. If you've ever been to an SAT tutoring class, you know that the only benefit the kids have is the benefit of being forced to take practice tests.
A top school is a good signal for how much time someone spent studying in high school, except for affirmative action students who get into top schools with much worse scores and GPA.
> A top school is a good signal for how much time someone spent studying in high school, except for affirmative action students who get into top schools with much worse scores and GPA.
For the truly competitive schools, the affirmative action students still need very high scores and GPA, they just tend to be given a little more of a pass on the extracurricular activities and essays in the application. I'd argue that for the most part, students of any demographic have to do some pretty crazy things to get the attention of top schools. The probabilities of an affirmative action student being granted admittance are just much higher than for an equivalent non-affirmative action student, but that doesn't mean that affirmative action students can get away with having poorer grades and scores. I'd argue they just have to be less well-rounded.
And the classes the student took. And the environment a student grew up in. And how much support the student received. And any one of about a million other things.
There are virtually an infinite number of variables that contribute to university admission and it's extremely reductive to suggest that it's purely a function of the amount of time a student spent studying in high school.
> Almost all resources can be found online nowadays.
While this is true it highlights a major misgiving in the attitude with regards to the reality of socioeconomic status. If your poor, yes you theoretically have the ability to utilize those resources, but what good are those resources if you don't have a conducive environment to engage in study?
Lower socioeconomic status is correlated with lower stages of Maslow's Hierarchy of Needs. Self-actualization i.e reaching one's potential is more dependent on the satisfaction of basic and psychological needs, which are generally more accessible to those of privilege.
It's not just about resources being available, it's resources being available and making sure people have the opportunity to engage in those resources.
> except for affirmative action students who get into top schools with much worse scores and GPA.
This dismissive "quota tokens" attitude really irks me. It's one thing getting in, it's a total different ballgame surviving and coming out the other end with a decent GPA. I've seen people of all socioeconomic backgrounds fail and some from remarkable poor backgrounds do exceptionally well.
I can see how it might follow from the assumption that intelligent people are more likely to succeed in their careers but keeping the GP in context, it should also be noted that not all intelligent people succeed because of various factors.
Given that many environmental conditions to which exposure (in many cases, particularly early childhood exposure) demonstrably adversely impacts intelligence are more likely to be avoided with wealth (that is, both inversely correlated and with a clear causative mechanism for that correlation), this isn't at all surprising.
It should be noted that these technical interviews are biased to a particular style, so the data only really is of relevance for these types of interviews.
Sad to see so much detail paid to the data, and so little to the setup of the experiment itself.
It shouldn't be surprising that an online technical screen favors candidates who've participated in a MOOC, but is blind, say, to years of experience. A screen like this is timed-performance-at-a-distance, which resembles MOOC participation. The full spectrum of qualities that comprise a Good Hire might incorporate the other signals from the post, but this type of interview won't test them.
(I'll be the first to admit I'm biased against performative coding in engineering interviews. Tech screens like this are often necessary, though, so they have their place.)
I wonder if "took courses" could be a stand in for "prepared heavily". It seems like people with all the other attributes might think they didn't need to study. People without them might think they did and took courses to "catch up". In my experience, preparation is the key driver of performance in these types of interviews.
It seems reasonable that a person who took a MOOC might have prepared in other ways as well while people who didn't probably didn't prepare much at all (since watching a few Algo lectures seems the most accessible refresher.)
I don't think it's entirely that black and white. If an undergraduate degree from a tier 1 school didn't cost approx. a quarter of a million dollars, along with the fact that your "peers" with rich parents will have attended a private school + tutoring + usually had a decent home-life; these things stack up, you can't claim it's a level playing field. Of course you need to be smart to get there in the first place (in most cases). But it's a far cry from a pure meritocracy.
you've described a range and complex combination. the author of the piece stated -- almost word for word -- that degrees from top schools are not earned.
Thanks for writing this Aline. As a recruiter for almost 20 years, I wish I had access to all my data and then the time to compile it, and anecdotally I'd expect the finding about MOOCs would be similar.
The most selective of my hiring clients over the years tended to stress intellectual curiosity as a leading criterion and factor in their hiring decisions, as they felt that trait had led to better outcomes (good hires) over the years. MOOCs are still a relatively recent development and new option for the intellectually curious, but it's not much different than asking someone about the books on their reading list.
Unfortunately, demonstrating intellectual curiosity often takes up personal time, so someone with heavy personal time obligations and a non-challenging day job is at a significant disadvantage. One could assume that those who have the time to take MOOCs also have time to study the types of interview questions likely favored by the types of companies represented in this study.
Thanks for continuing to share your data for the benefit of others.
> Unfortunately, demonstrating intellectual curiosity often takes up personal time, so someone with heavy personal time obligations and a non-challenging day job is at a significant disadvantage.
This is true, and one of the things that tends to stack disadvantages against people who are older in the industry. There's a good correlation between free time and age in some ways, as people get to mid-career they tend to marry, have kids, and so on. Maximum free time comes during college and immediately after, for many people.
Valuing intellectual curiosity also tends to mean that this curiosity is valued over direct experience, which is what the older more time-strapped people have to offer relative to younger people who are doing the MOOCs.
I see this pattern time and time again in both my recruiting work and in my resume business. Someone gets a rather non-challenging job that pays market rate or better, starts a family, pays little attention to any tech outside their day job, and suddenly finds that their marketability outside the current job has suffered greatly.
As developers get older, they need to be conscious of the relevance of their work in regards to current trends, and if the day job is largely irrelevant they may need to find access to projects (often unpaid) that will serve to impress new employers.
I don't think that the value of experience has truly diminished, but I think what is now classified as 'experience' may have shifted a bit.
I think my words may have been clumsy. My reference to "tech outside their job" was meant more about having relevant skills to a current job market (marketability), not the interview itself.
> Asking a person to traverse and modify a linked-list (a very common problem) or to implement a B-tree are not 'tech outside their job'
No. No one in their daily jobs implements a linked list or B-Tree. There are predefined libraries in the languages or a user created library that people use. Do you really think every person in the company has his/her own version of a linked list in the same code base? This is a fallacy perpetuated time and again. (I'm not against these type of questions, but they don't represent someone's true ability)
Building a product, app, project requires application of this knowledge: eg., I need to design a distributed data store, should I use hash map? What about collisions? Should the collisions be resolved by chaining (linked List) or another map? What kind of data am I storing? Can that be exploited to make this more efficient, synchronization of data across nodes (?), etc..
Many of these interviews completely overlook someone's design ability and harp on some straightforward (and some obscure) topics which IMO has little to no co-relation to someone's ability as a software engineer.
>Why is everyone stuck on getting someone to reinvent the wheel as against getting him to use the said wheel and get things done.
A person who can only use a pre-made wheel is a technician (regardless of how ostentatious their job title may be).
A person who can invent the wheel on demand can also usually invent any other type of rolling mechanism one might encounter a need for and that makes that person a full-fledged engineer.
No value judgement of either person is implied; it's simply a statement of capability. Which type of person is needed depends on the company and the position but it is becoming increasingly clear that the two roles in software are separate, just as they are in other engineering disciplines.
Asking a person to traverse and modify a linked-list (a very common problem) or to implement a B-tree are not 'tech outside their job'
Yes, it is.
More to the point, it's built-in bias toward recent college graduates. Once someone's actually been out in the real world programming for several years, any space in their brain's metaphorical working set that was ever dedicated to remembering how to do this stuff has long since been paged out in favor of the actual day-to-day knowledge they need to do their job. Implementing basic data structures and algorithms is not part of what they need to do their job; that stuff's already provided by the platforms they use.
So when you get to the interview, if you just put someone on the spot and ask them to do one of these things, you're inherently biasing toward an inexperienced recent college graduate who will have this stuff close to the top of their head (on account of having just done it in school and not yet swapped it out of working memory through time on the job).
Which is in turn why there are now books you study -- as an already-qualified programmer -- in order to freshen your memory and ability to quickly regurgitate problems that have nothing whatsoever to do with your day-to-day work, in order to be able to perform them on command in interviews.
I'll be honest, traversing a linked-list and implemented a B-tree (assuming someone reminds you what a B-tree is) are not particularly difficult problems. If you have trouble with those, I'm going to really wonder if your programming skills are really very good to begin with.
Those are bad examples, but real interview questions tend to be "implement from scratch, live and without reference to any materials, these things you've forgotten about because you could take them for granted for such a long time".
Also these days the trendy interview problem is longest common subsequence, which ends up testing "how recently did you review dynamic-programming techniques", not "can you code".
> I'll be honest, traversing a linked-list and implemented a B-tree (assuming someone reminds you what a B-tree is) are not particularly difficult problems
I'll be honest, this is indistinguishable from "I am good at X and I'm great therefore X is a good measure of ability".
They're not particularly difficult. You usually take the algorithm definition and implement one, as you did in the university or in the Coursera algorithms course. Fun, you might learn a thing or two about the language you're using.
If the point of the exercise is a TRIVIA question, leave the interview. On the other hand, if the point of the exercise is not to get the right solution but see how you think and how you use basic programming blocks, then you're in the right place.
I've used the linked list question. And the goal is not for me to check if it is "not difficult" for you. Because I might not be able to teach you what a linked list is in 5 minutes while you're nervous in a technical interview. But to work on technical problems and see the following:
- Can you speak your mind so your peers know what are you doing?
- Will you ask for help?
- Will you accept help and feedback?
- Can we have a discussion about something that there are many "right answers"?
- Do you know the basic building blocks of the language you decided to use? <-- This plus javascript plus references is very funny.
You refer to the interviewers as 'peers' but that's very much not the relationship that exists in an interviewer. The context is different, therefore the behaviour is different.
What you're describing is a confidence bias filter, dressed as an objective test.
Individually, with minor refreshing-study, perhaps no.
But in practice it's merely one of hundreds of things that might get instantly sprung on you as you stand before a whiteboard. In addition, developers with actual jobs can't stay in a continuous cycle of "exam cram" to the same extent that unemployed recent graduates can.
Right, as a metaphor, I'd almost compare it to grading a novelist or writer's grammatical knowledge of parts of speech and sentence structure rather than their ability to write a book or compelling article.
The problem is its possible to run a B-tree implementation and see if the unit tests pass or non-technical interviewers can compare the official solution to what the candidate is talking about fairly easily.
I've never used Go, don't really plan to unless I have to, nothing wrong with it just not my area of expertise and I'm busy with other toys although I'm sure its a very nice language, and if I went up against a non-technical or technically weak interviewer (aka almost all of them) I could probably BS my way thru based on what I do know about concurrency in general. Although after decades of experience, the reason why I'm not still programming in 6502/Z80 assembler is I'm good at that whole "BS my way thru till I'm an actual expert" thing, so maybe I'm talking myself into agreeing for non obvious reason!
BSing skill is under-rated. If you can't BS your way thru something that you don't entirely understand (because you're lazy), then when you're at the cutting edge developing or debugging you'll have trouble BSing about stuff that no one on the planet understands. Not all the worlds problems have a stack overflow question or a script reader at a support hotline, and how you deal with that gut check says a lot.
Actually BSing in tech makes you look very poor if you don't come out with a convincing answer.
(I listened to a training video given by a Marketing Cloud consultancy at my work. Someone asked "what is FTP" after seeing that as an option to upload files. Lots waffle including "the cloud" and no actual answer. The person asking the question was no wiser, and my opinion of the consultant went way down).
That's a problem with... "people looking for shiny things". There are many shiny things, and they loose their shine quite fast.
It is true that many managers, recruiting agencies (almost 100%), and HR, will look for the keywords of the technologies the company is using. So if they use the new Angular 4, they will look for that.
But there are companies that look for the basics: do you know javascript? Have you worked with SPAs? And these are the ones that IMHO deserve your time to apply.
Because tomorrow, today's current trends will be tomorrow's trends. Companies looking for people that can adapt say a lot about what kind of work place will you find there.
One thing that annoys me at present is NoSQL experience (or similar).
I know enough about relational databases and NoSQL to know that unless you have a specific use case, relational is likely a better choice. If you are not scaling beyond one server, and want to view your data from different angles its almost certainly a better choice. As such I haven't really got a great deal of NoSQL experience. I will build a better application than someone who does resume driven development, but won't get past many HR drones.
Interestingly, one of my 'no' signals is people relying on a DB to do the work for them. Don't know how to index this? Shove it in the DB and query it back.
If you don't have a relational query in mind a relational DB is a bad fit, or at least premature optimization.
I am not quite sure what you mean with your comment, as "relying on a DB to do the work for them" sound like a good thing to me. Its usually way faster than doing stuff in the application code, and the declarative nature of SQL usually means less bugs.
> I am not quite sure what you mean with your comment, as "relying on a DB to do the work for them"
Maybe "to do the thinking for them" would be more accurate.
> sound like a good thing to me.
You did what they do - you assumed based on some your usual workloads that a DB and the overhead of using another language, and a non-compile-time checked language, with poor integration (type mismatch), is the solution for the problem I'm working on.
What I expect them to do is ask questions. How many values are there? Are they complex types or something that would map to tables well? What are the querying requirements? Are there any relational elements to the data?
> Its usually way faster than doing stuff in the application code
If shoving things across two sets of pipes, to a general-purpose app not tuned to your workload, is faster - you're writing your main app incorrectly.
> the declarative nature of SQL usually means less bugs.
A declarative language usually means less bugs. But adding a second mismatched language usually means more. Especially when it comes with a whole new system that takes experts to properly tweak.
It'd be nice if we had on hand a lemma that stated that intellectual curiosity, once "proven" as a property of one phase of a person's life, will remain as a property of that person in all later phases, even if the indicator signal for it disappears. Then we could just look for people who look like they once were intellectually curious (perhaps in college) and then hire them on the basis that they'll be able to pick it right back up if given the time and the motivation.
Of course, nobody has ever done any experimentation to prove such a hypothesis so that it could be used as a lemma.
I often wish there was some sort of prediction market of I/O-psychology hypotheses that you could just throw a random prediction onto. The stakes on the most contentious results would then prompt the investors to leverage their bets with research organizations into grant-funding to study those questions.
It would be nice, but I think we'd be hard-pressed to ever agree on what intellectual curiosity even is. It's one of those "I know it when I see it" things. I could evaluate whether a person has it (to my standard) but would someone else accept my call, 5 years later, for their company?
I start a lot of things like that (currently trying to learn Rust). A lot of the time other things take over in my spare time and I never actually get anywhere. Having a job where you can put consistent practice into new technologies makes a big difference. I week at work is probably equal to more than a month of my spare time.
I don't think curiosity diminishes with age, even if the ability to explore that curiosity may. You can still talk to someone and find out if they are interested in experimental technologies, what they choose to tinker with in the limited discretionary time they have, and so on. You can ask about their opinion on new technical developments and whether they follow any trade publications or forums and the coolest things they've found from those places in the last few months.
Since for programmers at least, such information is not hard-separated from the course of their normal work, they will usually have the opportunity, if interested, to be at least partially informed about such things and have some sense of the overarching zeitgeist. For instance, a C# developer would be fully justified in reading about the changes slated for the next iteration of C# during his/her day job, especially as that iteration neared and achieved release.
On top of this, professional engagement and awareness IS something that deserves out-of-work nurture time, even when you have a family or other demanding non-work obligations. You don't have to have an impressive free time project like a hand-built CPU, but expecting someone to put in a few hours a month to tinker, learn, or explore new things related to their field is not unreasonably demanding IMO, even for experienced/busy professionals.
Obviously "participated in a MOOC [Massive Open Online Course, btw, for those curious]" may indicate curiosity (though it may also indicate someone trying to short-circuit their lack of skill by gaining Yet Another Credential to ride off), but it's far from the only signal.
The best employee is an experienced person who has retained their intellectual curiosity, even if they don't have the time to fully exploit it.
> I don't think curiosity diminishes with age, even if the ability to explore that curiosity may.
Anecdotally I'd assume that intellectual curiosity often increases with age, although that might be attributed to the availability of learning materials now compared to years ago. Some people here on HN would likely be shocked as to how many people in the industry consume almost zero content outside of what is necessary for their jobs.
> Some people here on HN would likely be shocked as to how many people in the industry consume almost zero content outside of what is necessary for their jobs.
Seriously. People significantly underestimate how much better the average HN user is than the average developer.
The median software developer sees programming as just a job and spends absolutely no time on developing their experience outside of work.
> I don't think curiosity diminishes with age, even if the ability to explore that curiosity may
I think it definitely does, depending on the person. Many, many attributes of the person change over time. That's what growth and evolution is about!
Some people take a career path where they become expert at a narrow topic. As they master their field, I think they lose a lot of curiosity because there's simply less to be curious about unless they're on the forefront of research and not just a practitioner.
Other people simply develop other life priorities.
What you do at your job during the day is usually imposed on the employee. Few employees get to choose the languages, tools, etc. that their company uses, so their interests aren't always relevant to their day jobs.
Your reading list on the other hand is entirely up to you. It's likely an indicator of something to the person asking the question. People who don't like math (or aren't skilled in math) aren't reading advanced math textbooks for pleasure.
Outside of research and for fields outside of compsci, how else can applicants demonstrate "intellectual curiosity", and can you just expand a bit on what that means?
To me, intellectual curiosity is basically "learning for the sake of learning". In other words, picking up a skill or researching a topic that interests you without having a real need to apply said skil/knowledge professionally or personally.
Often the research may be tangentially related to their work, but it doesn't have to be in order to be 'credited' as intellectual curiosity.
Outside of CS, demonstrating curiosity can be almost anything. A non-CS person learning a technical topic is actually a decent example. Non-programmers that buy Arduino and do some home automation project.
It can be almost anything I'd think. Many people don't dedicate much personal time to learning. Demonstrating that learning can be challenging.
I have clients that require interviewees to do a presentation on anything at all during the hiring process (both non-tech and tech employees). Some have demonstrated game strategies they've studied.
"Intellectual curiosity" was my primary attribute for many years. Particularly when I was young and didn't really know any better. It makes sense doesn't it? Someone who is interested in learning for learning's sake will be a better developer!
Except I'm convinced it's not really true. It's something that is horribly subjective and really self-selective. It's funny, intellectually curious people often have exactly the same interests as whomever is doing the interview. I find that nearly everyone loves to learn, you just have to find the thing they're interested in learning about.
The signal that I have found to be a great indicator of success on my teams isn't about curiosity at all. It's about attention to detail. In the world of scatter brained developers who never seem to really follow through on anything, it's those guys that are the real unicorns.
Our interview process is now designed to bubble that to the top. Vague programming problems with poorly defined requirements provide a platform by which we can see how someone digs into problems. I'll ask for them to send me a couple of things after the interview, it's a really good signal when they pull out their phone and add it to a to-do list.
Those guys may not always be the "smartest" or the most interesting, but man when you're going to spend months working down a really large project they get stuff done.
> It's about attention to detail. In the world of scatter brained developers who never seem to really follow through on anything, it's those guys that are the real unicorns.
I've done a few hundred technical interviews over the years and this is absolutely true. It's these guys who make production ready systems possible more than anybody else.
You had me until, "it's a really good signal when they pull out their phone and add it to a to-do list."
Because it's just one of many possible courses of action for staying organized. For all you know the guy added it to his to-do list on his way to the car, or maybe he employs some other mechanic. While you could argue that an interview is a showboating environment where one is expected to signal certain desireable traits, I personally resent that these attributes seem to manifest in recruiters as specific actions that reinforce their perceptions of a good candidate -- if someone is unconventional then how is looking for specific signals going to tell you anything useful about them?
It's a signal. It's not a binary yes/no, there are many observations we're making throughout the interview. This is just one, and the to-do list isn't the only correct answer. Asking me to email those details to them is another possible right answer. Writing a note in a notebook is fine.
The key thing to remember here is that we're asking this very much in the middle of the interview, and I just want to see that they have some sort of system in place for remembering these things.
In my experience, most of the time, simply going "sure, I'll do that" and depending on memory to actually get it done is a negative signal. Maybe they are particularly gifted in terms of memory, but in my experience that is rarely the case.
In hiring we're willing to accept false-negatives, but never false-positives. You are correct, I may let a perfectly good candidate walk, but I'm willing to accept that as the cost of hiring a bad one is just too high.
I would not have written that down, but I can assure you that I would remember to send you whatever you asked: An interview setting is very special, and tend to get hyper focus. I wouldn't go out from there wondering the typical "man, that was intense - wonder how it went??", and then forget to send you those things you asked!
Unless your request was very specific and strange ("send me the completed k3-45 and t9807 forms"), in which case I'd have to jot /that/ down.
Should you not rather evaluate whether your request was fulfilled?
So the requests are often a bit strange (relevant to the interview tho). Usually asking for a citation for some claim made during the interview. I also make a point to ask in the middle of the interview so there's lot of times to forget.
Again I do want to stress: This is a small detail as part of a larger interview. Screwing this one thing up isn't going to sink you, but we combine it with a bunch of other signals and observations to make decisions. I've been doing this for several years, and I'm very happy with the results.
I sincerely think the downsides of a false positive are vastly overrated.
IMHO the real problem is the strict barrier between people who are part of company and non-company. Imagine if we had a more permeable membrane with different departments hiring firing as they see fit (a faster flow than this structured hiring). This would test the candidates on working in the real environment rather than the whole magic and mysticism of made-up work scenarios in interviews.
In fact I wish we had a more libertarian system inside companies with more free markets rather than all the central planning which happens now
As it happens I have a poor temporal memory so the most important tool I have is I always carry my notebook and jot down everything that gets said, particularly as my previous role involved coordinating and managing teams and projects.
That graph is about the best evidence I've seen that interviews are garbage. Years of experience doesn't matter at all? Coursera courses are the best thing ever?
The author draws a hard distinction between Udacity/Coursera MOOCs (good) and traditional master's degrees (bad). I'll interject that with Georgia Tech's Online Master's in Computer Science program [0], which is delivered via Udacity and insanely cheap [1], you can get the best of both! (Their "Computability, Complexity and Algorithms" class is one of the top Udacity courses cited in the article.)
Keep in mind that a traditional degree program does have a huge advantage over a strict MOOC: accountability. It sounds good to say that anybody can go push themselves through one of these courses. Try pushing yourself through ten, and actually writing all the papers and implementing all the code, while working full time and having a family. That grade looming at the end of the semester really does wonders for your motivation. Plus you can get help from live professors and TAs, and the Piazza forums for OMSCS are full of smart, curious students who love talking about the subject at hand. There's a richness to the degree experience that I don't think you get with scattered classes.
Jumping in as another OMSCS student to second your post. The accountability really makes a difference finding the balance between work/life and the schooling (which can be nontrivial considering the courses are certainly rigorous).
These numbers still look very high from a European point of view. It seems it makes a huge difference whether a society deeply cares about educating the population, or cares just superficially.
For example, in Berlin you spend less than 300€ per semester, a CS master takes 10 semesters regular time. Add two semesters to make the time more realistic, and you end up with 3,600€ which are roughly $3,800.
Oh, and it contains a full time ticket for public transport (which would otherwise cost 970€/year, i.e. 485€/semester). In other words: University education is cheaper than regular public transport, even though it contains a full time ticket.
Oh, and if your parents don't have that money, you can get half of the university costs + half of the living costs + half of the rental costs from the state. [1]
And note that Germany is by far not the best one in Europe regarding education, in universities as well as all other types of schools. It is regularily and heavily criticized for cutting educational expenses more than is good for the country. [2] However, after reading statements like the parent comment, I suspect it is still pretty good.
[1] More precisely, you get a debt called "BAföG", from which you have to pay back only ~50% after finishing - either in rates or all at once.
[2] For example, this forces universities into projects financed by third parties (i.e. companies), which adds a strong bias to the research direction and even more so to the results. Even worse, if this research contains business internals (which is easy to claim by any company), this leads to results being only partly published, or not being published at all. To be fair, the latter is more a problem of the law and not of third-party project. There should be a law that demands everything that is fully or partly paid with public money is Open Access as well as Open Source.
One could very much argue that having educated people is more beneficial to the society, and therefore for everyone, than it costs the society. Also educated pay more taxes individually since they usually earn more.
But you'd have to argue it, it isn't self-evident that more education is simply better. Look at the number of PhDs who can't secure post-docs let alone tenure track. They may feel some personal satisfaction from the letters after their names, but it's very questionable that "society" benefits from churning them out. Once you educate people to the level of basic literacy and numeracy, say age 16, it's diminishing returns after that.
> But it's subsidized; so people who aren't getting an education are paying for those who do.
This is just another way of saying that the society cares about this issue!
To put an analogy: When a society cares about parenthood and children, non-parents pay some share to parents. This is how financial solidarity works. How else should that work? By telling parents that they do a great job, that you appreciate what they do for society, but not giving a single cent to them? That would be hypocritical, not solidary.
So why can't it voluntary via donations to universities?
> By telling parents that they do a great job, that you appreciate what they do for society, but not giving a single cent to them? That would be hypocritical, not solidary.
It's not hypocritical if you compliment someone without paying them money.
Education is perhaps more important for preventing crime and violence than police and courts are. Remember, police and courts just deal with the crime; they don't prevent it like having a proper education and employment does.
No, only some portion of the population (possibly not even the majority--see the U.S. 2016 election) decides for everybody including those who are against it.
Subsized - yes. But I don't agree to the second sentence: Generally the higher educated people are getting higher salaries afterwards and are paying way more taxes. These taxes are then used for providing education to the next generation (and also for lots of social programs which target the low/no income population).
If education was always profitable like that, governments wouldn't subsidize it, they wouldn't need to. Heck people would be fighting to give student loans.
The reality is lots of people study subjects that don't result in higher salaries but political correctness insists that people be able to study the arts easily, so governments "have" to subsidize education.
Well, no, the cost of education is ridiculous. Not many people get out of undergrad without any loans, and especially if you want to go on to get higher education (yes, even law degrees which pay nicely after you graduate), you'll be stuck with a mountain of debt that you won't be paying off any time soon.
I know someone in particular who went on to get a law degree (and was steadily employed in her field from her time of graduation) but was only managing to pay in the double digits toward her loan's principal on a monthly basis -- the rest went toward interest.
The benefits are ridiculous. Not the costs. Where I live it costs $28,000 to get a BS. I took out loans for every penny. Before college my salary was just under $18,000. After college I was making $48,000. I got my MS and PhD for free, fully funded by my work while in graduate school. I even made a decent salary, $18,000 during the year and $30,000 during summer internships.
Welcome to social state, where the state thinks the sustainable development of the whole society can be only accomplished by increasing the life level of everyone
When the government pays for health care (like what happens in most european countries) it means that people that are not sick are paying for people that are sick. Why should they, right?
I hate that sort of terminology, money only exists for you to trade because you are a member of a society that collectively decided it was a good idea.
Yes, but so are the trade schools/apprenticeships where people go that don't go to university, so what's your point? Your education is subsidized either way, unless you don't attend either of those options, in which case the individual is probably not going to pay for anything at all.
I agree that there are not-yet-mainstream concepts for schools/universities/etc. that should be covered by public money as well, at least partly. Currently, this exploration happens entirely in the private sector, which is simply inadequate (read: too small and too slow) for the society to move forwards with its educational system. A society should actively invest into improving their education the same way they improve on science, and that investment is clearly lacking in Germany and many other countries. We advance the educational topics, but not the educational system itself.
But: Investing into classic schools and universities is still better than not heavily investing into education at all, which is the only real-world alternative I've seen so far. (And I would be glad to be introduced into a real-world third alternative.)
> Yes there's some education conferred along with the schooling
Some? Maybe I was just lucky, but at school and especially at university I got a very good foundation and always felt well prepared to educate myself later on (through books, websites, technical manuals, and so on). Contrary to conventional wisdom, I learned more about critical thinking and judging sources in school/university than anywhere else. Not in all lectures, but in enough lectures that I would otherwise have missed. Without that initial foundation, educating myself later on would have been much harder.
Okay, I can introduce you to three real world alternatives I've seen in my own life—self study, direct mentorship and work experience. I'll share one example of each from different fields.
To be clear, I don't think there's a single one-size fits all solution for education. One of the core problems I see in formalized schooling is that by its nature it pushes large numbers of people through the same curricula. This may have been good in the early industrial era, but in today's world most job-related skills that can be commoditized are either outsourced or automated. Non-job related skills are also of great value, though it's not clear that it's best for people to build them in an factory-line style either.
Self-study: I put over a thousand hours into foreign language classes while growing up and got pretty bad results. It's an ancient discipline and curricula have had centuries to adapt, but it's just not well-suited to formalized schooling. I've met literally thousands of people with advanced degrees in English language study who don't speak that well. I've also met a lot of foreigners who graduated with degrees in Chinese who don't really speak or read comfortably. Though I've hired people for positions in which English language skills were important, I've never even considered looking at their related credentials rather than evaluating their results. Foreign languages are very learnable through self-directed study. This is even true for one's native tongue—most really good writers have gotten there through voracious reading and practicing their craft, not generallythrough advanced degrees.
Work experience: Another discipline I've seen schooling fall down is in sales. It's a core business skill, but those I've met who have excelled in it have come from a variety of backgrounds, not necessarily business schools. Almost invariably, the people who really know how to sell have gotten that way through work experience, either for themselves or on commission for someone else.
The third alternative, that of direct mentorship, is probably the most powerful I've encountered. Especially in music, athletics or other extremely competitive fields, there's nearly always a mentor behind the top performer, and often there is a series of several mentors over different stages of the learning process.
Now at this point, I suspect you're thinking about the fact that there are two types of educational goals—getting really good at something and getting to minimum level in all the core skills. Though my three examples were related to the first goal, schooling often fails in the second goal as well. It can succeed, but there are still a lot of people who do what must be done to get the credential they want and little else. On the other hand, it's exceedingly rare to meet someone who reads broadly and doesn't end up with at least a decent education.
Curious reasoning about society. For me, american society seems to care about education more, exactly because it's members are willing to pay for it themselves. In your sentence, you juxtapose 'society' and 'population', but those are exactly the same things.
For someone who moved from Australia to US, I don't think America cares about education for all. Only education for the elite rich. Universities should not cost as much as buying a house. The country literally puts a student in debt for life. That's nuts
I'm studying a 3 year coursework masters that allows people with no background in CS (just had to do an additional semester of courses) to participate and it aims to give a pretty well rounded CS education. Not a common thing, apparently.
I completely agree, the motivation and environment that it gives you, the reinforcement that you're actually doing something official and serious as opposed to doing something that very few people outside of the software industry take seriously, the structure it gives you, the fellow students you meet and befriend, it's all huge. Additionally, looking at the Udacity/Coursera material, a lot of it is shallow, poorly taught, and would not have taught me nearly as well as my master's program has taught me.
Other Masters have acceptance programs for this as well. I know that Delft TU has an 'introduction' year which covers everything you need for the masters years (two). It does require a technical (of sorts) bachelor and has a low success rate due to it containing all the difficult subjects in one instead of multiple years like in the regular CS Bachelors and skipping over any form of specialization courses. Not completely undoable, but difficult non the less.
I imagine there are multiple universities around the globe offering the same sort of educational entry.
While it was shown in the article it doesn't matter at interviews, we (at my company) do require technical masters degrees. Though it does not need to be a CS master. The form of individual thinking combined with analytics and methodology we found to be far better at those completing (any/a) Masters degree than those without. Especially self-reliance, which is important if you do not have groups of 10+ devs working together.
Is this UChicago's program? I did a masters there without a CS background, and it was amazing. Incredibly hard, but amazing. The beauty of it is that they ramp you up on the mathematics and programming if required with a few additional classes. They also let you test out of either if you already know it. Doing that masters was one of the best things I've ever done.
> Keep in mind that a traditional degree program does have a huge advantage over a strict MOOC: accountability.
I wonder if this is a contributing factor to the success of those who completed MOOCs. There was zero accountability but they STILL manage to complete the coursework and assignments.
I'm im the same camp. I've start studying at a university a few times now, and I keep failing because other oportunities come up that are more important than passing the course.
I need to be able to self study at my own pace according to how my free time allowes. I'm also not that interested in formal tests and exams. And even course projects.
I want to learn what I need for whatever project I'm currenty working on or going to work on next.
I so want to enroll and take the OMSCS; I know I could succeed at it if I took it, but the school requires you to have at least a Batchelor's degree to attend (they don't count work experience in software development - so my 25+ years mean nothing).
I found out about the OMSCS after I started my work on Udacity's "Self-Driving Car Engineer" nanodegree course, which I am currently taking. My plan (hopefully) is after I complete the nanodegree, I'm going to take some kind of online BS program (maybe accelerated if I can find one), then hop over to the OMSCS (hopefully it or something similar is still running). I mainly want to do it to prove to myself I can do it - I believe that I can.
I made some early life choices that have led me to where I am at, and while it hasn't hindered my job or career prospects, I have always wanted what I should have done in the first place.
Western Governors University (non-profit, accredited, affordable) offers a few different B.S. programs that you could look at to use as a buffer. I received my undergraduate degree from WGU and highly recommend it. It's self-paced and there are ways to fast-track the degree in less than the typical 4 years.
I remember when I graduated from a "Top School" and interviewed at "hot startups" from the valley. I aced a lot of the interviews - why? Because I had just taken classes on LinkedLists, Binary Trees, HashMaps, etc... So when they asked me to whiteboard a "shortest path algorithm" it was just rehashing what I did in school.
Years later, looking back, I fail to see the relevance in most of the technical questions. In fact, if I had to do those questions over again today I would probably fail miserably. Yet, I have been in the industry for a while now and have worked with countless more technologies and have accomplished far more than my younger self.
Just because someone performs well in a technical interview doesn't mean they will do a good job. That is the data that really matters. I've interviewed hundreds of candidates as a hiring manager for some big startups, and from my experience technical interviews are not a great indicator of success.
I'm saying this coming from someone who has gone to a "Top School" and done multiple Coursera/Udacity/etc classes.
Yes, someone might be able to whiteboard a random forest or write a merge sort, but do they know how to engineer a system? Can the candidate:
> Communicate well with others in a group?
> Solve unique technical problems?
> Research and learn new technologies effectively?
> Understand how to push back to product owners if there's scope creep?
etc...
These are all things that are not really analyzed in many technical interviews.
As I'm reading this analysis all I can think of is that it is pretty useless - if not dangerous for the industry.
What I've found is that it is critically important that someone knows how to code at some basic level. But their ability to code and explain algorithms on the fly, while probably relevant in academia/research, is such a minor part of the day-to-day of a programmer - At least from my experience.
That has been my experience, too. I remember deriving the optimal solution for an interview question that was a variant of a radix sort and impressing the interviewer for one of the hot startups at the time. That was partly the result of my ability to recognize the similarities between the problems but even more a consequence of having recently studied radix sorts. Twenty years later, I'm a much better developer than I was then and have shipped many projects with significant degrees of complexity, but I couldn't answer that interview question or ones like it as well today as I did then.
Interestingly, they suggest that if you attend a to school, the effect of Udacity is negligible. I'd argue that Udacity is this filling in gaps of a poor education.
Until recently I worked at a startup as Machine Learning Engineer/Data Scientist. There I got some experience interviewing people and looking at their resumes. In my experience, which is very limited compared to this post, people who put an MOOC on their resume are usually less qualified compared to people who don't.
There is nothing wrong with MOOCs, but they are almost always beginner-level. If you put them on your resume it kindof implies you don't have a lot of experience beyond that. Putting the Coursera Machine Learning course on your resume would be the equivalent of putting Java 101 on your resume for a Software Engineer.
I would recommend anyone to put projects on your CV instead. Even if you don't have a lot of work experience, just put side-projects and school projects on there.
I think the point here are folks taking MOOCs for enrichment, not as a gateway to get a job. So for instance someone taking the ML course just for knowledge, not to try to get in the door as a data scientist
Yeah, like I said, there is nothing wrong with MOOCs, but I would not put them on my resume as a candidate nor would I value them as a positive signal from an employer's point of view when I see them on a resume.
That's absurd. Everything in a masters degree is "beginner" when you've worked 10 years in a field. When I took my MSc in computer science, there hardly was any "big data" and deep learning around. That you actually would fail me for taking a course in this field is utterly beyond me.
Not to harp on the "technical interviews are disconnected from actual work!" angle too much, but I'm reminded of a comment from a thread about the creator of Homebrew failing a Google interview. Someone pointed out that it goes to show that it's possible to create widely-used software without an intimate knowledge of CS. I wonder if that's a disconcerting fact for some employers to grapple with.
To be fair, the creator of a bit of software that is widely used may not necessarily be a good fit for Google.
For many reasons.
I find some creators put a lot of ego into their products, not that they are egoists so much, but doing something on your own can be emotionally challenging. I don't mean 'ego' in a negative way - just that when you are on your own you have to make so, so many decisions. APIs, compatibility, etc. etc. you're often a little overwhelmed, you often have to 'just do it' and 'go with your gut' in many areas. Which may or may not be a good practice/attitude for a Google employee.
They might have a stronger sense of the 'outcome' (i.e. what it does) than the capacity for the underlying code. Maybe it 'does the job' but is poorly written? Which again, might not work in Google environment.
... and working on a very large team is night-and-day from working on your own.
I think we all want to 'feel it is unfair' that such an important contributor somehow 'can't get a job at Google' ... maybe he's worthy, maybe not, but I can totally grasp someone mighn't be a good contributor at Google.
Maybe it's easier to see it from a non-technical standpoint: it's often very difficult for founders of their own companies to get along in a place like Google. I think that's easier for us grasp. The same thing can apply in the technical domain.
I've always said there seems to be a weird relationship in tech between "the industry" and academia. On one hand, there's a growing disconnect between the everyday API gluing-frameworks galore-trudging through someone's code instead of rolling your own nature of day to day work vs. the clean algorithms of school. On the other, there's a youthful, even immature, vibe of all-nighters and casualness for the sake of casualness and Silicon Valley perks.
Maybe this is what happens when an industry is built around worshipping whiz kids. If CS in SV was about venerating NASA/aerospace code, less flashy but very very stable, well-engineered stuff that lives are dependent upon, it'd look a lot more like EE and other more traditional engineering fields.
Made a similar comment above, but that was entirely true for myself. I have worked on amazing things as we all have in life, petabyte databases, and just the most incredible setups.
But when it comes to an interview, I cannot get a job several levels below my own (without referrals).
With referrals, I am offered the moon and the stars, complete pick out of a tremendous number of amazing jobs.
But any random interview, and I totally bomb it. I wonder how many out there are like that as well. It feels rather strange from my perspective, but maybe it's fairly normal?
You cannot get a job at a top company without referral/recruiter looking you up. Any such company is totally swamped by the number of resumes submitted and the chance of getting an interview is practically zero.
While I do think that interviewing is broken, I would love to see the raw data with this. For example, did Udacity courses have other related traits associated with them, IE: did these candidates that also have a certain number of years of experience, degree etc.? 3000 is a small sample size and I'm wondering if there is some sampling bias here.
If you know me, or even if you’ve read some of my writing, you know that, in
the past, I’ve been quite loudly opposed to the concept of pedigree as a
useful hiring signal. With that in mind, I feel like I owe clearly
acknowledge, up front, that we found this time runs counter to my stance.
Did the interviewers have access to the applicant's resume? If so, to what extent do these results simply reflect the interviewers' bias for top schools and famous companies?
Interesting article! Some minor statistical pet peeves:
1. Setting non-significant bars to 0 seems fishy. Leaving them and putting confidence intervals on everything would let them speak for themselves.
2. Calling something effect size is ambiguous. That's like saying you measured distance in units (and the wiki article on effect size linked makes clear there are a billion measures of effect size).
I'm guessing their measure of effect size were the beta coefficients in a multiple regression?
Not minor at all. It's very easy to mislead intentionally or not with statistics and that's why rigorousness is needed. No details are given for the analysis, even basic things like definition of what "top university" or "top company job" even means.
It's just amazing to see how many positive comments a post like this gets without even the hint of methodology.
329 comments
[ 3.4 ms ] story [ 273 ms ] threadBut then this article seems to be measuring interview performance, not actual ability on the job. So is any of it actually relevant at all?
For instance, an interview I took last year came with a premise that I found downright bonkers. It was based on code the company had in production, but it stopped being a good problem to look at years before. I was having trouble coming up with the right tradeoffs for the implementation because all my experience was telling me that the entire approach was misguided in the first place, so the problem should not be solved. I passed, but it was a far rougher performance than I would have liked.
There's also how being far from college makes the least important knowledge gained fade away, and the least important thing I studied was memorizing algorithms. I write new algorithms at work sometimes, and I implement off the shelf ones too, but I don't have to recall them off the top of my head. Nobody has to implement distributed consensus algorithms under time pressure, or write HyperLogLog from memory.
So ultimately, there's what easy to measure, and then there's what is valuable and important. We go with what is easy, and those are things that are taught in college. Understanding the right level of testing or designing a system for observability are far more valuable in the long run: It's crazy how much downtime in well known companies comes from people not learning those things in college. But since we are bad at measuring those things, and kids right out of school don't know them, we don't interview for that.
And sadly this is why we all end up hiring by network so much: We can't tell if someone is good in a day, and we can't really ask people to dedicate two weeks to work with us in a probationary period if they have real jobs, but we sure can recall quality former coworkers and ask them to join in.
"...only 3 attributes emerged as statistically significant: top school, top company, and classes on Udacity/Coursera."
Universities don't teach programming, so it's not surprising you don't get applied skills out of the box with a degree.
'Good courses' are focused directly on teaching specific programming skills.
I think they are a good idea because learning programming by 'osmosis' (as you go along) can result in 'not knowing' a lot of key things, even when they are right in front of your face the whole time.
Most importantly - people who are going to take the time to specifically learn new skills, are displaying the kind of conscientiousness that you want in your talent - learning the actual skills is another benefit of that.
I disagree with this statement. Some universities teach programming, some not so much, some only need a Java 101 to graduate.
There is a huge variance. We'd need a statistical analysis accounting for the quality of education given.
Having done Udacity and also taking a couple CS classes at Cornell, this doesn't surprise me at all. People who take Udacity classes are doing so voluntarily on their own time, so they are going to generally be smarter and of higher socioeconomic status.
If you look at people who take CS classes over the summer at college when they don't have to be and when there are no student loans, you're going to get a similar population.
If anything the opposite is true. MOOCs have enabled vast swathes of economically challenged people to learn from high quality video material, whereas before they were limited to textbooks.
The key is that these kids are absolutely determined. And similarly if you really want that job at one of the Big Four, you might also consider soaking up as much prep material as you could.
So this is unsurprising. The thing people should really be thinking about is why, "how to pass a tech interview" is considered a separate and unrelated skill from "how to code", given how many people claim their interview processes are supposed to separate people who can code from people who can't.
(spoiler: it's because those interview processes don't accomplish their stated goal)
"I need to pass an interview where I'll be asked about specific technologies/algorithms/etc. I will take a free online course to memorize/rehearse that information before my interviews."
It is weird that so much effort is spent on interviewing and so much effort is spent on performance reviews. But AFAIK, hardly any organization tries to use performance evaluation of current employees to inform their hiring decisions.
1. From the perspective of a job seeker, the interview is what gets you the job, so it's in your interests as a job seeker to learn how to look better as a candidate based on data about how interviewers judge candidates.
2. The purpose of an interview is to be a predictor for job performance, so while this article doesn't address this part of the question, you want interviews to predict job performance, and how to make interviews do that is interesting.
If I am looking for a job, I care a whole lot about what would make me better at getting jobs.
I decided in my last change of positions to try interviewing for jobs like most people without referrals, since I had several other referral positions open.
And I was horrible. I was ridiculously bad, and couldn't get a job at jobs that paid less, and were for less skill. It was quite hilarious.
Thankfully, all of the referral jobs were begging for me, and that was fine. There was no real loss, but it was a bit concerning that if I didn't have an extensive connection network that I would be in a significantly worse role.
I've been interviewing a lot of developers lately, and while I'm fairly happy with my process (I've managed to avoid bad hires so far), I always have to wonder if I'm turning down people I shouldn't.
When I interview for jobs, I open up to any of the possible things in my background, of which it will take me perhaps 2 weeks to get back up to speed. But I am not going to take that 2 weeks for each job I interview for.
I have trained my brain to learn things, and discard them keeping only the current tasks in memory. The rest I know how to obtain again.
This means that I get asked questions I know that I've done repeatedly in the past, but can only vaguely explain without a refresher.
To the interviewer, this means I am underqualified at best, and at worst, lying about my past skills.
A lovely example of this during the process was that I was asked a fairly basic SQL question that I failed to explain when I had just landed from my second flight of the day. I didn't get the presales role for working with the database. (non referral job.)
However, I did get offered a job working on development for that database, as well as product management for the database.
So I couldn't get a presales role, that I've done in my past, because I couldn't answer a fairly basic sql question. I knew exactly how to answer it, the venn diagram of sql joins, a half second google, but in an interview, I blanked on the answer.
That is indicative of my experience with non-referral positions. I've given training on sql joins a good 30 times in my life, and couldn't explain the join in an interview.
Just the way it goes for me, thankfully it's not actually impactful in my life.
I'd be much more interested to see performance on other questions e.g. Google's typical curveball for new grads is an architecture question like "How would you implement YouTube?".
I'm perplexed as to how people don't think it has any relevance at all.
The entirety of every industry is based on 'interview performance' as being one fairly important factor of future success.
Interviews are obviously not perfect predictors, but they are pretty strongly correlated with outcomes.
Someone who does very well in a technical interview is more likely to be a 'good engineer' than someone who fails miserably - though I totally accept that is not always the case.
There's no doubt we could use a little more science to our approach, however.
Top company is a predictor for the obvious reason - it's selection bias for people who already passed those interviews at the company. You're not good at the interview because you worked at the company, you work for the company because you're good at the interview.
Master's degrees seem like largely for international students needing visas, career switchers, etc so not surprised they are not a strong predictor. And if anything the course material moves past the intro data structures stuff the whiteboard interviews tend to test.
The only huge surprise for me here is that Coursera is a stronger predictor than top company and top school. I would have predicted top company > top school > Coursera.
The post that I would be much more interested in is correlating performance reviews to interview performance. That gets suggested as a possible future post.
But Coursera stuff that gets into a CV is almost certainly recent.
I think there's a point - I would guess at about 8+ years of experience - where a technical interview will stop being a reliable indicator of an engineer's ability. It then becomes more important to evaluate what the candidate has been doing in their career and how that experience can be taken advantage of within the team that they'd be joining. Technical ability can still be judged by asking to see some existing code if it's that essential.
Except, remember, top company also means experience. If you are at a top company, your second-year CS stuff might be a little rusty.
Absolutely agree that some MS degrees are pretty much less rigorous cash cows by now, that allow students to skip the fundamentals such as data structures, operating systems, and compilers.
However, many CS MS degrees actually do require this as a background, to the point where some programs have emerged to prepare non-CS majors for MS degrees, kind of like those post-bac premed programs. It's hard to believe that those MS degrees, which require a decent GPA in those core courses, along with high GRE scores (sorry, but we are talking about interviewing skill, which may be more related to exam taking ability than job performance), wouldn't result in a similar profile to people with CS degrees from top schools.
This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading.
That's an aside, though. The value may very well be in the prep for these degrees (ie., the post-bac CS coursework required for admissions to a reputable MS program). If you can get that through online courses (udacity or coursera) through genuinely rigorous self-study? Yeah, that might do it, for far less money. I've audited a few of them, and they're the real deal, that's the real coursework there.
My guess is that pure MS would correlate positively from CS programs that require a background in more advanced math and algorithms. Info sys degrees typically don't.
I'm not knocking those degrees, keep in mind we're measuring performance on a technical interview, which often leans very heavily toward algorithms, trees, sometimes graphs, and occasionally numerical analysis or computing.
If the MS degree data includes a lot of programs that dont include this background, then this woukd not be a surprising result.
This leaves open the possibility rhat some categories of MS degrees might correlate positively, even though MS degrees overall correlate negatively.
>>>No surprises here. I’ve ranted quite a bit about the disutility of master’s degrees, so I won’t belabor the point.
is literally all she says about MS degrees. Wow the bias of this site is incredible..
http://blog.alinelerner.com/how-different-is-a-b-s-in-comput...
Is what she says about MS degrees in another post. No data or analysis is shown, just opinion.
At what point do we not consider operating systems and compilers "fundamental"? What percentage of CS/programming jobs require deep knowledge in these arenas?
By definition, these are the subjects that allow you to keep going when your abstractions leak[0].
To say a job "requires" them is pedantic, but what's the point of calling something a CS degree if it is not at least signaling an understanding of these fundamentals.
[0] https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
Now you'll say that I'll make a mess if I don't know what the compiler does. That's what best practices are for if say I pick TypeScript. Also, optimization is the compilers responsibility.
I agree that knowing fundamentals is not that essential for a large number of programming jobs. They're still fundamentals though.
My freshman year physics class had us building computer models of the solar system from Kepler's laws, constructing vocoders by manipulating raw sine waves, and discovering the equations that governed op-amp circuits through experimentation and a little deep thinking.
At my last job, one of my first tasks was to take raw data from truck engines computers (a data point ever 300ms) and turn point measurements like "instantaneous velocity" into data like "how far as the truck traveled over the last week"? Trivial. By combining that with engine speed and fuel usage, you could also say things like "what gear is the truck in?" "Has the engine performance declined recently?" "What percentage of the time has the driver idled in the last month?" And roll all that data up into an efficient database, build a super-flexible API around it, and create a series of web apps to display the useful stuff that managers wanted.
Having a deep understanding of the physical relationships between measurements, understanding what the data can and can't tell you, being able to spot potential problems in a proposed project (for instance, something that relies on an impossibly accurate GPS in a subtle way) -- these are all things that computer scientists absolutely should be able to do.
Yes, they should understand their computers from the electrons on up -- operating system included -- but they should understand the world that the computers interact with, as well.
On one extreme, people see CS as a kind of trade school housed in a research university, teaching only what is useful on the job. Some people see CS as professional degree, kind of like law - you must teach theory for students to understand the field, but in the end, you're producing practicing lawyers, not abstract thinkers. And then, at the other end of the distribution, you find people who see CS as an academic field, a branch of scholarship, where the question "what percentage of programmers need to understand compilers" would be akin to asking "what percentage of math majors who become actuaries use real analysis?".
I felt like my knowledge definitely cured during the process with graduate level data structures, architecture, operating systems and networking. The 2nd pass on some of these areas (despite getting mostly A's from a generally-considered challenging but now well known program). I took the opportunity to craft my program of study more than I would have been able to as an undergrad (both due to lack of knowledge and how undergraduates are given a lot less leeway) to include electronics and automation courses. It also gave me the opportunity to work with and mentor other students which was rewarding and I think has benefited me in and out of the workplace.
I wouldn't argue for doing it purely for career reasons and its not something to do just because you've been in school so long you don't know how to do anything else, but I found it worthwhile and you just need to consider what you want out of it.
The Bay Area had several shady MSCS-granting institutions. Some of them have gotten in trouble with ICE.
Even at programs run by major public research universities, the same economic incentives apply.
It's always hard to tell the value-add of a degree apart from selection bias. This case is no different.
"No surprises here. I’ve ranted quite a bit about the disutility of master’s degrees, so I won’t belabor the point."
And provides a link to another post the author has written. Look for ("disutility of masters degrees"). That page will contain more of the content we are discussing.
This is what I was referring to when I wrote:
"This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading."
I would agree that no data is provided here, the author simply states MS degrees are an indicator of poor technical performance "in my experience" and goes on to list reasons why this might be the case.
I'm sure there's a reasonable case to be made that an MS in CS isn't worth the time, effort, and money, even from a top school. But I don't think that grouping these degrees with MS degrees in Info Sys (among others) and then reporting experiences without that distinction provides much useful insight into the topic.
Perhaps you are right. This is a usual argument against canned exams.
EDIT: Wow, this is downvoted? Really?
The more I've grown as a developer the less I've had to do this.
I've found the exact opposite, the longer I dev the more I look for things. I've found it entirely impractical to keep the esoterica of dozens of different languages, frameworks and tools in my head at once. What little memory I have is typically reserved for thinking about abstractions, arch/features, and core concepts, and I let myself lean heavily on lookup resources for things that don't generalize. Just today I'm writing in py+full web stack+cpp+bash+powershell+sql(both PG and ms), and using full suites of associated libs. I at the very least am entirely unable to keep all of that in my head at once, I'd be curious if you find this same problem with bringing in a wide breadth of components.
(My honest fear is that I've pushed too far to become a generalist and am sacrificing strength in depth because of this, although I am somewhat proud of the end to end results it lets me accomplish)
I'm about ready to write my own manifesto about why man pages are worthless. When I do, I'm going to blog it and submit it here. I don't even know where to start about man pages. They are reference material, but totally useless at showing you how to use the tool. even though they tell you what EVERY dash-this and dash-that option means, you can STILL screw up by not ordering them properly, forgetting a required dash-this, or not formatting the arg properly (information that is either omitted completely or buried in thousands of words inside the man page, but could be explained concisely with one simple example). the corollary is also true: examples are so effective (in my opinion) that they could go ahead and omit the "dash-this and dash-that" from the man page, and I could likely infer what those options mean just by seeing the example, or an example with a one sentence comment saying "recursively scan directories" (and I see -r). the letters often don't correlate to an actionable-thing that the tool does, the man pages don't cover the most common, useful way that tool will be used, giving equal weight (or rather, equal ambiguity) to arcane, never-used options. They take 100 words to explain what a 10-word example could show. I honestly don't recall the last time I read a man page and actually found what I needed.
Those abstractions, identifying problems, talking with peers, deciding the next step, etc. The language is not the problem, you will learn it.
With time I got to know more core concepts but I don't know the syntax of whatever language the interviewer's company is using. It is nice to find people interviewing you who care more about what you're trying to say than the syntax.
At a distance, though, reality can take on funny appearances.
What you learned in class was, among other things, the style of communicating results, i.e., what a well written derivation looks like. I imagine that a CS course could be structured in this way, if a program in CS is analogous to a proof in math. Whether it's actually a good way of teaching is anybody's guess, but it's how things typically worked.
The students who thought that they were expected to mimic the "tricks" and opinions of the professor, were suspected of simply not grasping the material. On the other hand, though I grasped the formula for getting good grades, nobody ever told us that formula, and a student who never guessed it, would be faced with nothing but frustration and sorrow.
When I taught college math and engineering for a brief period, many years later, I told my students the trick, on the first day of class.
Of course I have no idea if this was a good preparation for the workforce, but I'm pretty sure students were expected to combine their textbook learning with experiences gained outside of the classroom, e.g., from research projects, internships, etc.
I totally believe you, but I've been trying to find good empirical studies and authors to read on interviewing.
Structured vs. unstructured are the fairly general categories, but that correlation number is accurate. The best predictor I can recall (~0.6 IIRC?) is a "work sample test". You can Google that and find all kinds of HR resources about it.
That is HR jargon for, in this case, writing code to solve a real problem under normal conditions for that company. That is, on an actual computer, not on a whiteboard, and not being constantly scrutinized throughout the process.
A top school is a good signal for how much time someone spent studying in high school, except for affirmative action students who get into top schools with much worse scores and GPA.
For the truly competitive schools, the affirmative action students still need very high scores and GPA, they just tend to be given a little more of a pass on the extracurricular activities and essays in the application. I'd argue that for the most part, students of any demographic have to do some pretty crazy things to get the attention of top schools. The probabilities of an affirmative action student being granted admittance are just much higher than for an equivalent non-affirmative action student, but that doesn't mean that affirmative action students can get away with having poorer grades and scores. I'd argue they just have to be less well-rounded.
Assuming all high schools are the same.
There are virtually an infinite number of variables that contribute to university admission and it's extremely reductive to suggest that it's purely a function of the amount of time a student spent studying in high school.
While this is true it highlights a major misgiving in the attitude with regards to the reality of socioeconomic status. If your poor, yes you theoretically have the ability to utilize those resources, but what good are those resources if you don't have a conducive environment to engage in study?
Lower socioeconomic status is correlated with lower stages of Maslow's Hierarchy of Needs. Self-actualization i.e reaching one's potential is more dependent on the satisfaction of basic and psychological needs, which are generally more accessible to those of privilege.
It's not just about resources being available, it's resources being available and making sure people have the opportunity to engage in those resources.
This dismissive "quota tokens" attitude really irks me. It's one thing getting in, it's a total different ballgame surviving and coming out the other end with a decent GPA. I've seen people of all socioeconomic backgrounds fail and some from remarkable poor backgrounds do exceptionally well.
Can you cite a source for this?
I can see how it might follow from the assumption that intelligent people are more likely to succeed in their careers but keeping the GP in context, it should also be noted that not all intelligent people succeed because of various factors.
Given that many environmental conditions to which exposure (in many cases, particularly early childhood exposure) demonstrably adversely impacts intelligence are more likely to be avoided with wealth (that is, both inversely correlated and with a clear causative mechanism for that correlation), this isn't at all surprising.
But there does seem to be a correlation between IQ and income. https://thesocietypages.org/socimages/2008/02/06/correlation...
This latter correlation could be explained by the wealth of the parents, as IQ may be a cultural artifact and income is correlated on education.
It shouldn't be surprising that an online technical screen favors candidates who've participated in a MOOC, but is blind, say, to years of experience. A screen like this is timed-performance-at-a-distance, which resembles MOOC participation. The full spectrum of qualities that comprise a Good Hire might incorporate the other signals from the post, but this type of interview won't test them.
(I'll be the first to admit I'm biased against performative coding in engineering interviews. Tech screens like this are often necessary, though, so they have their place.)
It seems reasonable that a person who took a MOOC might have prepared in other ways as well while people who didn't probably didn't prepare much at all (since watching a few Algo lectures seems the most accessible refresher.)
so a degree from a top school is not earned (nor are admissions i guess), but rather conferred at birth? i beg to differ.
the commentary on the "disutility" masters degrees is even worse.
The most selective of my hiring clients over the years tended to stress intellectual curiosity as a leading criterion and factor in their hiring decisions, as they felt that trait had led to better outcomes (good hires) over the years. MOOCs are still a relatively recent development and new option for the intellectually curious, but it's not much different than asking someone about the books on their reading list.
Unfortunately, demonstrating intellectual curiosity often takes up personal time, so someone with heavy personal time obligations and a non-challenging day job is at a significant disadvantage. One could assume that those who have the time to take MOOCs also have time to study the types of interview questions likely favored by the types of companies represented in this study.
Thanks for continuing to share your data for the benefit of others.
This is true, and one of the things that tends to stack disadvantages against people who are older in the industry. There's a good correlation between free time and age in some ways, as people get to mid-career they tend to marry, have kids, and so on. Maximum free time comes during college and immediately after, for many people.
Valuing intellectual curiosity also tends to mean that this curiosity is valued over direct experience, which is what the older more time-strapped people have to offer relative to younger people who are doing the MOOCs.
As developers get older, they need to be conscious of the relevance of their work in regards to current trends, and if the day job is largely irrelevant they may need to find access to projects (often unpaid) that will serve to impress new employers.
I don't think that the value of experience has truly diminished, but I think what is now classified as 'experience' may have shifted a bit.
Asking a person to traverse and modify a linked-list (a very common problem) or to implement a B-tree are not 'tech outside their job'
Most of these interviews are more like math problems than anything a practical developer would encounter.
Shit, if these interviews were "What do you think of node?" or "What is Go's concurrency model?" I think experienced people would do fine.
No. No one in their daily jobs implements a linked list or B-Tree. There are predefined libraries in the languages or a user created library that people use. Do you really think every person in the company has his/her own version of a linked list in the same code base? This is a fallacy perpetuated time and again. (I'm not against these type of questions, but they don't represent someone's true ability)
Building a product, app, project requires application of this knowledge: eg., I need to design a distributed data store, should I use hash map? What about collisions? Should the collisions be resolved by chaining (linked List) or another map? What kind of data am I storing? Can that be exploited to make this more efficient, synchronization of data across nodes (?), etc..
Many of these interviews completely overlook someone's design ability and harp on some straightforward (and some obscure) topics which IMO has little to no co-relation to someone's ability as a software engineer.
Counter case: HFT. Those guys really do implement this stuff. Again and again.
Why is everyone stuck on getting someone to reinvent the wheel as against getting him to use the said wheel and get things done.
I find the processes in a large company rather robotic (note: we're a startup, so my views might be biased).
A person who can only use a pre-made wheel is a technician (regardless of how ostentatious their job title may be).
A person who can invent the wheel on demand can also usually invent any other type of rolling mechanism one might encounter a need for and that makes that person a full-fledged engineer.
No value judgement of either person is implied; it's simply a statement of capability. Which type of person is needed depends on the company and the position but it is becoming increasingly clear that the two roles in software are separate, just as they are in other engineering disciplines.
Yes, it is.
More to the point, it's built-in bias toward recent college graduates. Once someone's actually been out in the real world programming for several years, any space in their brain's metaphorical working set that was ever dedicated to remembering how to do this stuff has long since been paged out in favor of the actual day-to-day knowledge they need to do their job. Implementing basic data structures and algorithms is not part of what they need to do their job; that stuff's already provided by the platforms they use.
So when you get to the interview, if you just put someone on the spot and ask them to do one of these things, you're inherently biasing toward an inexperienced recent college graduate who will have this stuff close to the top of their head (on account of having just done it in school and not yet swapped it out of working memory through time on the job).
Which is in turn why there are now books you study -- as an already-qualified programmer -- in order to freshen your memory and ability to quickly regurgitate problems that have nothing whatsoever to do with your day-to-day work, in order to be able to perform them on command in interviews.
Also these days the trendy interview problem is longest common subsequence, which ends up testing "how recently did you review dynamic-programming techniques", not "can you code".
I'll be honest, this is indistinguishable from "I am good at X and I'm great therefore X is a good measure of ability".
Maybe worth rephrasing
If the point of the exercise is a TRIVIA question, leave the interview. On the other hand, if the point of the exercise is not to get the right solution but see how you think and how you use basic programming blocks, then you're in the right place.
I've used the linked list question. And the goal is not for me to check if it is "not difficult" for you. Because I might not be able to teach you what a linked list is in 5 minutes while you're nervous in a technical interview. But to work on technical problems and see the following:
- Can you speak your mind so your peers know what are you doing?
- Will you ask for help?
- Will you accept help and feedback?
- Can we have a discussion about something that there are many "right answers"?
- Do you know the basic building blocks of the language you decided to use? <-- This plus javascript plus references is very funny.
What you're describing is a confidence bias filter, dressed as an objective test.
But in practice it's merely one of hundreds of things that might get instantly sprung on you as you stand before a whiteboard. In addition, developers with actual jobs can't stay in a continuous cycle of "exam cram" to the same extent that unemployed recent graduates can.
I've never used Go, don't really plan to unless I have to, nothing wrong with it just not my area of expertise and I'm busy with other toys although I'm sure its a very nice language, and if I went up against a non-technical or technically weak interviewer (aka almost all of them) I could probably BS my way thru based on what I do know about concurrency in general. Although after decades of experience, the reason why I'm not still programming in 6502/Z80 assembler is I'm good at that whole "BS my way thru till I'm an actual expert" thing, so maybe I'm talking myself into agreeing for non obvious reason!
BSing skill is under-rated. If you can't BS your way thru something that you don't entirely understand (because you're lazy), then when you're at the cutting edge developing or debugging you'll have trouble BSing about stuff that no one on the planet understands. Not all the worlds problems have a stack overflow question or a script reader at a support hotline, and how you deal with that gut check says a lot.
(I listened to a training video given by a Marketing Cloud consultancy at my work. Someone asked "what is FTP" after seeing that as an option to upload files. Lots waffle including "the cloud" and no actual answer. The person asking the question was no wiser, and my opinion of the consultant went way down).
It is true that many managers, recruiting agencies (almost 100%), and HR, will look for the keywords of the technologies the company is using. So if they use the new Angular 4, they will look for that.
But there are companies that look for the basics: do you know javascript? Have you worked with SPAs? And these are the ones that IMHO deserve your time to apply.
Because tomorrow, today's current trends will be tomorrow's trends. Companies looking for people that can adapt say a lot about what kind of work place will you find there.
I know enough about relational databases and NoSQL to know that unless you have a specific use case, relational is likely a better choice. If you are not scaling beyond one server, and want to view your data from different angles its almost certainly a better choice. As such I haven't really got a great deal of NoSQL experience. I will build a better application than someone who does resume driven development, but won't get past many HR drones.
If you don't have a relational query in mind a relational DB is a bad fit, or at least premature optimization.
Maybe "to do the thinking for them" would be more accurate.
> sound like a good thing to me.
You did what they do - you assumed based on some your usual workloads that a DB and the overhead of using another language, and a non-compile-time checked language, with poor integration (type mismatch), is the solution for the problem I'm working on.
What I expect them to do is ask questions. How many values are there? Are they complex types or something that would map to tables well? What are the querying requirements? Are there any relational elements to the data?
> Its usually way faster than doing stuff in the application code
If shoving things across two sets of pipes, to a general-purpose app not tuned to your workload, is faster - you're writing your main app incorrectly.
> the declarative nature of SQL usually means less bugs.
A declarative language usually means less bugs. But adding a second mismatched language usually means more. Especially when it comes with a whole new system that takes experts to properly tweak.
Of course, nobody has ever done any experimentation to prove such a hypothesis so that it could be used as a lemma.
I often wish there was some sort of prediction market of I/O-psychology hypotheses that you could just throw a random prediction onto. The stakes on the most contentious results would then prompt the investors to leverage their bets with research organizations into grant-funding to study those questions.
Since for programmers at least, such information is not hard-separated from the course of their normal work, they will usually have the opportunity, if interested, to be at least partially informed about such things and have some sense of the overarching zeitgeist. For instance, a C# developer would be fully justified in reading about the changes slated for the next iteration of C# during his/her day job, especially as that iteration neared and achieved release.
On top of this, professional engagement and awareness IS something that deserves out-of-work nurture time, even when you have a family or other demanding non-work obligations. You don't have to have an impressive free time project like a hand-built CPU, but expecting someone to put in a few hours a month to tinker, learn, or explore new things related to their field is not unreasonably demanding IMO, even for experienced/busy professionals.
Obviously "participated in a MOOC [Massive Open Online Course, btw, for those curious]" may indicate curiosity (though it may also indicate someone trying to short-circuit their lack of skill by gaining Yet Another Credential to ride off), but it's far from the only signal.
The best employee is an experienced person who has retained their intellectual curiosity, even if they don't have the time to fully exploit it.
Anecdotally I'd assume that intellectual curiosity often increases with age, although that might be attributed to the availability of learning materials now compared to years ago. Some people here on HN would likely be shocked as to how many people in the industry consume almost zero content outside of what is necessary for their jobs.
Seriously. People significantly underestimate how much better the average HN user is than the average developer.
The median software developer sees programming as just a job and spends absolutely no time on developing their experience outside of work.
I think it definitely does, depending on the person. Many, many attributes of the person change over time. That's what growth and evolution is about!
Some people take a career path where they become expert at a narrow topic. As they master their field, I think they lose a lot of curiosity because there's simply less to be curious about unless they're on the forefront of research and not just a practitioner.
Other people simply develop other life priorities.
It's 100% an individual thing.
Your reading list on the other hand is entirely up to you. It's likely an indicator of something to the person asking the question. People who don't like math (or aren't skilled in math) aren't reading advanced math textbooks for pleasure.
Thank you, and thanks Aline!
Often the research may be tangentially related to their work, but it doesn't have to be in order to be 'credited' as intellectual curiosity.
Outside of CS, demonstrating curiosity can be almost anything. A non-CS person learning a technical topic is actually a decent example. Non-programmers that buy Arduino and do some home automation project.
It can be almost anything I'd think. Many people don't dedicate much personal time to learning. Demonstrating that learning can be challenging.
I have clients that require interviewees to do a presentation on anything at all during the hiring process (both non-tech and tech employees). Some have demonstrated game strategies they've studied.
Except I'm convinced it's not really true. It's something that is horribly subjective and really self-selective. It's funny, intellectually curious people often have exactly the same interests as whomever is doing the interview. I find that nearly everyone loves to learn, you just have to find the thing they're interested in learning about.
The signal that I have found to be a great indicator of success on my teams isn't about curiosity at all. It's about attention to detail. In the world of scatter brained developers who never seem to really follow through on anything, it's those guys that are the real unicorns.
Our interview process is now designed to bubble that to the top. Vague programming problems with poorly defined requirements provide a platform by which we can see how someone digs into problems. I'll ask for them to send me a couple of things after the interview, it's a really good signal when they pull out their phone and add it to a to-do list.
Those guys may not always be the "smartest" or the most interesting, but man when you're going to spend months working down a really large project they get stuff done.
I've done a few hundred technical interviews over the years and this is absolutely true. It's these guys who make production ready systems possible more than anybody else.
Because it's just one of many possible courses of action for staying organized. For all you know the guy added it to his to-do list on his way to the car, or maybe he employs some other mechanic. While you could argue that an interview is a showboating environment where one is expected to signal certain desireable traits, I personally resent that these attributes seem to manifest in recruiters as specific actions that reinforce their perceptions of a good candidate -- if someone is unconventional then how is looking for specific signals going to tell you anything useful about them?
The key thing to remember here is that we're asking this very much in the middle of the interview, and I just want to see that they have some sort of system in place for remembering these things.
In my experience, most of the time, simply going "sure, I'll do that" and depending on memory to actually get it done is a negative signal. Maybe they are particularly gifted in terms of memory, but in my experience that is rarely the case.
In hiring we're willing to accept false-negatives, but never false-positives. You are correct, I may let a perfectly good candidate walk, but I'm willing to accept that as the cost of hiring a bad one is just too high.
Unless your request was very specific and strange ("send me the completed k3-45 and t9807 forms"), in which case I'd have to jot /that/ down.
Should you not rather evaluate whether your request was fulfilled?
Again I do want to stress: This is a small detail as part of a larger interview. Screwing this one thing up isn't going to sink you, but we combine it with a bunch of other signals and observations to make decisions. I've been doing this for several years, and I'm very happy with the results.
IMHO the real problem is the strict barrier between people who are part of company and non-company. Imagine if we had a more permeable membrane with different departments hiring firing as they see fit (a faster flow than this structured hiring). This would test the candidates on working in the real environment rather than the whole magic and mysticism of made-up work scenarios in interviews.
In fact I wish we had a more libertarian system inside companies with more free markets rather than all the central planning which happens now
Keep in mind that a traditional degree program does have a huge advantage over a strict MOOC: accountability. It sounds good to say that anybody can go push themselves through one of these courses. Try pushing yourself through ten, and actually writing all the papers and implementing all the code, while working full time and having a family. That grade looming at the end of the semester really does wonders for your motivation. Plus you can get help from live professors and TAs, and the Piazza forums for OMSCS are full of smart, curious students who love talking about the subject at hand. There's a richness to the degree experience that I don't think you get with scattered classes.
(Obvious disclaimer: I'm a current OMSCS student)
[0] http://omscs.gatech.edu [1] https://www.omscs.gatech.edu/program-info/cost-payment-sched...
[1] https://www.omscs.gatech.edu/prospective-students/faq
[2] http://grad-schools.usnews.rankingsandreviews.com/best-gradu...
For example, in Berlin you spend less than 300€ per semester, a CS master takes 10 semesters regular time. Add two semesters to make the time more realistic, and you end up with 3,600€ which are roughly $3,800.
Oh, and it contains a full time ticket for public transport (which would otherwise cost 970€/year, i.e. 485€/semester). In other words: University education is cheaper than regular public transport, even though it contains a full time ticket.
Oh, and if your parents don't have that money, you can get half of the university costs + half of the living costs + half of the rental costs from the state. [1]
And note that Germany is by far not the best one in Europe regarding education, in universities as well as all other types of schools. It is regularily and heavily criticized for cutting educational expenses more than is good for the country. [2] However, after reading statements like the parent comment, I suspect it is still pretty good.
[1] More precisely, you get a debt called "BAföG", from which you have to pay back only ~50% after finishing - either in rates or all at once.
[2] For example, this forces universities into projects financed by third parties (i.e. companies), which adds a strong bias to the research direction and even more so to the results. Even worse, if this research contains business internals (which is easy to claim by any company), this leads to results being only partly published, or not being published at all. To be fair, the latter is more a problem of the law and not of third-party project. There should be a law that demands everything that is fully or partly paid with public money is Open Access as well as Open Source.
This is just another way of saying that the society cares about this issue!
To put an analogy: When a society cares about parenthood and children, non-parents pay some share to parents. This is how financial solidarity works. How else should that work? By telling parents that they do a great job, that you appreciate what they do for society, but not giving a single cent to them? That would be hypocritical, not solidary.
> By telling parents that they do a great job, that you appreciate what they do for society, but not giving a single cent to them? That would be hypocritical, not solidary.
It's not hypocritical if you compliment someone without paying them money.
You could say that about any tax founded expense?
But, yes, I am in favor of a small, limited government.
The reality is lots of people study subjects that don't result in higher salaries but political correctness insists that people be able to study the arts easily, so governments "have" to subsidize education.
I know someone in particular who went on to get a law degree (and was steadily employed in her field from her time of graduation) but was only managing to pay in the double digits toward her loan's principal on a monthly basis -- the rest went toward interest.
My first post-grad school salary was $112,000.
My education was an investment. It paid off.
But definitely not for education.
I hate that sort of terminology, money only exists for you to trade because you are a member of a society that collectively decided it was a good idea.
Source?
Yes there's some education conferred along with the schooling, but credentialism is a huge part of it and even more so in Germany than most countries.
I agree that there are not-yet-mainstream concepts for schools/universities/etc. that should be covered by public money as well, at least partly. Currently, this exploration happens entirely in the private sector, which is simply inadequate (read: too small and too slow) for the society to move forwards with its educational system. A society should actively invest into improving their education the same way they improve on science, and that investment is clearly lacking in Germany and many other countries. We advance the educational topics, but not the educational system itself.
But: Investing into classic schools and universities is still better than not heavily investing into education at all, which is the only real-world alternative I've seen so far. (And I would be glad to be introduced into a real-world third alternative.)
> Yes there's some education conferred along with the schooling
Some? Maybe I was just lucky, but at school and especially at university I got a very good foundation and always felt well prepared to educate myself later on (through books, websites, technical manuals, and so on). Contrary to conventional wisdom, I learned more about critical thinking and judging sources in school/university than anywhere else. Not in all lectures, but in enough lectures that I would otherwise have missed. Without that initial foundation, educating myself later on would have been much harder.
To be clear, I don't think there's a single one-size fits all solution for education. One of the core problems I see in formalized schooling is that by its nature it pushes large numbers of people through the same curricula. This may have been good in the early industrial era, but in today's world most job-related skills that can be commoditized are either outsourced or automated. Non-job related skills are also of great value, though it's not clear that it's best for people to build them in an factory-line style either.
Self-study: I put over a thousand hours into foreign language classes while growing up and got pretty bad results. It's an ancient discipline and curricula have had centuries to adapt, but it's just not well-suited to formalized schooling. I've met literally thousands of people with advanced degrees in English language study who don't speak that well. I've also met a lot of foreigners who graduated with degrees in Chinese who don't really speak or read comfortably. Though I've hired people for positions in which English language skills were important, I've never even considered looking at their related credentials rather than evaluating their results. Foreign languages are very learnable through self-directed study. This is even true for one's native tongue—most really good writers have gotten there through voracious reading and practicing their craft, not generallythrough advanced degrees.
Work experience: Another discipline I've seen schooling fall down is in sales. It's a core business skill, but those I've met who have excelled in it have come from a variety of backgrounds, not necessarily business schools. Almost invariably, the people who really know how to sell have gotten that way through work experience, either for themselves or on commission for someone else.
The third alternative, that of direct mentorship, is probably the most powerful I've encountered. Especially in music, athletics or other extremely competitive fields, there's nearly always a mentor behind the top performer, and often there is a series of several mentors over different stages of the learning process.
Now at this point, I suspect you're thinking about the fact that there are two types of educational goals—getting really good at something and getting to minimum level in all the core skills. Though my three examples were related to the first goal, schooling often fails in the second goal as well. It can succeed, but there are still a lot of people who do what must be done to get the credential they want and little else. On the other hand, it's exceedingly rare to meet someone who reads broadly and doesn't end up with at least a decent education.
May be you meant 'government' instead?
I completely agree, the motivation and environment that it gives you, the reinforcement that you're actually doing something official and serious as opposed to doing something that very few people outside of the software industry take seriously, the structure it gives you, the fellow students you meet and befriend, it's all huge. Additionally, looking at the Udacity/Coursera material, a lot of it is shallow, poorly taught, and would not have taught me nearly as well as my master's program has taught me.
What courses are you taking?
I imagine there are multiple universities around the globe offering the same sort of educational entry.
While it was shown in the article it doesn't matter at interviews, we (at my company) do require technical masters degrees. Though it does not need to be a CS master. The form of individual thinking combined with analytics and methodology we found to be far better at those completing (any/a) Masters degree than those without. Especially self-reliance, which is important if you do not have groups of 10+ devs working together.
I wonder if this is a contributing factor to the success of those who completed MOOCs. There was zero accountability but they STILL manage to complete the coursework and assignments.
Accountability is overrated.
I need to be able to self study at my own pace according to how my free time allowes. I'm also not that interested in formal tests and exams. And even course projects.
I want to learn what I need for whatever project I'm currenty working on or going to work on next.
I found out about the OMSCS after I started my work on Udacity's "Self-Driving Car Engineer" nanodegree course, which I am currently taking. My plan (hopefully) is after I complete the nanodegree, I'm going to take some kind of online BS program (maybe accelerated if I can find one), then hop over to the OMSCS (hopefully it or something similar is still running). I mainly want to do it to prove to myself I can do it - I believe that I can.
I made some early life choices that have led me to where I am at, and while it hasn't hindered my job or career prospects, I have always wanted what I should have done in the first place.
I remember when I graduated from a "Top School" and interviewed at "hot startups" from the valley. I aced a lot of the interviews - why? Because I had just taken classes on LinkedLists, Binary Trees, HashMaps, etc... So when they asked me to whiteboard a "shortest path algorithm" it was just rehashing what I did in school.
Years later, looking back, I fail to see the relevance in most of the technical questions. In fact, if I had to do those questions over again today I would probably fail miserably. Yet, I have been in the industry for a while now and have worked with countless more technologies and have accomplished far more than my younger self.
Just because someone performs well in a technical interview doesn't mean they will do a good job. That is the data that really matters. I've interviewed hundreds of candidates as a hiring manager for some big startups, and from my experience technical interviews are not a great indicator of success.
I'm saying this coming from someone who has gone to a "Top School" and done multiple Coursera/Udacity/etc classes.
Yes, someone might be able to whiteboard a random forest or write a merge sort, but do they know how to engineer a system? Can the candidate:
> Communicate well with others in a group?
> Solve unique technical problems?
> Research and learn new technologies effectively?
> Understand how to push back to product owners if there's scope creep?
etc...
These are all things that are not really analyzed in many technical interviews.
As I'm reading this analysis all I can think of is that it is pretty useless - if not dangerous for the industry.
What I've found is that it is critically important that someone knows how to code at some basic level. But their ability to code and explain algorithms on the fly, while probably relevant in academia/research, is such a minor part of the day-to-day of a programmer - At least from my experience.
There is nothing wrong with MOOCs, but they are almost always beginner-level. If you put them on your resume it kindof implies you don't have a lot of experience beyond that. Putting the Coursera Machine Learning course on your resume would be the equivalent of putting Java 101 on your resume for a Software Engineer.
I would recommend anyone to put projects on your CV instead. Even if you don't have a lot of work experience, just put side-projects and school projects on there.
For many reasons.
I find some creators put a lot of ego into their products, not that they are egoists so much, but doing something on your own can be emotionally challenging. I don't mean 'ego' in a negative way - just that when you are on your own you have to make so, so many decisions. APIs, compatibility, etc. etc. you're often a little overwhelmed, you often have to 'just do it' and 'go with your gut' in many areas. Which may or may not be a good practice/attitude for a Google employee.
They might have a stronger sense of the 'outcome' (i.e. what it does) than the capacity for the underlying code. Maybe it 'does the job' but is poorly written? Which again, might not work in Google environment.
... and working on a very large team is night-and-day from working on your own.
I think we all want to 'feel it is unfair' that such an important contributor somehow 'can't get a job at Google' ... maybe he's worthy, maybe not, but I can totally grasp someone mighn't be a good contributor at Google.
Maybe it's easier to see it from a non-technical standpoint: it's often very difficult for founders of their own companies to get along in a place like Google. I think that's easier for us grasp. The same thing can apply in the technical domain.
Look at how many failed or poorly designed products and libs Google has released.
Maybe this is what happens when an industry is built around worshipping whiz kids. If CS in SV was about venerating NASA/aerospace code, less flashy but very very stable, well-engineered stuff that lives are dependent upon, it'd look a lot more like EE and other more traditional engineering fields.
But when it comes to an interview, I cannot get a job several levels below my own (without referrals).
With referrals, I am offered the moon and the stars, complete pick out of a tremendous number of amazing jobs.
But any random interview, and I totally bomb it. I wonder how many out there are like that as well. It feels rather strange from my perspective, but maybe it's fairly normal?
1. Setting non-significant bars to 0 seems fishy. Leaving them and putting confidence intervals on everything would let them speak for themselves.
2. Calling something effect size is ambiguous. That's like saying you measured distance in units (and the wiki article on effect size linked makes clear there are a billion measures of effect size).
I'm guessing their measure of effect size were the beta coefficients in a multiple regression?
It's just amazing to see how many positive comments a post like this gets without even the hint of methodology.