Author invokes comparisons with the medical industry.
Look, we really really really don't want to be taking anything from those folks. The hierarchies are rubbish, the talent questionable, and the egos immense. At least in software we can show that code does or doesn't work irrespective of who wrote it.
Look, we really really really don't want to be taking anything from those folks.
You don't want a doctor's salary? And her level of respect from society at large?
I also don't believe that the industry-wide talent level, or quality-of-work level, is lower in medicine than in private-sector software. In fact, I'd be shocked if it were the case. Look, there are some very smart people in our industry (as in theirs) but there are also a lot of idiots.
Even with the salary, the shitty hours, bureaucracy, student loans, and terrible customers kind of make it a wash.
The nice thing about software engineering is that we don't have to spend ten years getting educated enough to deal with customers that don't want to take our advice...we can basically start there.
I spend a lot of time thinking about big-O performance of my algorithm implementations. Not all the time, but the concern comes up frequently. If the job is just gluing framework bits and bobs together to stamp out CRUD apps, maybe we shouldn't be so surprised when low-skill programmers step up to take it?
I got caught up on that line as well, especially the authors use of "we". As far as I can tell, the author has certainly studied algorithms, but has never worked as a software engineer.
You're a 55-year old engineer who graduated from school in '81. You're being interviewed by a 24-year old engineer who graduated 2 years ago. You're asked, in an "algorithm interview", to explain some detail of the implementation of an AVL tree.
In reality, despite the fact that they're covered in CS courses, virtually nobody uses AVL trees in industry. In fact, people barely use balanced binary trees at all, and when they do, they use red-black trees, and they use someone else's implementation, because they're a bear to debug.
The 24-year old has an advantage with this question because they were recently taught about, and perhaps had to do exercises/homework/tests based on, AVL trees.
That this happens is stupid, because it's very unlikely that conversance with AVL trees is going to make much of a difference to your on-the-job performance. Almost all the narratives you can come up with about this involve reading tea leaves and are shot down easily by better selection/hiring/interviewing techniques that answer questions more directly.
This line of questioning can go too far; I don't, for instance, think knowing the big-O characteristics of an AVL tree is unreasonable (it's a balanced binary tree, and you should have the complexity of operations on those in resident set throughout your career).
(AVL trees are just the first example of algorithmic trivia that came into my head. Substitute Kruskal's Algorithm, or k-means clustering, or whatever,)
I think asking for particular details of an algorithm is pointless, and I wonder how many interviewers actually do that. The more common questions I've encountered in interviews are along the lines of "serialize a binary tree" where the data structures are basic but the problem has different possibly suboptimal ways of being solved. I'm curious if there really is an inherent and implicit bias to these sorts of questions that favors younger candidates. I'd be very sad if that's the case, as I'm more inclined to think other personal biases are in play.
You may think you're doing the candidate a favor by offering to handwave away the details, but the vaguer question might actually be worse. "Can you write a correct implementation of an AVL tree on this whiteboard, in any programming language, in the next thirty minutes" is a terrible interview question, but at least it has a factual answer. [1] "Can you riff on the subject of serializing a binary tree" is an open-ended question that will inevitably be graded on an imaginary curve. And the first rule of human psychology is: When you're asked to grade people on an imaginary curve, your intuition spits out a decision that is built on unconscious bias.
[1] edit: that is to say, it has a factual answer if you ask the question and then fall totally silent. Which would be evil, of course, so in practice you and the candidate will naturally turn this question into a back-and-forth repartee, and it will reduce to the same exercise of unconscious biases as the vague question is.
Well, the correct answer to questions like that is something like "I don't keep that stuff in my head, but the last time I had to do [foo], I looked at what kind of data was likely to be coming in, took down my copy of Knuth/CLR/Sedgewick, and decided that [bar] was likely to give good performance given that kind of data."
You won't get the job, but that's the correct answer nonetheless. :-)
I call this "the moonwalking fallacy". It's the problem where you're asked a dumb question, and you give the sensible non-answer that demonstrates the underlying competence the interviewer is ostensibly looking for, but you lose to the next candidate who has the exact answer they're looking for on tap. And they can moonwalk.
Interviews suck and we should all stop doing them as much as we possibly can. There are much better ways to handle this problem.
I like what you are onto here. Algorithm questions get universally panned or praised quite often, but there is a good deal of nuance in fact. Some are much, much better than others. Some are pure trivia.
I don't think most of us would bat an eyelash at the idea that most engineers need to know that a hash table is a O(c) look up, but it is an algorithm question that I've seen people fail. Conversely, your examples of what is a poor question are probably accurate in most domains.
"that most engineers need to know that a hash table is O(c) lookup"
O(1) is a leaky abstraction [1]. It's O(1) amortized assuming you have enough buckets and a good hashing algorithm that minimizes collisions. I think that distinction important to keep in your head, and good to mention/look for in interviews, since those details matter when things go awry.
I had an programmer that didn't know that. I went to the whiteboard, explained it, and now she is perfectly competent at it (and the rest of her job). What's the problem?
Where do we get the idea that if someone doesn't know some arbitrary fact that they are incapable of learning it? Hire people that are smart, have good work ethics, and play nicely with others, and everything else will take care of itself.
The last point is key, and overlooked by young insecure startup hirers too often. Hire smart people with good work ethics that play nice with others.
Having built technical teams numbering in the dozens this is literally the ONLY strategy that delivers long term success in my experience. Unfortunately experience is discounted nowadays...
I agree that AVL trees make a bad question. But if you're in an interview, you should just answer the question as best you can. If I was asking a question and someone tried to avoid it by changing the topic, I wouldn't feel threatened, but I would indicate in my feedback that they didn't answer the question.
"Why do you want to do that" might be appropriate when actually working, but not in an interview.
Starting to see how force of personality, self-confidence, and slickness play into interviews and determining who ends up with the best resume? And doesn't it suck how easily that cycle feeds itself? Slick interview -> high status job -> resume boost -> easier next interview -> another high status job.
I don't see how that relates to my post. If anything, the post I replied to was describing an attempt to pass the interview by force of personality and slickness, in that they were describing how to avoid answering the question the interviewer wanted to ask.
I'm not looking for personality, self-confidence or slickness. I just want a person to directly answer the question.
Yes, I understood what you were saying, and I was observing that even on a message board the question in the abstract was already a battle of wills.
I've done a lot of job interviews. It's not as simple as "candidate gave wrong answer, ding them and move on". If you're taking your job seriously, you're somewhat nervous too, and have to ask yourself if the answer they're giving is better than the one you're looking for. So now it's a debate, and now you're influenced by more than something you can look up in a book for the right answer.
But in my view there is no battle of wills. If the question is an algorithms type question, then there is no ambiguity, and either the interviewee answers it or they don't.
Now if I had asked for a good way to construct a map, and the answer had been that hash maps are good on paper but have high overhead, then I would have to consider their answer carefully. But if I ask how an AVL tree works and the answer is "why do you want to know that" then they have just avoided the question, and I have to insist they answer it.
Have you never had the experience of discovering that the answer you were looking for in an interview was not the best possible answer? You've never been surprised by a candidate?
If the question is an algorithms type question, then there is no ambiguity, and either the interviewee answers it or they don't.
Imagine that our interview goes like this:
---
Q: Can you implement a red-black tree?
A: I don't know. My Ph.D. is not in CS and I never actually seem to get around to reading the copies of Knuth and CLRS that I optimistically keep on my shelf. Nobody really wants to pay me to read about trees. So mostly I just use other people's trees, particularly the ones that are hiding inside Postgres and MySQL.
It would be fun to try to figure out how to implement a binary tree by asking you twenty questions -- it's one of my favorite party games -- but it will take a bit of time, and I'm in an interview and feeling nervous, so I'm not sure my odds of success are that high.
I did read on Wikipedia the other day that operations on balanced binary trees are O(log N) which makes intuitive mathematical sense, given that a branched structure with n levels will have 2^n leaves. So I guess balanced trees are pretty fast.
---
What do you do next?
I presume that, given that I've failed to answer the specific question, and demonstrated a degree of "slickness" and "force of personality" in the process, that you will ask me if I have any questions, shake my hand, send me out the door, inform your colleagues (including the one who referred me to your company) that I'm a NO HIRE because I couldn't remember the algorithm for constructing a red/black tree, and send me a polite rejection letter the next day?
Perhaps that strategy would have the virtue of being fair. But could you please get it over with during the phone screen? Better yet, could you make sure your recruiters never even email people who don't explicitly advertise CS degrees? Because I'd hate for you to waste a single minute thinking about me, and vice versa. Time is precious. If I find enough of it, I might even get around to reading Knuth!
i think many of us feel that questions that you can discuss is more meaningful and tells you more about the person being interviewed than strict algrorithmical question. In some cases those kind of questions only tell you if the other person has studied that exact question recently, nothing more.
I call those tests the airplane test. say you need to hire an airplane designer and you know d*ck all about aircraft design. You have two candidates both that tell you they have a new design being flow this week. So you go and watch, Candidate A's aircraft trundles down the runway and then crashes and burns. Candidate B's takes to the air and then lands. Candidate A could be a smooth talker and Candidate B could sound like a stuttering loon but B is the choice.
Much more importantly, trees are almost always the wrong answer (with a handful of exceptions). Hashmaps are the better data structure, because they are far more friendly to the processor cache. Consider a perfectly balanced tree with even 1k elements -- it will have height log2(1k) ~ 10, so approximately 11 reads, almost certainly from main memory, to access an element.
Exceptions are things like you have to iterate through elements in some order, a very high ratio of inserts/deletes to queries, etc -- the obvious stuff.
Much more importantly, vectors are almost always the wrong answer (with a handful of exceptions). Consider a perfectly balanced tree with even 1k elements -- it will have height log2(1k) ~ 10, so approximately 11 reads, almost certainly from main memory, to access an element.
Exceptions are things like you have to iterate through elements in some order, a very high ratio of inserts/deletes to queries, etc -- the obvious stuff.
This is the worst argument ever. You basically said, "trees are the wrong answer, except for when you want to do tree stuff." Want to have a priority queue? Or want to search through your collection without sorting it first? Or insert into your sorted collection? Trees are your friend. They're a tool, just like hashmaps.
When you quote things, it's customary to not modify the quote.
And no, that's not what I wrote. Trees are an associative data structure. Hashmaps are a better associative data structure, except when you need the few capabilities trees have that hashmaps don't.
ps -- if you run that quote through fold -s -w 77, it won't be so wide...
Whoops, I didn't mean to apply quote formatting to that. I DID mean to take the same argument and swap for vectors, but DIDN'T mean to make it look like your words.
My apologies. But, I maintain that trees have their uses. I would never use a hashmap to implement a priority queue, and i'd never use a tree to implement associative arrays!
Not a single interview that I've had that asked an obscure data structures question like this actually expected me to know how to implement the data structure. Saying "I don't know off the top of my head" has never cost me an interview. I could have just been lucky, of course.
Well, you're lucky. I was asked to do Bellman-Ford. I bombed that question, and was so shaken that I bombed the rest of the interview as well.
The irony was, I'd just spent the preceding 2 years writing multicast link-state routing code. I could have coded a decent C++ Bellman-Ford in a couple hours, but couldn't pull it out of my head on the spot in an interview without babbling. (I tried to dodge by describing link-state LSP forwarding and then Djikstra, but wasn't given credit for knowing the more sophisticated algorithm).
Algorithm interviews suck. We should stop doing them entirely.
Since the author ingratiatingly uses "we," but is actually a Public Policy student and Techcrunch writer, I predict this is part 2 in a maybe 5-part series where he ends up writing about tptacek's conclusions without mentioning him by name.
That said, it's interestingly telling to push "STE" as "secretly-terrible engineer" rather than "self-taught engineer." The author's ideal is for the negative case to be the soundbite, but then again: techcrunch.
He mentioned me by name in part 1 of this series, with a link to my post, and I didn't write anything about ageism. I'm pretty OK with the props I'm getting as is.
As we enter a more mature phase of the internet’s progress, it may be time for us to reconsider our strong bias toward the new. Maintainability and reliability are increasingly the edges needed to serve internet users accustomed to highly-reliable systems. New approaches to programming will still have their place, but they are entering a more crowded market where they really have to prove their value in order to be received.
We need to have a conversation about working with people of different experience and talent levels. We need to address our industry’s lust for shiny (i.e. young) over reliable. For in the process of that conversation, I believe that we will begin to address the odd inequities that plague our industry.
Wow.
And there exactly is a perfect example of the bias that older engineers complain about.
By using the same argument for older technologies and older engineers he conflates the two, implying that older engineers are really better off working on older tech.
Yes, this is not a good example of an intrinsic structural bias against older programmers in interviews. Plenty of older programmers also hopelessly captivated by the current NoSQL-du-jour. :)
If companies actually tracked the exact questions that are asked and how they are asked, then bias would be easy to spot. People they want to hire get softball binary tree or linked list questions. People they don't want to hire get disqualifier questions that are not in any textbook (and not in journals either) which definitely can't be white-boarded to their satisfaction in 40-45 minutes. I've gotten both. The latter I research afterwards to make sure that it's in none of the 30-40 textbooks I have or in any journals, ACM library, Dr Dobbs, etc.
The worst are interviewers who refuse to accept a correct answer. "Oops, I didn't expect the person to have the answer so I better make it more complicated." It's so ridiculous that it's hard to believe people would sink that low. Until you've experienced it though, you won't believe. I think it suffices to say anything that deals with distribution of money, which employment does, is going to have lots of bullshit.
Couldn't agree more. These type of questions/interviews are a total waste of time, and probably poison to most development teams.
I've been on more than one interview where I know that I've hit it out of the park, and I swear that the lead dev torpedoed my prospects out of fear that somebody might know enough to call him out on his BS.
Even in the best of scenarios, I think that this type of questioning has little value. Unless it is the most basic of questions to determine if the candidate actually knows the languages that they've listed on their resumes, what value is there in testing if someone knows something can be easily looked up and/or researched in a relatively short period of time if it was really important for the task at hand?
Please don't take my comments as minimizing the high level of detailed technical knowledge necessary for some tasks... but for most I would take a humble hacker who can quickly inhale new knowledge over the goofballs who end up doing these interviews, hands down.
I've never experienced that, although I once had someone give feedback that I didn't get hired because I didn't bring a notebook and a pen to take notes during the interview. I did bring a few extra copies of my resume, mind you. Just not a notebook.
Reading this, I couldn't help but think of why there is a shortage of programmers. I think a large part of it is the technical interview. So many places intend it to be a challenge, as if getting through it should be considered a right of passage. "We only hire the best."
No, you only hire people who apply, get through the process, and accept your offer. That's the best you can do.
People who interview a lot are going to be good at interviews. People that spend years at places will look at a job offer and see the process of proving themselves to new people. And that's a giant hurdle.
A part of me wants to do an interview, just to throw the interviewer a technical question back at them. I have a few good ones in my back pocket. When they start with whatever challenge they have, I stop them politely and say:
"Wait a minute. You invited me here, so before we go any further, I want to see if your company is worth my time."
And hit them with my question. Oh, that would be fun.
I wonder if, instead of focusing on the secretly terrible engineer, we focus on the secretly good engineer. This is the good engineer who you hire, despite your elaborate hiring process.
Your entire business could be staffed with SGE's, and you would never know it.
This is a good point. It would be useful to have a lower skilled, named role for a lot of the rote programming jobs that just involve gluing frameworks together.
I would like the difficult interviews if there were upside to them. Let's say that I'm a Level 6 engineer (on an abstract scale) and you're hiring me for a Level 7 position-- a small promotion to offset the risk of changing jobs. If the technical interview, I might fail (false negative). If that came with upside-- really nailing that machine learning brainteaser would make me eligible for Level 8 or 9 and the comp increase along with it-- then I'd be fine. My issue with the tough, multi-day interview processes is that there's often only downside. Why throw 4 hours into a company's take-home code test just to (possibly) get the same standard-issue, market-level offer I could get somewhere else?
A few years ago, I submitted a code sample that was ranked among the top 3 submissions ever-- I worked hard on that fucker-- only to receive a junior-level offer with <0.05% equity. So what was the point of doing the code sample?
On age and the quality of engineers, I generally find that older engineers are very good. That said, I'd be skeptical of someone who did "regular old" corporate engineering for 20 years. Why didn't he get out? I completely understand not wanting to move into management, but why didn't he look for an R&D job or move into an architectural role or try to become a consultant? (He may have good reasons, like wanting to stay in Ohio with his family, and therefore having fewer career options; I'd just try to figure out what they are.) Corporate engineering has a low ceiling and anyone good is going to have at least the desire (some may not have the ability, like someone tied to a geographical area) to break through it.
The cult of neoteny-- Agile and open-plan offices and the all of the other subtle signs of an age-discrimination/permanent-junior culture-- is somewhat of a self-fulfilling prophecy. Mainstream engineering, in most organizations, is a user-story ridden ghetto, a backlog-retrospective meeting that never ends. By 35 or so, the good engineers have either gotten out of it (possibly moving into R&D, data science, management, or independent consulting roles) or left the industry entirely. The culture of short-termism, open-plan offices, and business-driven engineering provides no exit. You don't get to escape it, in most companies, by being good at your job-- unless you become a manager.
What's happening is that tech's "thought leaders" are creating conditions that are hostile to older talent (which is necessary if you want your technology to be any good) with "Agile"/Scrum micromanagement and bullpen office layouts, then saying, "See, no one over 40 is any good". (In fact, there are a lot of good people over 40; they just don't want to work for jackasses.) This is congealing into a pernicious cultural assumption, based on shitty data, when we should actually be blaming the VCs and the mediocre middle-managers-I-mean-founders they fund instead of "old programmers".
So, I think that while there is certainly ageism going on, it cuts both ways. What is your advice to a "senior" engineer who, for whatever reason, ends up with a folks working under them that are markedly older than they are?
It's annoying, for example, to have good ideas or strong viewpoints written off as useful exuberance--and that happens too.
Bear in mind that agile was conceived by a bunch of veterans working on mega programs and was based on observation of what works best with mature teams
I've soured on work-sample tests. I noticed that I don't get an interview, even after I do them (and I used to take them seriously).
Not everyone can get an R&D job. Sometimes you have to get a job maintaining CRUD software, even when you're ridiculously overqualified. I don't see myself ever getting a job that doesn't suck unless I successfully bootstrap my own business.
So if qualified older engineers are undervalued, start a business, hire good people older than 40, and crush your competitors. One reason this does not happen in VC-funded startups is that the goal is to get something mediocre out quickly, hype it a lot, and then cash out.
1. Anything related to trees
2. Big-O trivia and errata
3. Reinventing the wheel
4. Bit-twiddling operations
Seriously. If I go into your interview and you ask me to reverse a string, and it's not <string>.reverse(), I'm not taking your stupid job. I don't care if you want me to know the tricks for an in-place string reverse with no temporary variables, that just shows that you don't have a clue how to interview.
Good interviewers? The best I've found so far:
1. Ask people to do real-life problems
2. No "gotcha" questions
3. Let people use the environment they'll be using for the job -- this means an IDE, the internet (including stack overflow), and any library they want.
What I want to see when I'm interviewing is whether or not you have the ability to solve a problem. I'm going to look at how you abstract and break down problems, how you solve them, and at what point you say, "I think this is done."
Yes, I expect good engineers to know when to use a tree or a list or set -- but if you're explicitly quizzing for that, you're probably filtering in more terrible developers than you're filtering out.
Google does none of what you prescribe for "good interviews" and they have interviewing down to a science. Yes they've admitted they err on false negatives, but that is acceptable because they have little to no false positives.
From what I gather you want an easy interview. What you wrote is impractical (though I agree gotcha questions can be a bit much).
As a candidate, I get 1-hour with you and I need to accomplish the following:
1. Does your resume reflect your experience?
2. Can you code?
3. What is it like to work with you to solve a challenge?
4. Sell you on joining us.
Real life problems may fit in an hour, but may not.
There are so many resources on programming interviews that spending an hour or two during the week prior would pay off dividends in a candidates success rate.
I interviewed with Google last year and the interviewer did two of the four things listed in killface's post. I actually do not agree with killface that those four things are necessarily bad (programmers should at least be able to make really good guesses at complexity, and know about trees), but at any rate, Google definitely had no fucking idea what my capabilities are as a developer after that interview, so incompetent was the interviewer.
It was actually one of the more terrible interviews I've been a part of, and I was not disappointed by the negative result. Google certainly believes they have interviewing down to a science, but I suspect that belief is itself not scientific, and instead just the usual bullshit and collective delusion.
Here's an often overlooked technique for older, self-taught engineers like me: study. Go get this book, The Algorithm Design Manual by Skienna. Also pick up a whiteboard, and a nice set of dry-erase markers.
Read the book, and do whatever exercises you can from the book on the whiteboard. Talk out loud, the way you will in the interviews. After a few weeks and 60 hours of doing this, you'll be ready to blow the minds of the people interviewing you.
When you go to your interview, you will bring your own markers. Then you don't have to deal with the fat, half dried out stuff you encounter during the interview. Also, warm up for an hour or two before you go to the interview, by doing problems from the book on the whiteboard, out loud. This helps you leave nothing to chance, and be ready for whatever they throw at you.
Yeah, it's a waste of time. But you have to play the game. Similarly, when they ask you why you left your last job, lie and tell them you'd accomplished all your goals there, got everything lined up and nailed down, and you're ready to make something happen somewhere else. Don't tell the truth if it's because your managers couldn't care less about doing what's best for the business. If you want to work at the circus, sometimes you gotta jump through some hoops. Big deal, you'll be ok!
Yes, the interview process is broken, but you can actually work at it and do well, even if you're old.
Buy this book even if you're not planning to take my advice and study. Even if you don't study, this should become one of your most treasured books. I myself was amazed at how many things turn out to be graph problems, and had a great time going through this book.
> Here's an often overlooked technique for [...] engineers like me: study.
This is fantastic advice, for everyone.
Preparing for interviews isn't wasted time, because ultimately the goal is to land a job that furthers your own personal goals in some fashion.
I wouldn't recommend lying. If you're leaving a job, you should stand by your convictions. In my market, it's small and everyone knows everyone, so claiming you nailed everything down is easy enough to verify. Just remember one simple truth: "You are the only common link between all your failed jobs."
The difference between "I nailed it all down" and "everybody between myself and the CEO was an idiot" may simply be a matter of perspective. Let's say both are true. Focus on the former, rather than the latter, in interviews. Be positive. Maybe that's a better way to phrase it, rather than saying "Lie." Heaven's no, never lie during a job interview; that simply won't do at all!
The best lies are the lies that can't be in principle disproven. I try to be reasonably honest most of the time, but if the question is something like "why did you leave" I'm going to give the "right" answer, not the honest one.
Instead, lawyers "speak colorably"; If one takes the pure white light of truth, and puts it through a prism, all of those colors in the spectrum of truth, while not 100% of the truth, still contain a very strong thread of it.
for example, I was in court recently settling a criminal charge for someone about two months ago, and that someone asked the crown prosecutor (during the hearing) why the crown has gone silent on the civil process settling the charges. The crown, unable to lie, or even acknowledge a commercial default, stated colorably that the paperwork was "not necessarily valid" in a criminal case. not only was this true, but this kept the prosecution in honor, the court in honor, and the defendant in honor, all while allowing him his "Section 11" remedy (canada criminal code).
I'd suggest using colorable language any time you feel the need to lie.
TL:DR - In the eyes of the law, telling a spectrum of truth is still considered telling the truth.
Implement algos in a new language. Last time I went thru CLR using Haskell. I might do Go or Rust this time and contribute some algos to the community.
I think algorithmic interviews are good, because they are a kind of intelligence test. In spite of liberal dogma, some people are more talented and intellegent than others, and tests can shed light on this. However, the article does expose some flaws. Here are two bad kind of algorithm questions
1) Questions that rely too much on background knowledge. Knowing the exact properties or implementation of the more complex algorithms in the canon of undergraduate CS knowledge is not a particularly useful skill in itself.
2) Questions that have some trick that you either get or don't get. These aren't always terrible, but they give a big advantage to someone who has seen the question before.
So what is a good algorithm based question?
1) Implementing something basic but non-trivial, like N-Queens with backtracking. Basically, something anyone should be able to do.
2) Something that mixes in domain knowledge. E.g. problems calculating probabilities than can be solved with dynamic programming.
3) Pure domain knowledge, but in programming form. E.g. how to implement k-means clustering.
The main problem I see with technical interviews is that they are based on an assumption of individual technical knowledge adding value to an organisation.
Working instead from an assumption that the whole is greater than sum when it comes to groups of people - I'd be far more interested in what a candidate is like to work with and what unique qualities they bring that we don't already have and which can act as a multiplier for the organisation.
There's definitely inherent bias in much of the interviewing process, whether deliberate or not, for many tech startups. But there's also a high degree of bias once inside them. Older engineers with families often can't really do a team bonding that involves Saturday night playing games getting drunk and stoned, for example. Many startups, as they grow, provide things like health insurance plans optimized for 25 year old healthy males, that are just not that good for families and make it harder on older employees. And there are a lot of younger engineering managers who feel uncomfortable leading teams with older more experienced members in them. I've seen teams grow and older engineers pushed out for fairly arbitrary reasons, despite being good coders. And yes, even in this crazy environment where it's hard to hire.
(And yes when I was young and early in my career I never really paid much attention to it and shrugged it off, but now it's my friends and I see myself getting up there in age now, it is so very obvious, and a lot more prevalent than I ever noticed.)
When someone has nearly 20 years engineering experience (not talking about me, I don't) it is ridiculous to make them jump through some of these interviewing hoops. Sure, some kid fresh out of college might pad his resume - because there's not much to it. But an older veteran with a long resume, not so much. Experience does count. The ins and outs of the latest technology might vary slightly, but the patterns repeat. So much of engineering goes beyond the code on the screen anyway.
71 comments
[ 7.7 ms ] story [ 307 ms ] threadLook, we really really really don't want to be taking anything from those folks. The hierarchies are rubbish, the talent questionable, and the egos immense. At least in software we can show that code does or doesn't work irrespective of who wrote it.
You don't want a doctor's salary? And her level of respect from society at large?
I also don't believe that the industry-wide talent level, or quality-of-work level, is lower in medicine than in private-sector software. In fact, I'd be shocked if it were the case. Look, there are some very smart people in our industry (as in theirs) but there are also a lot of idiots.
The nice thing about software engineering is that we don't have to spend ten years getting educated enough to deal with customers that don't want to take our advice...we can basically start there.
Do doctors have the problem that the stuff they worked on 5-10 years ago is considered obsolete now?
You're a 55-year old engineer who graduated from school in '81. You're being interviewed by a 24-year old engineer who graduated 2 years ago. You're asked, in an "algorithm interview", to explain some detail of the implementation of an AVL tree.
In reality, despite the fact that they're covered in CS courses, virtually nobody uses AVL trees in industry. In fact, people barely use balanced binary trees at all, and when they do, they use red-black trees, and they use someone else's implementation, because they're a bear to debug.
The 24-year old has an advantage with this question because they were recently taught about, and perhaps had to do exercises/homework/tests based on, AVL trees.
That this happens is stupid, because it's very unlikely that conversance with AVL trees is going to make much of a difference to your on-the-job performance. Almost all the narratives you can come up with about this involve reading tea leaves and are shot down easily by better selection/hiring/interviewing techniques that answer questions more directly.
This line of questioning can go too far; I don't, for instance, think knowing the big-O characteristics of an AVL tree is unreasonable (it's a balanced binary tree, and you should have the complexity of operations on those in resident set throughout your career).
(AVL trees are just the first example of algorithmic trivia that came into my head. Substitute Kruskal's Algorithm, or k-means clustering, or whatever,)
[1] edit: that is to say, it has a factual answer if you ask the question and then fall totally silent. Which would be evil, of course, so in practice you and the candidate will naturally turn this question into a back-and-forth repartee, and it will reduce to the same exercise of unconscious biases as the vague question is.
You won't get the job, but that's the correct answer nonetheless. :-)
Interviews suck and we should all stop doing them as much as we possibly can. There are much better ways to handle this problem.
O(1) is a leaky abstraction [1]. It's O(1) amortized assuming you have enough buckets and a good hashing algorithm that minimizes collisions. I think that distinction important to keep in your head, and good to mention/look for in interviews, since those details matter when things go awry.
[1] http://www.joelonsoftware.com/articles/LeakyAbstractions.htm...
I had an programmer that didn't know that. I went to the whiteboard, explained it, and now she is perfectly competent at it (and the rest of her job). What's the problem?
Where do we get the idea that if someone doesn't know some arbitrary fact that they are incapable of learning it? Hire people that are smart, have good work ethics, and play nicely with others, and everything else will take care of itself.
Having built technical teams numbering in the dozens this is literally the ONLY strategy that delivers long term success in my experience. Unfortunately experience is discounted nowadays...
* Tell me more about the problem are you trying to solve.
* There are well tested and tuned libraries for search trees, like the JVM TreeMap. What requirement prevents us from using it?
* The JVM TreeMap uses RedBlack trees, which are basically BTrees, and likely easier / faster to implement than AVL.
Now, you might not get the job because the dude might feel threatened. You just dodged a bullet, the job would have been shitty anyway.
"Why do you want to do that" might be appropriate when actually working, but not in an interview.
I'm not looking for personality, self-confidence or slickness. I just want a person to directly answer the question.
I've done a lot of job interviews. It's not as simple as "candidate gave wrong answer, ding them and move on". If you're taking your job seriously, you're somewhat nervous too, and have to ask yourself if the answer they're giving is better than the one you're looking for. So now it's a debate, and now you're influenced by more than something you can look up in a book for the right answer.
Now if I had asked for a good way to construct a map, and the answer had been that hash maps are good on paper but have high overhead, then I would have to consider their answer carefully. But if I ask how an AVL tree works and the answer is "why do you want to know that" then they have just avoided the question, and I have to insist they answer it.
Imagine that our interview goes like this:
---
Q: Can you implement a red-black tree?
A: I don't know. My Ph.D. is not in CS and I never actually seem to get around to reading the copies of Knuth and CLRS that I optimistically keep on my shelf. Nobody really wants to pay me to read about trees. So mostly I just use other people's trees, particularly the ones that are hiding inside Postgres and MySQL.
It would be fun to try to figure out how to implement a binary tree by asking you twenty questions -- it's one of my favorite party games -- but it will take a bit of time, and I'm in an interview and feeling nervous, so I'm not sure my odds of success are that high.
I did read on Wikipedia the other day that operations on balanced binary trees are O(log N) which makes intuitive mathematical sense, given that a branched structure with n levels will have 2^n leaves. So I guess balanced trees are pretty fast.
---
What do you do next?
I presume that, given that I've failed to answer the specific question, and demonstrated a degree of "slickness" and "force of personality" in the process, that you will ask me if I have any questions, shake my hand, send me out the door, inform your colleagues (including the one who referred me to your company) that I'm a NO HIRE because I couldn't remember the algorithm for constructing a red/black tree, and send me a polite rejection letter the next day?
Perhaps that strategy would have the virtue of being fair. But could you please get it over with during the phone screen? Better yet, could you make sure your recruiters never even email people who don't explicitly advertise CS degrees? Because I'd hate for you to waste a single minute thinking about me, and vice versa. Time is precious. If I find enough of it, I might even get around to reading Knuth!
This is why I believe that you, the interviewer, cannot interview someone above your "experience level" without some kind of objective test.
"Work sample test" is the traditional name for what you describe.
Exceptions are things like you have to iterate through elements in some order, a very high ratio of inserts/deletes to queries, etc -- the obvious stuff.
And no, that's not what I wrote. Trees are an associative data structure. Hashmaps are a better associative data structure, except when you need the few capabilities trees have that hashmaps don't.
ps -- if you run that quote through fold -s -w 77, it won't be so wide...
My apologies. But, I maintain that trees have their uses. I would never use a hashmap to implement a priority queue, and i'd never use a tree to implement associative arrays!
The irony was, I'd just spent the preceding 2 years writing multicast link-state routing code. I could have coded a decent C++ Bellman-Ford in a couple hours, but couldn't pull it out of my head on the spot in an interview without babbling. (I tried to dodge by describing link-state LSP forwarding and then Djikstra, but wasn't given credit for knowing the more sophisticated algorithm).
Algorithm interviews suck. We should stop doing them entirely.
That said, it's interestingly telling to push "STE" as "secretly-terrible engineer" rather than "self-taught engineer." The author's ideal is for the negative case to be the soundbite, but then again: techcrunch.
We need to have a conversation about working with people of different experience and talent levels. We need to address our industry’s lust for shiny (i.e. young) over reliable. For in the process of that conversation, I believe that we will begin to address the odd inequities that plague our industry.
Wow.
And there exactly is a perfect example of the bias that older engineers complain about.
By using the same argument for older technologies and older engineers he conflates the two, implying that older engineers are really better off working on older tech.
Though I think Stonebraker's VoltDB supports SQL as well.. ;)
The worst are interviewers who refuse to accept a correct answer. "Oops, I didn't expect the person to have the answer so I better make it more complicated." It's so ridiculous that it's hard to believe people would sink that low. Until you've experienced it though, you won't believe. I think it suffices to say anything that deals with distribution of money, which employment does, is going to have lots of bullshit.
I've been on more than one interview where I know that I've hit it out of the park, and I swear that the lead dev torpedoed my prospects out of fear that somebody might know enough to call him out on his BS.
Even in the best of scenarios, I think that this type of questioning has little value. Unless it is the most basic of questions to determine if the candidate actually knows the languages that they've listed on their resumes, what value is there in testing if someone knows something can be easily looked up and/or researched in a relatively short period of time if it was really important for the task at hand?
Please don't take my comments as minimizing the high level of detailed technical knowledge necessary for some tasks... but for most I would take a humble hacker who can quickly inhale new knowledge over the goofballs who end up doing these interviews, hands down.
I actually laughed.
No, you only hire people who apply, get through the process, and accept your offer. That's the best you can do.
People who interview a lot are going to be good at interviews. People that spend years at places will look at a job offer and see the process of proving themselves to new people. And that's a giant hurdle.
A part of me wants to do an interview, just to throw the interviewer a technical question back at them. I have a few good ones in my back pocket. When they start with whatever challenge they have, I stop them politely and say:
"Wait a minute. You invited me here, so before we go any further, I want to see if your company is worth my time."
And hit them with my question. Oh, that would be fun.
Your entire business could be staffed with SGE's, and you would never know it.
A few years ago, I submitted a code sample that was ranked among the top 3 submissions ever-- I worked hard on that fucker-- only to receive a junior-level offer with <0.05% equity. So what was the point of doing the code sample?
On age and the quality of engineers, I generally find that older engineers are very good. That said, I'd be skeptical of someone who did "regular old" corporate engineering for 20 years. Why didn't he get out? I completely understand not wanting to move into management, but why didn't he look for an R&D job or move into an architectural role or try to become a consultant? (He may have good reasons, like wanting to stay in Ohio with his family, and therefore having fewer career options; I'd just try to figure out what they are.) Corporate engineering has a low ceiling and anyone good is going to have at least the desire (some may not have the ability, like someone tied to a geographical area) to break through it.
The cult of neoteny-- Agile and open-plan offices and the all of the other subtle signs of an age-discrimination/permanent-junior culture-- is somewhat of a self-fulfilling prophecy. Mainstream engineering, in most organizations, is a user-story ridden ghetto, a backlog-retrospective meeting that never ends. By 35 or so, the good engineers have either gotten out of it (possibly moving into R&D, data science, management, or independent consulting roles) or left the industry entirely. The culture of short-termism, open-plan offices, and business-driven engineering provides no exit. You don't get to escape it, in most companies, by being good at your job-- unless you become a manager.
What's happening is that tech's "thought leaders" are creating conditions that are hostile to older talent (which is necessary if you want your technology to be any good) with "Agile"/Scrum micromanagement and bullpen office layouts, then saying, "See, no one over 40 is any good". (In fact, there are a lot of good people over 40; they just don't want to work for jackasses.) This is congealing into a pernicious cultural assumption, based on shitty data, when we should actually be blaming the VCs and the mediocre middle-managers-I-mean-founders they fund instead of "old programmers".
Nah, scratch that. You're too honest for politics.
It's annoying, for example, to have good ideas or strong viewpoints written off as useful exuberance--and that happens too.
Not everyone can get an R&D job. Sometimes you have to get a job maintaining CRUD software, even when you're ridiculously overqualified. I don't see myself ever getting a job that doesn't suck unless I successfully bootstrap my own business.
So if qualified older engineers are undervalued, start a business, hire good people older than 40, and crush your competitors. One reason this does not happen in VC-funded startups is that the goal is to get something mediocre out quickly, hype it a lot, and then cash out.
1. Anything related to trees 2. Big-O trivia and errata 3. Reinventing the wheel 4. Bit-twiddling operations
Seriously. If I go into your interview and you ask me to reverse a string, and it's not <string>.reverse(), I'm not taking your stupid job. I don't care if you want me to know the tricks for an in-place string reverse with no temporary variables, that just shows that you don't have a clue how to interview.
Good interviewers? The best I've found so far:
1. Ask people to do real-life problems 2. No "gotcha" questions 3. Let people use the environment they'll be using for the job -- this means an IDE, the internet (including stack overflow), and any library they want.
What I want to see when I'm interviewing is whether or not you have the ability to solve a problem. I'm going to look at how you abstract and break down problems, how you solve them, and at what point you say, "I think this is done."
Yes, I expect good engineers to know when to use a tree or a list or set -- but if you're explicitly quizzing for that, you're probably filtering in more terrible developers than you're filtering out.
Google does none of what you prescribe for "good interviews" and they have interviewing down to a science. Yes they've admitted they err on false negatives, but that is acceptable because they have little to no false positives.
From what I gather you want an easy interview. What you wrote is impractical (though I agree gotcha questions can be a bit much).
As a candidate, I get 1-hour with you and I need to accomplish the following:
1. Does your resume reflect your experience?
2. Can you code?
3. What is it like to work with you to solve a challenge?
4. Sell you on joining us.
Real life problems may fit in an hour, but may not.
There are so many resources on programming interviews that spending an hour or two during the week prior would pay off dividends in a candidates success rate.
It was actually one of the more terrible interviews I've been a part of, and I was not disappointed by the negative result. Google certainly believes they have interviewing down to a science, but I suspect that belief is itself not scientific, and instead just the usual bullshit and collective delusion.
Read the book, and do whatever exercises you can from the book on the whiteboard. Talk out loud, the way you will in the interviews. After a few weeks and 60 hours of doing this, you'll be ready to blow the minds of the people interviewing you.
When you go to your interview, you will bring your own markers. Then you don't have to deal with the fat, half dried out stuff you encounter during the interview. Also, warm up for an hour or two before you go to the interview, by doing problems from the book on the whiteboard, out loud. This helps you leave nothing to chance, and be ready for whatever they throw at you.
Yeah, it's a waste of time. But you have to play the game. Similarly, when they ask you why you left your last job, lie and tell them you'd accomplished all your goals there, got everything lined up and nailed down, and you're ready to make something happen somewhere else. Don't tell the truth if it's because your managers couldn't care less about doing what's best for the business. If you want to work at the circus, sometimes you gotta jump through some hoops. Big deal, you'll be ok!
Yes, the interview process is broken, but you can actually work at it and do well, even if you're old.
Buy this book even if you're not planning to take my advice and study. Even if you don't study, this should become one of your most treasured books. I myself was amazed at how many things turn out to be graph problems, and had a great time going through this book.
[1] http://www.amazon.com/Algorithm-Design-Manual-Steven-Skiena/...
This is fantastic advice, for everyone.
Preparing for interviews isn't wasted time, because ultimately the goal is to land a job that furthers your own personal goals in some fashion.
I wouldn't recommend lying. If you're leaving a job, you should stand by your convictions. In my market, it's small and everyone knows everyone, so claiming you nailed everything down is easy enough to verify. Just remember one simple truth: "You are the only common link between all your failed jobs."
Curious what the "right" answer is?
Instead, lawyers "speak colorably"; If one takes the pure white light of truth, and puts it through a prism, all of those colors in the spectrum of truth, while not 100% of the truth, still contain a very strong thread of it.
for example, I was in court recently settling a criminal charge for someone about two months ago, and that someone asked the crown prosecutor (during the hearing) why the crown has gone silent on the civil process settling the charges. The crown, unable to lie, or even acknowledge a commercial default, stated colorably that the paperwork was "not necessarily valid" in a criminal case. not only was this true, but this kept the prosecution in honor, the court in honor, and the defendant in honor, all while allowing him his "Section 11" remedy (canada criminal code).
I'd suggest using colorable language any time you feel the need to lie.
TL:DR - In the eyes of the law, telling a spectrum of truth is still considered telling the truth.
Algorithms by Dasgupta, Papadimitriou and Vazirani:
http://www.amazon.com/Algorithms-Sanjoy-Dasgupta/dp/00735234...
And Elements of Programming Interviews by Aziz, Lee and Prakash:
http://www.amazon.com/Elements-Programming-Interviews-Inside...
1) Questions that rely too much on background knowledge. Knowing the exact properties or implementation of the more complex algorithms in the canon of undergraduate CS knowledge is not a particularly useful skill in itself.
2) Questions that have some trick that you either get or don't get. These aren't always terrible, but they give a big advantage to someone who has seen the question before.
So what is a good algorithm based question?
1) Implementing something basic but non-trivial, like N-Queens with backtracking. Basically, something anyone should be able to do.
2) Something that mixes in domain knowledge. E.g. problems calculating probabilities than can be solved with dynamic programming.
3) Pure domain knowledge, but in programming form. E.g. how to implement k-means clustering.
Working instead from an assumption that the whole is greater than sum when it comes to groups of people - I'd be far more interested in what a candidate is like to work with and what unique qualities they bring that we don't already have and which can act as a multiplier for the organisation.
(And yes when I was young and early in my career I never really paid much attention to it and shrugged it off, but now it's my friends and I see myself getting up there in age now, it is so very obvious, and a lot more prevalent than I ever noticed.)
When someone has nearly 20 years engineering experience (not talking about me, I don't) it is ridiculous to make them jump through some of these interviewing hoops. Sure, some kid fresh out of college might pad his resume - because there's not much to it. But an older veteran with a long resume, not so much. Experience does count. The ins and outs of the latest technology might vary slightly, but the patterns repeat. So much of engineering goes beyond the code on the screen anyway.