I took the pill (what color were you supposed to take again?). I'm prepping for the leetcode interviews. If my opinion counts for anything, don't change the interview format now. How will i get my time back that I've spent learning how to use hashmaps with java-specific syntax? My notebooks full of pseudocode are waiting to be converted into a leetcode-time-limit friendly programming language. Dont tell me it's actually Python, not Java. A man can only stoop so low.
Begrudgingly accepting the mainstream way of doing things sounds a bit more in line with the blue pill than the red pill to me, idk. Though I guess realizing the silliness of the whole situation would give it a bit of a purplish tint maybe.
When taking into account the total amount of time involved in a traditional multi-round interview process (which can potentially go on for months depending on the company), the travel time required for any in-person interview rounds, and the time spent practicing leetcode drills in preparation for these interviews, 10 hours is nothing.
Like interviews aren't lengthy enough as-is. I had an onsite the other week with two sys design, two leetcode-esque, a hiring manager, and a project manager.
I'd take that any day over the take home project I had to do in a subset of common LISP fresh out of college. Twas NOT a fun weekend, especially since I had no familiarity with LISP. With few exceptions, I don't do takehomes anymore. I don't want to spend an hour or so fuddling with your/your vendor's poorly documented, buggy environment (or setting it up myself) on my own time, outside of office hours, talking to exactly zero human beings during the actual process. It teaches me nothing about the company or team I'd be working on, doesn't allow for any gauge of culture fit, and (unlike leetcode) doesn't make me a better programmer.
I see this complaint a lot, but 10 hours is crazy - I've been involved in setting, reviewing the results for, and doing such tests for many years and the longest I've ever expected anyone to take is 2 hours, which is more than enough to demonstrate whether you have at least basic capabilities as a programmer.
As it is in the "real world" my biggest problem with other devs is for them to spend days on a story, then finally put up a PR that shows they really didn't understand the requirements well or make any attempt to clarify them. It's actually pretty rare I'd ever expect a dev to be able to take a story away and put anything like 10 hours time into without needing to discuss it first, so why would I expect that for a take home test?
Literally any other process is better, in my experience. Short take-home exercises. A system design exercise around a whiteboard. Talking the interviewer through some previously-written or open-source code. Pairing up to fix a bug in the org’s code during the interview.
Leetcode-style problems as an interview technique are probably the one that’s furthest away from testing the from day-to-day software development challenges most organisations face.
As others have said, open to extreme abuse in terms of timeliness.
> A system design exercise around a whiteboard
I've worked with a frankly extreme number of architects who can talk the talk but cannot (or will not) code the code.
If you're hiring for someone to code the code system design isn't the signal you want.
> Pairing up to fix a bug in the org’s code during the interview
Open to cheating. Very difficult and time consuming to rotate the problem. With Leetcode you can take the problem out of a college textbook.
> Talking the interviewer through some previously-written or open-source code.
Works well for small companies where you're hiring someone who fits a team well. Probably the best approach for small companies.
For big companies, it's open to extreme racial and personal bias.
And I don't mean racism in an abstract sense. I mean that I've interviewed with a company that pairs with a particular coding exercise across all candidates.
They asked me back whilst knocking back my INFINITELY more qualified good friend who was interviewing with the same company.
We walked through the same code base, I'm a LOT less qualified than they are. We got far different feedback, to the extent I genuinely can only assume unspoken racism.
Often enough someone gives a take home project that actually is not even close to be capped in the time range requested unless you already work on this project.
Every such test is supposed to be calibrated. Leetcode... Is not.
Be that as it may, it will never even come close to the 100s of hours of leetcode grinding.
It's not even the same order of magnitude. Like I mentioned, it is common to spend months grinding leetcode. How can people rationalize this and prefer it to a closed scope assignment? I honestly don't get it.
I've personally learnt a lot from Leetcode, and found the process really valuable. I learnt a ton from bashing my head against the wall until dynamic programming made sense.
I anecdotally code much, much faster, in all languages, having done it. It's like doing college homework again. Something useful that makes you a better "coder," (not necessarily a better engineer). It was a damn hard slog, but having done the hard slog I really enjoy having done it.
I'm not going to pretend it's not time consuming or I'd be happy doing it if I was a busy parent.
But in hindsight I've really enjoyed having done it, and found it extremely valuable.
I suppose it's for people who just have that kind of time/energy/enthusiasm for programming/their career.
Even amongst the "top tier" companies, there are alternatives. Stripe essentially does pair programming on things you might actually do in your day job, using your own dev environment/etc.
That said, I'm partial to good leetcode questions. Some of them are worded terribly, some of them are essentially trivia, some of them are overly esoteric and not applicable to most roles. Some of them are great but have "optimal" solutions which take an absurd amount of research and effort to produce, like quickselect. These questions are bad and should not be asked.
Others I find fantastic questions. "Scheduling meetings", anything that requires you to walk a tree or do a binary search/BFS/DFS, and tasks that require you to combine two data structures like the "Hashmap with O(1) get random" question really helped me get better t my job.
There's others I'm lukewarm on, like many topological traversal, shortest/longest path, and knapsack/space filling questions. I personally love them and have gotten good at them, but I also don't expect joe schmo to remember how to implement the relevant algorithms off the top of their head. I'd consider this question cohort good for "open book" interviewing at "less prestigious" companies. I'm totally fine with Google/Meta/Amazon asking these closed-book for normal roles.
Simply having them work through a basic problem out loud as they code it with a decent engineer that understands the problem is much better than Leetcode.
While I'm not in favor of LeetCode-style interviews, I recently had a fairly pleasant interview process with a company that used these coding challenges. The twist was that they were all camouflaged as scenarios that resemble their own business, instead of some random unrelated concept. I found that their effort to make the questions look relevant made it much more pleasant for me to actually feel engaged and solve them.
I did get an offer from them though, not sure how I would have felt if rejected.
That's exactly how I frame my technical interviews as well. I only pick questions which I could tie to an actual, real-world problem that we've had.
A couple for example:
1. Given a set of number ranges (e.g. [1..3, 4..5, 9..20, 2..5]) consolidate them. Tying this to a real world scenario: we had a system that dealt with "time periods" and we needed to write a validator to ensure all future dates were covered by at least one configured "time period." One of the first steps to do this was basically the number-range consolidation.
2. Given a list of numbers output metrics like average, min, max, and mode. Now what if the list doesn't fit in memory and instead comes in as a stream (or from an iterator)? This one was inspired by an intern that got stuck when he needed to run calculations on data that didn't fit into memory. He was seriously about to request a host with 10+TB of memory before we stopped him.
On 1, you don’t really have to consolidate them just to check if the whole range is covered. You can just sort beginnings and ends together and then iterate over them with some tmp variable to check that you always have at least one beginning “open” (n log n).
The problem is that leetcode in itself is not a calibrated test. So, given any particular leetcode score, assuming the person is there, tells you literally nothing about skills involved.
Big corporations do not use it, and they would be most likely to have actual also likely untranslatable employment data.
Then there are specifics of any job - knowledge of particular system or rapid learning and experimentation skills tend to trump optimization and algorithmic knowledge. Leetcode does not really test any of that.
It is also particularly bad at weeding out cheaters. :)
Googles interviews works according to their internal data.
Google bans any questions that you can find on the internet though, including all questions you find on leetcode (they have people who check this, including the member only questions), they want to ensure that candidates only do problems they haven't seen before. Some accidents still happens, but since they have 5 interviews per candidate it is very unlikely for anyone to pass without solving a few problems they haven't seen before.
So the process works when the candidate hasn't seen the exact problems before. For those who just gives leetcode questions verbatim it might be less effective, or even counter productive, I haven't seen any data on that.
Honestly I think leetcode should be reserved for the jobs where you build the Lego bricks (the MANGAs).
Anywhere else where you use the Lego bricks to build some applications shouldn’t really use leetcode.
If you’re a good product engineer and you enjoy building good products, maybe you should start a company to build good products that solve problems for other people. Let the engineers who enjoy building the bricks do leetcode in peace so that we have solid pieces to build good software.
I’m doing the latter and trying to build a company. I realized I much prefer to build products that customers love, and I want to build useful software for people. I’m not that interested in low level tinkering and re-inventing the wheel all the time.
The Chrome browser and various protocols (SPDY/HTTP2 and others though I need to do some research to know exactly which ones). They've also invested a lot in optimizing the JS engine. The Go language, so indirectly contributed to any tool built in Go.
Happily employed and had no trouble with coding interviews myself. So my opinion isn’t from some personal angst about passing this bar: I found it easy.
That said: I’m tired of the attitude that there is no effective alternative to this style of interview. You do know we hired people to do engineering jobs for many decades before the industry started cargo culting what they observed a handful of Silicon Valley mega-companies doing. Plenty of very competent people were interviewed and hired before this “relive your algorithms class final on a whiteboard” style interview process became the thing to do.
These interview processes are not the only way. As I said: it’s cargo culting where insignificant companies try to act like Google or some other giant in hopes that perhaps the billions in revenue will start rolling in if they mimic them. To me this behavior is no different than “founders” who adopted dress styles and presentation mannerisms imitating Steve Jobs a decade or so ago. …And bloggers who can’t resist the “considered harmful” trope.
Well argued, but you didn't address the need for tests and/or certifications to prevent companies from being overwhelmed by a flood of unqualified candidates.
A doctor, lawyer, public accountant, and, well, many types of engineers obtain public or private certifications. As a result, it is much, much, much easier for hiring companies to find and interview qualified candidates.
I suppose, in the "old days" universities kind of, sort of, filled this role. A hiring manager looking at a resume might have said, "Oh, he's got an EE/CS from Berkeley, there's a good chance he's qualified.
That's because programming is nothing like any of those professions. How many certificates do you see for musicians, painters, photographers? Programming as at least as much an art as it is anything else and art is very hard to quantify in the types of tests that most certificate systems use. It requires a project based certificate which needs more organization that exists today. IMO it'd require a guild like setup that I don't see happening anytime soon.
On the other hand, open source culture is vastly different from proprietary culture.
So many times in proprietary culture I've churned out craptastic code to meet a deadline that didn't matter for a project whose customers were straw-persons. There's definitely some projects or features I've coded which I'm glad to hide from public view.
That said, artists can curate their portfolio, and unless they're presenting to a public forum, their turds won't have to see the light of day again.
Various non-software engineering disciplines implement chartered engineer schemes. I imagine there's an element of artistry there too, but they seem to cope.
Big difference is that there are standards and accepted practices in the fields in detail.
Nothing like that exists in programming, not even languages or platforms. Which is why there exist multiple proprietary certifications instead focusing on specific details.
Especially since the entire industry is constantly hiking on the St. James to find its inner self. Philosophies change and we notice tidbits that are good, tidbits that are bad and keep changing and renewing...
Almost like fashion. At least in Europe, young people dress like a Stranger Things inspired mashup of 70s, 80s and 90s nowadays picking the best parts of those decades while leaving the 2000s and 2010s fashion in the dust.
Are you gonna do a certificate on microservices? And what's that gonna help when suddenly everyone starts building some shape of monoliths again because people keep running into the "no free lunch" issues?
Except that would be like asking say a piano performer to run Liszt etudes mechanically, when you're hiring someone to play at your wedding.
Completely different skills. Sometimes, you do need someone with these skills. These times are few.
Often, a person forced to train to this test will neglect other skills or be unable to match top tier performance in the weird test.
Some programmers are like wedding singers, some play for a global audience.
If you hire a piano player for large symphonic orchestra, or a famous band, and they are not already famous, you want to hear them play. Maybe one piece they chose themselves, but maybe also a piece everyone in the genre is expected to know.
And if they have to improvize a bit, that may tell you a lot, too:
Sometime you meet programmer like this, than you can jam with on almost any topic. Those that can repeatedly improvize their way to "leetcode" solutions, while you observe their thinking process.
The "problem" with "leetcode" is that it has become too common and formalized. When people start to practice leetcode just to land the job, it suck the fun out of it, and it becomes harder to tell if the candidate is able to solve similar, but novel problems.
Nah. There is a longstanding tradition of competitions in classical music, especially for piano players. The vast majority of concert pianists got some notoriety through such competitions. And no, improvisation skills are not something classical pianists are judged upon.
Orchestra musicians indeed are routinely hired via a process that involves auditions, but my understanding is that applies only to those at the beginnings of their careers.
Popular/rock/jazz band formation process IMO is indeed similar to how start-ups are created (i.e. rather informal process generally based on networking, though audition jams also happen to fill a specific position). I fail to see how corporate staffing is similar.
My former teacher (he's been a concertmaster of a major city symphony for years, and is also well versed in the different ways in which one can ask "Did you actually practice?") wrote this explanation:
Think of it as management and labor. Management is Executive Director, Music Director, Principal Guest, staff, etc. Labor is orchestra musicians. Then you have the Board, which is there to raise money. Then you have the volunteers. Bear in mind, orchestras are non-profit, 501(c)(3).
So between management and musicians there is a CBA, or collective bargaining agreement. Just like the Major League Baseball has one between players and owners. In the CBA, you'll find chapters on hiring and firing. Each Symphony has its own CBA.
Usually, hiring is done by auditions behind a screen. If you're hired, your first year is the probationary period (sometimes two years). At the end of it, if management deems you fit for the job, you are granted tenure, just like at a university when you're applying for a teaching position. You don't need to re-audition, unless it is for a different title (from section to title chair, etc.), in which case you do. Firing, if it must come down to it, is also explained in the CBA.
The big symphonies are part of AFM (American Federation of Musicians). Then you have regional orchestras, community orchestras, etc. Some might be unionized, some not, but essentially it sort of works the same way.
Some concertmasters, like in England, are contracted outside the CBA, so they may be on a 3 or 4 yr contract, or more. But I think it the US, most are part of the CBA.
Most arts have a portfolio, not an audition. Auditions are specific to the performing arts which programming is definitely not. Musically I'd equate programming more to composition, not performance as there is no audience.
Well architects designing large systems through and through are maybe like composers
Your run of the mill dev would be more like a studio musician hired to play a specific instrument in a specific genre.
Surely, he/she needs to be able to come up with lines/hooks/riffs/solos (when asked) and play part in the arrangement but composing from scratch and for multiple instruments would be rather out of scope.
While there's no direct audience for studio musicians (only the output matters) there are definitely performance expectations and the studio musicians are expected to be efficient.
> If programming is an art, then the programmer should have an audition instead of an interview. Which is approximately what leetcode is.
Except that audition is to heavy on theory and minutia and the performer isn’t really showing (not given the chance) their better performance. There’s a variety of skills that leetcode style interviews flattens into a narrow category.
Engineering is overwhelmingly an applied science, not an art.
Sure, arts have elements of science and sciences have elements of arts. But, there's a reason we don't normally call scientists artists and vice-versa.
Birds are not fish, even though some birds swim and some fish fly. See?
>> That's because programming is nothing like any of those professions.
That assertion is blatantly wrong.
Software engineers are like civil engineers and mechanical engineers because, ummm, yeah, of that word engineer. See?
I found the following on The Internets...
"California licenses Professional Engineers in the following engineering disciplines: Agricultural, Chemical, Civil, Control System, Electrical, Fire Protection, Industrial, Mechanical, Metallurgical, Nuclear, Petroleum, and Traffic."
It's almost certain that software engineers will be licensed... just like many other engineers. Why? Because certification helps to ensure people who call themselves "software engineers" meet certain standards.
Just like blind people normally aren't able to obtain a driver's license, people who can't solve FizzBuzz shouldn't receive a license to be software engineers.
You are always free to launch a revolution in the hopes of overthrowing your national government so that you can make the regulations and laws you would like.
In the meantime, when the jurisdiction you are in creates a requirement for certain software engineers to become certified (not "if" but "when"), even if the law were to apply to software engineers like you (the arrogant artistic types, you know what I mean, right?), I suppose you'll probably be grandfathered in because governments usually don't want to fight too hard to initiate new regulations and laws.
Grandfathering the "crusy old-timers" like you keeps you off their collective back because, at the end of the day, you probably don't really care about the issue generally. You probably simply don't want to be bothered to spend the time and energy to become certified.
Agreed. That's what LC is filling these days. It's a kind of "qualification" in an unregulated industry. The doctor has the MD, the lawyer a JD, public accountant CPA, etc. There is no base minimum that a software engineer is required to have. Even tradesmen are required to be licensed which follows some certification.
The other aspect I would mention is that the LC style interview has made it possible for a candidate not to bother about their relationships with their existing employer. It is not so for other professions. A doctors career can be nuked by unfavorable recommendations by their existing employer.
This is what every "I don't like LC / LC is harmful" post doesn't address.
- How do you certify? (and no, a profile of github projects doesn't count)
- How do you make the candidate free of any obligation to his current employer?
LC interviews are the symptom, not the cause.
I think everyone gets that, and are still saying go get some more inspiration.
for example, go back to training people on the job.
If the company really thinks they need someone that can hit the ground running they could have trained someone that demonstrated other experience far faster than the same 7 months the job opening was there for.
Before the Web, we were paid like accountants and job hopping was frowned on, so nobody was highly motivated to scam his way into a brief stint he wasn’t qualified for.
Leetcode was created in part to be more equitable and objective.
Before leetcode, we had people asking puzzles which had nothing to do with your coding abilities, like "How many balls could fit on a Boeing 747" or "If you have 9 balls, a see-saw, and one ball weighs more/less than the others, what is the minimum number of weighings to find said ball".
Alma matter also used to matter a bunch more, and non-traditional schooling devs were much harder to come by.
Leetcode isn't perfect, but it's better than what we had before. That said, I don't think it's the "final form" or optimal template for interviewing. What I would love to see are more constructive suggestions for objectively evaluating candidates that don't devolve into things easily bullshitable or biased.
I don't think the certificate days would be much use today, given how fast frameworks change and just how many different technologies are involved with modern development.
How many times have you been asked to solve fizzbuzz at work? ;-)
And yet you can probably fluently juggle for loops and if statements to your heart's content; and can figure ways to bend them to your will (and to the problems at hand).
Similar goes for algorithms (the things leetcode questions ask about.) You'll probably never encounter the exact toy problems that get asked in an interview. The real world is always messy, you you need to be creative and imaginative. Knowing algorithms helps with that because you've already practiced, and have a head start in knowing what works and what not.
In fact, I am taking a short break from a problem that reduces to graph coloring with 2 colors. [1] If you interview with our team we will absolutely ask LC style algorithmic problems because one will definitely use that knowledge with us, a lot.
The problem is not LC style interviews per se. The problem is not enough companies tailoring their interview to the role they're trying to fill.
>> I don't think the certificate days would be much use today,
Think again.
>> given how fast frameworks change and just how many different technologies are involved with modern development.
That is not a problem.
There could be theoretical and applied certifications.
The theoretical certifications would be "one and done" (take the test once, and you are, for example a "Level 3 Software Architect). Theoretical certifications would be based on fundamental (fungible) abilities.
But yeah, if you had been a certified Level 5 Java software engineer, and then Java became eclipsed by "the new hot language" then you'd need to get certified in the new hot language to claim, "I'm a Level 2 New Hot Language engineer."
This might actually be much worse than just looking at the academical credentials. Most if not all those tests are utterly useless, at best can only tell how familiar the candidates with the test subject but nothing else. Let the team to review the resumes and do phone screening would be much better than those tests.
Honestly, what we define as a flood of unqualified candidates is kind of ridiculous, nowadays it refers to anyone who didn't got to a top state school or top 1% equivalent.
The reality is that there are too many qualified people at the lower end of the pipeline and this is because we don't hire enough people as it is for our current positions, why this happens is really simple. It's to save money. There's a reason why we have terms like rockstar developer and heavily advertise the idea of a developer doing 1000x work of another one, which simply put is not true.
Most improvements to software and technology occur over time with experience and expertise what we see nowadays is an extremely self destructive behavior, at most top companies it results in heavy loss in expertise as most individuals will leave once their stock options can be sold. In order to maximize their profit while providing little to nothing of value to said companies. The idea we pay developers six figure salaries is inflated and has been a net negative. Resulting in inflated egos and an elitist environment which it is debatable to which provides anything of value or close to little at all.
We don't really need certification programs as they're typically used to stop minorities from gaining access to fields due to barriers of entry and the fact they are systemically flawed as it results in people just chasing after their next certification as the "logical" next step instead of actually learning anything of value.
Honestly, companies just need to be comfortable with hiring someone with a credible degree and taking a chance on them, it's basically the same thing as hiring with leetcode anyways according to google there is very little positive correlation. Anything else is ripe with corruption. But even this process can easily be corrupted, e.g. hiring only people from elite universities or select ones you have deals with in the area which already occurs in many places.
> Honestly, companies just need to be comfortable with hiring someone with a credible degree and taking a chance on them.
I'd be all for it and I have seen some international non-profits hiring like that for junior roles.
But its a hard sell for googol and the likes who 1) have excess supply of qualified candidates 2) are so used to perpetuating the narrative of absolute exclusivity among their staff.
Is there a difference in the interview process between the US and the EU? Wherever I applied to in the EU I was almost always given a small, practical programming exercise relevant to the field I was applying for and for which I had a few days to hand in my solution (usually they were quick to solve, depending on the level of polish you would put in). In the interview itself we would then discuss the implementation and that way I got to know the team I'd be working with and they could see if we were able to discuss and explain code.
I'm in the EU as well and while I'm quite junior, I felt like I had lots of options during my recent job search. One company gave me a practical exercise (using Spring Boot) and since I'm not familiar with that framework there was simply no way for me to complete it in a reasonable amount of time (we didn't use Spring in the CS program at uni). I had to withdraw my application.
I can't preemptively learn every framework that someone might ask you to use in these practical assignments, but I still think that I'm a fairly strong candidate (let's say top 10% of my graduating class).
I ended up at a company that asked more generic (but harder) questions that I could actually answer.
The company probably was using Spring Boot in their production and wanted to find someone already familiar with the frameworks they use to ensure themselves with an easier onboarding. Of course you can’t preemptively learn every framework out there but chances are that you’d find a place to use the ones you already know, especially if said framework is popular. The alternative you ended up with was a company willing to test your general reasoning and train you on their SW stack. It’s great to have variety and the problem with leetcode is that standardized testing that doesn’t really test for anything practical but for which you have to put in months of preparation.
For my first programming job interview when I was just out of college I was asked to do a simple CRUD application with some sample database the team had. They gave me the general specs (do it in VB.net or C#) and left me alone for 3 hours. After that, I just talked a bit about my code .
This was back in 2003, way before the Leetcode crap had taken on small companies.
Very much so. The leetcode charade was invented in the US (and even ten years ago it wasn't common). Nowadays it's the only way they do it here even in regular companies with unremarkable tech and benefits.
The FAANGs brought this style to the EU but it's not as [dynamic programming-level] crazy yet. Interviewing in Europe feels more humane while here in the States it's gives you a distinct mechanical taste. And Europeans are more likely to discuss relevant tech and previous experience with it.
From what I have seen take home tasks had been popular in the EU before they became more common here.
It's not more humane, the entire process in europe is extremely "credential" based. Good luck getting any type of SWE interview in france without at least a good degree. And if you aren't from a "grande école" you will never get above team lead no matter how competent you are.
It's not even a glass ceiling, it's a full on concrete slab over your head. Management is completely sealed off depending on your Alma mater.
So yeah sure, the interview process is relaxed and gentle because you basically wouldn't even be given a chance if your education/credentials weren't enough for a given position/salary .
At least the American way gives you a chance to get your feet through the door at almost any level. That's why the interviews are harder, you get a chance to get a really high paying job without having to go to say, Polytechnique Paris. Obviously graduating from Stanford will probably help you a looot still, and your Alma Mater... matters. But you can get by just fine with a diploma from a state university, it doesn't completely shut you off from those very high end jobs. And if you don't want those SV-tier jobs and aim for something less competitive, the interview process is usually just as "humane" and laid back as in europe.
I don't know about the rest of europe but I don't think americans realize how "hierarchically stiff" France is. Like sure you won't get to leetcode for a 300k job, but that's because you won't even make it past the CV screening.
"There are only two kinds of languages: the ones people complain about and the ones nobody uses." -Bjarne Stroustrup
And so it is with interviewing techniques. You say "[leetcode] interview processes are not the only way," which is of course true. But it is also true that all other interviewing techniques are also unpopular on HN and with programmers in general. Of course each programmer has one or more techniques that they prefer, but there is no single technique that has even simple majority support amongst engineers as a whole.
Let's review some common techniques and what HN would say. NB: not expressing my opinion on any of these, just compiling common complaints you can find in any HN thread about interviewing.
Leetcode problems: rewards rote memorization over real ingenuity, not relevant to job, high stress.
Take home problem: company not incentivized to respect candidate's time; bias towards candidates that are unemployed, not engaged at current job, or otherwise able to devote large amounts of time to the task; easier to outsource; basically asking candidates to "work for free"
Review of github profile, OS projects, blogs, etc...: bias against candidates that "have a life" and don't program 24/7.
Discussion of previous work: highly subjective, susceptible to frauds and/or exaggerators, prone to false positives (complaint usually phrased in a form like: "we did this at my last company and we hired a bunch of people who couldn't code.")
“I don't know what the l̶a̶n̶g̶u̶a̶g̶e̶ interview technique of the year 2̶0̶0̶0̶ 2030 will look like, but I know it will be c̶a̶l̶l̶e̶d̶ ̶F̶o̶r̶t̶r̶a̶n̶ unpopular on HN.
You're missing my preferred option, pair program on a problem which requires skills relevant for the job. Have the candidate use their preferred tools, ideally their own laptop to minimize discomfort. Make the problem too large to get done within the time block but gives a variety of technical skills for the candidate to show off.
> You do know we hired people to do engineering jobs for many decades before the industry started cargo culting what they observed a handful of Silicon Valley mega-companies doing.
Very much this. I worry that those relatively new to this industry may now only have seen the leetcode-style interview disfunction madness and get the impression it's the only way.
Absolutely not! It's not how it used to be, it's not how it needs to be. I was closing in on 20 years into my career before ever hearing about this type of whiteboard algorithm memorization-style interview. It's common now, but it's not really normal and shouldn't be.
I for one take no part in that charade. I interview a lot of people and have never done a whiteboard algorithm dump question and never will.
I don't hire people to memorize algorithms from a book and write them on whiteboards all day, so I'm not going to interview for that irrelevant skill.
I'll read your resume and let's have a chat about projects you've worked on and what you'd like to work on.
I'm unpersuaded. To begin with, _is_ it such a waste of time to study programming problems, as, you know, a programmer?
Also, more specifically on this piece: can anyone explain how Theranos fits in here? I don't get the connection. One the one hand we have a massive fraud, and on the other... people studying techniques that are rarely used by normal software engineers? But that nevertheless are, still, programming techniques.
And please no more "considered harmful" titles! It's painful to read at this point.
>I'm unpersuaded. To begin with, _is_ it such a waste of time to study programming problems, as, you know, a programmer?
Me too, I've worked in teams without coding interviews, and they hired a lot more subpar engineers that made everybody's life harder.
If I interview I don't put so much emphasis on syntax or the algorithm itself, but whether people can tell me how the algorithms basically work and when to use them.
It seems like this interview glosses over the differences in hiring senior junior and student intern engineers. Or maybe I missed it, please enlighten.
The advantage of leetcode style interviews is that they appear to be objective. Anyone can learn to be good at them if they have the time and base intelligence to prepare for them.
Of course they are not really that fair for people that don't have the time to learn the circus tricks because they have family and stuff but at least there is some appearance of fairness.
Also, compared to other performance based testing, the time requirements on both sides are kind of reasonable. I get more annoyed when the interview process requires me to basically provide free work.
Leetcode is simple: performance on it is a proxy for a combination of two metrics, raw IQ and willingness to consistently jump through a series of arbitrary hoops to achieve some end, which is a predictor of success at large companies.
If you're hiring at scale, there's nothing else that matches it in terms of difficulty to game and cheapness of cost to implement. And for every engineer who says that they refuse out of principle to do Leetcode, there are ten who refuse to do time consuming take home projects and the nth implementation of a bespoke open source static site generator just to build a portfolio on Github.
Most smaller companies can and must figure out other hiring niches for overlooked/available candidates since they can't effectively compete with large companies for good conventional candidates using the same hiring process (e.g. by offering candidates more flexibility, taking more hiring risks), but pretending it's not a tradeoff isn't taking the problem seriously.
The problem is that we have no theory of intelligence that tells us what, if anything, IQ is actually measuring.
If you renamed IQ to "linear-correlated ability to take non-domain specific tests", or "GTTQ" (general test-taking quotient). Then you'd explain every observed correlation between IQ and non-IQ performance.
So, do businesses want intelligent software engineers, or those with a high GTTQ score? As research at google, iirc, has shown: high performers on coding challenges tend to do worse in practice.
Any plausible theory of "competent software engineering" would, I think, place an extremely minor emphasis on a GTTQ. Intelligence is a much richer phenomenon (need for cognition, curiosity, resilliance to failure, domain-specific depth-wise thinking, cross-domain lateral, etc.)....
That research was of candidates who had experience in competitive programming competitions, and IIRC it didn't show that they did worse but that they didn't do better.
There's a restriction of range effect at play here: if you look at the top "GTTQ" quartile, you're not going to see it have a huge effect in performance within that range (and the factors you mention would be far more predictive of on the job performance). But the relevant comparison isn't between how well actual candidates who barely got in perform versus how well candidates who steamrolled the GTTQ questions perform; the comparison is between the hypothetical performance of the candidates who didn't get in because they failed the GTTQ questions and the actual performance of candidates who did get in because they did well enough on GTTQ questions. For that you'd get a much stronger signal.
Though it would be interesting for Google to arbitrarily give offers to 1% of people who flunk the existing interview process just to track how well they perform on the job; that would give the world some really neat metrics to look at. (At the cost of giving everyone they hire even worse imposter syndrome.)
Basically there are two approaches: LeetCode vs project/test based.
The LC approach is fair if you only consider the fact that everybody will get these exercises. They are somewhat standard, and you know what to expect. However, you need to spend quite a lot of time to be able to solve difficult problems under a given time. And yet, emotions might get the best of you and you lose it, despite having solved the problem a bunch of times.
LC problems are not really useful as they don't resemble anything you'd do at work. However, if you do solve them, this shows to the company hiring you that you have been persistently training on LC problems, which shows you are consistent. A good quality, despite the uselessness of the problems.
The alternative, take home tests, are a better indicator of experience and engineering capability, although some might say one could "cheat". That's no different than outsourcing your job to somebody, which means this is fundamentally a trust issue -- so the problem lies elsewhere.
I've also seen variants of having candidates do some kind of trial where you get to experience what a typical day of work looks like at a company. This makes sense at a surface level, until you realise that the "typical day of work" is usually pretty atypical. They day of work trial generally can go two ways: the candidate works on an actual thing that needs to be done, or the candidate works on a fake thing users are supposedly asking for.
The latter is problematic, because you can do everything right, but since it's a) a fake problem, and b) you have no actual user feedback or requests to work on, this gets pretty subjective. Unless you read your interviewer's mind, you're left to wonder why you were rejected.
I'd take a LeetCode problem over a fake work day any time, as much as I dislike LC interviews. I guess all of this is to say, it's complicated :)
It is fascinating to me that people write books and articles on why "leetcode is harmful", just to have a justification for avoiding the hard work. It really makes me giggle. Considering that, if you are new to coding challenges, solving a hard leetcode problem can take up to or take more time, as it needs to write an article, I have to giggle even more. And it just shows me that the system works as intended in filtering out the lazy kind of people. Sorry for all the giggling but it is just so funny to me.
You sound like you haven’t been interviewing/interviewed recently.
Knowing your DSA is one thing: the task isn’t: “let’s write some code to balance this tree”
It’s the extremely cringy “John has 40 watermelons that he needs to take to Mars” style problems
If you only need to demo your understanding of the undergrad level DSA, that’s fine. But if people have to exclusively spend 3 months preparing for LC in particular that’s just madness
And in India, people are just taking this to new heights with people beginning to no-life with LC for 3 *YEARS* and skipping everything else
Whether Leetcode is necessary can be discussed but this person doesn't add anything to the discussion. Almost disconnected 7 paragraphs of text and the conclusion is FAANG and S & P 500 needs to find a way to assess without Leetcode.
It clearly works for them so they continue doing it and able to grow their eng organisations to thousands of engineers. Any other solution to this problem wouldn't be without its trade offs. If he really wants to contribute to tech industry he is more than welcome to found his own company and hire the way he wants, instead of saying "I don't like this FAANG find a new way!".
> If he really wants to contribute to tech industry he is more than welcome to found his own company and hire the way he wants, instead of saying "I don't like this FAANG find a new way!".
I don't think this is a very practical suggestion. FAANG are the winners and hold all the cash, which is why this is actually an issue. Asking someone to essentially try for a lottery ticket chance (in the best case) of repeating this success as a qualification for criticism is like telling people: "If you don't like America, you can get out"
A lot of people say that they never encounter leetcode-like problems in real life and therefore they are silly.
Am I an outlier? I feel like I actually encounter these kinds of things fairly often, or at least the insights that they represent. (Eg: this week I'm implementing a simple path-finder to deal with a sliiightly overgrown state machine from a vendor. )
So - at least because of that - I tend to think of leetcode problems as more or less a slightly harder and more practical fizzbuzz. Complaints about leetcode do tend to mirror complaints you read about fizzbuzz.
Am I being strange here? Perhaps not. I figure there's at least two or three ways to pass a leet-code-like test: Either you just happen to have just done a bunch of marginally interesting programming somewhere in your past (which is probably what the test was designed for?), OR you did a study in college that ideally allows you to avoid the worst parts of the school of hard knocks (also ~fine), OR you explicitly practice for the test.
I suspect Goodharts law might be at work here: In some cases the measure has become the target.
The history has it that the git prototype was written in about a week. If everyone is so familiar with algos and data structures nowadays why don't I see good and novel solutions popping up here and there?
Is that I'm blind? Or is it that the set of approachable impactful problems has been exhausted? Or all the innovation happens in closed source/saas space?
I find Leetcode discussions strange. I recently changed jobs so I was preparing by doing some Leetcode problems. The actual problems I was asked during the interview process were very simple compared to what I expected. Interviewers even explicitly stated that they wanted to see how I think, and the solution didn't even have to compile. So in my experience the whole idea that «Leetcode is harmful» is overblown a little bit.
To me the value of coding interviews are knowing enough algorithms/data structures to apply and adapt them to problems, being able to quickly sketch out your idea, and explain trade-ffs.
But I find the whole discussion not strange but rather dishonest. I realize I'm biased towards thinking that the blog posts condemning some special sort of interviewing come from the "losers", the people that didn't get the job. That isn't true in general, but there is still this aura of entitlement: "I am a good programmer! I have other skills than leetcode! I demand an interview process that is tailored to my specific skills!"
I don't believe the claims that the current style of interviews is not working. This claim usually is made in the blog posts and in the interview horror stories, but I haven't seen it any of the FAANGs I worked in. Sure, some questions give more signal than others, but that's the company's problem, not the candidate's. If anything, large companies realize that lack of diversity is a larger problem and are trying to combat that directly (not with changing the hiring process for the general case though).
What it feels like is the blog posts try to solve a different problem ("how can I get a FAANG job") under the disguise of making the world a better place. Meanwhile, companies try to solve a completely different problem altogether ("how do we weed out candidates that might not be a good fit for what we value at scale with low latency?").
As a hiring manager, I'm interested in applicants that are (among others) good at leetcode-style questions without having had to train a lot on leetcode: Technical problems that are not easy to understand (cognitive and communication abilities), have a simple and easy trivial solution (higher level view on problems and customer orientation) and have high technical depth at scale (specific technical knowledge). So that's where the post is right: don't spend so much time on leetcode. But also don't try to change how companies interview. It's unbecoming.
I guess it is strange to me, as you usually only hear about bad stories. So I was expecting it to be much worse.
I agree, as long as companies evaluate candidates by a combination of different tests, this should be the best for everybody.
Two companies for which I went through the whole interview process had 5-7 different interviews, and only one of them was leetcode-style problem. So even if somebody fails at leetcode problem they should excel at something else.
Lol almost all Leetcode is memorized, most of the algorithms developed tool years for researchers to understand, someone isn’t going to figure it out in 45 minutes without having seen it before
Depends on the company. FAANG and unicorns will absolutely ask for it to compile (except Google which asks questions on Google Docs of all things), and they want the optimal solution almost right away and will penalize you for not getting such a solution.
I just had this unfortunate experience with Google. The interviewer presented some badly-defined vague problem statement, and started asking probing questions about implementation details right away. I'm trying to ask exactly what the problem is, and he's there asking me for the runtime complexity of the solution I'm proposing. Was this dude entirely checked out? It felt like I was talking to a wall.
If I used Leetcode to interview someone I wouldn't even ask them to code a solution.
I'd show them a few problems that I thoroughly understand, ask them to pick two or three of them, and ask them for their thoughts on how to approach them. It would just be a discussion with no need to touch a computer. A whiteboard and scratch paper would be available if they wanted to use them.
I'd provide enough input to make sure the discussion moves along to figuring out a solution, but would try to let them take the lead only providing enough myself to make sure we make progress.
If I ended up providing most of the solution that would be fine. The most important part of the interview comes later.
Then what we'd do is look at the official answer and talk about it, seeing where it matches what we expected and where it does things differently, and maybe talk about whether those differences make it better than what we came up with.
Finally, and this is the most important part, we'd go to the Leetcode discussion for the problem.
In the discussion people post all kinds of solutions and those have all kinds of errors. Some are just flat out wrong. Some are mostly right but miss common edge cases (edge cases I would have made sure the interviewee and I talked about earlier). Some miss somewhat ridiculous edge cases such as failing if an input array is so large that all positive integers in the language they are using are valid array indexes. Some always give the right answer but don't meet the specified time or space constraints.
We'd do code reviews on some of those, again with me trying to just provide enough input to keep things moving along.
> Interviewers even explicitly stated that they wanted to see how I think
That's a common phrase, but it's hardly ever true. You'll actually need to almost perfectly regurgitate the memorized solution, stumbling only enough to give a convincing theatrical impression that you're developing this algorithm on the spot instead of having memorized it (even though the original development of the algorithm probably took years of research).
I used to have a physically demanding job. I didn’t often do movements at work that were identical to what I’d do in the gym. But it was helpful to fit.
I think of leetcode as the gym and I try to do one exercise everyday outside of work.
It depends on the nature of the test. I recently did an assessment for a company that wanted a DP leetcode problem done in 4 minutes.
It also doesn't help that, as previous HN comment sections prove, a staggeringly high number of people who ace these tests actually just outright cheat. And then they come here and talk about the necessity of using LC to filter out 'unqualified candidates' (ie not them).
This is no different than doctors who have to go through FRCS or EDAIC etc exams, these are very hard exams with very low pass percentage adding to the fact that you need to be a qualified MD when you give these exams.
my wife is an MD and has been giving exams for the past 3 years and will be giving exams for atleast another 3 to 4 years and she studies with whatever little time she has after work. The exams she gives have little to nothing to do with the work she does. The real life medical work does not require you to be smart, most average people could do surgeries if they are trained. But getting in is the hard part. The exams there are not qualifying rounds but elimination rounds.
Medicine is an old field, and I think these leetcode style interviews are only the beginning for the computer science based industry, the bar is only going to get higher as years pass which I feel may not be a bad thing overall for the software field.
When interviewers give Leetcode problems, what kind of solutions are they looking for?
I've used Leetcode personally as a source of problems I can think about at a leisurely pace in my head. I like to have a few such things (algorithm or coding problems, math problems, physics problems, music theory problems, electronics problems) that I can use when I'm lying in bed but not quite falling asleep yet or stuck in a slow line somewhere to pass the time.
For the Leetcode problems I've used for that, usually from the "hard" or "medium" categories, it is almost always the case that I can easily and quickly come up with an algorithm that gives the answer and I could easily code but is O(n^2).
It is getting down to O(n log n) or O(n) that is difficult. (I'll give a couple of examples below for your amusement of problems where O(n^2) was easy but doing better took me weeks or months).
Are most interviewers satisfied with O(n^2) that produces the right answer or are they looking for something better than that?
Now for your amusement two problems that took me ages to get past O(n^2).
1. You are given an array of integers and are to find the smallest positive integer that is not in the array. For example given [1, 9, 4, 7, 2, 5] the answer is 3. Your answer should run in O(n) time and use constant extra space.
2. Find the longest palindrome in a string. No time or space constraints are specified.
For the first missing positive integer problem without the time and space constraints the answer that quickly comes to mind is to sort the array (O(n log n)) then scan for the first i such that the element at the i'th position is not i ((O(n)).
What really threw me was the constant space requirement. I had always assumed the inputs were constant in Leetcode problems and could and so could not think of a way to do this without using O(n) space.
I even ended up spending a while trying to prove that it could not be done in constant space hoping that when I could not prove that seeing why I could not would point me toward what I was missing. But I was able to sort of convince myself that a Turing machine could not do it in constant space--"sort of convince" because I'm rusty enough with Turing machines I could not be sure I was missing something.
It turns out that inputs aren't constant in Leetcode problems. You are allowed to modify input arrays. It was then not too hard to figure out the O(n) time and constant extra space solution.
For the palindrome problem an O(n^2) is quite straightforward. But I've got a couple of other approaches that seem faster.
Let's call the two halves of a palindrome its "arms". So an even palindrome consists of two arms that are mirror images next to each other and an odd palindrome consists of two mirror image arms that are separated by one character.
If you have an even palindrome with arms of length n, and you want to check for an odd palindrome one over to the that is longer than the even palindrome, you can move the right arm over by one and check that it is still a mirror of the left. If it is you have found a longer odd palindrome.
It the shifted arm does not match, you can then shift the left arm over and check for a match. If they match you have an even palindrome of the same length. You can then check to see if the arms can be extended while still matching.
By inching the arms across the string this way, extending them when possible, you can sweep the string finding the longest polynomial. If verifying that the arms are mirror images was O(1) this inching approach would be O(n). But comparing the arms the most obvious way os O(L) where L is the length of the arm, which would be 1/2 the length of the longest palindrome found so far, which would be O(n), so the inching approach with the simplest arm matching is O(n^2).
But can we do better? What if we maintain for each arm some kind of hash. We can compare the hashes of the arms and only if the hashes match do we need to then...
131 comments
[ 2.9 ms ] story [ 207 ms ] threadWe're engineers.
If you don't like something, make something better.
Take home projects and how many marbles in a jar are extraordinarily worse than traversing a binary tree.
I suspect that any interviewing practice will always being in detractors. It's a process of culling, those culled will dislike it.
I dislike leetcode style puzzles so much that I will simply drop out of the interview loop rather than subject myself to them.
Like interviews aren't lengthy enough as-is. I had an onsite the other week with two sys design, two leetcode-esque, a hiring manager, and a project manager.
I'd take that any day over the take home project I had to do in a subset of common LISP fresh out of college. Twas NOT a fun weekend, especially since I had no familiarity with LISP. With few exceptions, I don't do takehomes anymore. I don't want to spend an hour or so fuddling with your/your vendor's poorly documented, buggy environment (or setting it up myself) on my own time, outside of office hours, talking to exactly zero human beings during the actual process. It teaches me nothing about the company or team I'd be working on, doesn't allow for any gauge of culture fit, and (unlike leetcode) doesn't make me a better programmer.
Leetcode-style problems as an interview technique are probably the one that’s furthest away from testing the from day-to-day software development challenges most organisations face.
As others have said, open to extreme abuse in terms of timeliness.
> A system design exercise around a whiteboard
I've worked with a frankly extreme number of architects who can talk the talk but cannot (or will not) code the code.
If you're hiring for someone to code the code system design isn't the signal you want.
> Pairing up to fix a bug in the org’s code during the interview
Open to cheating. Very difficult and time consuming to rotate the problem. With Leetcode you can take the problem out of a college textbook.
> Talking the interviewer through some previously-written or open-source code.
Works well for small companies where you're hiring someone who fits a team well. Probably the best approach for small companies.
For big companies, it's open to extreme racial and personal bias.
And I don't mean racism in an abstract sense. I mean that I've interviewed with a company that pairs with a particular coding exercise across all candidates.
They asked me back whilst knocking back my INFINITELY more qualified good friend who was interviewing with the same company.
We walked through the same code base, I'm a LOT less qualified than they are. We got far different feedback, to the extent I genuinely can only assume unspoken racism.
Take home project: typically capped at 4 hours.
Hmmm, this is a tough one...
Every such test is supposed to be calibrated. Leetcode... Is not.
It's not even the same order of magnitude. Like I mentioned, it is common to spend months grinding leetcode. How can people rationalize this and prefer it to a closed scope assignment? I honestly don't get it.
I've personally learnt a lot from Leetcode, and found the process really valuable. I learnt a ton from bashing my head against the wall until dynamic programming made sense.
I anecdotally code much, much faster, in all languages, having done it. It's like doing college homework again. Something useful that makes you a better "coder," (not necessarily a better engineer). It was a damn hard slog, but having done the hard slog I really enjoy having done it.
I'm not going to pretend it's not time consuming or I'd be happy doing it if I was a busy parent.
But in hindsight I've really enjoyed having done it, and found it extremely valuable.
I suppose it's for people who just have that kind of time/energy/enthusiasm for programming/their career.
That said, I'm partial to good leetcode questions. Some of them are worded terribly, some of them are essentially trivia, some of them are overly esoteric and not applicable to most roles. Some of them are great but have "optimal" solutions which take an absurd amount of research and effort to produce, like quickselect. These questions are bad and should not be asked.
Others I find fantastic questions. "Scheduling meetings", anything that requires you to walk a tree or do a binary search/BFS/DFS, and tasks that require you to combine two data structures like the "Hashmap with O(1) get random" question really helped me get better t my job.
There's others I'm lukewarm on, like many topological traversal, shortest/longest path, and knapsack/space filling questions. I personally love them and have gotten good at them, but I also don't expect joe schmo to remember how to implement the relevant algorithms off the top of their head. I'd consider this question cohort good for "open book" interviewing at "less prestigious" companies. I'm totally fine with Google/Meta/Amazon asking these closed-book for normal roles.
I did get an offer from them though, not sure how I would have felt if rejected.
A couple for example:
1. Given a set of number ranges (e.g. [1..3, 4..5, 9..20, 2..5]) consolidate them. Tying this to a real world scenario: we had a system that dealt with "time periods" and we needed to write a validator to ensure all future dates were covered by at least one configured "time period." One of the first steps to do this was basically the number-range consolidation.
2. Given a list of numbers output metrics like average, min, max, and mode. Now what if the list doesn't fit in memory and instead comes in as a stream (or from an iterator)? This one was inspired by an intern that got stuck when he needed to run calculations on data that didn't fit into memory. He was seriously about to request a host with 10+TB of memory before we stopped him.
I’d love to have an excuse to get a machine like that.
You can also expand the problem to be approximately internet sized.
https://web.engr.oregonstate.edu/~erwig/diet
All I see is anecdote after anecdote in both directions.
In god we trust, all others must bring data.
Big corporations do not use it, and they would be most likely to have actual also likely untranslatable employment data.
Then there are specifics of any job - knowledge of particular system or rapid learning and experimentation skills tend to trump optimization and algorithmic knowledge. Leetcode does not really test any of that.
It is also particularly bad at weeding out cheaters. :)
Google bans any questions that you can find on the internet though, including all questions you find on leetcode (they have people who check this, including the member only questions), they want to ensure that candidates only do problems they haven't seen before. Some accidents still happens, but since they have 5 interviews per candidate it is very unlikely for anyone to pass without solving a few problems they haven't seen before.
So the process works when the candidate hasn't seen the exact problems before. For those who just gives leetcode questions verbatim it might be less effective, or even counter productive, I haven't seen any data on that.
I think part of the problem is that really in order to get proper data.... You have to actually hire the people who failed the interview.
So i actually doubt google has any good internal data. Not unless they hired a bunch of people who failed.
Anywhere else where you use the Lego bricks to build some applications shouldn’t really use leetcode.
If you’re a good product engineer and you enjoy building good products, maybe you should start a company to build good products that solve problems for other people. Let the engineers who enjoy building the bricks do leetcode in peace so that we have solid pieces to build good software.
I’m doing the latter and trying to build a company. I realized I much prefer to build products that customers love, and I want to build useful software for people. I’m not that interested in low level tinkering and re-inventing the wheel all the time.
Coexistence is possible!
Is the linux kernel a tool you use in production?
Linux kernel isn't really proof of either lol.
Me: Points out obvious example to the contrary
You: That's getting pedantic imo
That said: I’m tired of the attitude that there is no effective alternative to this style of interview. You do know we hired people to do engineering jobs for many decades before the industry started cargo culting what they observed a handful of Silicon Valley mega-companies doing. Plenty of very competent people were interviewed and hired before this “relive your algorithms class final on a whiteboard” style interview process became the thing to do.
These interview processes are not the only way. As I said: it’s cargo culting where insignificant companies try to act like Google or some other giant in hopes that perhaps the billions in revenue will start rolling in if they mimic them. To me this behavior is no different than “founders” who adopted dress styles and presentation mannerisms imitating Steve Jobs a decade or so ago. …And bloggers who can’t resist the “considered harmful” trope.
A doctor, lawyer, public accountant, and, well, many types of engineers obtain public or private certifications. As a result, it is much, much, much easier for hiring companies to find and interview qualified candidates.
I suppose, in the "old days" universities kind of, sort of, filled this role. A hiring manager looking at a resume might have said, "Oh, he's got an EE/CS from Berkeley, there's a good chance he's qualified.
On the other hand, open source culture is vastly different from proprietary culture.
So many times in proprietary culture I've churned out craptastic code to meet a deadline that didn't matter for a project whose customers were straw-persons. There's definitely some projects or features I've coded which I'm glad to hide from public view.
That said, artists can curate their portfolio, and unless they're presenting to a public forum, their turds won't have to see the light of day again.
https://en.wikipedia.org/wiki/Chartered_Engineer_(UK)
As a software person I don't know enough about the scheme to comment on the quality of the article, but knowing Wikipedia I imagine it's on point.
edit For a broader context than just the UK, see https://en.wikipedia.org/wiki/Regulation_and_licensure_in_en...
Nothing like that exists in programming, not even languages or platforms. Which is why there exist multiple proprietary certifications instead focusing on specific details.
I agree.
After enough bridges fell down causing massive human fatalities, standards and accepted practices almost certainly "emerged."
Standards and accepted practices in software engineering will emerge, as if by magic, once enough "bridges fall down."
You are a diplomat sir; I'm merely a common street brawler.
Almost like fashion. At least in Europe, young people dress like a Stranger Things inspired mashup of 70s, 80s and 90s nowadays picking the best parts of those decades while leaving the 2000s and 2010s fashion in the dust.
Are you gonna do a certificate on microservices? And what's that gonna help when suddenly everyone starts building some shape of monoliths again because people keep running into the "no free lunch" issues?
Wrong audition can be worse than no audition.
If you hire a piano player for large symphonic orchestra, or a famous band, and they are not already famous, you want to hear them play. Maybe one piece they chose themselves, but maybe also a piece everyone in the genre is expected to know.
And if they have to improvize a bit, that may tell you a lot, too:
https://www.youtube.com/watch?v=HtI3IMlGRMY
Sometime you meet programmer like this, than you can jam with on almost any topic. Those that can repeatedly improvize their way to "leetcode" solutions, while you observe their thinking process.
The "problem" with "leetcode" is that it has become too common and formalized. When people start to practice leetcode just to land the job, it suck the fun out of it, and it becomes harder to tell if the candidate is able to solve similar, but novel problems.
Orchestra musicians indeed are routinely hired via a process that involves auditions, but my understanding is that applies only to those at the beginnings of their careers.
Popular/rock/jazz band formation process IMO is indeed similar to how start-ups are created (i.e. rather informal process generally based on networking, though audition jams also happen to fill a specific position). I fail to see how corporate staffing is similar.
https://www.quora.com/How-does-seating-and-hiring-firing-in-...
Some employers ask for your personal github. That’s just like a portfolio of past work.
Your run of the mill dev would be more like a studio musician hired to play a specific instrument in a specific genre.
Surely, he/she needs to be able to come up with lines/hooks/riffs/solos (when asked) and play part in the arrangement but composing from scratch and for multiple instruments would be rather out of scope.
While there's no direct audience for studio musicians (only the output matters) there are definitely performance expectations and the studio musicians are expected to be efficient.
Except that audition is to heavy on theory and minutia and the performer isn’t really showing (not given the chance) their better performance. There’s a variety of skills that leetcode style interviews flattens into a narrow category.
Engineering is overwhelmingly an applied science, not an art.
Sure, arts have elements of science and sciences have elements of arts. But, there's a reason we don't normally call scientists artists and vice-versa.
Birds are not fish, even though some birds swim and some fish fly. See?
This objection would be better addressed to the comment I was replying to, since I was just accepting the premise for the sake of argument.
That assertion is blatantly wrong.
Software engineers are like civil engineers and mechanical engineers because, ummm, yeah, of that word engineer. See?
I found the following on The Internets...
"California licenses Professional Engineers in the following engineering disciplines: Agricultural, Chemical, Civil, Control System, Electrical, Fire Protection, Industrial, Mechanical, Metallurgical, Nuclear, Petroleum, and Traffic."
It's almost certain that software engineers will be licensed... just like many other engineers. Why? Because certification helps to ensure people who call themselves "software engineers" meet certain standards.
Just like blind people normally aren't able to obtain a driver's license, people who can't solve FizzBuzz shouldn't receive a license to be software engineers.
You are always free to launch a revolution in the hopes of overthrowing your national government so that you can make the regulations and laws you would like.
In the meantime, when the jurisdiction you are in creates a requirement for certain software engineers to become certified (not "if" but "when"), even if the law were to apply to software engineers like you (the arrogant artistic types, you know what I mean, right?), I suppose you'll probably be grandfathered in because governments usually don't want to fight too hard to initiate new regulations and laws.
Grandfathering the "crusy old-timers" like you keeps you off their collective back because, at the end of the day, you probably don't really care about the issue generally. You probably simply don't want to be bothered to spend the time and energy to become certified.
The other aspect I would mention is that the LC style interview has made it possible for a candidate not to bother about their relationships with their existing employer. It is not so for other professions. A doctors career can be nuked by unfavorable recommendations by their existing employer.
This is what every "I don't like LC / LC is harmful" post doesn't address. - How do you certify? (and no, a profile of github projects doesn't count) - How do you make the candidate free of any obligation to his current employer? LC interviews are the symptom, not the cause.
for example, go back to training people on the job.
If the company really thinks they need someone that can hit the ground running they could have trained someone that demonstrated other experience far faster than the same 7 months the job opening was there for.
Before leetcode, we had people asking puzzles which had nothing to do with your coding abilities, like "How many balls could fit on a Boeing 747" or "If you have 9 balls, a see-saw, and one ball weighs more/less than the others, what is the minimum number of weighings to find said ball".
Alma matter also used to matter a bunch more, and non-traditional schooling devs were much harder to come by.
Leetcode isn't perfect, but it's better than what we had before. That said, I don't think it's the "final form" or optimal template for interviewing. What I would love to see are more constructive suggestions for objectively evaluating candidates that don't devolve into things easily bullshitable or biased.
I don't think the certificate days would be much use today, given how fast frameworks change and just how many different technologies are involved with modern development.
And yet you can probably fluently juggle for loops and if statements to your heart's content; and can figure ways to bend them to your will (and to the problems at hand).
Similar goes for algorithms (the things leetcode questions ask about.) You'll probably never encounter the exact toy problems that get asked in an interview. The real world is always messy, you you need to be creative and imaginative. Knowing algorithms helps with that because you've already practiced, and have a head start in knowing what works and what not.
In fact, I am taking a short break from a problem that reduces to graph coloring with 2 colors. [1] If you interview with our team we will absolutely ask LC style algorithmic problems because one will definitely use that knowledge with us, a lot.
The problem is not LC style interviews per se. The problem is not enough companies tailoring their interview to the role they're trying to fill.
[1] https://en.wikipedia.org/wiki/Graph_coloring
This is also the problem with Leetcode: it often doesn't test 'can you code' but 'have you learnt this trick'.
Think again.
>> given how fast frameworks change and just how many different technologies are involved with modern development.
That is not a problem.
There could be theoretical and applied certifications.
The theoretical certifications would be "one and done" (take the test once, and you are, for example a "Level 3 Software Architect). Theoretical certifications would be based on fundamental (fungible) abilities.
But yeah, if you had been a certified Level 5 Java software engineer, and then Java became eclipsed by "the new hot language" then you'd need to get certified in the new hot language to claim, "I'm a Level 2 New Hot Language engineer."
The reality is that there are too many qualified people at the lower end of the pipeline and this is because we don't hire enough people as it is for our current positions, why this happens is really simple. It's to save money. There's a reason why we have terms like rockstar developer and heavily advertise the idea of a developer doing 1000x work of another one, which simply put is not true.
Most improvements to software and technology occur over time with experience and expertise what we see nowadays is an extremely self destructive behavior, at most top companies it results in heavy loss in expertise as most individuals will leave once their stock options can be sold. In order to maximize their profit while providing little to nothing of value to said companies. The idea we pay developers six figure salaries is inflated and has been a net negative. Resulting in inflated egos and an elitist environment which it is debatable to which provides anything of value or close to little at all.
We don't really need certification programs as they're typically used to stop minorities from gaining access to fields due to barriers of entry and the fact they are systemically flawed as it results in people just chasing after their next certification as the "logical" next step instead of actually learning anything of value.
Honestly, companies just need to be comfortable with hiring someone with a credible degree and taking a chance on them, it's basically the same thing as hiring with leetcode anyways according to google there is very little positive correlation. Anything else is ripe with corruption. But even this process can easily be corrupted, e.g. hiring only people from elite universities or select ones you have deals with in the area which already occurs in many places.
I'd be all for it and I have seen some international non-profits hiring like that for junior roles.
But its a hard sell for googol and the likes who 1) have excess supply of qualified candidates 2) are so used to perpetuating the narrative of absolute exclusivity among their staff.
I can't preemptively learn every framework that someone might ask you to use in these practical assignments, but I still think that I'm a fairly strong candidate (let's say top 10% of my graduating class).
I ended up at a company that asked more generic (but harder) questions that I could actually answer.
This was back in 2003, way before the Leetcode crap had taken on small companies.
The FAANGs brought this style to the EU but it's not as [dynamic programming-level] crazy yet. Interviewing in Europe feels more humane while here in the States it's gives you a distinct mechanical taste. And Europeans are more likely to discuss relevant tech and previous experience with it.
From what I have seen take home tasks had been popular in the EU before they became more common here.
It's not even a glass ceiling, it's a full on concrete slab over your head. Management is completely sealed off depending on your Alma mater.
So yeah sure, the interview process is relaxed and gentle because you basically wouldn't even be given a chance if your education/credentials weren't enough for a given position/salary .
At least the American way gives you a chance to get your feet through the door at almost any level. That's why the interviews are harder, you get a chance to get a really high paying job without having to go to say, Polytechnique Paris. Obviously graduating from Stanford will probably help you a looot still, and your Alma Mater... matters. But you can get by just fine with a diploma from a state university, it doesn't completely shut you off from those very high end jobs. And if you don't want those SV-tier jobs and aim for something less competitive, the interview process is usually just as "humane" and laid back as in europe.
I don't know about the rest of europe but I don't think americans realize how "hierarchically stiff" France is. Like sure you won't get to leetcode for a 300k job, but that's because you won't even make it past the CV screening.
And so it is with interviewing techniques. You say "[leetcode] interview processes are not the only way," which is of course true. But it is also true that all other interviewing techniques are also unpopular on HN and with programmers in general. Of course each programmer has one or more techniques that they prefer, but there is no single technique that has even simple majority support amongst engineers as a whole.
Let's review some common techniques and what HN would say. NB: not expressing my opinion on any of these, just compiling common complaints you can find in any HN thread about interviewing.
Leetcode problems: rewards rote memorization over real ingenuity, not relevant to job, high stress.
Take home problem: company not incentivized to respect candidate's time; bias towards candidates that are unemployed, not engaged at current job, or otherwise able to devote large amounts of time to the task; easier to outsource; basically asking candidates to "work for free"
Review of github profile, OS projects, blogs, etc...: bias against candidates that "have a life" and don't program 24/7.
Discussion of previous work: highly subjective, susceptible to frauds and/or exaggerators, prone to false positives (complaint usually phrased in a form like: "we did this at my last company and we hired a bunch of people who couldn't code.")
“I don't know what the l̶a̶n̶g̶u̶a̶g̶e̶ interview technique of the year 2̶0̶0̶0̶ 2030 will look like, but I know it will be c̶a̶l̶l̶e̶d̶ ̶F̶o̶r̶t̶r̶a̶n̶ unpopular on HN.
But if this method was widely adopted, I’m certain HN would hate it.
Very much this. I worry that those relatively new to this industry may now only have seen the leetcode-style interview disfunction madness and get the impression it's the only way.
Absolutely not! It's not how it used to be, it's not how it needs to be. I was closing in on 20 years into my career before ever hearing about this type of whiteboard algorithm memorization-style interview. It's common now, but it's not really normal and shouldn't be.
I for one take no part in that charade. I interview a lot of people and have never done a whiteboard algorithm dump question and never will.
I don't hire people to memorize algorithms from a book and write them on whiteboards all day, so I'm not going to interview for that irrelevant skill.
I'll read your resume and let's have a chat about projects you've worked on and what you'd like to work on.
now there are leetcode grinding courses and books, it no longer serve any purpose than to dissuade those who don't have time, or motivation.
good way to identify the desperate willing for a small % chance in an 8 stage interview when someone may still reject you for some unknown reason.
Also, more specifically on this piece: can anyone explain how Theranos fits in here? I don't get the connection. One the one hand we have a massive fraud, and on the other... people studying techniques that are rarely used by normal software engineers? But that nevertheless are, still, programming techniques.
And please no more "considered harmful" titles! It's painful to read at this point.
https://news.ycombinator.com/item?id=9744916
Me too, I've worked in teams without coding interviews, and they hired a lot more subpar engineers that made everybody's life harder.
If I interview I don't put so much emphasis on syntax or the algorithm itself, but whether people can tell me how the algorithms basically work and when to use them.
If you wernt such an evil company you wouldnt need to pay people so much money to work on your ad data scam.
Of course they are not really that fair for people that don't have the time to learn the circus tricks because they have family and stuff but at least there is some appearance of fairness.
Also, compared to other performance based testing, the time requirements on both sides are kind of reasonable. I get more annoyed when the interview process requires me to basically provide free work.
If you're hiring at scale, there's nothing else that matches it in terms of difficulty to game and cheapness of cost to implement. And for every engineer who says that they refuse out of principle to do Leetcode, there are ten who refuse to do time consuming take home projects and the nth implementation of a bespoke open source static site generator just to build a portfolio on Github.
Most smaller companies can and must figure out other hiring niches for overlooked/available candidates since they can't effectively compete with large companies for good conventional candidates using the same hiring process (e.g. by offering candidates more flexibility, taking more hiring risks), but pretending it's not a tradeoff isn't taking the problem seriously.
If you renamed IQ to "linear-correlated ability to take non-domain specific tests", or "GTTQ" (general test-taking quotient). Then you'd explain every observed correlation between IQ and non-IQ performance.
So, do businesses want intelligent software engineers, or those with a high GTTQ score? As research at google, iirc, has shown: high performers on coding challenges tend to do worse in practice.
Any plausible theory of "competent software engineering" would, I think, place an extremely minor emphasis on a GTTQ. Intelligence is a much richer phenomenon (need for cognition, curiosity, resilliance to failure, domain-specific depth-wise thinking, cross-domain lateral, etc.)....
There's a restriction of range effect at play here: if you look at the top "GTTQ" quartile, you're not going to see it have a huge effect in performance within that range (and the factors you mention would be far more predictive of on the job performance). But the relevant comparison isn't between how well actual candidates who barely got in perform versus how well candidates who steamrolled the GTTQ questions perform; the comparison is between the hypothetical performance of the candidates who didn't get in because they failed the GTTQ questions and the actual performance of candidates who did get in because they did well enough on GTTQ questions. For that you'd get a much stronger signal.
Though it would be interesting for Google to arbitrarily give offers to 1% of people who flunk the existing interview process just to track how well they perform on the job; that would give the world some really neat metrics to look at. (At the cost of giving everyone they hire even worse imposter syndrome.)
The LC approach is fair if you only consider the fact that everybody will get these exercises. They are somewhat standard, and you know what to expect. However, you need to spend quite a lot of time to be able to solve difficult problems under a given time. And yet, emotions might get the best of you and you lose it, despite having solved the problem a bunch of times.
LC problems are not really useful as they don't resemble anything you'd do at work. However, if you do solve them, this shows to the company hiring you that you have been persistently training on LC problems, which shows you are consistent. A good quality, despite the uselessness of the problems.
The alternative, take home tests, are a better indicator of experience and engineering capability, although some might say one could "cheat". That's no different than outsourcing your job to somebody, which means this is fundamentally a trust issue -- so the problem lies elsewhere.
I've also seen variants of having candidates do some kind of trial where you get to experience what a typical day of work looks like at a company. This makes sense at a surface level, until you realise that the "typical day of work" is usually pretty atypical. They day of work trial generally can go two ways: the candidate works on an actual thing that needs to be done, or the candidate works on a fake thing users are supposedly asking for.
The latter is problematic, because you can do everything right, but since it's a) a fake problem, and b) you have no actual user feedback or requests to work on, this gets pretty subjective. Unless you read your interviewer's mind, you're left to wonder why you were rejected.
I'd take a LeetCode problem over a fake work day any time, as much as I dislike LC interviews. I guess all of this is to say, it's complicated :)
You sound like you haven’t been interviewing/interviewed recently.
Knowing your DSA is one thing: the task isn’t: “let’s write some code to balance this tree”
It’s the extremely cringy “John has 40 watermelons that he needs to take to Mars” style problems
If you only need to demo your understanding of the undergrad level DSA, that’s fine. But if people have to exclusively spend 3 months preparing for LC in particular that’s just madness
And in India, people are just taking this to new heights with people beginning to no-life with LC for 3 *YEARS* and skipping everything else
It clearly works for them so they continue doing it and able to grow their eng organisations to thousands of engineers. Any other solution to this problem wouldn't be without its trade offs. If he really wants to contribute to tech industry he is more than welcome to found his own company and hire the way he wants, instead of saying "I don't like this FAANG find a new way!".
I don't think this is a very practical suggestion. FAANG are the winners and hold all the cash, which is why this is actually an issue. Asking someone to essentially try for a lottery ticket chance (in the best case) of repeating this success as a qualification for criticism is like telling people: "If you don't like America, you can get out"
Am I an outlier? I feel like I actually encounter these kinds of things fairly often, or at least the insights that they represent. (Eg: this week I'm implementing a simple path-finder to deal with a sliiightly overgrown state machine from a vendor. )
So - at least because of that - I tend to think of leetcode problems as more or less a slightly harder and more practical fizzbuzz. Complaints about leetcode do tend to mirror complaints you read about fizzbuzz.
Am I being strange here? Perhaps not. I figure there's at least two or three ways to pass a leet-code-like test: Either you just happen to have just done a bunch of marginally interesting programming somewhere in your past (which is probably what the test was designed for?), OR you did a study in college that ideally allows you to avoid the worst parts of the school of hard knocks (also ~fine), OR you explicitly practice for the test.
I suspect Goodharts law might be at work here: In some cases the measure has become the target.
What do you tell a React developer?
You're going to need Knuth-Morris-Pratt to center a div?
95% of software jobs involve using a library for algorithm problems.
Is that I'm blind? Or is it that the set of approachable impactful problems has been exhausted? Or all the innovation happens in closed source/saas space?
To me the value of coding interviews are knowing enough algorithms/data structures to apply and adapt them to problems, being able to quickly sketch out your idea, and explain trade-ffs.
But I find the whole discussion not strange but rather dishonest. I realize I'm biased towards thinking that the blog posts condemning some special sort of interviewing come from the "losers", the people that didn't get the job. That isn't true in general, but there is still this aura of entitlement: "I am a good programmer! I have other skills than leetcode! I demand an interview process that is tailored to my specific skills!"
I don't believe the claims that the current style of interviews is not working. This claim usually is made in the blog posts and in the interview horror stories, but I haven't seen it any of the FAANGs I worked in. Sure, some questions give more signal than others, but that's the company's problem, not the candidate's. If anything, large companies realize that lack of diversity is a larger problem and are trying to combat that directly (not with changing the hiring process for the general case though).
What it feels like is the blog posts try to solve a different problem ("how can I get a FAANG job") under the disguise of making the world a better place. Meanwhile, companies try to solve a completely different problem altogether ("how do we weed out candidates that might not be a good fit for what we value at scale with low latency?").
As a hiring manager, I'm interested in applicants that are (among others) good at leetcode-style questions without having had to train a lot on leetcode: Technical problems that are not easy to understand (cognitive and communication abilities), have a simple and easy trivial solution (higher level view on problems and customer orientation) and have high technical depth at scale (specific technical knowledge). So that's where the post is right: don't spend so much time on leetcode. But also don't try to change how companies interview. It's unbecoming.
I agree, as long as companies evaluate candidates by a combination of different tests, this should be the best for everybody.
Two companies for which I went through the whole interview process had 5-7 different interviews, and only one of them was leetcode-style problem. So even if somebody fails at leetcode problem they should excel at something else.
I'd show them a few problems that I thoroughly understand, ask them to pick two or three of them, and ask them for their thoughts on how to approach them. It would just be a discussion with no need to touch a computer. A whiteboard and scratch paper would be available if they wanted to use them.
I'd provide enough input to make sure the discussion moves along to figuring out a solution, but would try to let them take the lead only providing enough myself to make sure we make progress.
If I ended up providing most of the solution that would be fine. The most important part of the interview comes later.
Then what we'd do is look at the official answer and talk about it, seeing where it matches what we expected and where it does things differently, and maybe talk about whether those differences make it better than what we came up with.
Finally, and this is the most important part, we'd go to the Leetcode discussion for the problem.
In the discussion people post all kinds of solutions and those have all kinds of errors. Some are just flat out wrong. Some are mostly right but miss common edge cases (edge cases I would have made sure the interviewee and I talked about earlier). Some miss somewhat ridiculous edge cases such as failing if an input array is so large that all positive integers in the language they are using are valid array indexes. Some always give the right answer but don't meet the specified time or space constraints.
We'd do code reviews on some of those, again with me trying to just provide enough input to keep things moving along.
That's a common phrase, but it's hardly ever true. You'll actually need to almost perfectly regurgitate the memorized solution, stumbling only enough to give a convincing theatrical impression that you're developing this algorithm on the spot instead of having memorized it (even though the original development of the algorithm probably took years of research).
I think of leetcode as the gym and I try to do one exercise everyday outside of work.
It also doesn't help that, as previous HN comment sections prove, a staggeringly high number of people who ace these tests actually just outright cheat. And then they come here and talk about the necessity of using LC to filter out 'unqualified candidates' (ie not them).
Medicine is an old field, and I think these leetcode style interviews are only the beginning for the computer science based industry, the bar is only going to get higher as years pass which I feel may not be a bad thing overall for the software field.
I've used Leetcode personally as a source of problems I can think about at a leisurely pace in my head. I like to have a few such things (algorithm or coding problems, math problems, physics problems, music theory problems, electronics problems) that I can use when I'm lying in bed but not quite falling asleep yet or stuck in a slow line somewhere to pass the time.
For the Leetcode problems I've used for that, usually from the "hard" or "medium" categories, it is almost always the case that I can easily and quickly come up with an algorithm that gives the answer and I could easily code but is O(n^2).
It is getting down to O(n log n) or O(n) that is difficult. (I'll give a couple of examples below for your amusement of problems where O(n^2) was easy but doing better took me weeks or months).
Are most interviewers satisfied with O(n^2) that produces the right answer or are they looking for something better than that?
Now for your amusement two problems that took me ages to get past O(n^2).
1. You are given an array of integers and are to find the smallest positive integer that is not in the array. For example given [1, 9, 4, 7, 2, 5] the answer is 3. Your answer should run in O(n) time and use constant extra space.
2. Find the longest palindrome in a string. No time or space constraints are specified.
For the first missing positive integer problem without the time and space constraints the answer that quickly comes to mind is to sort the array (O(n log n)) then scan for the first i such that the element at the i'th position is not i ((O(n)).
What really threw me was the constant space requirement. I had always assumed the inputs were constant in Leetcode problems and could and so could not think of a way to do this without using O(n) space.
I even ended up spending a while trying to prove that it could not be done in constant space hoping that when I could not prove that seeing why I could not would point me toward what I was missing. But I was able to sort of convince myself that a Turing machine could not do it in constant space--"sort of convince" because I'm rusty enough with Turing machines I could not be sure I was missing something.
It turns out that inputs aren't constant in Leetcode problems. You are allowed to modify input arrays. It was then not too hard to figure out the O(n) time and constant extra space solution.
For the palindrome problem an O(n^2) is quite straightforward. But I've got a couple of other approaches that seem faster.
Let's call the two halves of a palindrome its "arms". So an even palindrome consists of two arms that are mirror images next to each other and an odd palindrome consists of two mirror image arms that are separated by one character.
If you have an even palindrome with arms of length n, and you want to check for an odd palindrome one over to the that is longer than the even palindrome, you can move the right arm over by one and check that it is still a mirror of the left. If it is you have found a longer odd palindrome.
It the shifted arm does not match, you can then shift the left arm over and check for a match. If they match you have an even palindrome of the same length. You can then check to see if the arms can be extended while still matching.
By inching the arms across the string this way, extending them when possible, you can sweep the string finding the longest polynomial. If verifying that the arms are mirror images was O(1) this inching approach would be O(n). But comparing the arms the most obvious way os O(L) where L is the length of the arm, which would be 1/2 the length of the longest palindrome found so far, which would be O(n), so the inching approach with the simplest arm matching is O(n^2).
But can we do better? What if we maintain for each arm some kind of hash. We can compare the hashes of the arms and only if the hashes match do we need to then...