I never understood why Big Tech never setup contracts with all the SAT and ACT test centers across the country. Even before Zoom with Codepads it would have made sense for the recruiters to send potential candidates to a test center to do a pre assessment rather than waste time with engineers sitting on prescreen calls all day.
What exactly are you suggesting here? A standardized test that applies to all your job applications? Or, a candidate having to drive to a test center for every company they apply to? Or something else?
Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs. Company either flys you in for in-person or sends you back to test center to do live remote interview in controlled environment.
It's been tried and failed: Sun Microsystems pushed certifications in the 90s. Pass the test on some technology, get the certification. Then they studied performance. The result? More certifications implied a worse employee. The reason was the top performing employees had no time to study for the exams, but the managers of the bottom performing employees were happy to send them off to training and testing. And then the certification fad came mostly to an end.
That was something quite different that got tried. This would be more based on aptitude rather than knowledge.
Of course, they'd miss out on some good talent. But in the article where it shows the quote of someone getting rejected for not inverting a binary tree on a whiteboard, that doesn't seem like a terrible thing to test for.
the idea is sound. create a basic standardized test targeted at tech/engineering jobs. not actually SAT -- operated by a vendor like The College Board. There are plenty of standardized test operators
> The truth is I don’t trust anyone else to run evals.
It's a common sentiment.
But compare https://www.cambridge.org/core/journals/judgment-and-decisio... . ("People predicting the future performance of college students state that interviewing the students aids prediction, although in fact the interviews make predictions less accurate.")
When I'm interviewing, I'm putting about 30% of the weight towards "would I enjoy working with this person on a daily basis?", but in the context of technical discussions. Standardized testing won't be able to replicate it.
If the candidate is a protected class and they are rejected for "cultural fit" it will be an easy case for EEOC to raise a discrimination case.
This is effectively how Harvard was rejecting Asian applicants. They created a "personal fit" / cultural fit quality that Asians scored low on . Supreme Court found this to be discrimination.
It doesn't matter if you are truly discriminating, it matters how well you have tangible evidence of the employee not meeting the qualifications for the role.
Discriminate against... a personality that will negatively impact the team dynamics? It's not that easy, to be honest, as every team has its own requirements.
You just send out a generic decline, and document it as there's a better candidate fit for the role.
I'm not sure if you guys have been in charge of hiring, but there's no real alternative. In my most recent experience, we had one open position, and after interviewing 10 candidates, 3 of them were basically identical in terms of technical qualifications. How do you choose one over the other, other than the "vibes"? Anyone suggesting otherwise is either living in a weird alternate reality, or doesn't want to accept that working is a cooperative job and interpersonal relationships are very important.
There always will be exceptions for different type of roles and specializations, but that's not what I'm talking about.
I just looked them up, as I recalled those cases differently, and it doesn’t look like anything has to do with declining an applicant due to them not being the right fit for the team.
A stereotypical Asian interviewing a stereotypical German might find the German rude in some interactions. While another German interviewer would find it being frank.
Interviews based on personal feelings have hidden biases not even the interviewer is aware of.
Here's another question - stereotypical Japanese interviewer, interviewing, back-to-back, a stereotypical Indian and a stereotypical German for the role. Both are capable and equally technically proficient. How do you choose, other than looking at the team you're hiring for, and thinking how the person would fit in?
given the in-person interview is at the end of the funnel by a factor of 500-1000, standardized testing might even open up opportunity for under represented candidates.
Think of how poor the screening process is at the recruiter & CTS (left side) of the funnel, and how many false negatives there are .
If you could offer standardized test at that level, you may be able to keep viable candidates in the funnel longer.
Interviewing fads are set by these large companies that have the problem of systematically evaluating many thousands of candidates. A standardized test is all they want. Then they could do the rest like college admissions, and at a fraction of the cost.
I have done interviews with Karat which just outsources technical interviews to engineers elsewhere.
All of these technical interviews still suck though! I basically never code with someone watching me and find it very difficult to do in interviews. I also find it hard to find the time to actually practice this skill
In the US and Canada, to get into university, there is, or at least used to be, a standard entrance exam or something close to it (the SAT in the US, OAC scores if you were from Ontario, Canada, etc..).
Additionally, Undergraduate programs in the US and Canada, at least used to, despite their varied reputation, have a pretty standard program.
Maybe things have deteriorated so far at the high school and university level that new standardized exams are needed. But we also have a plethora of verifiable certifications whose exams are held in independent test facilities.
That maybe end up happening. Send them to a test center to do their remote hacker rank interview. Doesn’t even need to be standardized. I don’t like it but it’s one of the lower friction options.
Im revisiting technical interview prep because it has been awhile and it seems like a good time for a refresher, and it is striking how similar it all is to SAT and GMAT prep these days. A pretty cookie-cutter performance that is mostly about demonstrating that you have the time and means to properly prepare. Might as well just go the extra step at that point and have it be exactly like those standardized tests… take them once at a test center, get a score that is valid for a few years that you can just send in with your application…
So there’s AI that’s really good at doing the skills we’re hiring for. We want you to not use AI so we can hire you for a job that we’re saying we’re going to replace with AI. Sounds like a great plan.
No. Using AI requires a depth of knowledge to spot the mistakes in the generated. code, and to know how to fit all the snippets of code in to something that works.
We need to know that the developer actually has skills and isn't just secretly copying the answer off of a hidden screen. We are interviewing now, and some cantidates are obviously cheating. Our interview process is not leet code based, and reasonably chill, but we will probably have to completely rethink the process.
Since we are hiring contractors, in theory we can let them go after a couple months if they suck, but we haven't tested out how this will work in practice.
Interestingly I find AI is actually better at that kind of CS whiteboard question (implementing a binary search tree) than that "connecting middlewares to API" type task. Or at least, it's more straightforward to apply the AI, when you want a set of functions with clearly defined interfaces written - rather than making a bunch of changes across existing files.
I’ve wondered about the kind of person who starts white boarding with the pros and cons of several AI offerings. As if, confronted with a problem domain, they are choosing or hiring AI before architecture. As an interviewer, how should I adapt my questions? Something like, “How would you prompt it to add fuzzing?” “How would you prompt it to describe how each change might affect our stack?”
I've been arguing that "AI" has very little impact on meaningful technical interviews, that is ones that don't test for memorization of programming trivia: https://blog.sulami.xyz/posts/llm-interviews/
A couple of weeks ago I interviewed at a place where I had to do a take-home exercise. It's fine, I don't mind. No Leetcode. Just my own IDE, my own shortcuts, and write a piece code that solves a problem.
I was asked whether I used AI/LLM for the solution. I didn't. I felt like using an LLM to solve the problem for me wasn't the right way of showcasing knowledge. The role was for some form of 'come in with knowledge and help us'.
The response to that was basically: everybody here uses AI.
I declined the follow-up interview, as I felt that if all you have is the speed of AI to be ahead of your competitors, you're not really building the kind of things that I want to be a part of. It basically implies that the role is up in the air as soon as the AI gets better.
When I started coding I did it in notepad. I thought it was hardcore and cool. I was young and stupid. Then I adopted an IDE and I became much better at writing code.
To me AI is just another tool that helps me solve problems with code. An auto complete on steroids. A context aware stack overflow search. Not wanting to adopt or not even work somewhere where colleagues use it, sounds to me like coding in notepad AND in the process scoffing those who use an IDE.
Besides, if AI gets to the point it can replace you, it will replace you. Better to start learning how to work with it so you can fill whatever gap AI can't.
I still mainly use a text editor after several decades, and do a lot of thinking and initial design with pencil and paper. IDEs just get in the way.
I've seen the type of code AI generates. It might work, but if you think that's good or that massaging it so it works will make you any better, I have some bad news for you...
We have been interviewing people who are obviously using covert AI helper tools. Ask them a question and they respond with coherent response, but they are just reading off of a window we can't see.
In some cases it is obvious they are blathering a stream of words they don't understand. But others are able to hold something resembling a coherent conversation. We also have to allow for the fact that most people we interview aren't native English speakers, and are talking over Teams. It can be very hard to tell if they are cheating.
Asking questions to probe their technical skills is essential, otherwise you are just selecting for people who are good at talking and self promotion. We aren't just asking trivia questions.
We also give a simple code challenge, nothing difficult. If they have a working knowledge of the language, they should be able to work through the problem in 30 minutes, and we let them use an IDE and google for things like regex syntax.
Some of them are obviously using an AI, since they just start typing in a working solution. But in theory they could be a Scala expert who remembers how to use map plus a simple regex...
Unless the job you're interviewing for is remote-only, this makes perfect sense. If you expect your candidates to be able to work in your office, they should be interviewed there.
Or you can just given them a way to bypass all of that, and ask them about any significant project that the candidate did build (which is relevant to the job description, open or closed source that is released) or even open source contributions towards widely used and significant projects. (Not hello world, or demo projects, or README changes.)
Both scenarios are easily verifiable (can check that you released the project or if you made that commit or not) and in the case of open-source, the interviewer can lookup at how you code-review with others, and how you respond and reason about the code review comments of others all in public to see if you actually understand the patches you or another person submitted.
A conversation can be started around it and eliminates 95% of frauds. If the candidate cannot answer this, then no choice but give a leetcode / hackerrank hard challenge and interview them again to explain their solution and why.
A net positive to everyone and all it takes to qualify is to build something that you can point to or contribute to a significant open source project. Unlike Hackerrank which has now become a negative sum race to the bottom quest with rampant cheating thanks to LLMs.
After that, a simple whiteboard challenge and that is it.
This would be a nice interview for candidates who have open source contributions, but many who have day jobs do not. Or their open source code is 5 years old and not representative of their current skill set.
There is no shame in taking time off after leaving a job to develop or contribute to an open source project or two. The world would be a better place for it.
Well the reason I don’t has nothing to do with shame and everything to do with time. I’m allocating my extra time to work, which (on a good day) makes my company money.
For a candidate who does have OS contributions, that’s great but most will not. And the more senior they are the less likely I would imagine.
If you already have a job, you don't strictly need a different one. If you really need one, it should be okay to quit the old one first, perhaps work on open source as needed, then get a new one. As it is often said, looking for a job is a full-time job.
That is a lot like saying that a college degree is financially straining or outright impossible. In many respects, developing open source is a lot less straining, as there are no large fees, with the main expenses being living expenses. This is why it's important to live far below one's means when one does have a job.
This take totally disregards very common family situations that are much less likely during college years: putting kids through school, health issues, caring for/supporting aging parents.
Remember not that not everyone is a coastal techie with loads of extra cash to put away.
My last interview, for the job I'm currently employed in, asked for a take home assignment where I was allowed to use any tool I'd use regularly including AI. Similar process for a live coding interview iterating on the take home that followed. I personally used it to speed up wirting initial boilerplate and test suites.
I fail to see why this wouldn't be the obvious choice. Do we disallow linters or static analysis on interviews? This is a tool and checking for skill and good practices using it makes all sense.
As someone on the other side of the table, I don't care if you used AI to complete a take-home project. I care if you can explain the strengths and weaknesses of the approach it took, or if you chose to steer it in one direction or another. It usually becomes quite clear those who actually understand what the AI actually did for them.
I've been considering using a second webcam stream focused on my screen just to assure hiring managers that I don't have ChatGPT on my screen, or anywhere else. Kind of like chess players do it sometimes on online tournaments. I've been hearing people complain about cheating a lot.
I've been interviewing a bunch of developers the past year or so, and this:
> Architectural interviews are likely safe for a few years yet. From talking to people who have run these, it’s evident that someone is using AI. They often stop with long pauses, do not quite explain things succinctly, and do not understand the questions well enough to prompt the correct answer. As AI gets better (and faster), this will likely follow the same fate as the rest but I would give it some years yet.
Completely matches my experience. I don't do leet code BS, just "let's have a talk". I ask you questions about things you tell me you know about, and things I expect of someone at the level you're selling yourself at. The longest it's taken me to detect one of these scumbags was 15 minutes, and an extra 5 minutes to make sure.
Some of them make mistakes that are beyond stupid, like identity theft of someone who was born, raised and graduated in a country whose main language they cannot speak.
The smartest ones either do not know when to stop answering your questions with perfect answers (they just do not know what they're supposed to not know), or fumble their delivery and end up looking like unauthentic puppets. You just keep grinding them until you catch em.
I'm sure it's not infallible, but that's inherent to hiring. The only problem with this is cost, you're going to need a senior+ dev running the interview, and IME most are not happy to do so. But this might just be what the price of admission for running a hiring pipeline for software devs is nowadays. Heck, now feels like a good time to start a recruitment process outsourcing biz focused on the software industry.
I think this approach is not very favoured by hacker news but it's also what I prefer. It's so much easier to quickly gauge a minimum level of the basic programming knowledge and other sw knowledge by just asking some simple directed questions.
I once got a guy that claimed to have implemented multiple default HTTP JSON REST APIs and somehow had never:
- tested his API with JSON payloads, serialise serialise
- never queried his APIs manually or semi automatically (no knowledge of curl, postman or anything similar)
I don't think it did, if anyone cares.
The way I've been advocating to my colleagues who are concerned about "cheating" is that there's probably a problem with the interview process.
I prefer to focus on the think, rather than the solve.
Collaborate, as opposed to just do.
Things that really tell me if I can work with that person and if together, we can make good things.
In my uni days, I respected professors who designed exam in a way where students can utilize whatever they could to complete the assignment, including internet, their notes, calculators, etc.
I think the same applies to good tech interview. Company should adapt hiring process to friend with AI, not fight.
Ask questions that are tricky to cheat, evaluate candidates by their thought process in solving a problem and ability to clarify, discuss and justify their decisions.
To test their inherent thinking skills and base knowledge.
There's a huge difference between occasionally looking up something, and practically leaning on it. Ironically, the mass degradation of search engine result quality within the past ~decade has made it much harder for people to do the latter, and when they do, it shows much more clearly.
Show the remote candidate an AI's deficient answer to a well-asked question, and ask the candidate if they understand what exactly is wrong with the AI's assessment, or what the follow-up/rewritten prompt to the AI should be. Compile a library of such deficient chats with the AI.
Very funny :) I too failed an interview at google, also related to binary search on a white board. I never write with pens. I'm on keyboards the whole time, my handwriting is terrible.
I've built a search engine for two countries and then I was failed by a guy that wears cowboy hats to work at google in Ireland. Not a lot of cows there I'm guessing. (No offence to any real cowboys that work at google of course).
I did like the free flight to Ireland though and the nice lunch. Though I was disappointed I lost "Do no evil" company booklet.
Prediction: faangs will come up with something clever or random or just fly everyone onsite, they are so rich and popular, they can filter by any arbitrary criteria.
Second-rate companies will keep some superficial coding, but will start to emphasize more of the verbal parts like system design and retrospective. Which sucks, because those are totally subjective and mostly filters for whoever can BS better on the spot and/or cater to the interviewer's mood and biases better.
My favorite still: in-person pair programming for a realistic problem (could be made-up or shortened, but similar to the real ones on the job). Use whatever tools you want, but get the correct requirements and then explain what you just did, and why.
A shorter/easier task is to code review/critique a chunk of code, could even just print it out if in person.
I've taken this approach, and found that it's trivially easy to distinguish people relying on LLMs from people who have thought the problem through and can explain their own decision-making process.
I had a couple of people who, when asked to explain specific approaches reflected in their code, very obviously typed my question right back into ChatGPT and then recited its output verbatim. Those interviews came to an end rather quickly.
One of my favorite ones was when I asked a candidate to estimate the complexity of their solution, and ChatGPT got it wrong, giving O(log(n)) for an O(n) algorithm. When I asked leading questions to see if the candidate could see where the error came in, they starting verbatim reciting a dictionary definition of computational complexity, and could not address the specifics of the problem at all.
Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.
How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview?
Sounds like you're only selecting Foss devs and nothing more.
Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep
I don't code much outside of work. I have hobby projects from 10+ years ago, but they're not much more than landing pages copied from templates and wordpress installs. I mostly work in backend/data/platform engineering professionally.
If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which seems to defeat the purpose as it relates to this discussion)
This points to another issue - when I do code outside of work, it's often specifically to try out things I don't do at work. After a day of doing backend work, I'll maybe put together a basic web UI for something. That code is likely awful because I just need it to be functional more than good, and also probably not related to the work I'd be hired to do.
My most recent real "side projects" are a terrible OSS monte carlo simulator tool that I contributed to, but cannot explain most of the code for, and a half-working React application that has performance issues I never fixed. Both are years old at this point. I'm not sure what an interviewer would gain from those.
This relates to another issue of using people's public github as a hiring signal. I don't share any of these repos because at a glance the code is ugly, broken, incomplete.
Below the surface, I'm probably scratching a very interesting itch. Exploring a specific idea or problem, and then I stop when I get my answer.
Who are these "most people"? School was over 10 years ago for me when schoolwork was not posted on GitHub nor is it relevant to my current job anymore, and I don't do hobby coding since I have other hobbies and responsibilities.
WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
It's that trimodal-comp thing. The 6-hour leetcode gauntlet is only the overwhelming norm in the top hump of that graph. It's not normal in the other two (not that it's never seen, but the companies dumb enough to do it without offering FAANG-tier comp are shooting themselves in the foot). I've had, IDK, ten plus tech jobs and have been solidly in the middle comp tier for most of those, and have exactly once encountered anything resembling a leetcode question in the wild.
I've also done hiring, and have no idea what good leetcode would do me. I'm convinced these people thinking they "dodged a bullet" on "fakers" either spotted a real faker who they would have caught with a normal conversation anyway, or else are assuming someone with a good resume who failed their test did so because they were lying and not because they choked under the unique sort of pressure interviews present when you turn them into a dancing-monkey routine (it's approximately the same kind of stress as doing an open mic night or karaoke in front of a crowd of strangers—most people have trouble with that and lots of them fall apart at least some of the times they try it). Meanwhile, anyone who can talk about the job and their career in any depth, convincingly, without giving away that they're actually either very-green or have no idea what they're doing, while in fact not knowing how to do the job, possesses a skill at least as valuable as programming, and I find it hard to believe most such folks haven't figured out they can apply that skill directly in exchange for money instead of trying to fake their way through conversational tech interviews.
I do see how leetcode is valuable if you want to ensure that most of the candidates in your pipeline would do fine before you even evaluate them, because you offer high comp and need a way to discourage candidates who definitely can't make it before they even apply, and/or if you want to make job-hopping painful as a wink-and-nudge collusion way to keep comp suppressed. It makes sense for FAANG, in a certain way, but not because it's a good way to evaluate candidates per se.
> WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
I think programming has more commonality with other creative, 'soft' jobs like graphic design (which itself can involve programming), architecture, media, marketing, etc than meets the eye.
Many of these roles require that applicants have some sort of portfolio that can be perused by the interviewer freely. I feel co-opting that word—'portfolio'—would do us software developers a big favour instead of trivialising outside-of-work programming as 'side projects' or 'hobbies'.
>architecture, media, marketing, etc than meets the eye.
I disagree. Programing is more engineering than art. Art doesn't have source code. You can show the final painting and I can show the final product I worked on but not the source code I wrote as that belongs to my employer. Also, most art like paintings are not done by large teams, so you can show what you did in that painting but in a large SW projects, I can't show what exactly form the final product I did and what else was done by my team.
Most of my valuable work in programing is engineering, especially fixing bugs, not creating portfolios to show off. I have nothing publicly to show off, mostly because firstly, it's private to my former employers, and secondly because code gets outdated and replaced fast, most of what I worte in the past probably doesn't run today anymore, but have made my employers happy and wealthy.
I said neither 'art' nor 'paintings' which you have fixated on for some reason. I mentioned creative endeavours that are generally team-based but all generate some sort of portfolio. Whether that portfolio is from work or done in one's personal time, it is still a portfolio of past work.
Plus, software engineering is absolutely a creative endeavour. And I daresay normal 'engineering' (civil, mechanical, aero, etc) is a creative endeavour too; it's just a matter of egos and that seem to separate STEM versus non-STEM. There are portfolios for everything. I don't understand the desire for software engineers to just waltz into an interview, claim to have done X, Y, Z, with no proof, and secure a job.
The proof part is interesting. Civil is easy to prove because of its artifacts. Someone from Netflix or Meta layoffs, what proof do any of them have? Do some people defensively maintain background proof other than paycheck stubs?
Sounds like you just treat programming more like engineering than art. Some art does have source code, there is plenty of room for creative exploration with code.
>there is plenty of room for creative exploration with code.
That also pays the bills? That's not my experience. That's what hobbies are for. Jobs are for paying bills. Paying bills with hobbies an art are a luxury for privileged.
It could be considered similar to scaffolding or boilerplate in code, except usually none of this is visible in the end product, while the code boilerplate is always there. These lines are drawn light and completely covered up by the end result - sometimes even manually erased depending on the medium.
I can empathize with your position if you are also against expecting candidates "prepare for the interview" by leet code grinding or "brush up on CS concepts".
Hobby coding is million times better than that crap.
At least leetcode is something somewhat standardized (algorithms don't change). Hobby coding is not, it's something subjective and varying between the interests of each candidates and often have nothing to do with a job.
I don't like writing code for the sake of it, and have gotten a lot better, over 25 years of writing code, at evaluating whether I need to write code or whether I'd be better off using something that already exists and putting up with its limitations, or even just doing nothing (see that XKCD comic with the time-savings payoff chart).
The result is that I don't think I've written anything longer than about a ten-line shell or python or JS script for my personal use in... a decade or more.
Frankly I probably think you shouldn't be paying anyone to do the thing you're wanting to pay me to do, because computers are likely just an expensive distraction that management's pursuing because the promise of legibility, even if in-fact pointless in this case, is incredibly enticing to them, but also I like money and will build the thing you shouldn't be building for you if you pay me. I'll even do it well, if you let me. But I don't make the same mistake (much) in my own life, any more.
Would I write a bunch of code on my own if I thought it'd be worth it? Yes, but that'd almost certainly mean I had a product idea. If I were any good at thinking of product ideas, I'd long since have had my own business. I'm terrible at it. That's literally the only reason I'm applying for a job. If I had a pile of decent code to show you, it'd be because I didn't need your job.
> Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show.
First: they might have private code, but not necessarily code to show (I, for example, am rather not willing to show quite some of the code that I wrote privately).
Second: the kind of "code" that I tend to write privately (and into which I invest quite a lot of time) is really different from what I do at work, and what is actually considered "code" by many. It's more like (very incomplete) drawings and TeX notes about observations and proofs of properties and symmetries between some algorithms. Once finished, they will be very easy to systematically transform into a program in a computer language.
This is about very novel stuff, which to explain would take quite a lot of time.
The objective in an interview like this shouldn't be to grade the quality of the code you bring in any sort of scale, but to have a discussion about the options you took. In that sense, it really matters very little what you present as long as we can do a small back-and-forth that lets me into what sort of person you are.
> but to have a discussion about the options you took
I can clearly state that this is not I commonly think about code that I write privately (and also for code that I write for the job only if I must). For private code, I rather commonly start with a "gut feeling" about some unexpected symmetry that the problem that I am working on likely has, then try to formulate these "gut feelings" as mathematical properties, and later theorems. At the end, everything "fits (for outsiders: unexpectly) together".
Thus, there is hardly ever a "option that I took", but rather a "I let everything flow: from the source [my gut feeling] to the sea [which is - ironically - the source (code)]".
I’ve been working professionally for almost 30 years. I have never written a single line of code “for fun”. I write code for money. I then take that money to fund my hobbies. The absolutely last thing I want to do when I get off work is stare at a computer.
If I already have a job, unless you are paying top of market, why would I spend my weekend writing code?
It looks like you professionally sold 30 years of your life for money with no fun. You could have done something for fun all this time, and got payed for it, too. Much better that way.
I did no such thing, I spent the first 15 years as a part time fitness instructor mostly for the social aspect, hanging out with friends and training for and doing group runs and a little travel.
I spent the next 8 years married (still married) and raising two step sons and spent the last two and half years traveling extensively including over a year doing the “digital nomad thing”.
We have been averaging getting on a plane to do something on average over a dozen times a year since late 2021.
Of course the Covid lockdown slowed us down for two years.
When I am at home in Florida, I go swimming at one of the multiple pools or workout at one of the two gyms that’s part of our complex. It’s warm enough most of the year at least during the day.
During the weekends, I go downstairs and hang out at the bar and just sip soda while hanging out with my friend the bartender and whoever else is down there.
I “retired my wife” in 2020 when I was 46 and she was 44 8 years into my marriage so she could pursue her passion projects and we could pick up and travel as often as we wanted to - the joys of working remotely.
School was a few decades ago, and the code I have on Github is mostly toy stuff I do in rainy weekends, most of us have a life without room to code outside work most of the time.
> Most people have still written code for school or a hobby project
School was years and years ago, and has nothing to do with my current skills.
From the people i personally know, most do _not_ have a hobby project, even fewer have hobby projects that showcases their technical skills. Nor should they be expected to. Most people have non-programming hobbies.
> I cannot understand how some developers have no code to show.
It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
> let them make a small project over the weekend and then do another interview where you ask stuff about what they've made
If I'm paid for it, sure why not I could do that. I won't love it but hey I'm looking for a job, I'll put the legwork in.
But if this is the only or the "preferred" interview process for a company, I need to point out that it is deeply discriminatory as it advantages people who have the time to do a weekend project: for example it benefits males disproportionally (women do most of the care work in any country, also the most house work, also have a higher chance to be a single parent, all of which impacts the time they can put in a "weekend project" if they can do it at all).
> It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
It's like asking a dentist interview candidate to show you examples of fillings and crowns they did at home as a hobby. I don't understand why there is this automatic assumption that people who program at work also do it outside of work.
I ask for code, and if they have none prepared I ask that they spend at most two hours building something they enjoy.
I expect a decent developer to be able to bootstrap and write most of a fun toy project in a domain they know well or at worst some kata from the Internet within half an hour. Then we spend some time screen sharing and talking about it, similar to pair programming but less problem focused.
If you can't do it you'll likely struggle a lot when working with us because we commonly use throwaway prototypes.
Sure. A bigger organisation with more layers in their technology department could treat candidates better that would struggle with a hiring process like mine. More detailed tickets, more mentors available, bigger cashflow, things like that.
In such settings it's also somewhat common to hire consultants in bulk, like 5-10 at a time, try them out for six months and keep the ones that enjoy the work and fit well in the organisation, and over time try to employ some of them directly.
I realize it's a rhetorical question, but pretty much everybody in the arts? I'd be very surprised to find a professional musician that doesn't play as a hobby.
Artists and musicians surely have portfolios to show, but I don't think those are usually composed of things they're doing in their free time. Maybe? I'm not an artist so maybe I'm wrong. I guess then we're back to "you should have a portfolio" rather than specifically "you should do your job in your free time." I can agree with that!
If you're not interested, you're not interested. Not even about "passion" at that point, but the bare minimum interest in your industry. I said it better previously: https://news.ycombinator.com/item?id=15553482
Like many of the other commenters, I have no code to show. I'm strongly motivated at work to solve problems and create correct, performant, maintainable code. I appreciate a job well done.
Outside of work, I just don't have the motivation to code anything. I don't have sufficient at-home problems where code will fix them.
In an interview, ask me anything! ... except to show you code on Github.
I started writing code when I was 12 and started doing it professionally at 22. I'm now in my mid-30s and outside of work, I haven't written anything more than one-off scripts for my homelab in close to a decade. I'm already spending upwards of 50 hours with code each week and I need to do something else at night and on the weekends to release my brain from it. I also didn't go to school for CS, and even if I did... it was over a decade ago. So I have ~25 years of experience writing code but could not show you a single line of it. And even if I could, how would you know I was the one to write it?
This is an extremely flawed interview process in my opinion and the last time I encountered it led to an awkward scenario that led to me walking out. Personally, when I conduct interviews, it's a mix of things. We talk about your past work, I quiz you a bit on some topics you'd encounter in your day-to-day here, and then we'll spend an hour doing some combination of a code review of a working-but-flawed demo project I created, a 30-40 minute coding exercise, and/or a problem-solving scenario where I give you a problem and then we talk through how, as a pair, how we could solve it.
Maybe you forked a library because of reasons. You can tour the original repo and explain the problems. I have at least one of those examples for each time the legal or confidentiality department stepped in.
The word maybe is doing a lot of heavy lifting here. What if you never had to do that? Not everyone's work is public. Inf act I'd say most people's work is not public. Sometimes even the product is not public since it's B-2-B.
But if you’ve worked on something mature and nontrivial, you’ve forked a dependency and are able to tour it. Looks like I’ve done it on average twice per year.
My worst code is always what I wrote yesterday. Often what’s missing is context, unless I comment ad nauseam. Sure I didn’t write complete test, obey open closed principles abstract into factory functions. The code I send from my hobby projects is likely a mess, because finishing on my own time by my own unpaid constraints wills it to be so
We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
Edit: From the replies I can see people are a little defensive about not knowing it. Not knowing it is ok because it was a question I asked people 20 years ago relevant to a language long dead in the US. I blame the defensiveness on how FUBAR the current landscape is. Giving a nuanced answer to show your depth of knowledge is actually preferred. A once sentence answer is minimal.
I'm editing this because HN says I'm posting too fast, which is super annoying, but what can I do?
Here's an interesting thought on your "gotcha" - I'm 57 years old, been programming as a career for over 30 years, a lot of languages and I have no idea what the difference is.
It's a question to filter out people who don't know a lick of Delphi for a position where the person was coding Delphi. It's like level 2 after the hello world chapter. It's easier than asking a database developer the difference between a query and a view. It's a bonehead simple question that 70% of applicants couldn't answer.
You would be surprised how many bad applicants interviewers get. It's only gotten worse over the last 20 years.
I have no idea either. I can easily look it up though. You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
Lol a bit touchy aren't we?
Like I said, it's not really relevant in today's languages. It was for a Delphi/Pascal position. If you do any type of database code (like T-SQL), you would also know it. If your experience is mainly in C type languages, everything is a function so it doesn't apply.
If you hired a guy for a Delphi position who didn't know the difference between a function and a procedure, you hired the wrong guy.
procedure Hello;
begin
ShowMessage ('Hello world!');
end;
function Double (Value: Integer) : Integer;
begin
Double := Value * 2;
end;
Function or procedure is defined in every subroutine. It's a very basic question for Delphi, like what's the difference between an integer and a string.
Words like functions/procedures tend to have different connotations across languages and once one crosses one's 15th language, and each having some 20 different keywords, it become difficult to remember what the exact connotation of a word is, in a specific language/framework. This is the most likely situation of the guy whose post I responded to.
The exception to the rule is, if you have been working quite a bit _recently_ on a specific language. You are presumably talking about this situation.
No. I just looked up other responses to your post. It's obvious you got exposed as being inexperienced (or an idiot), while posing to know the definitive with your "gotcha". Being inexperienced (or ignorant) is not a problem, but being cocky is.
I think you're projecting. You aren't posting this using your real name are you?!
Here's something to consider, Dennis. Instead of using any type of reasoning that maybe I'm interviewing for a language you aren't familiar with where functions and procedure differences matter, you decided to just go off the handle and call me inexperienced and/or an idiot. This is what we call in the hiring business a "huge red flag." I recommend maybe use some of that big brain you have and apply some deductive reasoning instead of just calling people names.
Look up my other responses - I decided to call you names _later_ . Other people also pointed that out to you. I'm sure you would thick twice before asking your 'gotcha' again (the question is fine in general but not as a gotcha).
>>You can often tell an inexperienced interviewer from the extremely domain specific question they ask which _they_ are familiar with.
>Lol a bit touchy aren't we?
You lost your composure and decided to start calling names after this. I haven't asked it since the late 90s, early 2000s. It was a for a Delphi position. It's a bonehead easy question any Delphi developer who got to chapter 2 of any Delphi book would have understood. It's still an applicable question for a SQL developer and it's just as easy. I even showed you sample code. I don't see why you aren't getting it.
I'm not getting it, I also see that 2 other people are also not getting it.
>I haven't asked it since the late 90s, early 2000s.
You overall gave the impression that you are currently asking it.
And a personal rhetorical question - aren't you too old to even state this 'gotcha' business _today_ about what you did in the past? What made you state it? If I gave you the benefit of doubt - that was slip, where you omitted the past tense.
(If I did that in the 90's I'd be a embarrassed to even mention it today.)
> Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
If you're asking this question (by virtue of the present-tense "is") in the year 2025 even though by your own admission
> it's not really relevant in today's languages.
then you aren't giving candidates a good impression. Even though I would have nailed this question I would have serious reservations about any job that would ask it in an interview because it means that the person interviewing me has more concern for legacy minutiae than broad technical knowledge or problem-solving skills.
There are some absolutely ridiculous qs I've been asked like this and they've all had no followup question to illuminate why it would have been relevant
1. what version of java do you use? we used 8 at the time
2. what is the engine and version underneath your sql db?this was not for a dba role, just standard backend engineer
3. why did you use python instead of r for x project? this was about a gui automation script
It’s ok to say that it’s never professionally mattered. No one has ever been paid to know that. “Are side effects a bad pattern?” Lotsa people have needed to know that on day one.
If I'm applying for a Java position and I claim to have Java experience on my resume, it's perfectly valid for them to ask me the difference between an int, an Integer, and a BigInteger.
But it's certainly not a universal question applicable to all programming languages.
Likewise, Clubber says in their post that their 'gotcha' question doesn't apply to most languages.
> We do this too, works fine. We ask open ended questions like, "What's your favorite thing you've done in your career and why?" and "What was the most challenging project in your career and why?" If you listen, you can get a lot of insight from just those two questions. If they don't give enough detail, we'll probe a little.
The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what
interviewers want to hear. :-(
> "What's the difference between a function and a procedure." It's a one sentence answer
The terminology here differs quite a lot in different "programming communities". For example
says: "Procedure (computer science), also termed a subroutine, function, or subprogram",
i.e. there is no difference. On the other hand, Pascal programmers strongly distinguish between functions and procedures; here functions return a value, but procedures don't. Programmers who are more attracted to type theory (think Haskell) would rather consider "procedures" to be functions returning a unit type. If you rather come from a database programming background, (stored) procedures vs functions are quite different concepts.
I could go on and on. What I want to point out is that this topic is much more subtle than a "one sentence answer".
>The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis.
This is unfortunate that you would get that response. FWIW, I would be interested in hearing all this in an interview and I would look at it favorably.
>What I want to point out is that this topic is much more subtle than a "one sentence answer".
Yes, you would definitely get bonus points for nuance. The one sentence answer was minimal. What it filters out are people who don't know anything about Delphi but applying for the job with highly embellished resumes hoping to get lucky. This was for software used in hospitals, so bugs or errant code could have pretty drastic consequences.
> I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. [...] I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear.
In my experience you'll be fine giving that answer assuming you're going for the kind of programming job that hires PhDs.
You remind them you have a PhD - and in something deeply algorithmic. You can successfully answer any follow-up questions from them, as you literally have a PhD in the topic they're asking about. There's no shame in entering industry because you want jobs and money - in fact, those things are precisely what the hiring manager is able to offer you.
You'd rather be in academia but it doesn't have the pay and job security? Well, the hiring manager would rather be a snowboard instructor in Aspen but doesn't for the same reason. So you've got common ground with them.
> Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear. :-(
I would love to hear that from a candidate I'm interviewing. Who can't relate to the distinction between your ideal job and the job that will actually pay you money?
> Single interview, we ask the candidate about *his* CV and what *his* expectations are, what are *his* competences and we ask *him* to show us some code *he* has written
You... might want to think about what implicit biases you might be bringing here
That process might work for your company precisely because you are not FAANG. You don't get hundreds of applicants that are dying to get in, so people don't have that strong of a motivation to do anything it takes (including lying) to get the job.
I’ve worked at a company with 150,000 employees. The interview process was pretty much as described here. There is absolutely no reason a Big Co needs to operate any differently.
The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it.
Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
> Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
Candidate would be compensated, obviously. That's why it's expensive.
You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
> Candidate would be compensated, obviously. That's why it's expensive
Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview.
I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes.
Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this.
> You don't need him to become efficient
So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
And so that self-selects for people who already are unemployed then, right? Most developers I know (including myself) look for a new job while still having a job, as to not create a financial hole in-between. I'd be curious if that doesn't then end up with lower quality candidates who ended up unemployed to begin with?
> self-selects for people who already are unemployed then
You can say that about all forms of hiring process. If you're unemployed, you obviously have more time: to spend more time on the take-home assignments (which I hate, see another thread [1]), to add more stuff to your GitHub profile, to go to more interviews, etc.
> You can say that about all forms of hiring process
Yes, but there's a significant difference between spending a few hours on a take-home assignment and dropping your current employment to spend 4 weeks potentially in another city working full time.
And, additionally, it encourages your candidates to still be interviewing while they're on their probationary period with you, since they may be back to unemployed after 4 weeks or whatever. Which creates even more potential issues if they get a much better offer while they're onboarding with you.
I'd argue the bigger expense is on the team having to onboard what could potentially be a revolving door of temporary hires. Getting a new engineer to the point where they understand how things work and the specific weirdness of the company and its patterns is a pretty big effort at anywhere I've worked.
If you work with Boring Technology, your onboarding process has no reason to be longer than a week, unless you're trying to make the non-tech parts of the role too interesting.
> unless you're trying to make the non-tech parts of the role too interesting.
Unless your role is trivial to replace with an LLM, you need to understand the business. Maybe not for really junior role, but everything above - you need to solve issues. Tech is just a tool.
I am not sure I follow - when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Let's say you're hiring manager for a company that compares flight tickets, something similar to Google Flights or Skyscanner. You need three additional Rust engineers. You're located in Palermo, Italy.
How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Even if you're willing to have people remotely, in the same region, how many unemployed Rust developers that know that business are on the market? 0?
> when you hire you search for someone who has 100% coverage of the tech you're using and also already works for your direct competitor?
Ideally, yes. It's a common occurrence among large organisations. Google and Apple used to even have an anti-poaching agreement.
> How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Rust isn't Boring, which is why you don't do that and hire one of many Java developers and do Java, unless the tradeoff is really worth it.
How do you control for confounders and small data?
For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy.
For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave early. Human factors are big.
Sure, psychological research is hard because of this, but that's not what I'm proposing - I'm talking about just having some data on predictive validity of the hiring process. If there's some coding test: is it reliable and valid? Aren't some items redundant because they're too easy or too hard? Which items have the best discrimination parameter? How the total scores correlate with e.g. length of the test takers tenures?
Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.
Maybe some companies already do this, but I didn't see it (though my sample is small).
I've accidentally been using an AI-proof hiring technique for about 20 years: ask a junior developer to bring code with them and ask them to explain it verbally. You can then talk about what they would change, how they would change it, what they would do differently, if they've used patterns (on purpose or by accident) what the benefits/drawbacks are etc. If they're a senior dev, we give them - on the day - a small but humorously-nasty chunk of code and ask them to reason through it live.
Works really well and it mimics the what we find is the most important bit about coding.
I don't mind if they use AI to shortcut the boring stuff in the day-to-day, as long as they can think critically about the result.
This is a fine way. I’ll say that the difference between a senior and a principal is that the senior might snicker but the principal knows that there’s a chance the code was written by a founder.
And if the Principal is good, they should stand up and say exactly why the code is bad. If there's a reason to laugh because it is cliche bad, they should say so.
If someone gave me code with
if (x = 7) { ... } as part of a C eval.
Yeah, you'll get a sarcastic response back because I know it is testing code.
What I think people ignore is that personality matters. Especially at the higher levels. If you are a Principal SWE you have to be able to stand up to a CEO and say "No, sir. I think you are wrong. This is why." In a diplomatic way. Or sometimes. Less than diplomatic, depending on the CEO.
One manager that hired me was trying to figure me out. So he said (and I think he was honest at the time). "You got the job as long an you aren't an axe murderer."
To which I replied deadpan: "I hope I hid the axe well." (To be clear to all reading, I have never killed someone, nevermind with an axe! Hi FBI, NSA, CIA and pals!)
Got the job, and we got along great, I operated as his right hand.
Yep. I've also been using an AI-proof interview for years. We have a normal conversation, they talk about their work, and I do a short round of well-tested technical questions (there's no trivia, let's just talk about some concepts you probably encounter fairly regularly given your areas of expertise).
You can tell who's trying to use AI live. They're clearly reading, and they don't understand the content of their answers, and they never say "I don't know." So if you ask a followup or even "are you sure" they start to panic. It's really obvious.
Maybe this is only a real problem for the teams that offloading their interviewing skills onto some leetcode nonsense...
I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice. That discussion went on for a good few minutes and by the end of it I was teetering on leaving because the fit wasn't good.
I agreed to do it as long as they understood that I felt it was a terrible way of assessing someone's ability to code. I was allowed to use any programming language because they knew them all (allegedly).
The solution was a pretty obvious bit-shift. So I wrote memory registers up on the board and did it in Motorola 68000 Assembler (because I had been doing a lot of it around that time), halfway through they stopped me and I said I'd be happy to do it again if they gave me a computer.
>I was allowed to use any programming language because they knew them all (allegedly).
After 30 years of doing this, I find that typically the people who claim to know a lot often know very little. They're insecure in their ability so much that they've tricked themselves into not learning anything.
Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
It isn't that your interviewer knew all the languages, but that the language didn't matter.
I didn't get this until I was giving interviews. The instructions on how to give them are pretty clear. The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
> We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Except, that's not what happens. In basically every coding interview in my life, it's been a gauntlet: code this leetcode medium/hard problem while singing and tapdancing backwards. Screw up in any way -- or worse (and also commonly) miss the obscure trick that brings the solution to the next level of algorithmic complexity -- and your interview day is over. And it's only gotten worse over time, in that nowadays, interviewers start with the leetcode medium as the "warmup exercise". That's nuts.
It's not a one off. The people doing these interviews either don't know what they're supposed to be looking for, or they're at a big tech company and their mandate is to be a severe winnowing function.
> It isn't that your interviewer knew all the languages, but that the language didn't matter.
I've done enough programming interviews to know that using even a marginally exotic language (like, say, Ruby) will drastically reduce your success rate. You either use a language that your interviewer knows well, or you're adding a level of friction that will hurt you. Interviewers love to say that language doesn't matter, but in practice, if they can't know that you're not making up the syntax, then it dials up the skepticism level.
They generally do not know what they are looking for. They are generally untrained, and if they are trained, the training is probably all about using leetcode-type problems to give out interviews that are sufficiently similar that you can run stats on the results and call them "objective", which is exactly the thing we are all quite correctly complaining about. Which is perhaps anti-training.
The problem is that the business side wants to reduce it to an objective checklist, but you can't do that because of Goodhart's Law [1]. AI is throwing this problem into focus because it is basically capable of passing any objective checklist, with just a bit of human driving [2]. Interviews can not consist of "I'm going to ask a question and if you give me the objectively correct answer you get a point and if you do not give the objectively correct answer you do not". The risk of hiring someone who could give the objectively correct answers but couldn't program their way out of a wet paper bag, let alone do requirements elicitation in collaboration with other humans or architecture or risk analysis or any of the many other things that a real engineering job consists of, was already pretty high before AI.
But if interviewing is not a matter of saying the objectively correct things, a lot of people at all levels are just incapable of handling it after that. The Western philosophical mindset doesn't handle this sort of thing very well.
[2]: Note this is not necessarily bad because "AI bad!", but, if all the human on the other end can offer me is that they can drive the AI, I don't need them. I can do it myself and/or hire any number of other such people. You need to bring something to the job other than the ability to drive an AI and you need to demonstrate whatever that is in the interview process. I can type what you tell me into a computer and then fail to comprehend the answer it gives is not a value-add.
It is a gross oversimplification but you can look at the Western mindset as being a reductionistic, "things are composed of their parts" sort of view, and the Eastern mindset as a holistic mindset where breaking things into their components also destroys the thing in the process.
The reality isn't so much "in between" as "both". There is a reason the West developed a lot of tech and the East, despite thousands of years of opportunity, didn't so much. But there is also a limit to the reductionistic viewpoint.
In this case, being told that the only way to hire a truly good developer is to make a holistic evaluation of a candidate, that you can not "reduce" it to a checklist because the very act of reducing it to a checklist invalidates the process, is something that a lot of Western sorts of people just can't process. How can something be effectively impossible to break into parts?
On the other hand, it is arguably a Western viewpoint that leads to the idea of Goodhart's law in the first place; the Eastern viewpoint tends to just say "things can't be reduced" and stop the investigation there.
This is highly stereotypical, of course, and should be considered as an extremely broad classification of types of philosophy, and not really associated directly with any individual humans who may happen to be physically located in the east or west. Further as I said I think the "correct" answer is neither one, nor the other, nor anything in between, but both, so I am not casting any shade on any country or culture per se. It is a useful, if broad, framework to understand things at a very, very high level.
When I joined my current team I found they had changed the technical test after I had interviewed but before I joined. A couple of friends also applied and got rejected because of this new test.
When I finally got in the door and joined the hiring effort I was appalled to find they’d implemented a leetcode-esque series of challenges with criteria such as “if the candidate doesn’t immediately identify and then use a stack then fail interview”. There were 7 more like this with increasingly harsh criteria.
> So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
Everybody says that, but reality is they don't imho. If you don't pass the pet question quiz "they don't know how to program" or are a "faker", etc.
I've seen this over and over and if you want to test a real conversation you can ask about their experience. (I realize the challenge with that is young interviewers aren't able to do that very well with more experienced people.)
Do I? yes. I also teach my students that the goal of an interview is to convince the interviewer you are a good candidate, not to answer the questions correctly. Sometimes they correlate. Give the customer what they need not what they asked for.
Do I see others doing so? sadly no.
I feel like a lot of the replies to my comment didn't read to the end, I agree the implementation is bad. The whiteboard just isn't actually the problem. The interviewers are.
Unless they change mentality to "did this candidate show me the skills i am looking for" instead of "did they solve puzzle" the method doesn't matter.
The replies are addressing the reality of the interview landscape that fails to live up to your theory of how whiteboarding interviews should be.
It's all well and good that you and other "wise interviewer" commenters on HN actually grok what the point of interviews are, but you are unicorns in the landscape.
I don't think you made it to the last paragraph either:
> I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
+1 to all this. It still surprises me how many people, even after being in the industry for years, think the goal of any interview is to “write the best code” or “get the right answer”.
What I want to know from an interview is if you can be presented an abstract problem and collaboratively work with others on it. After that, getting the “right” answer to my contrived interview question is barely even icing on the cake.
If you complain about having to have a discussion about how to solve the problem, I no longer care about actually solving the problem, because you’ve already failed the test.
I think you're severely underestimating how much just about every software company has bought into the FAANG philosophy, and how many candidates they get who can answer those questions correctly.
Yes if you don't communicate clearly, you will get points deducted. But if you can't answer the question nearly perfectly, its basically an immediate fail.
Unfortunately I used to think this was the main purpose of the interview as well, but have been proven wrong time and time again.
The only thing that matters in most places is getting to the optimal solution quickly. It doesn't matter if you explain your thought process or ask clarifying questions, just get to the solution and answer the time and space complexity correctly and you pass.
Like others have said I think this is a symptom of the sheer number of people applying and needing to go through the process, there is no time for nuance or evaluating people on if you would actually like to work with them or not.
> The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
...while being closely monitored in a high-stakes performance in front of an audience of strangers judging them critically.
> Why are you on a thread about Google-style interviews?
For the same reason you wrote "Google-style". Because this thread is specifically about those interviews happening not at Google.
Oh, maybe you misunderstood their question. When they suggested Google wasn't relevant, they meant the company culture at Google itself because that's what you were talking about.
Perhaps. I'd even say it's part of what is taught as part of a PhD.
But if someone was ready for your exact question by having the right interview practice/experience, or they just don't care about your job so there's no stakes. Then you still aren't measuring what you think you are.
What is the functional difference between copying an AI answer and copying a StackOverflow answer, in terms of it being "cheating" during an interview?
I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts like OOP. I ask about what they've worked on previously, what they found interesting, what they found frustrating, etc.
Languages are largely teachable, it's just syntax and keywords. What I can't teach people is how to think like programmers need to: how to break down big, hard problems into smaller problems and implement solutions. If you know that, I can teach you fucking Swift, it isn't THAT complicated and there's about 5 million examples of "how do I $X" available all over the Internet.
> Languages are largely teachable, it's just syntax and keywords.
This is like "learning a natural language is just 'cramming vocabulary and grammar' - voila, you've become a fluent C1 speaker". :-)
Seriously: if you argue this way, you have only seen a very biased set of programming languages, and additionally, your knowledge of these programming languages is very superficial (i.e. you have never gotten to the "interesting"/"deep" concepts that make this particular programming language special, and which are hard to replicate in most other programming languages).
I think the argument is that for a good chunk of business work, you don't need to use the "interesting"/"deep" concepts. Sure, you'll need time to adapt to the idioms of the language you're using, but following examples you can be just as productive as others in a relatively short time.
But companies don't pay high salaries for the 80% mundane and easy tasks you do day to day. They pay for the 20% that is challenging and especially for that 1% of problems that occur only once every few months or years. If that 80% was 100% of the job then the company could pay 1/2 to 1/3rd the amount by outsourcing it.
I disagree, companies pay based on the problems you can solve to make them money or help achieve organizational goals. One of those ways can be coding, but there are many others.
No, the comparison to natural languages is what is whack. If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works. That's not really how natural language works at all, but it does work with programming languages.
> If you understand the underlying concepts that programming languages pick and choose from as features, all you have to learn is what keywords map to those concepts and the language's syntax.
If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
> The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works.
There do exist books on this topic (though more commonly for language families). See for example
> If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
Yes, and then do that in real time while you're having a conversation with someone who's been learning the language since they were a baby. It is an unreasonable comparison.
In C, implicit type narrowing/widening behavior stumped me as a noob working on noob problems. “Deep C Secrets” was a revelation when I finally found it.
> Languages are largely teachable, it's just syntax and keywords.
That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
I have to deal with students using AI to cheat on homework and exams, and I can't allow them to not even learn the basic concepts.
They could convince you with buzzwords, get hired, and then feed all problems to the AI until it reaches a point where the codebase is too big for the context, and then all their prompt “engineering” experience is basically useless.
That is the future I am trying to prevent.
Until the AI can code a full system like SAP, or an Operating System, or a Photoshop clone, by itself, we need some people in the loop, and the more knowledgeable the people, the better.
It's not useless because some people will lie and cheat. Over the years, I've interviewed hundreds of people and a substantial minority could not write even the simplest code. Many will say they would be able to filter out such people after a conversation. But, IMO, the fact that they are still able to get hired at some places shows how wrong that often is.
> That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
In this context, to a large extent it holds. Yeah. It’s probably more true of mainstream languages roughly related to each other, but in an interview, you’re not testing for knowledge of syntax and keywords. You’re trying to test for ability to build and problem solve.
I share your concern about prompt “engineers” who don’t understand the underlying codebase, language or system. But I don’t know what to do about it in the context of a technical interview.
I work for a faang subsidiary. We pay well below average salary and equity. We finally got one nice perk, a very good 401k match. A few months later it was announced that the 401k match would be scaled back "to come in line with what our parent company offers". I thought about asking "will be getting salaries or equity in line with what our parent company offers?" but that would have been useless. Management doesn't care. I'm job hunting.
> I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice.
This looks more like a culture fit test than a coding test.
I am so happy that you did this. We vote with our feet and sadly, too many tech folks are unwilling to use their power or have golden handcuff tunnel vision.
Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code.
A junior, mid, senior, staff are going to see very different things in the same codebase.
Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness.
I went through one interview with a YC company that had a first round code review. I enjoyed it so much that I ended up making a small open source app for teams that want to use code reviews: https://coderev.app (repo: https://github.com/CharlieDigital/coderev)
This is harder than it sounds, although I agree in a vacuum the idea is a good one.
So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd.
The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.
You can do code review exercises without larger context.
An example from the interview: the code included a python web API and SQL schema. Some obvious points I noticed were no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case.
You might be thinking about it in the wrong way; what you want to see is that someone can spot these types of logic errors that either a human or AI copilot might produce regardless of the larger context.
The juniors will find formatting and obvious bad practices; the senior and staff will find the real gems. This format works really well for stratification.
> no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case
I'd say all this stuff is junior-level (maybe ~mid for things like user ID integers). It's just a checklist of "obvious bad practices", it doesn't require experience.
The senior stuff is much higher-level: domain modelling, code architecture, consistency guarantees, system resilience... system design in general.
You can do all of that in a code review; the point is that it actually allows for better stratification because you can incorporate different challenges in a reasonable time frame and without having to do take homes and get working environments (you'll end up reviewing their code anyways in a followup session).
You can do it in a real code review. I think his point was that you can't do stuff like "instead of loading a YAML file at runtime this should be generated during build time using the existing infrastructure we have here" type stuff.
But I'm not sure you really need to in a job interview. It's not like you can do that with any other interview method anyway - leetcode also doesn't really touch high level architecture type stuff, and take home problems are also too small (or they should be anyway!)
In my experience you only learn how good developers' architectural taste is by working with them for a long time.
In a previous job we did code review interviews. And went the route you said due to the problem I said. And yes, it's a lot better. But what also happened over time was that the bar slowly raised. Because over time the "harder" pieces of that session started to seem rote to interviewers, they became viewed as table stakes.
Mind you this is true of any interview scheme that has a problem solving component to it. I'm not saying that the code review style is extra bad here, just that it doesn't solve this problem.
I think the way to avoid the interviewer's expectations being raised over time is to write down some guidelines for what a successful candidate should be able to do. Even if you don't know how high to set the bar at the beginning, once you've hired someone you'll have at least one example of a good answer.
In theory, you can do code reviews without larger context if the code is carefully selected. Apparently, some companies think any badly written code from their code base can just be selected though.
It's not so hard. One of the interview stages I did somewhere well known used this.
Here's the neural net model your colleague sent you. They say it's meant to do ABC, but they found limitation XYZ. What is going on? What changes would you suggest and why?
Was actually a decent combined knowledge + code question.
There are so many interesting ways to use code reviews like subtly introducing defects and bugs and see if people can follow the logic, read the code, find where the reasoning comes up short.
Some of the most interesting interviews that I felt like accurately assessed my skills (even non-live ones) where debugging and code review assessments. I didn't get offers from these cos later on because I failed the leetcodes they did later in the process but I felt the review interviews were a good way to be assessed.
Yeah it's really tempting when you discover an interesting fact to think "that would make an interesting interview question" and turn the interview into some kind of pub quiz. Happens with all forms of technical interview though. I mean 90% of leetcode questions are "this one weird trick".
Yep, I've done a lot of SQL interviews and it is always interesting to see the folks who've crash and burned at code review and killed it at writing individual queries and sometimes the unexpected, the opposite happened, the person would fly through a code review and do really subpar on writing it, a signal I usually took to mean that the person was nervous as hell in the interview.
The two folks who showed this behavior I hired anyway (they were contractors so nbd) and they were excellent hires, so I really love the code review approach for climbing up bloom's taxonomy.
I feel like identifying problems is the most important skill for success. Especially with AI, (but even before that) SEs are more often "editing" rather than "writing" code, and most of your time is either fixing odd states or anticipating them.
I like the code review approach and tried it a few times when I was needed to do interviews.
The great thing about code reviews is that there are LOTS of ways people can improve code. You can start with the basics like can you make this code run at all (i.e. compile) and can you make it create the right output. And there's also more advanced improvements like how to make the code more performant, more maintainable, and less error-prone.
Also, the candidates can talk about their reasoning about why or why not they'd change the code they're reviewing.
For example, you'd probably view the candidates differently based on their responses to seeing a code sample with a global variable.
Poor: "Everything looks fine here"
Good: "Eliminate that global variable. We can do that by refactoring this function to..."
Better: "I see that there's a global variable here. Some say they're an anti-pattern, and that is true in most but not all cases. This one here may be ok if ..., but if not you'll need to..."
100% it is more conducive to a conversational exchange that actually gives you better insight into how a developer thinks much more so than leetcode.
Coding for me is an intensely focused activity and I work from home to boot so most of the time, I'm coding in complete silence. It's very awkward to be talking about my thought process while I'm coding, but not talking is just as awkward!
I loved the idea of code reviews interviews, i've had several good ones, until yesterday when I had my first bad code review interview.
They asked me to review a function for a residential housing payment workflow, which I'm unfamiliar with. From an actual snippet of their bad production code (which has since been rewritten). In Go which I've never used (I've never professionally used the language that doesn't have error handling built-in, for example).
I had to spend more than half of my time asking questions to try and get enough context about Go error handling techniques, the abstractions they were using which we only had the import statements to and the way that the external system was structured to handle these requests to review the hundred lines of code they shared.
I was able to identify a bunch of things incidentally, like making all of the DB changes as part of a transaction so that we don't get inconsistent state or breaking up the function into sub functions, because the names were extremely long, but this was so far outside my area of expertise and comfort zone that I felt like shooting in the dark.
So just like any other interview style, they can be done very poorly.
Honestly, it was also a red flag for me that they don’t actually know what they want and have bad communication between leadership and engineering. Prior to this interview I was already on the fence about them.
They don’t work mostly in Go. Even the interviewer said that he’s vaguely familiar with this area of the code, but he doesn’t work and Go. They work mostly in Kotlin and they explicitly are advertising for solid generalists.
I don't know. A cold code review on a codebase they never saw is not super informative about how the candidate would interact with you and the code once they're in known territory.
I know nobody likes doing tech interviews but how has AI killed it ? Anyways you do want to know basics of computer science, it is a helpful thing to know if you ever want to progress beyond CRUD shitshovelling.
Also wtf is inverting a binary tree ? Like doing a "bottom-view". That shit is easy.
nah ai killed stupid tech interviews. you can easily get an idea of someones competence by literally just talking to them instead of making them do silly homework exercises and testing their rote memorisation abilities.
This is the real answer. However to gauge competence you must first have it. The fact that most people don't is why we are in this position in the first place.
I just ask to share a text editor and write down my questions. Its critical anyway because often then not its not always clear for tech questions what exactly i asked (linux command for example).
This blocks their screen too.
and yes we do know very soon if you look somewere else, take time or rephrase the question to get more time.
If you able to fake it, at that point you should just get th ejob anyway :P
650 comments
[ 3.0 ms ] story [ 333 ms ] threadBig Tech / Unicorn / Wannabe Unicorn prescreens are all basically standardized now anyway.
Of course, they'd miss out on some good talent. But in the article where it shows the quote of someone getting rejected for not inverting a binary tree on a whiteboard, that doesn't seem like a terrible thing to test for.
but the screening cost for companies is eye watering so something should be done.
It's a common sentiment.
But compare https://www.cambridge.org/core/journals/judgment-and-decisio... . ("People predicting the future performance of college students state that interviewing the students aids prediction, although in fact the interviews make predictions less accurate.")
This is effectively how Harvard was rejecting Asian applicants. They created a "personal fit" / cultural fit quality that Asians scored low on . Supreme Court found this to be discrimination.
It doesn't matter if you are truly discriminating, it matters how well you have tangible evidence of the employee not meeting the qualifications for the role.
Likely. But haters gonna hate, and lawyers gonna sue.
I'm not sure if you guys have been in charge of hiring, but there's no real alternative. In my most recent experience, we had one open position, and after interviewing 10 candidates, 3 of them were basically identical in terms of technical qualifications. How do you choose one over the other, other than the "vibes"? Anyone suggesting otherwise is either living in a weird alternate reality, or doesn't want to accept that working is a cooperative job and interpersonal relationships are very important.
There always will be exceptions for different type of roles and specializations, but that's not what I'm talking about.
Interviews based on personal feelings have hidden biases not even the interviewer is aware of.
Think of how poor the screening process is at the recruiter & CTS (left side) of the funnel, and how many false negatives there are .
If you could offer standardized test at that level, you may be able to keep viable candidates in the funnel longer.
All of these technical interviews still suck though! I basically never code with someone watching me and find it very difficult to do in interviews. I also find it hard to find the time to actually practice this skill
Additionally, Undergraduate programs in the US and Canada, at least used to, despite their varied reputation, have a pretty standard program.
Maybe things have deteriorated so far at the high school and university level that new standardized exams are needed. But we also have a plethora of verifiable certifications whose exams are held in independent test facilities.
We need to know that the developer actually has skills and isn't just secretly copying the answer off of a hidden screen. We are interviewing now, and some cantidates are obviously cheating. Our interview process is not leet code based, and reasonably chill, but we will probably have to completely rethink the process.
Since we are hiring contractors, in theory we can let them go after a couple months if they suck, but we haven't tested out how this will work in practice.
I was asked whether I used AI/LLM for the solution. I didn't. I felt like using an LLM to solve the problem for me wasn't the right way of showcasing knowledge. The role was for some form of 'come in with knowledge and help us'.
The response to that was basically: everybody here uses AI.
I declined the follow-up interview, as I felt that if all you have is the speed of AI to be ahead of your competitors, you're not really building the kind of things that I want to be a part of. It basically implies that the role is up in the air as soon as the AI gets better.
To me AI is just another tool that helps me solve problems with code. An auto complete on steroids. A context aware stack overflow search. Not wanting to adopt or not even work somewhere where colleagues use it, sounds to me like coding in notepad AND in the process scoffing those who use an IDE.
Besides, if AI gets to the point it can replace you, it will replace you. Better to start learning how to work with it so you can fill whatever gap AI can't.
I've seen the type of code AI generates. It might work, but if you think that's good or that massaging it so it works will make you any better, I have some bad news for you...
In some cases it is obvious they are blathering a stream of words they don't understand. But others are able to hold something resembling a coherent conversation. We also have to allow for the fact that most people we interview aren't native English speakers, and are talking over Teams. It can be very hard to tell if they are cheating.
Asking questions to probe their technical skills is essential, otherwise you are just selecting for people who are good at talking and self promotion. We aren't just asking trivia questions.
We also give a simple code challenge, nothing difficult. If they have a working knowledge of the language, they should be able to work through the problem in 30 minutes, and we let them use an IDE and google for things like regex syntax.
Some of them are obviously using an AI, since they just start typing in a working solution. But in theory they could be a Scala expert who remembers how to use map plus a simple regex...
Unless the job you're interviewing for is remote-only, this makes perfect sense. If you expect your candidates to be able to work in your office, they should be interviewed there.
Both scenarios are easily verifiable (can check that you released the project or if you made that commit or not) and in the case of open-source, the interviewer can lookup at how you code-review with others, and how you respond and reason about the code review comments of others all in public to see if you actually understand the patches you or another person submitted.
A conversation can be started around it and eliminates 95% of frauds. If the candidate cannot answer this, then no choice but give a leetcode / hackerrank hard challenge and interview them again to explain their solution and why.
A net positive to everyone and all it takes to qualify is to build something that you can point to or contribute to a significant open source project. Unlike Hackerrank which has now become a negative sum race to the bottom quest with rampant cheating thanks to LLMs.
After that, a simple whiteboard challenge and that is it.
For a candidate who does have OS contributions, that’s great but most will not. And the more senior they are the less likely I would imagine.
Remember not that not everyone is a coastal techie with loads of extra cash to put away.
I fail to see why this wouldn't be the obvious choice. Do we disallow linters or static analysis on interviews? This is a tool and checking for skill and good practices using it makes all sense.
> Architectural interviews are likely safe for a few years yet. From talking to people who have run these, it’s evident that someone is using AI. They often stop with long pauses, do not quite explain things succinctly, and do not understand the questions well enough to prompt the correct answer. As AI gets better (and faster), this will likely follow the same fate as the rest but I would give it some years yet.
Completely matches my experience. I don't do leet code BS, just "let's have a talk". I ask you questions about things you tell me you know about, and things I expect of someone at the level you're selling yourself at. The longest it's taken me to detect one of these scumbags was 15 minutes, and an extra 5 minutes to make sure.
Some of them make mistakes that are beyond stupid, like identity theft of someone who was born, raised and graduated in a country whose main language they cannot speak.
The smartest ones either do not know when to stop answering your questions with perfect answers (they just do not know what they're supposed to not know), or fumble their delivery and end up looking like unauthentic puppets. You just keep grinding them until you catch em.
I'm sure it's not infallible, but that's inherent to hiring. The only problem with this is cost, you're going to need a senior+ dev running the interview, and IME most are not happy to do so. But this might just be what the price of admission for running a hiring pipeline for software devs is nowadays. Heck, now feels like a good time to start a recruitment process outsourcing biz focused on the software industry.
I once got a guy that claimed to have implemented multiple default HTTP JSON REST APIs and somehow had never:
- tested his API with JSON payloads, serialise serialise - never queried his APIs manually or semi automatically (no knowledge of curl, postman or anything similar)
Collaborate, as opposed to just do.
Things that really tell me if I can work with that person and if together, we can make good things.
I think the same applies to good tech interview. Company should adapt hiring process to friend with AI, not fight.
There's a huge difference between occasionally looking up something, and practically leaning on it. Ironically, the mass degradation of search engine result quality within the past ~decade has made it much harder for people to do the latter, and when they do, it shows much more clearly.
I've built a search engine for two countries and then I was failed by a guy that wears cowboy hats to work at google in Ireland. Not a lot of cows there I'm guessing. (No offence to any real cowboys that work at google of course).
I did like the free flight to Ireland though and the nice lunch. Though I was disappointed I lost "Do no evil" company booklet.
Dang! I knew it was a mistake leaving my hat at home. Little things like that people tend to forget.
Second-rate companies will keep some superficial coding, but will start to emphasize more of the verbal parts like system design and retrospective. Which sucks, because those are totally subjective and mostly filters for whoever can BS better on the spot and/or cater to the interviewer's mood and biases better.
My favorite still: in-person pair programming for a realistic problem (could be made-up or shortened, but similar to the real ones on the job). Use whatever tools you want, but get the correct requirements and then explain what you just did, and why.
A shorter/easier task is to code review/critique a chunk of code, could even just print it out if in person.
I had a couple of people who, when asked to explain specific approaches reflected in their code, very obviously typed my question right back into ChatGPT and then recited its output verbatim. Those interviews came to an end rather quickly.
One of my favorite ones was when I asked a candidate to estimate the complexity of their solution, and ChatGPT got it wrong, giving O(log(n)) for an O(n) algorithm. When I asked leading questions to see if the candidate could see where the error came in, they starting verbatim reciting a dictionary definition of computational complexity, and could not address the specifics of the problem at all.
How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview?
Sounds like you're only selecting Foss devs and nothing more.
If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep
If I were asked to make a small project over a weekend, I'd be likely to decline rather than doing a more standard interview, or I'd use AI to do it in a reasonable timeframe (which seems to defeat the purpose as it relates to this discussion)
My most recent real "side projects" are a terrible OSS monte carlo simulator tool that I contributed to, but cannot explain most of the code for, and a half-working React application that has performance issues I never fixed. Both are years old at this point. I'm not sure what an interviewer would gain from those.
Below the surface, I'm probably scratching a very interesting itch. Exploring a specific idea or problem, and then I stop when I get my answer.
WTF is this hobby coding bullshit expectations? What other professions expect you do more work after work as a hobby and show it? Do bus drivers film themselves driving busses after work as a hobby? Do surgeons cut up people in their spare time as a hobby?
I've also done hiring, and have no idea what good leetcode would do me. I'm convinced these people thinking they "dodged a bullet" on "fakers" either spotted a real faker who they would have caught with a normal conversation anyway, or else are assuming someone with a good resume who failed their test did so because they were lying and not because they choked under the unique sort of pressure interviews present when you turn them into a dancing-monkey routine (it's approximately the same kind of stress as doing an open mic night or karaoke in front of a crowd of strangers—most people have trouble with that and lots of them fall apart at least some of the times they try it). Meanwhile, anyone who can talk about the job and their career in any depth, convincingly, without giving away that they're actually either very-green or have no idea what they're doing, while in fact not knowing how to do the job, possesses a skill at least as valuable as programming, and I find it hard to believe most such folks haven't figured out they can apply that skill directly in exchange for money instead of trying to fake their way through conversational tech interviews.
I do see how leetcode is valuable if you want to ensure that most of the candidates in your pipeline would do fine before you even evaluate them, because you offer high comp and need a way to discourage candidates who definitely can't make it before they even apply, and/or if you want to make job-hopping painful as a wink-and-nudge collusion way to keep comp suppressed. It makes sense for FAANG, in a certain way, but not because it's a good way to evaluate candidates per se.
I think programming has more commonality with other creative, 'soft' jobs like graphic design (which itself can involve programming), architecture, media, marketing, etc than meets the eye.
Many of these roles require that applicants have some sort of portfolio that can be perused by the interviewer freely. I feel co-opting that word—'portfolio'—would do us software developers a big favour instead of trivialising outside-of-work programming as 'side projects' or 'hobbies'.
I disagree. Programing is more engineering than art. Art doesn't have source code. You can show the final painting and I can show the final product I worked on but not the source code I wrote as that belongs to my employer. Also, most art like paintings are not done by large teams, so you can show what you did in that painting but in a large SW projects, I can't show what exactly form the final product I did and what else was done by my team.
Most of my valuable work in programing is engineering, especially fixing bugs, not creating portfolios to show off. I have nothing publicly to show off, mostly because firstly, it's private to my former employers, and secondly because code gets outdated and replaced fast, most of what I worte in the past probably doesn't run today anymore, but have made my employers happy and wealthy.
Plus, software engineering is absolutely a creative endeavour. And I daresay normal 'engineering' (civil, mechanical, aero, etc) is a creative endeavour too; it's just a matter of egos and that seem to separate STEM versus non-STEM. There are portfolios for everything. I don't understand the desire for software engineers to just waltz into an interview, claim to have done X, Y, Z, with no proof, and secure a job.
That also pays the bills? That's not my experience. That's what hobbies are for. Jobs are for paying bills. Paying bills with hobbies an art are a luxury for privileged.
Drawn art absolutely does have something like it:
https://www.reddit.com/r/learntodraw/comments/nibjjn/any_adv...
It could be considered similar to scaffolding or boilerplate in code, except usually none of this is visible in the end product, while the code boilerplate is always there. These lines are drawn light and completely covered up by the end result - sometimes even manually erased depending on the medium.
I can empathize with your position if you are also against expecting candidates "prepare for the interview" by leet code grinding or "brush up on CS concepts".
Hobby coding is million times better than that crap.
The result is that I don't think I've written anything longer than about a ten-line shell or python or JS script for my personal use in... a decade or more.
Frankly I probably think you shouldn't be paying anyone to do the thing you're wanting to pay me to do, because computers are likely just an expensive distraction that management's pursuing because the promise of legibility, even if in-fact pointless in this case, is incredibly enticing to them, but also I like money and will build the thing you shouldn't be building for you if you pay me. I'll even do it well, if you let me. But I don't make the same mistake (much) in my own life, any more.
Would I write a bunch of code on my own if I thought it'd be worth it? Yes, but that'd almost certainly mean I had a product idea. If I were any good at thinking of product ideas, I'd long since have had my own business. I'm terrible at it. That's literally the only reason I'm applying for a job. If I had a pile of decent code to show you, it'd be because I didn't need your job.
First: they might have private code, but not necessarily code to show (I, for example, am rather not willing to show quite some of the code that I wrote privately).
Second: the kind of "code" that I tend to write privately (and into which I invest quite a lot of time) is really different from what I do at work, and what is actually considered "code" by many. It's more like (very incomplete) drawings and TeX notes about observations and proofs of properties and symmetries between some algorithms. Once finished, they will be very easy to systematically transform into a program in a computer language.
This is about very novel stuff, which to explain would take quite a lot of time.
I can clearly state that this is not I commonly think about code that I write privately (and also for code that I write for the job only if I must). For private code, I rather commonly start with a "gut feeling" about some unexpected symmetry that the problem that I am working on likely has, then try to formulate these "gut feelings" as mathematical properties, and later theorems. At the end, everything "fits (for outsiders: unexpectly) together".
Thus, there is hardly ever a "option that I took", but rather a "I let everything flow: from the source [my gut feeling] to the sea [which is - ironically - the source (code)]".
If I already have a job, unless you are paying top of market, why would I spend my weekend writing code?
I spent the next 8 years married (still married) and raising two step sons and spent the last two and half years traveling extensively including over a year doing the “digital nomad thing”.
We have been averaging getting on a plane to do something on average over a dozen times a year since late 2021.
Of course the Covid lockdown slowed us down for two years.
When I am at home in Florida, I go swimming at one of the multiple pools or workout at one of the two gyms that’s part of our complex. It’s warm enough most of the year at least during the day.
During the weekends, I go downstairs and hang out at the bar and just sip soda while hanging out with my friend the bartender and whoever else is down there.
I “retired my wife” in 2020 when I was 46 and she was 44 8 years into my marriage so she could pursue her passion projects and we could pick up and travel as often as we wanted to - the joys of working remotely.
Worse actually. There is more to life than code - unless you are a savant. Most of us aren't.
But it is the way you are, you probably know no better and you are doing your best, what you can do is to refuse to interview.
Friends, family, stuff to take care of.
Not everyone gets to do so.
I’m a data engineer, so at work I mostly use SQL, Python and Bash. There’s not much overlap.
School was years and years ago, and has nothing to do with my current skills.
From the people i personally know, most do _not_ have a hobby project, even fewer have hobby projects that showcases their technical skills. Nor should they be expected to. Most people have non-programming hobbies.
> I cannot understand how some developers have no code to show.
It's really not that deep, I'm worried if you really cannot understand. I don't code outside of work, I'm not interested in doing it. I'm good at software engineering, not passionate about it. I have a bunch of other hobbies. There's no reason I'd have any code to show now or at any point in the future.
> let them make a small project over the weekend and then do another interview where you ask stuff about what they've made
If I'm paid for it, sure why not I could do that. I won't love it but hey I'm looking for a job, I'll put the legwork in. But if this is the only or the "preferred" interview process for a company, I need to point out that it is deeply discriminatory as it advantages people who have the time to do a weekend project: for example it benefits males disproportionally (women do most of the care work in any country, also the most house work, also have a higher chance to be a single parent, all of which impacts the time they can put in a "weekend project" if they can do it at all).
It's like asking a dentist interview candidate to show you examples of fillings and crowns they did at home as a hobby. I don't understand why there is this automatic assumption that people who program at work also do it outside of work.
I expect a decent developer to be able to bootstrap and write most of a fun toy project in a domain they know well or at worst some kata from the Internet within half an hour. Then we spend some time screen sharing and talking about it, similar to pair programming but less problem focused.
If you can't do it you'll likely struggle a lot when working with us because we commonly use throwaway prototypes.
In such settings it's also somewhat common to hire consultants in bulk, like 5-10 at a time, try them out for six months and keep the ones that enjoy the work and fit well in the organisation, and over time try to employ some of them directly.
Outside of work, I just don't have the motivation to code anything. I don't have sufficient at-home problems where code will fix them.
In an interview, ask me anything! ... except to show you code on Github.
This is an extremely flawed interview process in my opinion and the last time I encountered it led to an awkward scenario that led to me walking out. Personally, when I conduct interviews, it's a mix of things. We talk about your past work, I quiz you a bit on some topics you'd encounter in your day-to-day here, and then we'll spend an hour doing some combination of a code review of a working-but-flawed demo project I created, a 30-40 minute coding exercise, and/or a problem-solving scenario where I give you a problem and then we talk through how, as a pair, how we could solve it.
The word maybe is doing a lot of heavy lifting here. What if you never had to do that? Not everyone's work is public. Inf act I'd say most people's work is not public. Sometimes even the product is not public since it's B-2-B.
Our "gotcha," which doesn't apply to most languages anymore is, "What's the difference between a function and a procedure." It's a one sentence answer, but people who didn't know it would give some pretty enlightening answers.
Edit: From the replies I can see people are a little defensive about not knowing it. Not knowing it is ok because it was a question I asked people 20 years ago relevant to a language long dead in the US. I blame the defensiveness on how FUBAR the current landscape is. Giving a nuanced answer to show your depth of knowledge is actually preferred. A once sentence answer is minimal.
I'm editing this because HN says I'm posting too fast, which is super annoying, but what can I do?
https://stackoverflow.com/questions/721090/what-is-the-diffe...
... in Pascal/Delphi.
You would be surprised how many bad applicants interviewers get. It's only gotten worse over the last 20 years.
Lol a bit touchy aren't we?
Like I said, it's not really relevant in today's languages. It was for a Delphi/Pascal position. If you do any type of database code (like T-SQL), you would also know it. If your experience is mainly in C type languages, everything is a function so it doesn't apply.
If you hired a guy for a Delphi position who didn't know the difference between a function and a procedure, you hired the wrong guy.
Function or procedure is defined in every subroutine. It's a very basic question for Delphi, like what's the difference between an integer and a string.Words like functions/procedures tend to have different connotations across languages and once one crosses one's 15th language, and each having some 20 different keywords, it become difficult to remember what the exact connotation of a word is, in a specific language/framework. This is the most likely situation of the guy whose post I responded to.
The exception to the rule is, if you have been working quite a bit _recently_ on a specific language. You are presumably talking about this situation.
No. I just looked up other responses to your post. It's obvious you got exposed as being inexperienced (or an idiot), while posing to know the definitive with your "gotcha". Being inexperienced (or ignorant) is not a problem, but being cocky is.
Here's something to consider, Dennis. Instead of using any type of reasoning that maybe I'm interviewing for a language you aren't familiar with where functions and procedure differences matter, you decided to just go off the handle and call me inexperienced and/or an idiot. This is what we call in the hiring business a "huge red flag." I recommend maybe use some of that big brain you have and apply some deductive reasoning instead of just calling people names.
>Lol a bit touchy aren't we?
You lost your composure and decided to start calling names after this. I haven't asked it since the late 90s, early 2000s. It was a for a Delphi position. It's a bonehead easy question any Delphi developer who got to chapter 2 of any Delphi book would have understood. It's still an applicable question for a SQL developer and it's just as easy. I even showed you sample code. I don't see why you aren't getting it.
>I haven't asked it since the late 90s, early 2000s.
You overall gave the impression that you are currently asking it.
And a personal rhetorical question - aren't you too old to even state this 'gotcha' business _today_ about what you did in the past? What made you state it? If I gave you the benefit of doubt - that was slip, where you omitted the past tense.
(If I did that in the 90's I'd be a embarrassed to even mention it today.)
Yes, but you're the only one who threw a tantrum. Enjoy your career.
If you're asking this question (by virtue of the present-tense "is") in the year 2025 even though by your own admission
> it's not really relevant in today's languages.
then you aren't giving candidates a good impression. Even though I would have nailed this question I would have serious reservations about any job that would ask it in an interview because it means that the person interviewing me has more concern for legacy minutiae than broad technical knowledge or problem-solving skills.
But it's certainly not a universal question applicable to all programming languages.
Likewise, Clubber says in their post that their 'gotcha' question doesn't apply to most languages.
The problem is: there is a very negative incentive to give honest answers. If I were to answer these questions honestly, I'd bring up some very interesting theorems (related to some deep algorithmic topics) that I proved in my PhD thesis. Yes, I would have loved to stay in academia, but I switched to industry because of the bad job prospects in academia - this is not what interviewers want to hear. :-(
> "What's the difference between a function and a procedure." It's a one sentence answer
The terminology here differs quite a lot in different "programming communities". For example
> https://en.wikipedia.org/w/index.php?title=Procedure&oldid=1...
says: "Procedure (computer science), also termed a subroutine, function, or subprogram",
i.e. there is no difference. On the other hand, Pascal programmers strongly distinguish between functions and procedures; here functions return a value, but procedures don't. Programmers who are more attracted to type theory (think Haskell) would rather consider "procedures" to be functions returning a unit type. If you rather come from a database programming background, (stored) procedures vs functions are quite different concepts.
I could go on and on. What I want to point out is that this topic is much more subtle than a "one sentence answer".
This is unfortunate that you would get that response. FWIW, I would be interested in hearing all this in an interview and I would look at it favorably.
>What I want to point out is that this topic is much more subtle than a "one sentence answer".
Yes, you would definitely get bonus points for nuance. The one sentence answer was minimal. What it filters out are people who don't know anything about Delphi but applying for the job with highly embellished resumes hoping to get lucky. This was for software used in hospitals, so bugs or errant code could have pretty drastic consequences.
In my experience you'll be fine giving that answer assuming you're going for the kind of programming job that hires PhDs.
You remind them you have a PhD - and in something deeply algorithmic. You can successfully answer any follow-up questions from them, as you literally have a PhD in the topic they're asking about. There's no shame in entering industry because you want jobs and money - in fact, those things are precisely what the hiring manager is able to offer you.
You'd rather be in academia but it doesn't have the pay and job security? Well, the hiring manager would rather be a snowboard instructor in Aspen but doesn't for the same reason. So you've got common ground with them.
I would love to hear that from a candidate I'm interviewing. Who can't relate to the distinction between your ideal job and the job that will actually pay you money?
My answer would be along the lines of "It's 2025, no one has talked about procedures for 20+ years"
You... might want to think about what implicit biases you might be bringing here
Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
[1] https://en.wikipedia.org/wiki/Psychometrics
What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
Because you have zero context of what the org is working on.
Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview.
I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes.
Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this.
> You don't need him to become efficient
So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
You can say that about all forms of hiring process. If you're unemployed, you obviously have more time: to spend more time on the take-home assignments (which I hate, see another thread [1]), to add more stuff to your GitHub profile, to go to more interviews, etc.
[1] https://news.ycombinator.com/item?id=40200397
Yes, but there's a significant difference between spending a few hours on a take-home assignment and dropping your current employment to spend 4 weeks potentially in another city working full time.
Unless your role is trivial to replace with an LLM, you need to understand the business. Maybe not for really junior role, but everything above - you need to solve issues. Tech is just a tool.
Let's say you're hiring manager for a company that compares flight tickets, something similar to Google Flights or Skyscanner. You need three additional Rust engineers. You're located in Palermo, Italy.
How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Even if you're willing to have people remotely, in the same region, how many unemployed Rust developers that know that business are on the market? 0?
Ideally, yes. It's a common occurrence among large organisations. Google and Apple used to even have an anti-poaching agreement.
> How do you hire people that would not only know Rust, be willing to move to Palermo, or at least visit occasionally, but also know the airfare business?
Rust isn't Boring, which is why you don't do that and hire one of many Java developers and do Java, unless the tradeoff is really worth it.
For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy.
For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave early. Human factors are big.
Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.
Maybe some companies already do this, but I didn't see it (though my sample is small).
Works really well and it mimics the what we find is the most important bit about coding.
I don't mind if they use AI to shortcut the boring stuff in the day-to-day, as long as they can think critically about the result.
If someone gave me code with
if (x = 7) { ... } as part of a C eval.
Yeah, you'll get a sarcastic response back because I know it is testing code.
What I think people ignore is that personality matters. Especially at the higher levels. If you are a Principal SWE you have to be able to stand up to a CEO and say "No, sir. I think you are wrong. This is why." In a diplomatic way. Or sometimes. Less than diplomatic, depending on the CEO.
One manager that hired me was trying to figure me out. So he said (and I think he was honest at the time). "You got the job as long an you aren't an axe murderer."
To which I replied deadpan: "I hope I hid the axe well." (To be clear to all reading, I have never killed someone, nevermind with an axe! Hi FBI, NSA, CIA and pals!)
Got the job, and we got along great, I operated as his right hand.
You can tell who's trying to use AI live. They're clearly reading, and they don't understand the content of their answers, and they never say "I don't know." So if you ask a followup or even "are you sure" they start to panic. It's really obvious.
Maybe this is only a real problem for the teams that offloading their interviewing skills onto some leetcode nonsense...
I agreed to do it as long as they understood that I felt it was a terrible way of assessing someone's ability to code. I was allowed to use any programming language because they knew them all (allegedly).
The solution was a pretty obvious bit-shift. So I wrote memory registers up on the board and did it in Motorola 68000 Assembler (because I had been doing a lot of it around that time), halfway through they stopped me and I said I'd be happy to do it again if they gave me a computer.
The offered me the job. I went elsewhere.
After 30 years of doing this, I find that typically the people who claim to know a lot often know very little. They're insecure in their ability so much that they've tricked themselves into not learning anything.
Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem.
It isn't that your interviewer knew all the languages, but that the language didn't matter.
I didn't get this until I was giving interviews. The instructions on how to give them are pretty clear. The goal isn't to "solve the puzzle" but instead to demonstrate you can reason about it effectively, communicate your knowledge and communicate as part of problem solving.
I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
Except, that's not what happens. In basically every coding interview in my life, it's been a gauntlet: code this leetcode medium/hard problem while singing and tapdancing backwards. Screw up in any way -- or worse (and also commonly) miss the obscure trick that brings the solution to the next level of algorithmic complexity -- and your interview day is over. And it's only gotten worse over time, in that nowadays, interviewers start with the leetcode medium as the "warmup exercise". That's nuts.
It's not a one off. The people doing these interviews either don't know what they're supposed to be looking for, or they're at a big tech company and their mandate is to be a severe winnowing function.
> It isn't that your interviewer knew all the languages, but that the language didn't matter.
I've done enough programming interviews to know that using even a marginally exotic language (like, say, Ruby) will drastically reduce your success rate. You either use a language that your interviewer knows well, or you're adding a level of friction that will hurt you. Interviewers love to say that language doesn't matter, but in practice, if they can't know that you're not making up the syntax, then it dials up the skepticism level.
The problem is that the business side wants to reduce it to an objective checklist, but you can't do that because of Goodhart's Law [1]. AI is throwing this problem into focus because it is basically capable of passing any objective checklist, with just a bit of human driving [2]. Interviews can not consist of "I'm going to ask a question and if you give me the objectively correct answer you get a point and if you do not give the objectively correct answer you do not". The risk of hiring someone who could give the objectively correct answers but couldn't program their way out of a wet paper bag, let alone do requirements elicitation in collaboration with other humans or architecture or risk analysis or any of the many other things that a real engineering job consists of, was already pretty high before AI.
But if interviewing is not a matter of saying the objectively correct things, a lot of people at all levels are just incapable of handling it after that. The Western philosophical mindset doesn't handle this sort of thing very well.
[1]: https://en.wikipedia.org/wiki/Goodhart%27s_law
[2]: Note this is not necessarily bad because "AI bad!", but, if all the human on the other end can offer me is that they can drive the AI, I don't need them. I can do it myself and/or hire any number of other such people. You need to bring something to the job other than the ability to drive an AI and you need to demonstrate whatever that is in the interview process. I can type what you tell me into a computer and then fail to comprehend the answer it gives is not a value-add.
Mind elaborating on that?
The reality isn't so much "in between" as "both". There is a reason the West developed a lot of tech and the East, despite thousands of years of opportunity, didn't so much. But there is also a limit to the reductionistic viewpoint.
In this case, being told that the only way to hire a truly good developer is to make a holistic evaluation of a candidate, that you can not "reduce" it to a checklist because the very act of reducing it to a checklist invalidates the process, is something that a lot of Western sorts of people just can't process. How can something be effectively impossible to break into parts?
On the other hand, it is arguably a Western viewpoint that leads to the idea of Goodhart's law in the first place; the Eastern viewpoint tends to just say "things can't be reduced" and stop the investigation there.
This is highly stereotypical, of course, and should be considered as an extremely broad classification of types of philosophy, and not really associated directly with any individual humans who may happen to be physically located in the east or west. Further as I said I think the "correct" answer is neither one, nor the other, nor anything in between, but both, so I am not casting any shade on any country or culture per se. It is a useful, if broad, framework to understand things at a very, very high level.
When I finally got in the door and joined the hiring effort I was appalled to find they’d implemented a leetcode-esque series of challenges with criteria such as “if the candidate doesn’t immediately identify and then use a stack then fail interview”. There were 7 more like this with increasingly harsh criteria.
I would not have passed.
Everybody says that, but reality is they don't imho. If you don't pass the pet question quiz "they don't know how to program" or are a "faker", etc.
I've seen this over and over and if you want to test a real conversation you can ask about their experience. (I realize the challenge with that is young interviewers aren't able to do that very well with more experienced people.)
And do you frame the problem like that when giving interviews? Or the candidates are led to believe working code is expected?
Do I see others doing so? sadly no.
I feel like a lot of the replies to my comment didn't read to the end, I agree the implementation is bad. The whiteboard just isn't actually the problem. The interviewers are.
Unless they change mentality to "did this candidate show me the skills i am looking for" instead of "did they solve puzzle" the method doesn't matter.
It's all well and good that you and other "wise interviewer" commenters on HN actually grok what the point of interviews are, but you are unicorns in the landscape.
> I know many interviewers also didn't get it, and it became just "do you know the trick to my puzzle". That pattern of failure is a good reason to deprecate white board interviews, not "I don't write on a whiteboard when i program in real life".
What I want to know from an interview is if you can be presented an abstract problem and collaboratively work with others on it. After that, getting the “right” answer to my contrived interview question is barely even icing on the cake.
If you complain about having to have a discussion about how to solve the problem, I no longer care about actually solving the problem, because you’ve already failed the test.
Yes if you don't communicate clearly, you will get points deducted. But if you can't answer the question nearly perfectly, its basically an immediate fail.
The only thing that matters in most places is getting to the optimal solution quickly. It doesn't matter if you explain your thought process or ask clarifying questions, just get to the solution and answer the time and space complexity correctly and you pass.
Like others have said I think this is a symptom of the sheer number of people applying and needing to go through the process, there is no time for nuance or evaluating people on if you would actually like to work with them or not.
...while being closely monitored in a high-stakes performance in front of an audience of strangers judging them critically.
So why is Google relevant to this in any way?
Sucks for you, then. Why are you on a thread about Google-style interviews?
For the same reason you wrote "Google-style". Because this thread is specifically about those interviews happening not at Google.
Oh, maybe you misunderstood their question. When they suggested Google wasn't relevant, they meant the company culture at Google itself because that's what you were talking about.
But if someone was ready for your exact question by having the right interview practice/experience, or they just don't care about your job so there's no stakes. Then you still aren't measuring what you think you are.
Today? Now that's when it is tricky. How can we know you are not one of these prompt "engineers" copy paster? That's the issue being discussed.
20 years and many new technologies of difference.
I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts like OOP. I ask about what they've worked on previously, what they found interesting, what they found frustrating, etc.
Languages are largely teachable, it's just syntax and keywords. What I can't teach people is how to think like programmers need to: how to break down big, hard problems into smaller problems and implement solutions. If you know that, I can teach you fucking Swift, it isn't THAT complicated and there's about 5 million examples of "how do I $X" available all over the Internet.
This is like "learning a natural language is just 'cramming vocabulary and grammar' - voila, you've become a fluent C1 speaker". :-)
Seriously: if you argue this way, you have only seen a very biased set of programming languages, and additionally, your knowledge of these programming languages is very superficial (i.e. you have never gotten to the "interesting"/"deep" concepts that make this particular programming language special, and which are hard to replicate in most other programming languages).
This is not something nice to say about the colleagues. :-)
That's what the MBA people want to believe. To lower costs, or if they see writing code as an operating expense, instead of R&D.
If this is true or not, it depends on many, many factors, and it can change over the course of the business life.
The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works. That's not really how natural language works at all, but it does work with programming languages.
If you understand the grammatical topics that a natural language picks, all you have to learn is what word transformation rules map to those concepts, and the natural language's vocabulary.
> The comparison to natural languages would be if you could learn one language and then quickly pick up every other language of that "class" after learning how that single language works.
There do exist books on this topic (though more commonly for language families). See for example
https://www.quadrilingual.com/
or the book
EuRom 5. Leggere e capire 5 lingue romanze
> That's not really how natural language works at all, but it does work with programming languages.
... it might give you some shallow knowledge in a very limited subset of programming languages.
Yes, and then do that in real time while you're having a conversation with someone who's been learning the language since they were a baby. It is an unreasonable comparison.
Then again, that’s C.
That's only true for a subset of programming languages, and it requires you to already know how to program in at least another language of the same family. Knowing Java will not help you with Haskell, but it will help you with C#.
I have to deal with students using AI to cheat on homework and exams, and I can't allow them to not even learn the basic concepts.
They could convince you with buzzwords, get hired, and then feed all problems to the AI until it reaches a point where the codebase is too big for the context, and then all their prompt “engineering” experience is basically useless.
That is the future I am trying to prevent.
Until the AI can code a full system like SAP, or an Operating System, or a Photoshop clone, by itself, we need some people in the loop, and the more knowledgeable the people, the better.
That's true, but most of the industry is running on a subset of programming languages.
In this context, to a large extent it holds. Yeah. It’s probably more true of mainstream languages roughly related to each other, but in an interview, you’re not testing for knowledge of syntax and keywords. You’re trying to test for ability to build and problem solve.
I share your concern about prompt “engineers” who don’t understand the underlying codebase, language or system. But I don’t know what to do about it in the context of a technical interview.
I have yet to come across an interviewer who has a clue about anything that interests me.
This looks more like a culture fit test than a coding test.
I am so happy that you did this. We vote with our feet and sadly, too many tech folks are unwilling to use their power or have golden handcuff tunnel vision.
Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code.
A junior, mid, senior, staff are going to see very different things in the same codebase.
Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness.
I went through one interview with a YC company that had a first round code review. I enjoyed it so much that I ended up making a small open source app for teams that want to use code reviews: https://coderev.app (repo: https://github.com/CharlieDigital/coderev)
I guess it would degrade to stackoverflow-like poems eventually, but still interesting.
It would be interesting, but I agree it would need to be content moderated to some extent.
https://codereview.stackexchange.com
So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd.
The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.
An example from the interview: the code included a python web API and SQL schema. Some obvious points I noticed were no input validation, string concatenating for the database access (SQL injection), no input scrubbing (XSS), based on the call pattern there were some missing indices, a few bad data type choices (e.g. integer for user ID), a possible infinite loop in one case.
You might be thinking about it in the wrong way; what you want to see is that someone can spot these types of logic errors that either a human or AI copilot might produce regardless of the larger context.
The juniors will find formatting and obvious bad practices; the senior and staff will find the real gems. This format works really well for stratification.
I'd say all this stuff is junior-level (maybe ~mid for things like user ID integers). It's just a checklist of "obvious bad practices", it doesn't require experience.
The senior stuff is much higher-level: domain modelling, code architecture, consistency guarantees, system resilience... system design in general.
But I'm not sure you really need to in a job interview. It's not like you can do that with any other interview method anyway - leetcode also doesn't really touch high level architecture type stuff, and take home problems are also too small (or they should be anyway!)
In my experience you only learn how good developers' architectural taste is by working with them for a long time.
In a previous job we did code review interviews. And went the route you said due to the problem I said. And yes, it's a lot better. But what also happened over time was that the bar slowly raised. Because over time the "harder" pieces of that session started to seem rote to interviewers, they became viewed as table stakes.
Mind you this is true of any interview scheme that has a problem solving component to it. I'm not saying that the code review style is extra bad here, just that it doesn't solve this problem.
Here's the neural net model your colleague sent you. They say it's meant to do ABC, but they found limitation XYZ. What is going on? What changes would you suggest and why?
Was actually a decent combined knowledge + code question.
I wrote up 7 general strategies for teams that are interested: https://coderev.app/blog/7-strategies-for-using-code-reviews...
Each code sample should have multiple things wrong. The best people will find most (not necessarily all) of them. The mediocre will find a few.
The two folks who showed this behavior I hired anyway (they were contractors so nbd) and they were excellent hires, so I really love the code review approach for climbing up bloom's taxonomy.
The great thing about code reviews is that there are LOTS of ways people can improve code. You can start with the basics like can you make this code run at all (i.e. compile) and can you make it create the right output. And there's also more advanced improvements like how to make the code more performant, more maintainable, and less error-prone.
Also, the candidates can talk about their reasoning about why or why not they'd change the code they're reviewing.
For example, you'd probably view the candidates differently based on their responses to seeing a code sample with a global variable.
Poor: "Everything looks fine here"
Good: "Eliminate that global variable. We can do that by refactoring this function to..."
Better: "I see that there's a global variable here. Some say they're an anti-pattern, and that is true in most but not all cases. This one here may be ok if ..., but if not you'll need to..."
Coding for me is an intensely focused activity and I work from home to boot so most of the time, I'm coding in complete silence. It's very awkward to be talking about my thought process while I'm coding, but not talking is just as awkward!
They asked me to review a function for a residential housing payment workflow, which I'm unfamiliar with. From an actual snippet of their bad production code (which has since been rewritten). In Go which I've never used (I've never professionally used the language that doesn't have error handling built-in, for example).
I had to spend more than half of my time asking questions to try and get enough context about Go error handling techniques, the abstractions they were using which we only had the import statements to and the way that the external system was structured to handle these requests to review the hundred lines of code they shared.
I was able to identify a bunch of things incidentally, like making all of the DB changes as part of a transaction so that we don't get inconsistent state or breaking up the function into sub functions, because the names were extremely long, but this was so far outside my area of expertise and comfort zone that I felt like shooting in the dark.
So just like any other interview style, they can be done very poorly.
Language and domain experience are things id like to know after an interview process.
They don’t work mostly in Go. Even the interviewer said that he’s vaguely familiar with this area of the code, but he doesn’t work and Go. They work mostly in Kotlin and they explicitly are advertising for solid generalists.
So yeah, I think it's the opposite: explicitly testing for their ability to read code is probably kinda important.
(OBS Elements other times)
Also wtf is inverting a binary tree ? Like doing a "bottom-view". That shit is easy.
This blocks their screen too.
and yes we do know very soon if you look somewere else, take time or rephrase the question to get more time.
If you able to fake it, at that point you should just get th ejob anyway :P