This is the third Jane Street-related thing I've seen on here this week after not seeing much about them that I can remember any time recently beforehand I'm aware of their use of OCaml and probably do other things that might be of interest to the HN community, but it seems like a lot in a very short time out of nowhere. Is there some sort of marketing push or something going on?
People like to talk about Jane Street because it is this Very High IQ™ company that pays people astronomical salaries and is extremely hard to get into.
They don't have to do marketing, though. It's not like they sell anything to Hacker News readers, they make their money trading finantial assets
You don't think that companies who want to hire engineers ever try to post stuff to Hacker News as a way to market themselves as a potential employer? Where did you get this impression of "Very High IQ" and "astronomical salaries"? I'm guessing at least part of it is from reading stuff from them or from talking to other people who read stuff from them.
The fact that you seem to adamantly disagree that there's any plausible angle here doesn't mean it's not marketing, but it might mean that it is marketing and that it's working on you.
Such strong wording, you got that from me saying "They don't have to do marketing"?
> where did you get this impression of [...] "astronomical salaries"?
From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+. Even for Manhattan + SWE jobs, surely that merits "astronomical". Other devs making that much are usually working at Anthropic or OpenAI, aren't they?
> where did you get this impression of [...] "Very High IQ"?
You didn't copy my ™ there, which was meant to be snarky.
In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.
People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?
Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?
I don't ever see anyone saying Paul Graham is botting posts about his "essays"
> Such strong wording, you got that from me saying "They don't have to do marketing"?
The only alternative interpretation I can think of for what you said is "they waste resources on marketing they don't need to do". I didn't think that was more likely to be your intended meaning, but you're welcome to correct me on it.
> From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+.
Can you clarify where these numbers are coming from? Real world facts tend to have sources that can verify them.
> In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.
> People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?
Yes, that's exactly what I'm implying. I honestly would have stated it explicitly in the first place, but I made the incorrect assumption that it was not controversial to claim that this phenomenon exists at least for some companies and that it can be described as a form of marketing.
> Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?
> I don't ever see anyone saying Paul Graham is botting posts about his "essays"
If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument. That said, I also don't bother trying to discuss his writing much because I mostly find it insufferable. I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.
> Real world facts tend to have sources that can verify them.
Sure. What I said is a real world fact is the base salary, which can be verified by any of their job postings, e.g. www.janestreet.com/join-jane-street/position/4274288002, where it says "Base salary is $300,000. Base salary is only one part of Jane Street total compensation, which includes an annual discretionary bonus".
The claim that has less direct sources is how much the TC ends up being. The figure I said is the one I've heard multiple times over, sometimes by JS employees themselves, so I assume it to be true
> If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument.
That's fair, but I used Graham just as an example of source that always gets upvotes here. Uncle Bob hasn't posted anything in a while, but if he did, it'd certainly trend here, so that might be a better example
> I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.
Yes, that's fair. I don't think it's impossible that JS would bot/astroturf like you're implying, I just don't think it's the case. By Occam's razor I find it's more likely that people do care for what they have to say, similar to P. Graham, Uncle Bob, or any sort of blogpost from Anthropic/OpenAI, etc. It's quite possible that they got to this stage due to intentional marketing, though, so I'll give you that
According to algolia, there were 57 posts with jane street somewhere in the title or url in the last year, so a bit more than one per week. For a Poisson distribution with rate = 57/52, P(X >= 4) ~= 2.5%. So there should on average be ~5 weeks with 4 or more jane street related posts at that rate.
> Poisson is such an under-utilized distribution for these kinds of phenomena.
Yeah, Poisson distributions show up everywhere if you are paying attention. You can even use the theory as the basis for a fast, elegant algorithm to take parallel, weighted random samples. You can even do it in SQL!:
SELECT *
FROM Population
WHERE weight > 0
ORDER BY -LN(1.0 - RANDOM()) / weight
LIMIT 100 -- Sample size.
Except I'm pretty sure there are three from the last 24 hours. Two live on the front page right now and one about some almost flooding just dropped off.
Working in quantitative finance (as a "quant") became the new big thing after main street discovered that those firms pay more than your usual big tech firms. It has explode the past few years.
Quant firms are also notorious for being some of the most selective in the world, so that alone has become a status symbol. The idea that if you land a job at firms like Jane Street, you must be some 150+ IQ math/physics/cs Olympiad machine.
I had lunch with a couple friends this week, and we each work in different tech environments. Most of the conversation was about the workflows we're each using, and seeing our colleagues use.
It's a really interesting time to work in this field. People who are paying attention to workflows are using a range of tools, and constantly refining their own workflow. For example a whole team might use Claude, but one person collects data about their own usage on an ongoing basis and develops more advanced workflows than everyone else on the team. Their focus isn't just about working faster, but about doing better work.
I don't think the post is vague; I think we're at a time where you need flexibility in your interview process to find candidates who are doing the best work these days.
They're basically doing what everyone else has been doing for years (minus the LLM part). Just focusing on software and system design rather than coding riddles.
This sort of announcement has made me very happy I left the software world. I still code as part of my job, but it's not my main part and luckily I don't have to use LLMs are part of my job. I still code 100% by hand.
In some ways, although I detest AI, it has clarified the sort of people and organizations I want to associate with. Any latent characteristics that I abhor have been made obvious with the existence of AI and of course, I would want nothing to do with Jane Street.
so, i've gone through a bunch of iterations on how to interview in days of LLMs and i came up with a process that is quite good:
first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.
BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.
the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.
(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).
the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".
one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).
in general, this flow helped us a ton and i really really like running it :)
Well, I've been using git at work for 6 or 7 years, but I can't out of hand open a PR; at work we very rarely use different branches and the last time I touched Github I was a student.
Do you mean specifically Github, or any code forge at all? When you say "we rarely use different branches", what do you mean - you're all pushing to master, or you have branches for each team or something? I would regard "code forge" and "separate branch per reviewable unit of work" as extremely normal, nearly universal.
Yes, we're all pushing in the same branch (not called master, we have one branch per version (usually we have one branch that's the current version deployed in production and the development branch)). To group up commits for review, we add in the commit message the Jira issue number and Jira will keep a list of all commits through Bitbucket. Bitbucket supports PRs, but I don't think it's ever been used here.
I can gather that the way we're using is not the most usual ;).
I rarely use anything beyond checkout, clone, pull, commit, cherry-pick, revert, and push in a job. If I have to, I'm probably asking the AI what to do by giving it my situation. And hopefully the AI can read about whatever git standards the company would like to be enforced so the engineer doesn't have to interrupt someone for that.
This is more about opportunity cost because instead of asking about git, you could ask about other valuable skills in your limited time with the interviewee, but I guess if git is really that important, then it's justified.
Something similar for senior devs: code reviews. Reading and understanding code has always been important. I find offering simple code reviews very telling. In a way, your LLM approach matches that. It's a way to do a code review. It's a test of understanding, and not just memorization.
I feel like git operations are one of the few things that I truly don't care at all about whether someone uses an LLM for and doesn't tell me. PR descriptions and commit messages should be written by humans for humans, but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care? Beyond the code itself, I care about the things that will continue existing in the long run fitting my team's conventions (the number of commits, the commit messages, the PR description and comments, and then whether there's a merge commit at the end), but the process used to create those things doesn't matter to me.
> but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care?
Because LLMs aren't a reason to not learn your tools. A determinist CLI command vs using a statistical machine for it. It's just a waste of tokens at the very least.
I happen to have memorised how to add new remotes a long time ago, but I would 100% not care if someone I worked with googled it every time. That's how it was for me for several years. I don't think that has anything to do with "learning the tools". And getting claude to do it is not that much different from copy-pasting from google.
Some 6-figure per year developer spending a quarter’s worth of tokens to do something they do once a month or so is not something I’ll ever care to optimize away…
I misread this as a "quarter" as in "three month period of the calendar year" and was very confused until I remembered that was also the name of a coin
No, the reason not to learn your tools is if they're not your tools anymore, and something else is. I don't spend time trying to manually invoke filesystem APIs to create files in my work projects and instead will often just have my LSP do it when I declare a module and the file isn't found. I could memorize the exact sequence of system calls needed, but my employer would prefer I spend time on things that are more worthwhile to them.
I don't disagree that having a deterministic CLI is useful, but I'd argue that's not the question here, because the CLI is what the statistical machine will be using. The question is whether the time spend doing it manually is worthwhile compared to the tokens spend on it. I'd argue that if you instead learn how to pick the right model for the right job so that you don't use something way more expensive than needed for the task at hand, the token overhead will be extremely minimal for stuff like git operations; knowing that you can use something cheaper than like, Astra or Opus if all you want to do is a quick git cleanup and push is going to save your employer a lot more money in the long run than if you insist on doing every messy rebase by hand.
While using LLMs is a great idea, it has become crucial to put safeguards in place to ensure that their use doesn't deviate from team practices or compromise security.
I'd argue that these are not really specific to git (or even LLMs honestly); in terms of security, don't put company data into anything that your company hasn't given you explicit permission to put it into.
In terms of team practices, that basically seems like a code review thing. If my team has conventions around git, and a PR doesn't fit them, that's something I would point out regardless of how the PR was generated. If a team member is repeatedly not meeting the expectations we've all agreed upon, I would deal with it in the same way I always have; reach out and offer guidance for how to do things the way they're supposed to, notify my manager if they continue having trouble with it, and eventually just not bother looking at their PRs if they absolutely refuse to do things the way everyone else on the team has agreed they should be done.
The point of knowing your tools is to stop escalating to other people every time something doesn't work with them.
Even if an LLM can hold your hand through everything, the disruption I've had because people don't understand what the LLM tool is doing, and ultimately does damage (we've had repos corrupted, systems taken offline, systems brought their knees, numerous data security incidents... it's been a fucking nightmare) is immense.
Repos corrupted? From arbitrary usage of a git client? I feel like it's been generally a bad idea to allow arbitrary force pushes for anything shared for a while before LLMs entered the picture.
Even worse, it gave a developer a command to run that basically crushed all history across all branches, it was a mess to clean up and lost a significant amount of work. It wasn't Claude running it on its own, but was entirely driven by a mix of a developer's lack of understanding their tools, and trusting Claude's output too much.
This was also the dev trying to "remove a key he had accidentally committed" again, just letting Claude take the wheel.
Of course, the new team _also_ really loved leaning on AI so they were all behind his plan to "clean up the repo" and not just reset the key and move on.
The idea of saying the LLM is "my tool" as opposed to all the actual tools it calls, is about as appealing to me as saying the same about, say, Remote Desktop. "The statistical machine" is there to generate candidate actions, not to implement them.
In the before time, this is the sort of thing people would just google.
One time setup commands are the sort of thing people do once when setting up the repo, but dont use day to day. its understandable that people dont have that memorized and will reach for a man page, google or an AI.
Tokens are cheaper than brain power. Why waste my time trying to recall the details of git's byzantine, ill-begotten command line interface when I can delegate that boring task to a robot?
Wasn't this a think before_ai? Enough people are using IDEs which handle this for them. I still have to check a few commands now and then.
Though ideally interviewers should appreciate this - unless its a hard requirement you do it by hand day to day - the rest of us can look this up quick enough.
At previous jobs, I was the one coworkers would come to when they had messy git problems and needed help cleaning them up. I've put in the hours in the past to learn how to do all of that, and then later forgotten it because it's just not worth it anymore. I switched to jj a few years ago and had the majority of the annoying rebase issues go away, and whatever's left goes into my LLM to spit out a potential solution that I can then manually tweak if the result isn't perfect.
My OS professor in college once told us about what he considered the difference between "knowledge" and "information", with "knowledge" being the underlying facts about how something works and "information" being the terminology/labeling/bucketing/etc. of how to refer to it. The point he made was that knowledge was more important than information, and that he would try to test us on knowledge rather than information. To me, knowing how the underlying version control model works and what "shapes" you might want to make with it is useful knowledge. The exact set of commands, interactions with an IDE, or specific prompts you can use to ask an LLM to produce them is information. Arguably the entire benefit of LLMs is that you can ask them to do things even if you don't have the exact information memorized; if you can describe it in a rigorous way, and recognize whether what it gives you as an output fits that or not, you don't need to know any of the actual information of how to do it manually.
I hadn't heard of this either, but they have an interesting reason for the name:
> The command-line tool is called jj for now because it's easy to type and easy to replace (rare in English). The project is called "Jujutsu" because it matches "jj".
Problem for me is that unless I see someone being proficient at using git commands to rebase, merge etc.
…
I have hard time believing they understand underlying model. Even if someone knows all the right words.
i guess your difficulty is that half the commands i can write off hand and the other half i cant, and i only really care to understand the parts of the data model that i actually touch.
like, i dont really need to know all the stash details when i only stash one thing at a time, very temporarily
i also dont really care to know all the origin and remote definitions stuff when i only touch it once per project or so.
i could certainly draw graphs of what different git repos look like, but what makes it salient to know the rest of the details unless im actually building git itself? if youre using every git feature regularly you're probably doing something wrong
> like, i dont really need to know all the stash details when i only stash one thing at a time, very temporarily
If I were hiring, I would care about whether you can `git help stash`, skim over it (bonus points for actually using the pager competently) and figure out what you need to do.
I do care about … if someone understands origins and not by definition because developer needs to know what is his local version and what is on the server and how those two affect each other. Bonus points if they understand server is not special in Git and their colleagues copy on their laptop can be origin as well.
Drawing graphs of what different repos look like? That’s irrelevant.
Irrelevant if one understands that Git uses DAG and is content addressable store.
You see someone might know those hard words but when working daily with Git reasoning about rebase or merge, cleaning up their own work with interactive rebase, making pull/merge requests or resolving conflicts it is much easier if one knows what is local what is remote and how Git stores data.
You also learn it once. In my 15 years of using Git I basically didn’t learn much new commands but having mental model of how it works was my tool to fix any problems other people had. Knowing commands or definitions is secondary.
I used to go to stackoverflow to resolve issues with git. The trouble was, I'd get 15 answers, all different, all saying "this might work", "try this", "try that".
Now I just ask google, and I get a straightforward "do this", and it works.
That's largely because the OP couldn't clearly describe a specific situation to ask about (the question underlying the problem, but also a clearly recognizable, reproducible example); and solving your problem might depend on things that are different in your setup vs. the OP's.
That's the thing, though. Stack Overflow is not and never was intended to solve your problem, nor to solve the OP's problem. It was intended to answer questions. It was your responsibility, both as someone posting a question and as someone reading the Q&A later, to adapt knowledge to your circumstances. That's why they're called questions and answers, and are not called problems and solutions.
People who use a web site to get a solution to a problem are looking for a help desk, or a forum. Stack Overflow was specifically created to be not that, and that choice was specifically motivated by the frustration people had with trying to search those kinds of sites for information, and the frustration that experts had with trying to help people (for free) in that kind of environment.
That's pretty much where I'm at, so long as they're not filling the code with "///CLAUDE ADDITION" type guards (ie noise).
Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR. I'm getting half the speed the High Speed Sloperators are getting but I was the only person submitting PRs the other day when the LLM service was down.
For nasty multi-way git merges and rebases, it's been a game changer. And of course that's reviewable by doing a delta on the final rebase branch vs. the original branch.
With all that said... having to use LLMs at work is making me want to use them less on my own projects.
Oh, I was not even attempting to discuss using them for the code itself in my above comment. Code that's checked into a repo has a much more complex set of tradeoffs that I'm much less convinced have a one-size-fits-all answer. My point was more that even if reasonable people can disagree on how much you need to care about the code itself being LLM generated, I can't wrap my head around why anyone should care whether an LLM happened to execute the `git commit` and `git push` shell commands!
> Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR
For the more important changes, I find it easier to stay on my toes by taking a pass at the codebase before comparing notes with the LLM. Peeking at the answer key first makes it too easy to nod uncritically.
I think that actual direct knowledge about how git merges things and what ends up surfacing as "conflicts" as a result of that is not particularly well understood by quite a large number of people who in practice do not have any trouble using git every day. Moreover, there's nothing specific to git in terms of reviewing code, which is fundamentally what you get from the result of fixing conflicts; once the conflicts are all resolved, there's a diff from the old state (pre-merge) to the new state, and you already should be reviewing it to see if it's what you want even if there were never any conflicts.
Should you resolve the conflicts in a bad way we're going to see the result in the MR, will we not?
I don't think git knowledge is essential anymore.
Back in the days of the crappy git GUIs this was a problem, because people would regularly misunderstand even the basics of what they were doing and managed to lose their local changes.
I had a coworker a few years ago who was very stuck up and the first place I saw it was his opinions how people should use git. I use the terminal for some stuff and the gui for others, but he was weirdly judgemental about doing anything other than full terminal.
Omzsh user checking in, I always stumble a little in interviews about this because they usually have a default git configured and commands like gco -b and gp don’t work, and I have to think about it for a second.
> but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care?
Because someone who's familiar with those sorts of things can get the commands typed and running before someone who isn't has even figured out the prompt.
(Well, not so much if there's a need to identify the base commit for a squash, do some cherry-picking, etc. But in those cases I'd rather see direct human judgment of what to do than hoping the LLM understands a vague description.)
Git is a simple tool. I expect any engineer can learn to use it. While git is common, I would not expect an engineer who worked at a company using a different version control system to know how to use git (which is a really awful - I still miss mercurial). I expect you to know the general concept of version control, but the specific on is a tool I expect you can learn quickly.
There is nothing simple about git. 'ls' is a simple tool. git is probably the most complicated console command (in widespread use) I have ever used, although I'm open to other examples
It is not simple like a hammer, but imo the bar for being a software engineer is being intelligent enough to be able to understand it well enough to do things with it, since it is a tool of the trade.
initializing a repo, staging files, adding them to the changeset, commiting then pushing those changes are easy to do. solving merge conflicts is a little complicated but IDEs make that much easier to do.
Agreed, the amount of stuff checkout can do is crazy. And the concepts of hard and soft resets mixed with garbage collection make deletion non trivial to understand. Or using interactive rebase to variously remove, squash, and rewrite commits.
Fundamentally git is conceptually complex in a way that almost no other tool is. Others might have scarier syntax to newcomers but rarely expose the same level of underlying complexity.
Version control is simple for the typical user. Git is a terrible tool though and doesn't properly abstract the complexity that should be hidden away from users. Even still, the typical user only needs to know how to clone, stage, commit, push, and merge (I'd like to add branch, but git doesn't have real branches, so I guess tag which should be an advanced topic nobody needs)
honestly, for a senior, the coding challenge is just a checkbox -- because they can use llms, the main thing is making sure whatever code they accept, they can explain from top to bottom.
the coding challenge itself is just a way to make sure that people can understand a request within an existing codebase. as an example, i pay attention to the questions the candidate asks -- questions asked always mean something.
my general assumption with seniors are "they can think high level", which means i "mark them" with that assumption, so for me, the architecture talk is much more important.
at the end of the day, you need to understand what your org needs and adapt.
The trick is to get a good sense of the algorithms associated with basic data structures (list, stack, queue, graph, hashmap,...). They are primitives and almost any standard problem can be solved with those.
What you need to train most is the capability to break down problems: Like I have this input (and associated metadata), I have this context (may be null), I have those constraints (values, ranges, memory space, time, both, or neither), and I need these outputs. Then it's just a puzzle.
I no longer prepare for coding problems other than a quick refresh of the syntax of the main language. And if it's a whiteboard type of things, I go with pseudocode.
Github was like Yahoo's directory, a way to share content in a searchable place and as a backup/bounce point. In the early days you could also upload to 'downloads' so it was a quick share host. Before that we had email, exposed ssh and cgit/gitweb but Github gave us `PR` to promote social interactions with forking. So it could be argued that Github tainted the spirit of git.
Having personally evolved from making date stamped zip files, to subversion and then to git, it earned its place quite quickly for myself and my roaming nature, didn't hurt who hacked it together either. When Linux adopted it from Bitkeeper, it became the way for many.
But, like... wouldn't you want them to at least know the steps of `git add` -> `git commit` -> `git push` and then click the button on Github? And Gitlab has basically the same web UI.
I have no problem with letting an IDE or LLM handle those details in practice... but I don't think I'd want to work with someone who doesn't understand the very basics of what's going on under the hood.
Years ago I was the “git expert” on my team, the person you went to when you got into trouble. But I have long since moved to jj, and I basically don’t remember git commands anymore. I also don’t use GitHub much, so when I need to open a PR I prompt an LLM to do it for me.
Well, I can't read the GP's mind, but I'm fairly certain they didn't literally mean "candidate knows how to click one button in the web UI for a specific proprietary developer platform and type some stuff into a text box".
When I tell my LLM to "open a PR for those changes" (which is how I do it nowadays), my LLM understands that what I mean is "make a commit, push it to a remote and raise a PR".
Does it mean we've reached AGI when machines understand what we mean better than some humans?
> When I tell my LLM to "open a PR for those changes" (which is how I do it nowadays), my LLM understands that what I mean is "make a commit, push it to a remote and raise a PR".
This makes it seem like all of those things are part of a single operation to you. That doesn't make sense to me.
On the occasions where I've submitted a PR to a project (anywhere, not just on Github), making the commit was not part of the same operation. I'd already made the commit and satisfied myself that it captured everything I wanted, before making the PR. I don't see those things as part of a single operation. I don't think the poster you were responding to does either.
So you think the OP is asking candidates a question, where the solution is:
1. Navigate your browser to GitHub.mycompany.example.com
2. Click the repo name
3. Click pull requests
4. Click the source branch[1] and destination branch
5. Click "create pull request"
???
I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.
[1] Presumably already extant, because they never mentioned anything about committing code, nor pushing it to github (and that's not the same operation!).
So I think the OP meant to have the candidate, um, open a PR. Which is not the same thing as making a commit and pushing it to a remote. Maybe you always conflate those operations, but that doesn't make them the same operation. If the OP actually meant that they ask the candidate to make a commit, push it to the remote, and then open a PR, I would have expected the OP to say, um, all of those things, instead of just the last one.
I couldn't say how the OP expected the candidate to open a PR, since I don't know what their standard workflow is. There are ways of doing it from the CLI, as well as through the web interface, as well as through various IDEs.
> I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.
Please save your patronizing for another time. And stop trying to put words in my mouth.
> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).
How does that work? Git has no concept of pull requests.
I run a live coding challenge on a rather benign coding problem with the explicit expectation that candidates will use an LLM.
Interestingly, an LLM involved feels like it’s made it easier to assess candidates. Senior candidates know exactly what they’re doing and strongly driving the LLM. They pre-communicating design decisions, gotchas, and requirements. They also ask a lot of clarifying questions and dial in the scope (after all not building something is better than building it).
Less experienced candidates tend to jump right in, build way too much, and stumble because they delegated all of the critical thinking to the agent. They often end up in a place where the agent doesn’t align with their vision and they can’t really reign it back in.
> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).
Worked with a "super senior", lead architect. Management was raving about the guy before he joined. Insane resume. Skipped the regular interview process because of this.
Had to show him how to open a PR and setup his git config so his commits are linked to his account. And then his PR's failed the automated ci checks for descriptions and titles, and he came to me to ask how to fix it. The failing gh action literally says what was broken.
This is all with Claude available btw, so one could literally just ask.
I guess it's a bit of a modern fizzbuzz. Where it's so simple but still manages to stump a lot of candidates.
What I though of was to give candidates a not-so-smart LLM, one that is almost guaranteed to make mistakes.
The idea is to see how the candidates deals with it. Maybe he will just say "fuck it" and do it by hand, or patiently point out every mistake so that the LLM gets it right. Both indicate competence, what you don't want is someone who doesn't see the mistakes and hand you slop.
> we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it
Will you allow jj instead. I switched to jj 1.5 years ago (as many others have), and my git is now rusty.
I mean sure, I can do add, commit, basic diff, etc. But rebase/merge - oh dear, I forget which one is source/target (jj made the infinite design choice that the ordering doesn't matter - you have two parents making a child - it's symmetric).
> we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it
This part seems weird. "[B]asics of git"? You can Google for that and find a fossilized answer on StackOverflow. Do you really have candidates who cannot do git commit/push? Also, if you allow LLMs, then any idiot LLM will also know git basics.
If you are asking any human to write code, you are effectively asking them to write byte code. Humans should not be writing code in this era, and only in rare circumstances should humans be reading code.
They pay a LOT of money, have some high level of “nerd” prestige for solving interesting technical problems, have good outreach with things like problem of the month, and have not been impacted by massive layoffs.
There are pretty few companies out there like this. Being relatively small probably helps; everyone knows someone that worked at Mag7 and maybe has a bad story to share, but this is more rare.
> They pay a LOT of money, have some high level of “nerd” prestige for solving interesting technical problems, have good outreach with things like problem of the month, and have not been impacted by massive layoffs.
I never heard of them.
If not for HN, I would never know they existed.
I don't think the remark about "nerd prestige" holds, but beyond that your points boil down to marketing.
And how an unknown company changed its hiring practices is hardly front page news.
They are unknown because they choose to be unknown. The first rule of finance is that, you never tell anyone how you make money. Even more so if you make billions of dollars annually.
I posted this. I'm just a regular dev, without any kind of attachment to Jane Street, in fact i'm a little wary of them, and would subscribe to Evan Czaplicki's understanding of the faustian bargain working there would involve. I think they're banned from India, if you want an example of the kind of raiding they are equipped to do.
In general - I am wondering how engineering interviews change in the age of LLMs, Jane Street, tech-wise is a "thought leader", at least having opinions, sharing them is part of how they recruit "the best", i think oxide does this too.
So in some way, in this time of change it makes sense that they are noisy... It also makes sense that the noises they make may filter down...
Yeah, sorry, I didn't mean to imply anything nefarious on your part (or anyone else's part). It's just odd that a single financial firm keeps coming up.
OCaml like a lot of niche functional programming languages has a lot of weight on HN. Or used to for many many years, mostly before LLMs. Jane street is single-handedly keeping OCaml alive and in use in real applications and as such they get a lot of credence.
Whether you agree with what they do or not, they have had a sort of mystique about them (tons of money, quant vests, _and_ a niche functional programming language as a daily driver??? Sign me up!) that HN likes.
Their ban from India isn't particularly sinister. What they did would have been perfectly legal here in the US as "supplying liquidity". They just successfully arbitraged the liquidity gap between the Indian options market and the long position stock market.
> The cross-market manipulation involves buying and selling stocks at a loss, allegedly for the purpose of moving the prices of corresponding options, so that Avalon could make a profit by trading those options at artificial prices.
> The Court found that the two alleged manipulative trading schemes-if proven-would violate Sections 10(b) and 9(a)(2) of the Exchange Act and Section 17(a)(1) of the Securities Act.
There are three Jane Street stories on the front page right now. This one, "Socratic Tutor (github.com/janestreet)", and "A false smoke alarm almost drowned Jane Street's only NYC datacenter (bacarella.com)".
What I learned over time is that first, the pool of people who regularly submit to HN is surprisingly small, so if they have interest in a particular set of topics, these topics are overrepresented in the queue; and second, relatively few people patrol new submissions and the the first 3-4 upvotes in the first 15 minutes decide if something bubbles up to the front page or not. These folks often go by keywords / vibes / employer associations, so for example, anything that mentions any frontier lab is more likely to bubble up.
More generally, I think there's been a significant drop in topic diversity on HN over the past year. Almost everything on the front page is related in some way to AI or to companies on the forefront of AI.
The closest to "HN lite" is probably lobste.rs. There's a fair amount of overlap, but it's closer to 20% AI than 80% AI.
Reddit? General-interest subreddits such as /r/technology are mostly mainstream news, but there's plenty of specialized ones, such as /r/math, /r/programming, /r/c_programming, /r/rust, etc.
Many of the aggregators "killed" by HN are still alive, just not as popular. For example, boingboing.net is up and running. RSS feeds from places like Ars Technica are a fair choice for industry gossip too.
All three stories fit the criteria of gratifying one's intellectual curiosity, which is goal of Hacker News submissions.
Consider this: Would you find anything unusual about seeing those stories on the front page if each came from a different company or didn't mention the company name at all? I wouldn't, because each of the submissions is primarily interesting for the content, not the company associated with it.
I think Jane Street has gained prestige and notoriety in the last few years. For people like Math majors who are split between tech and finance, it’s the most famous I think.
Back then, people who were bad at interviews were often undervalued. I think interview culture itself will soon be devalued and eventually become obsolete. People who build real technology and solve real problems will be unaffected, while the glorified Valley slop built around interview performance will become obsolete.
My optimism comes from AI's ability to solve genuinely hard mathematical problems this year. September and October could go down as two of the most remarkable months in the history of AI If, and only if, those proofs are correct and independently verified by real experts. Even if the proofs have been formally verified in Lean, expert scrutiny still matters.
There are a lot of people telling you how they interview and why. Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview, much less that they apply it. They also are not showing any indication that they have done any testing - to see if their questions help hire better people.
If I said they could get equal results by picking resumes at random and hiring them without any other screening could they refute that? If so have they justified every part of their process or just one step that is useful and the rest they don't know?
Even Steve Yegge got selected (or maybe nearly selected) out by his own process (unwittingly, mind you)
> One day, the recruiters gave us a special round of packets to review. In these special packets, we were able to read the interviewer notes and candidate responses. All personal details were stripped out, and we were told it was a “calibration exercise.” We had to do our regular voting job with these special packets, and see how it went. I think we may have assumed they were from another site, since cross-site calibration was common.
> Our group did our job, and voted not to hire about 2/3 of the packets. This was about par for the course.
> But surprise surprise, this time, those were our own packets from when we had all interviewed at Google. The recruiters had tricked us into reviewing our own interview packets, and we had voted not to hire most of our own group.
They do say "Not every onsite includes these new rounds yet, and when you come on-site we’ll give you a heads-up about what to expect."
So that implies that they are testing this, trying to determine if it works better than the current process. Whether they are doing this in a scientifically rigorous way is unclear, however.
> Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview,
The topic of the article is about adapting interviews in the age of LLMs.
Scientific research on interview techniques requires years of retrospective analysis after the hired employees have been evaluated in their roles for a long time.
That research does not exist on this topic. It's too new. I don't think this is a valid criticism of the actual article.
I'd add that even those years of retrospective analysis will be insufficient, because "hiring successfully" is fundamentally about measuring membership in a particular social group. You can never pick the right observable dimensions to measure in your analysis, it will ALWAYS be lacking. Sure, you can measure work done, tenure, etc, etc, but that does not mean a hire was successful.
You might have 2 people with opposite personality traits ace the interview then get put on a team together, now all of a sudden your whole team is worse overall. That's a massive effect but your study will just see it as noise.
The reality is your best hires will just be intelligent people who belong in the group and who push the mission forward.
I used to interview people and would generally just have a chat with them about previous problems, projects, or experience at school. I would work out a pseudocode problem with them (designing an elevator). From what I can tell everyone I said yes to has had a fruitful career.
The idea of forcing a candidate to give a third party access to their screen, transcribing AI notes, and forcing them to do 3+ leetcode, and an architecture round, and a behavioral round while I watch over them for hours on end sounds completely idiotic and antisocial. This entire industry is high on its own supply when it comes to interviews.
A friend recently got through Google after getting interviewed in Meta and several other companies. He said interview styles are changing in most places but there is no common pattern. Every company is taking their own custom approach. Some are giving open access to AI IDE, some are asking to not ask everything and monitoring what you ask. One company give access to LLM (probably custom) that would provide wrong code intentionally and candidate has to correct it.
the thought of going into an interview and getting asked to write code is a joke at this point. i haven't written more than a handful of lines in over a year. the interview should be related to the job.
We're going to get downvoted for this opinion unfortunately. HN is both a place where people say the state of software interviews is terrible and yet will downvote any attempts to actually point out the obvious. Sorry stranger, I took many a downvote also for the same opinion.
I keep seeing job adverts for specific languages, I think that’s honestly ridiculous now. I’ve built tools for myself that work well in rust and swift and while I can read the code I couldn’t write code in them. I’d say devops is completely over now as a job, you can build almost anything and it’s not hard anymore.
I would say my skill is now working things through with people in the business/users to figure things out and improve what the software does.
This will always be the shortcoming of how "knowledge work" sort of job interviews put so much focus on trying to get the exact unicorn candidate from the outset, instead of just seeing what happens and either keeping the worker or letting them go like how every other job works. Most shift work on the other hand doesn't have to worry about you lying on your resume or interview. If you lie, and you can't do the job the next day, you are fired. That's it. If you lie, and you can do the job the next day, all is well anyhow, right?
It is like an unhackable way to do it. Just hiring the person, seeing how it goes, and letting them go or not. It isn't rocket science. The guys digging a ditch outside the office know more about hiring than the career HR professionals inside I guess.
My guess is that they'll end up hiring the same people regardless of the interview format. They want to hire top graduates from elite institutions anyway.
Leetcode was stupid before LLMs and now it’s even less useful as a signal.
What I want to know is whether someone has basic DSA understanding. Is the person familiar with any programming language. Can they translate business requirements into code? Can they disambiguate weird things PMs say into a viable plan?
You do not get a better signal about that from asking harder leetcode problems.
Never once in my career has writing a tricky algorithm under time pressure been relevant. The closest thing I can think of is debugging complex systems in an outage, but then it’s more like familiarity with bash commands and general debug skills.
As for LLMs and coding, companies need to figure out how to differentiate good and bad outputs here and what lasting engineer inputs to select for. And I don’t have any idea about this.
You missed the point. I’m not saying I support dsa but you are arguing against the weakest version of the reasoning behind why companies use dsa. It’s used as an iq test. IQ is highly correlated with many things associated with intelligence yet even though we don’t take iq tests on a daily basis it still correlates.
The argument was dsa functions like an iq test. It’s not relevant to the job (in fact it’s harder than the job, far harder) such that if you do well on it companies can be confident you can do well on the job just like how scoring high on an iq test makes people confident you’re intelligent.
There are plenty of arguments against what I said above and going after this argument is the better target because the irrelevancy of dsa to the job is widely known and obvious.
Is DSA knowledge really pointless or just a stand-in for an IQ test?
I've been engineering systems for 25 years. On a macro level, most of my time is spent considering the right tooling and stack for a job, balancing budget and time constraints and efficiency against how things may need to scale in the future. On a micro level, in the data model and in the code itself, choosing the best structures for a task is important for maintainability and overall coherence.
In my experience with LLMs, I often need to guide them toward choosing the right ways to model data on the code and database level. LLMs seem to prefer json blobs of unstructured data, naming conventions that drift over time, argument structure patterns that change from one piece of code to another. The net result is a lot of tables and code that an LLM needs to read back through, find all the linkages, and recommit to its window before any debugging can take place. It makes human debugging very difficult if, for example, an LLM decides to just store all the userdata in some json column and then decides to add some kind of stateful preferences to that, and other state to something in memory, and then parse the memory falling back to the database state. I've seen a ton of things like this where weird ways of handling state kinda work, until they don't, and it's extremely difficult to figure out what is taking precedence when or where a thread race is coming from.
All this is to say that DSA knowledge may be even more important when LLMs are writing most of the code, because an unguided LLM will write stuff that can practically only be maintained by an LLM, and make design decisions that don't cohere or scale. If that's not really a problem, and you're certain that the code you're investing in writing will never need human intervention, then I guess it doesn't matter. But with a large enough system designed that way, the boundaries and edges of poor DSA choices aren't going to be apparent until either something breaks or you need to grow.
To get into Marine boot camp, there's a minimum strength test.
To get into college, there's the SAT test with a minimum score.
To get a high paying software job, there's the leetcode test.
These are gates meant to filter out the obviously unqualified candidates. Unqualified candidates waste the time of the candidate and the company. The real interview comes after that.
A friend of mine wanted a job at a FAANG company. He knew there'd be first a leetcode interview, and complained that he'd have to spend 3 weeks studying for it. I said those three weeks would be the best investment he'd ever make.
So he did the three weeks, passed the leetcode, passed the interviews, and now makes 400k.
When I was 10 or 11, I asked my dad what MIT was looking for in a student application. He (an MIT alum) told me. I picked out the things I could do (being captain of the football team was never going to work for me), applied to MIT, and got rejected. But Caltech, Dartmouth and Johns Hopkins did admit me.
I wholeheartedly agree with this, and I always thought leetcode questions and tech interviewing in general was broken, but as a filter for unqualified candidates, they were pretty effective. Which really matters when you have hundreds of candidates to sift through.
The biggest problem, of course, is that they also filtered qualified candidates (I personally got filtered out multiple times and I'd like to believe I was qualified!) But it is an understandable compromise whenever there is an extreme asymmetry in the volume of candidates and volume of interviewers, and the fix for that is simple: you study for the test (as you pointed out.)
Note also that as grandparent comment indicates, this test is a necessary but not sufficient condition; it's just to get your foot in the door for a deeper evaluation.
I think the complaint is that the test has nothing to do with the job.
It's like "to get into Marine boot camp, there's a minimum crochet requirement".
Solving leetcode problems has nothing to do with commercial coding. It's an unrelated hoop that we make folks jump through, because it's easy to measure. It's like measuring developer productivity by LoC written.
Yes, it'll measure how much time the candidate spent preparing for the interview, and how well they can jump through hoops. But it'll also deter the actual people you want in the team, the folks who know that leetcode has nothing to do with commercial coding, and see the leetcode test as a sign that your organisation has no clue and a bad management culture.
And if the test is dancing in a monkey outfit while the examiner throws bananas at you like in Zoolander, it's just "fair game", right?
People don't like Leetcode because it's insulting to treat them like freshmen that know nothing but algorithms that related poorly to the job at hand; can't judge them on coding prowesses, for most of them.
Wait, why am I explaining this to you like you're ready to listen beyond your monologue?
> Leetcode was stupid before LLMs and now it’s even less useful as a signal.
You got my vote right there. I also find it funny that LLMs can easily do leetcode problems, because there were so many examples and easy test cases.
> You do not get a better signal about that from asking harder leetcode problems.
I think this is a thing that needs bigger formatting for this statement or a feature to vote twice.
---
Only response I have to this:
There are many different schools of thought on how to architect systems. In many cases these different opinions don't have a lot of solid backing on why you'd do this. On top of that there is level restrictions that prevent talented individuals from being more influential. I.e. an Architect is less about technical experience and excellence and more about the abiltiy to communciate to high level stakeholds and having systems that can solve the problem [some what].
> Can they translate business requirements into code?
I have been looking for resources that tackle/teach/learn this translation part, but so far hasn't gotten any hits. Can anyone here point me to such a resource, if it exists?
Leetcode made perfect sense very early on, before leetcode, the company, existed. It was, at the same time, part a test of smarts, having some capacity to work under pressure, whether a new grad had paid any attention to the data structures and algorithms classes, while we got to see that they could code at all. It was an improvement over whiteboard interviews. And thus, it was what separated a green college student from very high compensation at generous tech companies. The ideal candidate wouldn't even prep for this: they'd just show up, remember the right things, let their talent shine a little, and go past. The questions weren't even that hard back then: You might not even touch what now is a leetcode medium in a typical interview loop. It came from the DSA classes because it would be easy to explain, and something you might recall and implement in 30-50 minutes. It couldn't all just be fizzbuzz.
The problem is that the compensation was so generous, and the kind of questions being asked so public, that your typical somewhat less talented, yet hard working grads would just cram for the interviews, and get past them, thus lowering the value as a filter. Leetcode and the like decided that there was an industry in helping you cram. The filtering got worse and worse, and the companies then decided to keep upping demands, hoping to still get the most talented. But there's nothing in a leetcode interview that cannot be trained. So the metric became the target, and we ended up in the current mess.
> What I want to know is whether someone has basic DSA understanding. Is the person familiar with any programming language. Can they translate business requirements into code? Can they disambiguate weird things PMs say into a viable plan?
I agree with you but this is a "simpler" test with a subjective analysis (simpler in the sense that the candidate might make some progress rather than completely stuck). Big Tech works at a scale where they probably want to quickly evaluate based on objective results. Did they crack the problem or not? Imagine having a pool of 1000+ candidates for a single role.
We laid off most of the company and just offshore everything anyway so I don’t care, least I can do is give these randos a chance in Brazil and hope it works out better for them
191 comments
[ 4.4 ms ] story [ 34.5 ms ] threadok thanks for this slop then. Dont see why this stupid article is on frontpage.
They seem to belive in 'organic' marketing like doing a fake podcasts on dwarkesh. So maybe they have some bots upvoting their name.
Also one of my absolute favorite programming talks ever was hosted by Jane street:
https://www.janestreet.com/tech-talks/algorithm-for-precisio...
It’s like half talk, half story. Got me super interested in logic programming, and minikanren in particular.
They don't have to do marketing, though. It's not like they sell anything to Hacker News readers, they make their money trading finantial assets
The fact that you seem to adamantly disagree that there's any plausible angle here doesn't mean it's not marketing, but it might mean that it is marketing and that it's working on you.
Such strong wording, you got that from me saying "They don't have to do marketing"?
> where did you get this impression of [...] "astronomical salaries"?
From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+. Even for Manhattan + SWE jobs, surely that merits "astronomical". Other devs making that much are usually working at Anthropic or OpenAI, aren't they?
> where did you get this impression of [...] "Very High IQ"?
You didn't copy my ™ there, which was meant to be snarky.
In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.
People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?
Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?
I don't ever see anyone saying Paul Graham is botting posts about his "essays"
The only alternative interpretation I can think of for what you said is "they waste resources on marketing they don't need to do". I didn't think that was more likely to be your intended meaning, but you're welcome to correct me on it.
> From the real world fact that the base salary is US$300k/yr, and often said that the total compensation for the first year ends up being US$700k+.
Can you clarify where these numbers are coming from? Real world facts tend to have sources that can verify them.
> In any case, what I was trying to say is that they don't have to do marketing in the usual sense. They don't have to get the Jane Street logo in your subconscious so you buy a Jane St. product when you go to Kroger's, and you can't sign a Jane St. subscription plan.
> People in this thread seem to be implying that the company is botting/astroturfing, but for what? To maintain the image that they're a highly sought after employer?
Yes, that's exactly what I'm implying. I honestly would have stated it explicitly in the first place, but I made the incorrect assumption that it was not controversial to claim that this phenomenon exists at least for some companies and that it can be described as a form of marketing.
> Anything Paul Graham posts gets upvoted here instantly, because it's Paul Graham who posted it. For Jane Street there's the same halo effect, because many believe Jane Street's work is impressive. Is it really easier to believe the idea of botting to get an even longer list of candidates to interview, than it is to believe people just like to hear what this company has to say?
> I don't ever see anyone saying Paul Graham is botting posts about his "essays"
If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument. That said, I also don't bother trying to discuss his writing much because I mostly find it insufferable. I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.
Sure. What I said is a real world fact is the base salary, which can be verified by any of their job postings, e.g. www.janestreet.com/join-jane-street/position/4274288002, where it says "Base salary is $300,000. Base salary is only one part of Jane Street total compensation, which includes an annual discretionary bonus".
The claim that has less direct sources is how much the TC ends up being. The figure I said is the one I've heard multiple times over, sometimes by JS employees themselves, so I assume it to be true
> If we were discussing this on a website originally created by Jane Street, I would find this a more compelling counterargument.
That's fair, but I used Graham just as an example of source that always gets upvotes here. Uncle Bob hasn't posted anything in a while, but if he did, it'd certainly trend here, so that might be a better example
> I find the motives of Jane Street to be more interesting precisely because my opinion on them is more malleable.
Yes, that's fair. I don't think it's impossible that JS would bot/astroturf like you're implying, I just don't think it's the case. By Occam's razor I find it's more likely that people do care for what they have to say, similar to P. Graham, Uncle Bob, or any sort of blogpost from Anthropic/OpenAI, etc. It's quite possible that they got to this stage due to intentional marketing, though, so I'll give you that
https://hn.algolia.com/?q=jane+street
Probably just random fluctuation.
Yeah, Poisson distributions show up everywhere if you are paying attention. You can even use the theory as the basis for a fast, elegant algorithm to take parallel, weighted random samples. You can even do it in SQL!:
For more on the Poission connection, see: https://blog.moertel.com/posts/2024-08-23-sampling-with-sql....the book is underutilized?
Quant firms are also notorious for being some of the most selective in the world, so that alone has become a status symbol. The idea that if you land a job at firms like Jane Street, you must be some 150+ IQ math/physics/cs Olympiad machine.
Looks like two things:
1. them publishing a lot more programmer-facing material
2. more news coverage (SEBI ban, Terra suit, ...)
Don't know if it's a deliberate marketing push though, tbh.
It's a really interesting time to work in this field. People who are paying attention to workflows are using a range of tools, and constantly refining their own workflow. For example a whole team might use Claude, but one person collects data about their own usage on an ongoing basis and develops more advanced workflows than everyone else on the team. Their focus isn't just about working faster, but about doing better work.
I don't think the post is vague; I think we're at a time where you need flexibility in your interview process to find candidates who are doing the best work these days.
In some ways, although I detest AI, it has clarified the sort of people and organizations I want to associate with. Any latent characteristics that I abhor have been made obvious with the existence of AI and of course, I would want nothing to do with Jane Street.
first, a coding problem that is solved live (right now, online) -- it's not a hard one and most LLMs can easily one shot it.
BUT: if you are not a senior developer, you need to do it without LLM, and if you are a senior+ and do it via LLM, the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.
the main component here is to know if the candidate understands coding in general, understands problem solving and how to dive into an existing (although quite small) codebase. solving the problem is not really the goal -- in fact, most candidates we hired did not solve it 100%, but were able to understand and explain everything that they put it on code. this usually takes 1h30.
(we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).
the second is an architecture interview, where we build a system together BUT instead of doing the whole spec, miro board, whatever, we do it in more of a free way: we start with a really simple, MVP level, spec and we ask how the candidate would build it. then, we add another feature, and we ask how they would go for it. during this whole thing, i ask random questions like "why would you use x instead of y?" or "why do you prefer this approach?".
one important thing: we allow the candidate to say "i don't know" and we tell the candidate that there's no gotcha questions (which is true, i hate these kind of questions). this lets the candidate be super comfortable (feedback we had from real people).
in general, this flow helped us a ton and i really really like running it :)
(edit: spelling)
I've heard this multiple times now and it still boggles my mind that git is something you have to test for.
I can gather that the way we're using is not the most usual ;).
This is more about opportunity cost because instead of asking about git, you could ask about other valuable skills in your limited time with the interviewee, but I guess if git is really that important, then it's justified.
Because LLMs aren't a reason to not learn your tools. A determinist CLI command vs using a statistical machine for it. It's just a waste of tokens at the very least.
Some 6-figure per year developer spending a quarter’s worth of tokens to do something they do once a month or so is not something I’ll ever care to optimize away…
I don't disagree that having a deterministic CLI is useful, but I'd argue that's not the question here, because the CLI is what the statistical machine will be using. The question is whether the time spend doing it manually is worthwhile compared to the tokens spend on it. I'd argue that if you instead learn how to pick the right model for the right job so that you don't use something way more expensive than needed for the task at hand, the token overhead will be extremely minimal for stuff like git operations; knowing that you can use something cheaper than like, Astra or Opus if all you want to do is a quick git cleanup and push is going to save your employer a lot more money in the long run than if you insist on doing every messy rebase by hand.
In terms of team practices, that basically seems like a code review thing. If my team has conventions around git, and a PR doesn't fit them, that's something I would point out regardless of how the PR was generated. If a team member is repeatedly not meeting the expectations we've all agreed upon, I would deal with it in the same way I always have; reach out and offer guidance for how to do things the way they're supposed to, notify my manager if they continue having trouble with it, and eventually just not bother looking at their PRs if they absolutely refuse to do things the way everyone else on the team has agreed they should be done.
Even if an LLM can hold your hand through everything, the disruption I've had because people don't understand what the LLM tool is doing, and ultimately does damage (we've had repos corrupted, systems taken offline, systems brought their knees, numerous data security incidents... it's been a fucking nightmare) is immense.
Even worse, it gave a developer a command to run that basically crushed all history across all branches, it was a mess to clean up and lost a significant amount of work. It wasn't Claude running it on its own, but was entirely driven by a mix of a developer's lack of understanding their tools, and trusting Claude's output too much.
This was also the dev trying to "remove a key he had accidentally committed" again, just letting Claude take the wheel.
Of course, the new team _also_ really loved leaning on AI so they were all behind his plan to "clean up the repo" and not just reset the key and move on.
It was a nightmare.
One time setup commands are the sort of thing people do once when setting up the repo, but dont use day to day. its understandable that people dont have that memorized and will reach for a man page, google or an AI.
Though ideally interviewers should appreciate this - unless its a hard requirement you do it by hand day to day - the rest of us can look this up quick enough.
My OS professor in college once told us about what he considered the difference between "knowledge" and "information", with "knowledge" being the underlying facts about how something works and "information" being the terminology/labeling/bucketing/etc. of how to refer to it. The point he made was that knowledge was more important than information, and that he would try to test us on knowledge rather than information. To me, knowing how the underlying version control model works and what "shapes" you might want to make with it is useful knowledge. The exact set of commands, interactions with an IDE, or specific prompts you can use to ask an LLM to produce them is information. Arguably the entire benefit of LLMs is that you can ask them to do things even if you don't have the exact information memorized; if you can describe it in a rigorous way, and recognize whether what it gives you as an output fits that or not, you don't need to know any of the actual information of how to do it manually.
jj? what's that?
I wish better names would be chosen by the tech industry, that would make search engine discoverability easier.
> The command-line tool is called jj for now because it's easy to type and easy to replace (rare in English). The project is called "Jujutsu" because it matches "jj".
https://github.com/jj-vcs/jj
There is no way to verify it just by talking.
like, i dont really need to know all the stash details when i only stash one thing at a time, very temporarily
i also dont really care to know all the origin and remote definitions stuff when i only touch it once per project or so.
i could certainly draw graphs of what different git repos look like, but what makes it salient to know the rest of the details unless im actually building git itself? if youre using every git feature regularly you're probably doing something wrong
If I were hiring, I would care about whether you can `git help stash`, skim over it (bonus points for actually using the pager competently) and figure out what you need to do.
I couldn’t care less about stashing commands.
I do care about … if someone understands origins and not by definition because developer needs to know what is his local version and what is on the server and how those two affect each other. Bonus points if they understand server is not special in Git and their colleagues copy on their laptop can be origin as well.
Drawing graphs of what different repos look like? That’s irrelevant.
Irrelevant if one understands that Git uses DAG and is content addressable store.
You see someone might know those hard words but when working daily with Git reasoning about rebase or merge, cleaning up their own work with interactive rebase, making pull/merge requests or resolving conflicts it is much easier if one knows what is local what is remote and how Git stores data.
You also learn it once. In my 15 years of using Git I basically didn’t learn much new commands but having mental model of how it works was my tool to fix any problems other people had. Knowing commands or definitions is secondary.
Now I just ask google, and I get a straightforward "do this", and it works.
That's the thing, though. Stack Overflow is not and never was intended to solve your problem, nor to solve the OP's problem. It was intended to answer questions. It was your responsibility, both as someone posting a question and as someone reading the Q&A later, to adapt knowledge to your circumstances. That's why they're called questions and answers, and are not called problems and solutions.
People who use a web site to get a solution to a problem are looking for a help desk, or a forum. Stack Overflow was specifically created to be not that, and that choice was specifically motivated by the frustration people had with trying to search those kinds of sites for information, and the frustration that experts had with trying to help people (for free) in that kind of environment.
Personally I ask the LLM to summarise what the commit does and explain it to me, make sure I understand it, then review the PR. I'm getting half the speed the High Speed Sloperators are getting but I was the only person submitting PRs the other day when the LLM service was down.
For nasty multi-way git merges and rebases, it's been a game changer. And of course that's reviewable by doing a delta on the final rebase branch vs. the original branch.
With all that said... having to use LLMs at work is making me want to use them less on my own projects.
For the more important changes, I find it easier to stay on my toes by taking a pass at the codebase before comparing notes with the LLM. Peeking at the answer key first makes it too easy to nod uncritically.
Git is an essential tool; not knowing how a key tool works is dangerous IMO
I have not run a single git command in like the entire 2026.
Should you resolve the conflicts in a bad way we're going to see the result in the MR, will we not?
I don't think git knowledge is essential anymore.
Back in the days of the crappy git GUIs this was a problem, because people would regularly misunderstand even the basics of what they were doing and managed to lose their local changes.
That shouldn't happen with LLMs.
Because someone who's familiar with those sorts of things can get the commands typed and running before someone who isn't has even figured out the prompt.
(Well, not so much if there's a need to identify the base commit for a squash, do some cherry-picking, etc. But in those cases I'd rather see direct human judgment of what to do than hoping the LLM understands a vague description.)
initializing a repo, staging files, adding them to the changeset, commiting then pushing those changes are easy to do. solving merge conflicts is a little complicated but IDEs make that much easier to do.
Fundamentally git is conceptually complex in a way that almost no other tool is. Others might have scarier syntax to newcomers but rarely expose the same level of underlying complexity.
the coding challenge itself is just a way to make sure that people can understand a request within an existing codebase. as an example, i pay attention to the questions the candidate asks -- questions asked always mean something.
my general assumption with seniors are "they can think high level", which means i "mark them" with that assumption, so for me, the architecture talk is much more important.
at the end of the day, you need to understand what your org needs and adapt.
The trick is to get a good sense of the algorithms associated with basic data structures (list, stack, queue, graph, hashmap,...). They are primitives and almost any standard problem can be solved with those.
What you need to train most is the capability to break down problems: Like I have this input (and associated metadata), I have this context (may be null), I have those constraints (values, ranges, memory space, time, both, or neither), and I need these outputs. Then it's just a puzzle.
I no longer prepare for coding problems other than a quick refresh of the syntax of the main language. And if it's a whiteboard type of things, I go with pseudocode.
A candidate may be able to open a PR and not be able to make a commit without an IDE.
Another candidate might know plenty of git, but haven't worked with Github.
That seems very unlikely. Github is the entire reason git is so popular, vs. any of the other distributed VC systems.
Github was like Yahoo's directory, a way to share content in a searchable place and as a backup/bounce point. In the early days you could also upload to 'downloads' so it was a quick share host. Before that we had email, exposed ssh and cgit/gitweb but Github gave us `PR` to promote social interactions with forking. So it could be argued that Github tainted the spirit of git.
Having personally evolved from making date stamped zip files, to subversion and then to git, it earned its place quite quickly for myself and my roaming nature, didn't hurt who hacked it together either. When Linux adopted it from Bitkeeper, it became the way for many.
Personally I prefer GitLab over GitHub. The CI is better, and it doesn't go down every other week unless you self host on Azure.
I have no problem with letting an IDE or LLM handle those details in practice... but I don't think I'd want to work with someone who doesn't understand the very basics of what's going on under the hood.
The GP said "open a PR", not "make a commit and push it to a remote".
When I tell my LLM to "open a PR for those changes" (which is how I do it nowadays), my LLM understands that what I mean is "make a commit, push it to a remote and raise a PR".
Does it mean we've reached AGI when machines understand what we mean better than some humans?
This makes it seem like all of those things are part of a single operation to you. That doesn't make sense to me.
On the occasions where I've submitted a PR to a project (anywhere, not just on Github), making the commit was not part of the same operation. I'd already made the commit and satisfied myself that it captured everything I wanted, before making the PR. I don't see those things as part of a single operation. I don't think the poster you were responding to does either.
1. Navigate your browser to GitHub.mycompany.example.com
2. Click the repo name
3. Click pull requests
4. Click the source branch[1] and destination branch
5. Click "create pull request"
???
I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.
[1] Presumably already extant, because they never mentioned anything about committing code, nor pushing it to github (and that's not the same operation!).
The OP said:
"we also ask the candidate to open a PR"
So I think the OP meant to have the candidate, um, open a PR. Which is not the same thing as making a commit and pushing it to a remote. Maybe you always conflate those operations, but that doesn't make them the same operation. If the OP actually meant that they ask the candidate to make a commit, push it to the remote, and then open a PR, I would have expected the OP to say, um, all of those things, instead of just the last one.
I couldn't say how the OP expected the candidate to open a PR, since I don't know what their standard workflow is. There are ways of doing it from the CLI, as well as through the web interface, as well as through various IDEs.
> I started out by assuming this was some kind of good-faith disagreement, but I guess you're just trying to be funny.
Please save your patronizing for another time. And stop trying to put words in my mouth.
How does that work? Git has no concept of pull requests.
Doesn't surprise me at all that many developers can't do this, most professional software development isn't happening on Gitub.
Interestingly, an LLM involved feels like it’s made it easier to assess candidates. Senior candidates know exactly what they’re doing and strongly driving the LLM. They pre-communicating design decisions, gotchas, and requirements. They also ask a lot of clarifying questions and dial in the scope (after all not building something is better than building it).
Less experienced candidates tend to jump right in, build way too much, and stumble because they delegated all of the critical thinking to the agent. They often end up in a place where the agent doesn’t align with their vision and they can’t really reign it back in.
Worked with a "super senior", lead architect. Management was raving about the guy before he joined. Insane resume. Skipped the regular interview process because of this.
Had to show him how to open a PR and setup his git config so his commits are linked to his account. And then his PR's failed the automated ci checks for descriptions and titles, and he came to me to ask how to fix it. The failing gh action literally says what was broken.
This is all with Claude available btw, so one could literally just ask.
I guess it's a bit of a modern fizzbuzz. Where it's so simple but still manages to stump a lot of candidates.
The idea is to see how the candidates deals with it. Maybe he will just say "fuck it" and do it by hand, or patiently point out every mistake so that the LLM gets it right. Both indicate competence, what you don't want is someone who doesn't see the mistakes and hand you slop.
You have missed the point entirely.
Will you allow jj instead. I switched to jj 1.5 years ago (as many others have), and my git is now rusty.
I mean sure, I can do add, commit, basic diff, etc. But rebase/merge - oh dear, I forget which one is source/target (jj made the infinite design choice that the ordering doesn't matter - you have two parents making a child - it's symmetric).
Today: https://news.ycombinator.com/item?id=50467297
Yesterday: https://news.ycombinator.com/item?id=50455394
Two days Ago: https://news.ycombinator.com/item?id=50438261
It's a bit odd (?!).
There are pretty few companies out there like this. Being relatively small probably helps; everyone knows someone that worked at Mag7 and maybe has a bad story to share, but this is more rare.
I never heard of them.
If not for HN, I would never know they existed.
I don't think the remark about "nerd prestige" holds, but beyond that your points boil down to marketing.
And how an unknown company changed its hiring practices is hardly front page news.
In general - I am wondering how engineering interviews change in the age of LLMs, Jane Street, tech-wise is a "thought leader", at least having opinions, sharing them is part of how they recruit "the best", i think oxide does this too.
So in some way, in this time of change it makes sense that they are noisy... It also makes sense that the noises they make may filter down...
Whether you agree with what they do or not, they have had a sort of mystique about them (tons of money, quant vests, _and_ a niche functional programming language as a daily driver??? Sign me up!) that HN likes.
eg: https://www.sec.gov/enforcement-litigation/litigation-releas...
> The cross-market manipulation involves buying and selling stocks at a loss, allegedly for the purpose of moving the prices of corresponding options, so that Avalon could make a profit by trading those options at artificial prices.
> The Court found that the two alleged manipulative trading schemes-if proven-would violate Sections 10(b) and 9(a)(2) of the Exchange Act and Section 17(a)(1) of the Securities Act.
or https://www.google.com/search?client=firefox-b-1-d&q=banging... per https://www.steel-eye.com/news/jane-street-fine-566.3m-marke...
What I learned over time is that first, the pool of people who regularly submit to HN is surprisingly small, so if they have interest in a particular set of topics, these topics are overrepresented in the queue; and second, relatively few people patrol new submissions and the the first 3-4 upvotes in the first 15 minutes decide if something bubbles up to the front page or not. These folks often go by keywords / vibes / employer associations, so for example, anything that mentions any frontier lab is more likely to bubble up.
More generally, I think there's been a significant drop in topic diversity on HN over the past year. Almost everything on the front page is related in some way to AI or to companies on the forefront of AI.
Reddit? General-interest subreddits such as /r/technology are mostly mainstream news, but there's plenty of specialized ones, such as /r/math, /r/programming, /r/c_programming, /r/rust, etc.
Many of the aggregators "killed" by HN are still alive, just not as popular. For example, boingboing.net is up and running. RSS feeds from places like Ars Technica are a fair choice for industry gossip too.
Consider this: Would you find anything unusual about seeing those stories on the front page if each came from a different company or didn't mention the company name at all? I wouldn't, because each of the submissions is primarily interesting for the content, not the company associated with it.
My optimism comes from AI's ability to solve genuinely hard mathematical problems this year. September and October could go down as two of the most remarkable months in the history of AI If, and only if, those proofs are correct and independently verified by real experts. Even if the proofs have been formally verified in Lean, expert scrutiny still matters.
There are a lot of people telling you how they interview and why. Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview, much less that they apply it. They also are not showing any indication that they have done any testing - to see if their questions help hire better people.
If I said they could get equal results by picking resumes at random and hiring them without any other screening could they refute that? If so have they justified every part of their process or just one step that is useful and the rest they don't know?
> One day, the recruiters gave us a special round of packets to review. In these special packets, we were able to read the interviewer notes and candidate responses. All personal details were stripped out, and we were told it was a “calibration exercise.” We had to do our regular voting job with these special packets, and see how it went. I think we may have assumed they were from another site, since cross-site calibration was common.
> Our group did our job, and voted not to hire about 2/3 of the packets. This was about par for the course.
> But surprise surprise, this time, those were our own packets from when we had all interviewed at Google. The recruiters had tricked us into reviewing our own interview packets, and we had voted not to hire most of our own group.
From https://yegge.ai/essays/the-last-technical-interview/ (May 2026)
its not like they instantly reviewed their own docs; the hiring bar and comparisons changed
So that implies that they are testing this, trying to determine if it works better than the current process. Whether they are doing this in a scientifically rigorous way is unclear, however.
In my experience (both as interviewer and interviewee), interviews primarily serve to satisfy the ego of the interviewer.
The topic of the article is about adapting interviews in the age of LLMs.
Scientific research on interview techniques requires years of retrospective analysis after the hired employees have been evaluated in their roles for a long time.
That research does not exist on this topic. It's too new. I don't think this is a valid criticism of the actual article.
You might have 2 people with opposite personality traits ace the interview then get put on a team together, now all of a sudden your whole team is worse overall. That's a massive effect but your study will just see it as noise.
The reality is your best hires will just be intelligent people who belong in the group and who push the mission forward.
The idea of forcing a candidate to give a third party access to their screen, transcribing AI notes, and forcing them to do 3+ leetcode, and an architecture round, and a behavioral round while I watch over them for hours on end sounds completely idiotic and antisocial. This entire industry is high on its own supply when it comes to interviews.
ie. Not actually interviewing software engineers.
Perhaps one day, interviews will require candidates to "bring your own harness/prompt/skill/mcp"
How ‘wrong’ are we talking? Like the code is correct, but slow? Fails easy-to-spot edge cases? Uses poor taste in variables names?
This is probably genius and also hilarious!
"You are a helpful coding assistant. Make some mistakes."
I would say my skill is now working things through with people in the business/users to figure things out and improve what the software does.
It is like an unhackable way to do it. Just hiring the person, seeing how it goes, and letting them go or not. It isn't rocket science. The guys digging a ditch outside the office know more about hiring than the career HR professionals inside I guess.
What I want to know is whether someone has basic DSA understanding. Is the person familiar with any programming language. Can they translate business requirements into code? Can they disambiguate weird things PMs say into a viable plan?
You do not get a better signal about that from asking harder leetcode problems.
Never once in my career has writing a tricky algorithm under time pressure been relevant. The closest thing I can think of is debugging complex systems in an outage, but then it’s more like familiarity with bash commands and general debug skills.
As for LLMs and coding, companies need to figure out how to differentiate good and bad outputs here and what lasting engineer inputs to select for. And I don’t have any idea about this.
The argument was dsa functions like an iq test. It’s not relevant to the job (in fact it’s harder than the job, far harder) such that if you do well on it companies can be confident you can do well on the job just like how scoring high on an iq test makes people confident you’re intelligent.
There are plenty of arguments against what I said above and going after this argument is the better target because the irrelevancy of dsa to the job is widely known and obvious.
I've been engineering systems for 25 years. On a macro level, most of my time is spent considering the right tooling and stack for a job, balancing budget and time constraints and efficiency against how things may need to scale in the future. On a micro level, in the data model and in the code itself, choosing the best structures for a task is important for maintainability and overall coherence.
In my experience with LLMs, I often need to guide them toward choosing the right ways to model data on the code and database level. LLMs seem to prefer json blobs of unstructured data, naming conventions that drift over time, argument structure patterns that change from one piece of code to another. The net result is a lot of tables and code that an LLM needs to read back through, find all the linkages, and recommit to its window before any debugging can take place. It makes human debugging very difficult if, for example, an LLM decides to just store all the userdata in some json column and then decides to add some kind of stateful preferences to that, and other state to something in memory, and then parse the memory falling back to the database state. I've seen a ton of things like this where weird ways of handling state kinda work, until they don't, and it's extremely difficult to figure out what is taking precedence when or where a thread race is coming from.
All this is to say that DSA knowledge may be even more important when LLMs are writing most of the code, because an unguided LLM will write stuff that can practically only be maintained by an LLM, and make design decisions that don't cohere or scale. If that's not really a problem, and you're certain that the code you're investing in writing will never need human intervention, then I guess it doesn't matter. But with a large enough system designed that way, the boundaries and edges of poor DSA choices aren't going to be apparent until either something breaks or you need to grow.
To get into college, there's the SAT test with a minimum score.
To get a high paying software job, there's the leetcode test.
These are gates meant to filter out the obviously unqualified candidates. Unqualified candidates waste the time of the candidate and the company. The real interview comes after that.
And leetcode is not something you prep for once in your life, but every time you switch jobs.
So he did the three weeks, passed the leetcode, passed the interviews, and now makes 400k.
When I was 10 or 11, I asked my dad what MIT was looking for in a student application. He (an MIT alum) told me. I picked out the things I could do (being captain of the football team was never going to work for me), applied to MIT, and got rejected. But Caltech, Dartmouth and Johns Hopkins did admit me.
You do what you gotta do.
The biggest problem, of course, is that they also filtered qualified candidates (I personally got filtered out multiple times and I'd like to believe I was qualified!) But it is an understandable compromise whenever there is an extreme asymmetry in the volume of candidates and volume of interviewers, and the fix for that is simple: you study for the test (as you pointed out.)
Note also that as grandparent comment indicates, this test is a necessary but not sufficient condition; it's just to get your foot in the door for a deeper evaluation.
It's like "to get into Marine boot camp, there's a minimum crochet requirement".
Solving leetcode problems has nothing to do with commercial coding. It's an unrelated hoop that we make folks jump through, because it's easy to measure. It's like measuring developer productivity by LoC written.
Yes, it'll measure how much time the candidate spent preparing for the interview, and how well they can jump through hoops. But it'll also deter the actual people you want in the team, the folks who know that leetcode has nothing to do with commercial coding, and see the leetcode test as a sign that your organisation has no clue and a bad management culture.
People don't like Leetcode because it's insulting to treat them like freshmen that know nothing but algorithms that related poorly to the job at hand; can't judge them on coding prowesses, for most of them.
Wait, why am I explaining this to you like you're ready to listen beyond your monologue?
You got my vote right there. I also find it funny that LLMs can easily do leetcode problems, because there were so many examples and easy test cases.
> You do not get a better signal about that from asking harder leetcode problems.
I think this is a thing that needs bigger formatting for this statement or a feature to vote twice.
---
Only response I have to this:
There are many different schools of thought on how to architect systems. In many cases these different opinions don't have a lot of solid backing on why you'd do this. On top of that there is level restrictions that prevent talented individuals from being more influential. I.e. an Architect is less about technical experience and excellence and more about the abiltiy to communciate to high level stakeholds and having systems that can solve the problem [some what].
I have been looking for resources that tackle/teach/learn this translation part, but so far hasn't gotten any hits. Can anyone here point me to such a resource, if it exists?
The problem is that the compensation was so generous, and the kind of questions being asked so public, that your typical somewhat less talented, yet hard working grads would just cram for the interviews, and get past them, thus lowering the value as a filter. Leetcode and the like decided that there was an industry in helping you cram. The filtering got worse and worse, and the companies then decided to keep upping demands, hoping to still get the most talented. But there's nothing in a leetcode interview that cannot be trained. So the metric became the target, and we ended up in the current mess.
I agree with you but this is a "simpler" test with a subjective analysis (simpler in the sense that the candidate might make some progress rather than completely stuck). Big Tech works at a scale where they probably want to quickly evaluate based on objective results. Did they crack the problem or not? Imagine having a pool of 1000+ candidates for a single role.
We laid off most of the company and just offshore everything anyway so I don’t care, least I can do is give these randos a chance in Brazil and hope it works out better for them