Ask HN: How to get a job as a compiler engineer?

https://www.simplyhired.com/search?q=compiler

https://www.ziprecruiter.com/jobs-search?search=compiler

https://www.linkedin.com/jobs/search?keywords=compiler

https://www.dice.com/jobs?q=compiler

Good luck!

> My goal is to actually use the concepts I spent so much to learn in my PhD in my career.

Do you mean you're specifically looking for a job that lets you focus on the grammar/parsing of programming languages (possibly with attribute grammars in particular!)? Or do you mean front-end (or general) compiler development more generally? Because if you specifically want to apply research on parsing techniques, as much as I love that area and wish there was more interest in parsing, I dare say you're unlikely to find such businesses hiring for something like this specifically (unless there's some research institution working on projects like this), given it's not exactly a money-making opportunity. Though if you somehow think there might be such a job, I imagine the best people to ask would be PL faculty.

Maybe have a snoop around these types of firms?

I think you may find that lots of the job is ‘regular software engineering’ work if it involves maintaining a production compiler rather than being all about specific compiler techniques there will be lots of time on eg making the compiler faster, improving debugging information, coming up with better error messages, etc. I guess something more on the research end is going to be totally different.

Google has a lot of compiler stuff. XLA for machine learning: ML graph compilation is a huge area right now because ML is so costly and optimizing ever-growing ML models is something compilers help a lot with. Closure for javascript. A bunch of their own languages: {a few versions of SQL that optimize for different workloads}, {go for servers}, {dart which is both AOT + JIT compiled for rapid app development but performant serving}, {carbon because rust is hard to adopt at a company like google}, ...and this is just off the top of my head. Teams continue to develop and improve these compilers & devtools around them every year, so there are definitely jobs. It feels like big companies like Google / MSFT are most likely to be doing interesting things with compilers because they don't have as much direct linkage to product and thus it's hard to get budget to pay people to do these things unless you're a money-printing behemoth. I'm assuming you don't want to take a big pay cut just to work on compilers :)

In terms of landing the job, I don't think knowing specific tools/libraries/frameworks are that important. I actually flirted with joining one of the teams mentioned above years ago despite not having compiler background, so that's the other thing. They care a lot more about you being a solid engineer and teammate than anything else. PhD will help you get an interview and if the _perfect_ role opens up for you, you'd have an edge. But more often I see folks with PhDs work on things unrelated to their academic research. And actually I think it can be more fun if you're open to doing something a little different, if still in the domain of compilers/programming languages. Your experience will definitely become useful if you are close-ish to the subject, but you'll learn more if it's not _too_ close.

LLVM / MLIR / GCC all have significant numbers of employees working on them as well as volunteers, partly because hardware companies employ software devs to make them work for their hardware. I'm not as clear who funds front end work, seems to be big companies that like internal tooling.

Compilers are great. Exciting is in the eye of the beholder though, you'd have to expand on what interests you before we could guess a more specific direction to try.

Overall, I think the job market for compilers/program analysis is pretty strong right now. It's also well spread between companies of different sizes and scopes: MS and Apple are doing serious LLVM-based optimization work, while you also see medium-sized companies (Fastly and CloudFlare) doing interesting work on WebAssembly. On the research side, companies like ToB and Galois have been doing LLVM and other compiler-based program analysis for over a decade.

[1]: https://www.trailofbits.com

[2]: https://www.trailofbits.com/careers

Source: I know people on both the Scala and Swift teams and this was my experience. Everyone wants to work on these teams, and it was generally seen as the most enjoyable work. You pretty much had to have good connections/solid OSS contributions/relevant PhD to get a placement on these teams. I imagine it's the same at MS.

Some companies solve problems that have a larger component of parsing and evaluating things.

One example is Hasura's SQL-to-GraphQL layer: https://hasura.io/

Another example is GitHub's CodeQL: https://github.com/github/codeql

A third example -- I forget the name of the company -- parses Counter-Strike games real-time and restructures this data for better analytical introspection, e.g. for betting, time-scrollable replay, 2D rendering, etc. There appears to be a lot of hard-earned knowledge going from a stream of events monkey-patched over two decades, to a complete model of a game (who's on what team, who is dead, what round is this, etc.)

Microsoft does a lot of interesting compiler-related stuff, too, of course.

Then there's blockchain: A lot of programming-language enthusiasts have been employed to write VMs and DSLs to express safe application-level environments. An example is Anoma's Juvix: https://github.com/anoma/juvix

A job I was looking at involved building a more programmatic interface to some legacy SCADA systems, i.e. make old factory monitoring systems interoperate via a DSL. The idea, I think, is to transform and manage the configuration files from a dynamic GUI system, but not employ people to drag-and-drop; rather, express this as code.

tl;dr: If your main tool is a compiler, there are compiler problems everywhere.

Generally, you can try these companies:

* really big software companies, like Microsoft, IBM, Google...

* chip makers

* developer tool makers, like JetBrains

Also, search engines are a thing :-) https://www.dice.com/jobs?q=compiler or try linkedin...

On the topic of finding a job … I would be of no help. The only internships/jobs I found so far were all outside the usual applications forms:

* By emailing to the author of an R&D language that I was already using, asking if they could make a position for me in their lab.

* From a teacher recommending me in the start-up where he worked for building an interpreted language.

* By going to one of the Firefox 4 party, and giving a visit card after seeing a presentation about SpiderMonkey.

Then, if you are interested in applying for a job opening on compilers. There is one right now at Mozilla, on WebAssembly: https://www.mozilla.org/en-US/careers/position/gh/4437801/ … but there is no attribute grammar.

Until recently I was heading a compiler team. We found it impossible to hire experienced compiler engineers despite paying very well, so we essentially switched to hiring promising university graduates and trained them on the job. My peers in other companies are all in the same situation.

Note that there is a mismatch between academic and industrial compiler work. A university compilers course is necessary, but not sufficient to be considered experienced, you should have good knowledge of things like ABI design, ELF, DWARF, LLVM's tablegen, C++, compilation of varargs etc. This stuff is not typically taught in compiler courses (for good reason, since it is a bit arbitrary) but it is absolutely vital to producing a working, industrial strength compiler. You should also have some idea about computer architecture, so you can read ISA manuals and understand what individual machine instructions do, and what performance characteristics they have. Otherwise you won't be able to write good instruction selection and scheduling. Given the increasing prevalence of JIT compilers (eg eBPF in the Linux kernel), it's also useful to know about that. Another dimension is security: modern compilers to need to worry about stack canaries, shadow stacks etc. This too is not taught at universities.

Regarding what tool/library/framework, skills are rather transferrable, so I would not worry too much about this. LLVM is by far the most widely used open framework and is easier to get into than other open stuff. If you are knowledgable about security and verification, doing hacking on e.g. CakeML [1] or CompCert [2] might be interesting.

A good way of being hired is to go to e.g. LLVM or GCC meetup, and make it clear that you are looking for a job. Put it on your Github that you are looking for a job. Put a link to an LLVM or GCC (or similar) compiler phase that you have written. The next LLVM dev meeting is November 8-9 in San Jose, CA, hang out there, rest assured there will be many companies seeking to hire. Bonus points if you give a presentation, even if it's whimsy like compiling Brainfuck to Intel 4004 ... Another avenue is RISCV startups which are currently 10-a-penny, they all need and hire compiler engineers. Cold-contact them ...

PS, since you are a PhD student. I suggest finishing your PhD in some form or other. A lot of companies are loath to hire PhD students before they finish, or make the job offer contingent upon successful graduation. That certainly was our policy. If you are not yet near graduation, I suggest to do a compiler related internship.

[1] https://cakeml.org/

[2] https://github.com/AbsInt/CompCert

[1] Shy bairns go hungry, goes some saying.

Hmmm... That was me, 12 years ago, after having done a Ph.D. focussing on computational semantics (the nlp-kind, not the compiler-kind). Trying to apply it anywhere in industry has been my white whale ever since. I'd recommend de-emphasizing the "I'd like to work here to apply this incredibly specific thing that I've been working on for my Ph.D." angle in job interviews.