The point is there are limits on how much better the compilers can get, without modifying the language to be closer to the target environments. And there are multiple target environments, so a one-size-fits-all solution is not really possible. You have to compromise somewhere - you can’t have a single high level language that automatically produces maximally efficient code for every target, because the targets are not general-purpose computing environments, they have significant constraints.
If what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet.
One of the reasons why I have been thinking and writing this topic is because there is indeed a malaise among the team, which is along the lines of "AI agents are blowing up coding everywhere, what place does a compiler have in this brave new world?" If compilers truly are pointless, then I would like to update on this and save people time and grief!
But I also don't think compilers are pointless. In this post, which looks at the maximalist AI situation, the compiler resurrects itself as the verifier: the two types of systems do very similar things! You can also defend compilers on more banal grounds: not everyone has a ton of kernel engineers to hand optimize your kernels; token efficient ways of getting good baselines is useful; there's more to ML than transformer models and compiler leverage is really useful there. There's also a historical point which is that torch.compile... by all objective metrics, has really been quite successful!
I do agree it's hard to tell the future. These days, I ask myself, "Do I think this is likely to happen in six months." I don't think compilers die in six months. Ask me again in six months :)
I think this idea is exactly right. Many ML papers that include pseudocode basically just write out a minimal PyTorch training loop - down to the loss.backward(). It has, along with Python, become something of a linga franca for research in the field.
8 comments
[ 4.8 ms ] story [ 28.6 ms ] threadIf what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet.
But I also don't think compilers are pointless. In this post, which looks at the maximalist AI situation, the compiler resurrects itself as the verifier: the two types of systems do very similar things! You can also defend compilers on more banal grounds: not everyone has a ton of kernel engineers to hand optimize your kernels; token efficient ways of getting good baselines is useful; there's more to ML than transformer models and compiler leverage is really useful there. There's also a historical point which is that torch.compile... by all objective metrics, has really been quite successful!
I do agree it's hard to tell the future. These days, I ask myself, "Do I think this is likely to happen in six months." I don't think compilers die in six months. Ask me again in six months :)