It's a bold strategy - let's see if it works out for them.
But from what I remember, this still uses the LLVM backend, right? Sure, you can beat LLVM on compilation speed and number of platforms supported, but when it comes to emitting great assembly, it is almost unbeatable.
LLVM is a trap. You bootstrap extra fast, you get all sorts of optimization passes and platforms for free, but you lose out on the ability to tune the final optimization passes and performance of the linking stages.
I think we'll see cranelift take off in Rust quite soon, though I also think it wouldn't be the juggernaut of a language if they hadn't stuck with LLVM those early years.
Go seems to have made the choice long ago to eschew outsourcing of codegen and linking and done well for it.
Additionally any language that wants to be a C++ replacement, keeps a dependency on C++ as long as LLVM is part of its tooling infrastructure.
For me beyond the initial adoption hump, programming languages should bootstrap themselves, if nothing else, reduces the urban myth that C or C++ have to always be part of the equation.
Not at all, in most cases it is a convenience, writing usable compilers takes time, and it is an easy way to get everything else going, especially when it comes to porting across multiple platforms.
However that doesn't make them magical tools, without which is impossible to write compilers.
That is one area where I am fully on board with Go team's decisions.
Is it? I think Rust is a great showcase for why it isn't. Of course it depends somewhat on your compiler implementation approach, but actual codegen-to-LLVM tends to only be a tiny part of the compiler, and it is not particularly hard to replace it with codegen-to-something-else if you so desire. Which is why there is now codegen_cranelift, codegen_gcc, etc.
The main "vendor lock-in" LLVM has is if you are exposing the tens of thousands of vendor SIMD intrinsics, but I think that's inherent to the problem space.
Of course, whether you're going to find another codegen backend (or are willing to write one yourself) that provides similar capabilities to LLVM is another question...
> You bootstrap extra fast, you get all sorts of optimization passes and platforms for free, but you lose out on the ability to tune the final optimization passes and performance of the linking stages.
You can tune the pass pipeline when using LLVM. If your language is C/C++ "like", the default pipeline is good enough that many such users of LLVM don't bother, but languages that differ more substantially will usually use fully custom pipelines.
> I think we'll see cranelift take off in Rust quite soon, though I also think it wouldn't be the juggernaut of a language if they hadn't stuck with LLVM those early years.
I'd expect that most (compiled) languages do well to start with an LLVM backend. Having a tightly integrated custom backend can certainly be worthwhile (and Go is a great success story in that space), but it's usually not the defining the feature of the language, and there is a great opportunity cost to implementing one.
I hope cranelift continues to grow too. LLVM tragically has become fork hell. Every cpu vendor now seems to have a fork with enhancements for their cpu arch in some closed source package. It’s hellish as soon as you want to use another language frontend or find the inevitable compiler bug.
A trap? New language projects would never be able to develop the architecture support and optimizer. There's nothing new in the linking stage you couldn't submit as a patch to LLVM upstream.
Seems more like a tr-ade off than a tr-ap. Everything has a price. I think LLVM is well worth the tradeoffs for all the stuff that it gives you. I'm sure that some better tech will replace it but calling it a trap seems like an exaggeration, it seems to work well enough for even the biggest companies and countries in the world.
The gold standard for compiler speed so far is TCC. (Fabrice Bellard)
It is not even super optimized (single thread, no fancy tricks) but it is so far unbeaten by a large margin. Of course I use Clang for releases, but the codegen of tcc is not even awful.
Yes. I once was thinking I should write a x86-64 codegen for my own compiler, but using tcc I was able to reach 3 million lines in a second (using multi-threading and multiple instances of tcc) so I called it a day. I never did check out if the linking was in parallel
Is there a straightforward path to building Zig with polyglot build systems like Bazel and Buck2? I'm worried Zig's reliance on Turing complete build scripts will make building (and caching) such code difficult in those deterministic systems. In Rust, libraries that eschew build.rs are far preferable for this reason. Do Zig libraries typically have a lot of custom build setup?
FYI, build scripts are completely optional. Zig can build and run individual source code files regardless of build scripts (`build.zig`). You may need to decipher the build script to extract flags, but that's pretty much it. You can integrate Zig into any workflow that accepts GCC and Clang. (Note: `zig` is also a drop-in replacement C compiler[1])
I've really appreciated the incremental builds with zig - I really like having a single static binary, and I have an application which uses a few different libraries like SQLite and luau and it compiles at nearly Go speeds.
Tangential: I've been writing a Java library that is pretty much a JNI shim to a C library. Looking at building a dynamic library for each of the platform / architecture combo is looking like a huge pain in the arse; I'm toying with the idea of using the zig compiler to do the compile / cross compiles and simply be done with it.
do yourself a favor and try it, Zig ships with headers and libc sources and its own LLVM distribution and makes cross compilation so easy it's just ridiculous, you literally don't have to do anything
I've been building a shared lib for both Android and iOS/iPadOS and using jni and jin_fn crate in Rust has been a God sent. Using cargo-ndk simplifies the building process so much.
I'm surprised that whatever IO primitive is used on Mac doesn't work on OpenBSD. Mac has select and some variant of kqueue, right? What is ghostty doing there that doesn't work on OpenBSD?
Might not be that different to add OpenBSD. Someone would begin here: https://github.com/mitchellh/libxev/blob/main/src/backend/kq...
It's about 1/3 tests and 2/3 mostly-designed-to-be-portable code. Some existing gaps for FreeBSD, but fixing those (and adding OpenBSD to some switch/enums) should get you most of the way there.
If you care about compilation speed b/c it's slowing down development - wouldn't it make sense to work on an interpreter?
Maybe I'm naiive, but it seems the simpler option
Compiling for executable-speed seems inherently orthogonal to compilation time
With an interpreter you have the potential of lots of extra development tools as you can instrument the code easily and you control the runtime.
Sure, in some corner cases people need to only be debugging their full-optimization-RELEASE binary and for them working on a interpreter, or even a DEBUG build just doesn't makes sense. But that's a tiny minority. Even there, you're usually optimizing a hot loop that's going to be compiling instantly anyway
For me, Zig the language is "meh", but Zig the toolchain is very interesting. It's really ambitious, to the point I think it can't work in reality in the long term, but I want it to succeed.
Honestly a bit surprised that he waits 16 seconds every minor change to Ghostty while developing. That sounds painful. Glad Zig build times are getting faster though, and I understand he is still on LLVM.
It's finally as snappy as recompiling the "Borland Pascal version of Turbo Vision for DOS" was on an Intel 486 in 1995, when I graduated from high school...
They C version of Turbo Vision was 5-10x slower to compile at that time too.
Turbo Vision is a TUI windowing framework, which was used for developing the Borland Pascal and C++ IDEs. Kinda like a character mode JetBrains IDE in 10 MB instead of 1000 MB...
Are their ghostty builds statically linked by default or does zig follow in the footsteps of the C/C++ world and dynamically link everything? If statically linking (like rust) then a huge part of the incremental build times for apps with external dependencies is the lack of incremental linking in any of the major linkers. Wild linker is working on becoming the answer to this: https://github.com/davidlattimore/wild
While LLVM certainly is a problem, consuming over 90% of compile times both for C3 and Odin, it should be noted that both languages compile MUCH faster than Zig, achieving in general sub second compile times while using LLVM and without incremental compilation. So the question could be asked: ”what is making Zig so slow to compile” as both languages have similar feature sets compared to that of Zig.
34 comments
[ 3.7 ms ] story [ 64.8 ms ] threadBut from what I remember, this still uses the LLVM backend, right? Sure, you can beat LLVM on compilation speed and number of platforms supported, but when it comes to emitting great assembly, it is almost unbeatable.
I think we'll see cranelift take off in Rust quite soon, though I also think it wouldn't be the juggernaut of a language if they hadn't stuck with LLVM those early years.
Go seems to have made the choice long ago to eschew outsourcing of codegen and linking and done well for it.
For me beyond the initial adoption hump, programming languages should bootstrap themselves, if nothing else, reduces the urban myth that C or C++ have to always be part of the equation.
Not at all, in most cases it is a convenience, writing usable compilers takes time, and it is an easy way to get everything else going, especially when it comes to porting across multiple platforms.
However that doesn't make them magical tools, without which is impossible to write compilers.
That is one area where I am fully on board with Go team's decisions.
Is it? I think Rust is a great showcase for why it isn't. Of course it depends somewhat on your compiler implementation approach, but actual codegen-to-LLVM tends to only be a tiny part of the compiler, and it is not particularly hard to replace it with codegen-to-something-else if you so desire. Which is why there is now codegen_cranelift, codegen_gcc, etc.
The main "vendor lock-in" LLVM has is if you are exposing the tens of thousands of vendor SIMD intrinsics, but I think that's inherent to the problem space.
Of course, whether you're going to find another codegen backend (or are willing to write one yourself) that provides similar capabilities to LLVM is another question...
> You bootstrap extra fast, you get all sorts of optimization passes and platforms for free, but you lose out on the ability to tune the final optimization passes and performance of the linking stages.
You can tune the pass pipeline when using LLVM. If your language is C/C++ "like", the default pipeline is good enough that many such users of LLVM don't bother, but languages that differ more substantially will usually use fully custom pipelines.
> I think we'll see cranelift take off in Rust quite soon, though I also think it wouldn't be the juggernaut of a language if they hadn't stuck with LLVM those early years.
I'd expect that most (compiled) languages do well to start with an LLVM backend. Having a tightly integrated custom backend can certainly be worthwhile (and Go is a great success story in that space), but it's usually not the defining the feature of the language, and there is a great opportunity cost to implementing one.
It is not even super optimized (single thread, no fancy tricks) but it is so far unbeaten by a large margin. Of course I use Clang for releases, but the codegen of tcc is not even awful.
[1]: https://andrewkelley.me/post/zig-cc-powerful-drop-in-replace...
That said, there are definitely still bugs to their self hosted compiler. For example, for SQLite I have to use llvm - https://github.com/vrischmann/zig-sqlite/issues/195 - which kinda sucks.
E.g., here it is kqueue-aware on FreeBSD: https://github.com/mitchellh/libxev/blob/34fa50878aec6e5fa8f...
Might not be that different to add OpenBSD. Someone would begin here: https://github.com/mitchellh/libxev/blob/main/src/backend/kq... It's about 1/3 tests and 2/3 mostly-designed-to-be-portable code. Some existing gaps for FreeBSD, but fixing those (and adding OpenBSD to some switch/enums) should get you most of the way there.
If you care about compilation speed b/c it's slowing down development - wouldn't it make sense to work on an interpreter? Maybe I'm naiive, but it seems the simpler option
Compiling for executable-speed seems inherently orthogonal to compilation time
With an interpreter you have the potential of lots of extra development tools as you can instrument the code easily and you control the runtime.
Sure, in some corner cases people need to only be debugging their full-optimization-RELEASE binary and for them working on a interpreter, or even a DEBUG build just doesn't makes sense. But that's a tiny minority. Even there, you're usually optimizing a hot loop that's going to be compiling instantly anyway
It's finally as snappy as recompiling the "Borland Pascal version of Turbo Vision for DOS" was on an Intel 486 in 1995, when I graduated from high school...
They C version of Turbo Vision was 5-10x slower to compile at that time too.
Turbo Vision is a TUI windowing framework, which was used for developing the Borland Pascal and C++ IDEs. Kinda like a character mode JetBrains IDE in 10 MB instead of 1000 MB...
https://en.m.wikipedia.org/wiki/Turbo_Vision