I'm curious about motivation - bounds checks for corrupted inputs seems like it would be one, but it also seems that fixing corrupted input handling in a C/C++ code base would not be too hard, and probably less of an effort? So why did you choose the rewrite?
And second, did you use any AI tools for the rewrite?
I suspect fearless concurrency is another motivating factor. Better perf can be achieved by squeezing more parallelism, but without borrow checking it's difficult to do fine-grained parallelism correctly.
> fixing corrupted input handling in a C/C++ code base would not be too hard
The best programmers on the planet have tried and failed with this task for 50 years now, so I don't think this is true.
The main disadvantage of Rust right now is not supporting some more obscure platforms, but because mold wouldn't support them anyway I don't see that as a problem.
> The main disadvantage of Rust right now is not supporting some more obscure platforms
Last time I checked, I got impressed by the wide platform support, once you go down the tier list (https://doc.rust-lang.org/nightly/rustc/platform-support.htm...). What "obscure platform" specifically are you thinking about, that is currently missing from those lists?
Just to be clear, I think this is a very small disadvantage. GCC and therefore C/C++ supports some old stuff like SuperH, Intel Itanium, PA-RISC and a bunch of microcontroller archs that LLVM does not.
However, there is now a Rust codegen plugin for GCC, so even this disadvantage is now basically moot.
LLVM doesn't have an equivalent annotated list, but it is missing alpha, bfin, c6x, fr30, frv, gcn, h8300, ia64 (aka Itanium), lm32, m32c, m32r, mcore, mep, microblaze, mmix, mn10300, moxie, nds32, nios2, pa (aka PA-RISC), pdp11, pru, rl78, rs6000, rx, sh (aka SuperH), storm16, v850, vax, and visium. For its part, LLVM does have some targets that GCC doesn't have (mostly related to GPU compilation).
The "big" targets that GCC has that LLVM lacks are Alpha, Itanium, PA-RISC, and SuperH, with Itanium being sufficiently weird that it's pretty firmly in the "fuck this" category from a maintainer's perspective, and people are actively ripping out support for it.
My very naive understanding is that a part of what makes mold fast is concurrency, which I'd expect to be a lot more error-prone in C/C++. Not having to worry about data races might give more confidence with trying out more complex techniques for how to split up work in a way that ends up making things faster
It's possible to write safe code in C or C++ but it's extremely difficult to read existing code and prove it's safe, without using as much effort as it takes to write it in the first place. This includes the code you wrote last month whose surrounding code has changed. And you have to be right every time while the attacker only needs you to be wrong once.
As someone who worked professionally with both C++ and Rust, I mostly agree with this, but I would say that writing the concurrent code in C++ is also hard.
The only way I found that works reliably is stick to a small well defined set of mostly safe primitives. E.g. at work we use message passing / event bus architecture everywhere, which works great for a robotics / industrial context. But even then, if you somehow mess up and have a variable accessed from event handlers in two different threads, it is tough to spot other than if you get lucky and observe it with a build using TSAN.
With rust that class of mistakes is just entirely eliminated, which makes it easier to to concentrate on the hard things that actually matter (like the domain specific logic).
Apparently the old mold version had a couple of plain C source files in the source tree (both vendored 3rd party libs but also in the 'regular' source code), so technically C/C++ is correct in this case ...and looks like the Rust version also has some (very minimal) C code left (looks like mostly varargs stuff, I guess Rust doesn't have a concept of varargs?), so it could be called a C/Rust project ;)
What warrants an off the cuff comment like this?
I can take any piece of software and just say a vague statement like that. All of software... What does it even bring to the conversation? Just vague "they"s again we keep on propping up? Who is unethical? The mold team? The AI? The rust foundation? The ASCII character set? The cloud system it has been compiled on? Oh no obviously the backdoors in there?! Oh let us guess!!
Was the conversion assisted? I get the sense that Mold's author is an extremely competent guy and it would be quite feasible for someone like him to do it on his own.
It's also his third(!) time writing a linker: lld, mold 1-2, now mold 3. mold 3 is also supposed to do what he never did before: implement all of the GNU ld features (notably linker scripts, I think it also has some optimization features not available elsewhere) so that it can finally be replaced, never needing it as a fallback anymore.
When I announced the rewrite, it was actually already mostly done.
AI's coding ability is truly amazing. We've spent decades inventing languages, tools, and methodologies to help us write better code more efficiently, but AI-assisted coding is the biggest breakthrough in programming productivity I've seen in my lifetime. It's genuinely incredible.
Last time I checked it didn't make any difference or whatsoever when compared to lld so I would not go that far by saying "mold is a reference in the world of linkers" - made a test just few days ago on 10M LoC C++. So, rewriting the codebase in another language just because doesn't seem like an investment worth doing when there's much bigger fish to fry.
LLMs are very good at taking your existing structure and converting it as-is to a new language. But rewriting by hand gives you the opportunity to rethink previous design decisions or optimize the code for the new language. So I was disappointed that this appears to just be a straight port and no additional performance improvements were gained.
The days when AI always means slop are gone. It still often does, but the latest models can be used to get very high quality code in many situations. I haven't checked the code here but I would be surprised if it's bad.
The linux distribution I co-maintain uses mold as our bootstrap linker to bootstrap rust itself, and it saved us -hours- on long version-by-version build chains. Mold being in c meant we could build it very early and use it as the default linker distro wide and enjoy build speedups everywhere.
Now sadly we will have to fork and maintain the c version as mold2 forever.
Rust is not actually the right tool for all problems.
If the need is well justified, maybe there is great chance to ask adding #[no_mangle] and extern C support? Since release is very fresh. If that is causing the problem. Is some dependency as the issue?
It is of extreme relevance for us as we have to re-bootstrap every time we change any dependency of rust, and rust depends on almost everything that is expensive to build in a toolchain tree.
I sure hope so. Go is the gold standard to copy here, which is not surprising given the involvement of Ken Thompson who authored Reflections on Trusting Trust.
Unfortunately Rust does not even do full source bootstrapped deterministic builds themselves. Their release strategy is entirely the honor system, where everyone trusts their pinky swear that a single computer or person will never be compromised and allow for the injection of malware into the binaries everyone downloads from rustup.
The only reason we have even the crazy long bootstrap path we have today at all is the hard work of mutabah, an individual independent hacker that does it as a hobby.
The rust team seemingly considers memory safety as the only security problem that matters, and supply chain attacks out of scope.
We use the llvm linker, lld, to link mold at the earliest point we can, then use mold exclusively after that. We are an LLVM native distro so we build llvm right away as the system compiler used to build the whole tree instead of gcc.
LLD would work fine for the whole tree, and did previously, but is much much slower than mold which is why we switched to mold.
Having to build all dependencies of rust including python and openssl and everything else before being able to use mold erases most of the full-tree build speedups.
We are a distro that mandates 100 percent deterministic builds from source for any given commit of the tree, so we have to build the whole tree several times for every release.
Can't you just keep using Mold2 for the initial toolchain bootstrap and then build mold3 with Rust as part of the "final" set of toolchains that are to be used to compile everything beyond that? It isn't perfect but it's congruent with how many other components in most open reproducible bootstrapping flows work; ie using GCC 4.4 or whatever just to bootstrap GCC 10. It's annoying but it only needs to be done once and then you just use it to bootstrap newer components forever. Doesn't help with compile times, though.
I would agree if not for the fact rust depends on almost the entire tree, so doing what you propose would increase our full tree verification time from 6ish hours to 12ish hours erasing any wins mold was giving us in the first place.
That is what we do, and it takes several hours on most machines. Having to do that once to get mold and a second time to then build with the benefits of mold defeats any advantages for us.
I may be misinterpreting the parents point, but you're not building E.g. tinycc for it to be a final artefact; why not continue to build mold2 in its current place in the build chain? It's not going anywhere, much like tinycc isn't.
We have to rebuild tinycc any time a dependency of tinycc changes. Likewise we would have to rebuild mold every time a dependency of mold changes, including rust and all dependencies of rust. It is a problem of almost full tree dependency recursion.
Yes. I'm saying that mold3 gets added to the end of the build tree, and mold2 remains where it is currently, which doesn't change how often you need to build those and let's you continue to use mold2 to speed up the builds (which I've now seen is what you'll likely do).
My point though was that you already build many end-of-life projects just so they can be used in the build, mold2 can now one of those rather than being built for its use in the final distro.
I indicated as much in my original comments. Pinning mold2 and backporting relevant fixes to it forever is our responsibility now that upstream has basically abandoned the linux-distro-bootstrapping use case. It is just unfortunate as that was supposed to be one of the goals.
>Rust is not actually the right tool for all problems.
"my problems (that are not mold's) are not solved by mold. How dare mold make those decisions?"
Maybe you should rewrite more of your linux distribution in rust so it's available earlier in the build process and get back to it being the default linker.
Our tree is entirely reproducible and hermetic. In fact we are the only distro that does this 100%.
The problem is changing any dependency of rust, even python or perl or musl or openssl or the llvm stack or any dependencies of the llvm stack have the potential of resulting in a different hash for rust.
Any time a dependency is changed all decedents must be rebuilt. Which we must do very frequently. That is where mold saved us a ton of time.
That's why you build a stage2 compiler at least with stage0 being the latest prebuilt.
Your stage1 will be based on whatever version is your stage0, but your stage2 should end up identical to any other build with any other stage0. So you don't have to rebuild the full chain all the time.
And then preferably you build a stage3 with FDO, you can easily get 20% more speed with a good corpus of tests.
Do you have at least a distributed cache with I guess sccache to save time for each rebuild?
If we added rust to our stage3, it would add 6 hours of wall time with the fastest 192 core CPU on the market to the time it takes people to reproduce and validate stage3. Add a couple days for someone with just a 16 core consumer CPU.
Our goal is to allow people to reproduce the entire tree from source in the shortest time possible, so no one has to trust us, thus encouraging as many people to do it as possible, thus preventing us from having the means to inject supply chain attacks without anyone noticing.
It is literally a goal for new users to be able to run a server that bootstraps itself, and then bootstraps and signs all future releases, with remote attestation proofs. The shorter that initial build window is the better, which is why things like fast linkers written in early-bootstrappable-languages are so important.
How long are all the rust builds in your distro? If the compiler is 20% faster (my LLVM Clang results), you "just" need 30 hours of building time to get even, and I'm sure there is enough rust around (maybe not all in your distro) to make this worthwhile.
In any case, people could be using the stage2 or stage3 binary and still get the same binaries in the end, they are just alternative versions of the same package that should be compatible.
And using a 192 core CPU is a bit futile, have you looked at how well the work can parallelize? Isn't there some good remote worker solution you could use to have more cores available for the whole fleet without locking a 192 cores machine for a single build? It doesn't look very efficient to me as it is.
Building from zero to rust is about 6 hours on my 192 core local workstation. The extra cores absolutely help. Each linux kernel takes 16 seconds. Rust still takes a few hours. And on a typical laptop, a day or more.
Building rust that far to build mold to then build rust again faster negates any benefits.
I have been thinking about Rust bootstrapping recently. Couldn't a Rust compiler without borrow checker be put together relatively easily?
Assuming the source code contains no issues (which can be checked later once the Rust compiler is built), one could leave that piece behind, and take the shortest path from .rs to executed code (C transpilation, or even an interpreter).
Ah, good to know, thank you. Though since we are talking about bootstrapping, it's a bit of a shame that the project is C++ though, this is also complex to bootstrap (at least it supports GCC 5.4 from 2016, which is a bit easier). Ideally it could target a C dialect, such as one supported my GNU MES. See also M2-Planet.
There are classes of virus that are hard to detect. One is a compiler virus that passes itself from compiler to compiler. You only get rid of the vector by bootstrapping from 0.
We only re-bootstrap the layers which had dependencies change under them. Early parts of the tree rarely change so we often do not have to rebuild these across releases, but very late tree things like rust depend on almost everything and something in the rust dependency graph changes almost every release.
Using binaries from past releases is a strict downgrade in terms of verification speed, as it means a new independent reproducible build verifier must now build both trees, doubling the release verification time, and erasing any wins mold3 could otherwise offer.
Google can rely on lots of centralized internal provenance tooling to prove cached binaries are not tampered with to other Googlers but when the goal is proving end to end full source bootstrapped build integrity to any interested user from the public in the least time possible, the requirements are significantly higher.
Sure but that's a compiler bootstrapping problem. It doesn't answer the question: why do you need to bootstrap the toolchain to build the distro? You can reuse a trusted toolchain that's been safely bootstrapped .
No user should trust them. If someone threatened me to put in a backdoor, I would. But it would not matter. Trust happens when many independent signed builds from reputable people all get the same result. That is why we make it easy and inexpensive for users to build the whole world from 0 from source at any given commit.
Luckily Mr Clanker can do most of the work for you at least (I myself have already had Claude/Codex rewrite a couple of Rust projects in C, worked great)
You could use https://github.com/thepowersgang/mrustc to get relatively modern Rust compiler (1.90 currently apparently, but every now and then that is updated). Then build newer rustc from there to reach the current 1.99.
But I don't get why some people are obsessed with bootstrapping. Yes it is good to be able to do it, but it isn't something you need to do regularly.
Especially since rust had a much better cross compilation story than C or C++ (not as good as go or zig though), so you don't need to bootstrap on a new architecture, just cross compile to it. Furthermore, new architectures for hosting a compiler (as opposed to just a target, like microcontrollers) is a rare event. Just something that happens every few years.
> But I don't get why some people are obsessed with bootstrapping.
It only matters if you have supply chain attacks in your threat model. Given they are up 400x since 2019, they should probably be in almost every threat model. Most distros operate on the honor system and that is not going to survive the post AI world.
That's why you have signed packages that everyone can rebuild and verify from the very early stages of bootstrapping to speed up the process and get back to a synchronization point that is reasonable.
Distros that are not fully hermetic and don't have reproducible packages (and there are many layers of reproducibility) will certainly have issues, but that's not a huge problem as long as you have a documented path to getting back to the current state. It doesn't need to be the fastest path, just a verifiable chain of trust.
We do sign our packages, but the threat model is to minimize the time it takes for users to not have to trust us.
It is critical to encourage many independent verifications that it be as fast as possible that someone can go from a clone of our tree of pure source code to the exact release hashes we publish.
Adding any binaries to that means someone that distrusts us must now go build those past releases as well, and if they rely on binaries, they must build those past past releases as well. This approach would make verification time go up dramatically every release.
You don't need to verify the chain of trust all the time, you can remember up to where you had trusted everything when you adopt your solution.
Also, while you can use the latest prebuilt as a stage0, you could also use any other prebuilt from earlier down the chain as a stage0 and still get the same stage2 binaries. This is the next trust checkpoint you have, and it should be fine to have multiple ways to get there too, one for quick iterative releases and one that is reusing the minimal amount of bootstrapped packages.
Or you just try to only upgrade your stage0 once in a while when extremely necessary and it would build your new stage2.
So many ways to optimize the system, it's a choice to refuse to reuse what was trusted yesterday in order to build the next stage.
>it's a choice to refuse to reuse what was trusted yesterday in order to build the next stage.
If the point is to let independent people verify the full build chain, then there is no "trust from yesterday" to reuse.
Even worse, if every release N depends on trusting release N-1, then you need to do a full rebuild of everything in the whole history, instead of just building the current release. This doesn't sound reasonable, or at least sounds incompatible with the project goals.
Bootstrapping rust with mold2 is going to be the likely play. My complaint is that now we must maintain mold2 forever now as mold3 rust edition is now incompatible with most of our tree which is made up of dependencies of rust.
Hopefully not much. I am just lamenting it is an extra dependency and chore we must carry and think about forever, and will have no official upstream support.
Like some other commenters, I do not understand why using a cached binary isn’t sufficient. Why not use the mold3 binary and build that first, and then cache it and never rebuild it again? I’m guessing it has to do with hermiticity and build provenance guarantees in your distro.
Do you have any reading material that you can share that might help me understand better? Docs for the distro, or an issue tracker I can search through?
In short, the entire distro is always built in one-shot at any given commit, and we can only rely on cached binaries from a past release if they or nothing in their supply chains changed. Given rustc depends on almost everything, mold3 would be built far too late to be useful for the most expensive build in the whole tree, which is rust.
Recursive dependencies would break our threat model, so we cannot use any rust tools to bootstrap rust. We bootstrap rust from llvm which we bootstrap from gcc which we bootstrap from tinycc which we bootstrap from M2Planet and so on back to 186 bytes of machine code.
Anyone getting a rust package from stagex must be able to build the entire tree up until that package and get the same hash, with no binary dependencies, thus removing any trust in maintainers.
If you can boostrap one version of stagex (so that its mold is trustworthy), I don't see why you can't use that to build another version of stagex.
As you say, you can rely on the cached binary because nothing in the old version's supply chain would change; it is pinned, and as long as you can build that, everything is fine.
The threat model is trust no single person or computer.
In order for you to not have to trust us, you must clone our repo, of only source code, and build from zero to our released binary hashes. The shorter we can make the time for that to be possible, the more people we can convince to do it and ideally sign and publish their matching hashes. The more people we convince to do it, the less risk of us as maintainers being able add a backdoor without anyone noticing.
Mold was a tool to shave hours off the time most people have to spend doing a from-scratch verification. Forcing them to build a whole tree to get to rust to et to mold3, to then use that to build the whole tree a second time, would directly work against the goal of minimizing full tree verification time.
> In order for you to not have to trust us, you must clone our repo, of only source code
> rust from llvm which we bootstrap from gcc which we bootstrap from tinycc which we bootstrap from M2Planet and so on back to 186 bytes of machine code.
While the effort is laudable, the fact that these are built from source is not enabling me to verify them. Rust, LLVM, gcc, tinycc, all the way back to that 186 seed are prohibitively large to verify.
I'm not even in a position to verify the delta to these projects over a single day. I'm having to trust _someone_, many someones.
An operating system distribution owns specifically the compiling and distribution links of the supply chain. It is our job to prove we produce artifacts accurately and faithfully from upstream source code without giving ourselves any ability to tamper with the sources or the resulting artifacts.
Our specific responsibility is to limit the number of people users have to trust to only the actual authors of the software they are installing.
Self inflicted because we want to protect as many people as possible from supply chain attacks. That is a very real engineering problem we have to solve that extremely high risk end users of our work depend on.
Most popular Linux distros take a position of hoping and praying supply chain attacks do not target them. I am not convinced this will go well for them in the post AI world, but hey, I also hope I am wrong.
There are plenty of other points in the design space here. For example, I think once a sufficient number of people have built a binary and verified that it is the same hash (publishing the hashes to an irrevocable transparency log), caching those binaries doesn't increase the supply chain surface. The point you have chosen is quite out there, and it isn't reasonable to expect others to conform to it, and it certainly isn't reasonable to say that Rust isn't the right choice for a project because it negatively impacts you.
> caching those binaries doesn't increase the supply chain surface.
Okay, but what if all the signers are people that a new user does not know? Who is to say all those signatures are not a bunch of made up AI identities?
This is why we make it trivial for anyone to clone our tree and build from zero and get the same result at any release with no required binaries of any kind anyone has to verify the provenance of. This is called full source bootstrapping. The cheaper we make that, the more people that will do it and the higher the chances users will see a signature from someone they personally trust.
I think you might be fundamentally misunderstanding distro maintainers' role in the ecosystem. Your job is not to question your upstreams' development practices or technologies used, beyond the basics like it being FOSS -- it is to adapt to them.
Upstreams naturally know a lot less about supply chain security than we do as that is our core expertise. We leave the coding to them, and handle supply chain integrity.
Often having to make lots of small patches to preserve upstream functionality while fixing their obvious security and determinism bugs. Or we have to ignore autogenned code and figure out bootstrapping ourselves. Many upstreams just put binaries in their source code and call it reproducible.
For instance, XZ published a malicious hand-packed archive of their code that many distros use because it has all the auto-generated code already. We totally ignored that archive and pulled the code that was actually reviewed, and ran autogen ourselves. In doing so we were never impacted by the XZ attack.
That seems rather one-sided to me. In a healthy FOSS ecosystem, shouldn't the upstream developers and distro maintainers be peers, with the upstream developers respecting the needs and values of the distro maintainers, rather than unilaterally insisting that the distro maintainers adapt?
I have sympathy for overburdened distro maintainers who feel that recent trends in upstream development practices are going in the wrong direction, as expressed in this blog post earlier this year: https://blogs.gentoo.org/mgorny/2026/03/07/money-isnt-going-...
Edit to add: And yet, I would be very reluctant to give up the safety of Rust to go back to C or C++. I don't know what the answer to that one is, but I don't think it's to tell the distro maintainers that their role (decided by whom?) is to adapt to what upstream developers are doing.
Distro maintainers ought _not_ tell upstreams whether they can or cannot use static linking, pin their dependencies, etc, and they certainly ought not call otherwise-successful software development methods "bad practices" just because they make Linux packagers' lives harder.
I wonder if a rust-to-c tool would help you, your mold2 would just be the cached result of that. Would that C code still be treated as a binary dependency since it would not be very human readable?
How about a rust-to-wasm process, you store the wasm binary, but have a human readable wasm interpreter that only implements enough for mold to work? Would that be trusty enough?
If a rust-to-c tool could fully handle this automatically, and said tool only depended on an early language like go or c, that would be very useful for many problems including this one.
Yep that is what we are going to have to do. Is what it is. Thanks for maintaining an amazing linker, and we will surely ship v3 at the last mile for end users.
If it was transpiled there would be what GNU calls a "preferred form" of the linker source in which it wasn't written in Rust.
This is true for - as an example - the WUFFS GIF decoder. You can get C which decodes GIFs and was transpiled from WUFFS, but that's awful code and nobody wants to modify that code, whereas the WUFFS source code for the decoder is fine.
When we look at rui314's changes to mold today after 3.0 release, they just modify the mold source code in Rust, as you'd expect if this was in fact now written in Rust.
> If it was transpiled there would be what GNU calls a "preferred form" of the linker source in which it wasn't written in Rust.
No, that's only sometimes true. If you like the transpiled version and start editing it directly then that's the new preferred form, but it was still a transpile to get from one language to the other.
Oh what? That was not on my bingo card for 2026. Mold already was incredible when it still was in C++ and I assume it is even better now that it is in rust
Pleasantly surprised to see a ~600 lines Cargo.lock (vs 2200 for wild, for example) here! Hope it can slim it down further.
Something I really look into because the crates situation (due to a too small stdlib à la R5RS Scheme) is Rust's Achilles' heel, since it directly affects its security claims.
Maybe the lead can use the time he didn’t use to translate mold into rust to write chibirustc for fast local rust compilation and we can all forget this was ever an issue.
136 comments
[ 1.7 ms ] story [ 24.1 ms ] threadAnd second, did you use any AI tools for the rewrite?
IMO, given the recent commits: almost certainly.
The best programmers on the planet have tried and failed with this task for 50 years now, so I don't think this is true.
The main disadvantage of Rust right now is not supporting some more obscure platforms, but because mold wouldn't support them anyway I don't see that as a problem.
Last time I checked, I got impressed by the wide platform support, once you go down the tier list (https://doc.rust-lang.org/nightly/rustc/platform-support.htm...). What "obscure platform" specifically are you thinking about, that is currently missing from those lists?
However, there is now a Rust codegen plugin for GCC, so even this disadvantage is now basically moot.
Rewrite all the things!
LLVM doesn't have an equivalent annotated list, but it is missing alpha, bfin, c6x, fr30, frv, gcn, h8300, ia64 (aka Itanium), lm32, m32c, m32r, mcore, mep, microblaze, mmix, mn10300, moxie, nds32, nios2, pa (aka PA-RISC), pdp11, pru, rl78, rs6000, rx, sh (aka SuperH), storm16, v850, vax, and visium. For its part, LLVM does have some targets that GCC doesn't have (mostly related to GPU compilation).
The "big" targets that GCC has that LLVM lacks are Alpha, Itanium, PA-RISC, and SuperH, with Itanium being sufficiently weird that it's pretty firmly in the "fuck this" category from a maintainer's perspective, and people are actively ripping out support for it.
The only way I found that works reliably is stick to a small well defined set of mostly safe primitives. E.g. at work we use message passing / event bus architecture everywhere, which works great for a robotics / industrial context. But even then, if you somehow mess up and have a variable accessed from event handlers in two different threads, it is tough to spot other than if you get lucky and observe it with a build using TSAN.
With rust that class of mistakes is just entirely eliminated, which makes it easier to to concentrate on the hard things that actually matter (like the domain specific logic).
It's like "in a Zodiac boat / aircraft carrier navy". This customary putting C and C++ into the same bucket is as amusing as it is unproductive.
https://github.com/rui314/mold/tree/main/c
Hope you're posting this from a GNU Hurd system or some other luddite-appoved zeroes and ones. You wouldn't want to be a hypocrite.
Edit: this seems to have been cooking for a while when the first commit dropped: https://github.com/rui314/mold/commit/f41bfcd5c72ca30cce6498...
Yes. Comment by mold's author:
https://www.reddit.com/r/rust/comments/1w45j6n/comment/p7ac5...
AI's coding ability is truly amazing. We've spent decades inventing languages, tools, and methodologies to help us write better code more efficiently, but AI-assisted coding is the biggest breakthrough in programming productivity I've seen in my lifetime. It's genuinely incredible.
(Wild is another fast linker, that only supports Linux)
Zig isn’t even in the same league as Rust regarding these things. Zig may still be around and active 10 years from now, Rust is guaranteed to be.
Now sadly we will have to fork and maintain the c version as mold2 forever.
Rust is not actually the right tool for all problems.
Would that meant that building rust is slower, but everything else is the same, and you don't have to maintain a fork of another complex project?
Rust is the single most expensive thing to bootstrap in any given linux distro.
It is the thing you need mold the most for to speed things up.
Unfortunately Rust does not even do full source bootstrapped deterministic builds themselves. Their release strategy is entirely the honor system, where everyone trusts their pinky swear that a single computer or person will never be compromised and allow for the injection of malware into the binaries everyone downloads from rustup.
The only reason we have even the crazy long bootstrap path we have today at all is the hard work of mutabah, an individual independent hacker that does it as a hobby.
The rust team seemingly considers memory safety as the only security problem that matters, and supply chain attacks out of scope.
LLD would work fine for the whole tree, and did previously, but is much much slower than mold which is why we switched to mold.
Having to build all dependencies of rust including python and openssl and everything else before being able to use mold erases most of the full-tree build speedups.
We are a distro that mandates 100 percent deterministic builds from source for any given commit of the tree, so we have to build the whole tree several times for every release.
My point though was that you already build many end-of-life projects just so they can be used in the build, mold2 can now one of those rather than being built for its use in the final distro.
"my problems (that are not mold's) are not solved by mold. How dare mold make those decisions?"
Maybe you should rewrite more of your linux distribution in rust so it's available earlier in the build process and get back to it being the default linker.
Then you can just use the latest built version to build mold and the next version of rust.
The problem is changing any dependency of rust, even python or perl or musl or openssl or the llvm stack or any dependencies of the llvm stack have the potential of resulting in a different hash for rust.
Any time a dependency is changed all decedents must be rebuilt. Which we must do very frequently. That is where mold saved us a ton of time.
Your stage1 will be based on whatever version is your stage0, but your stage2 should end up identical to any other build with any other stage0. So you don't have to rebuild the full chain all the time.
And then preferably you build a stage3 with FDO, you can easily get 20% more speed with a good corpus of tests.
Do you have at least a distributed cache with I guess sccache to save time for each rebuild?
Our goal is to allow people to reproduce the entire tree from source in the shortest time possible, so no one has to trust us, thus encouraging as many people to do it as possible, thus preventing us from having the means to inject supply chain attacks without anyone noticing.
It is literally a goal for new users to be able to run a server that bootstraps itself, and then bootstraps and signs all future releases, with remote attestation proofs. The shorter that initial build window is the better, which is why things like fast linkers written in early-bootstrappable-languages are so important.
In any case, people could be using the stage2 or stage3 binary and still get the same binaries in the end, they are just alternative versions of the same package that should be compatible.
And using a 192 core CPU is a bit futile, have you looked at how well the work can parallelize? Isn't there some good remote worker solution you could use to have more cores available for the whole fleet without locking a 192 cores machine for a single build? It doesn't look very efficient to me as it is.
Building rust that far to build mold to then build rust again faster negates any benefits.
Assuming the source code contains no issues (which can be checked later once the Rust compiler is built), one could leave that piece behind, and take the shortest path from .rs to executed code (C transpilation, or even an interpreter).
I know this is common but it seems like either an aesthetic decision, or glibc cruft.
Bootstrapping at every build does not save you from the threat you think it does.
Using binaries from past releases is a strict downgrade in terms of verification speed, as it means a new independent reproducible build verifier must now build both trees, doubling the release verification time, and erasing any wins mold3 could otherwise offer.
Google can rely on lots of centralized internal provenance tooling to prove cached binaries are not tampered with to other Googlers but when the goal is proving end to end full source bootstrapped build integrity to any interested user from the public in the least time possible, the requirements are significantly higher.
Is this not just verifying hashes? What further effort do they go to to prove a binary hasn't been modified?
But I don't get why some people are obsessed with bootstrapping. Yes it is good to be able to do it, but it isn't something you need to do regularly.
Especially since rust had a much better cross compilation story than C or C++ (not as good as go or zig though), so you don't need to bootstrap on a new architecture, just cross compile to it. Furthermore, new architectures for hosting a compiler (as opposed to just a target, like microcontrollers) is a rare event. Just something that happens every few years.
It only matters if you have supply chain attacks in your threat model. Given they are up 400x since 2019, they should probably be in almost every threat model. Most distros operate on the honor system and that is not going to survive the post AI world.
Distros that are not fully hermetic and don't have reproducible packages (and there are many layers of reproducibility) will certainly have issues, but that's not a huge problem as long as you have a documented path to getting back to the current state. It doesn't need to be the fastest path, just a verifiable chain of trust.
It is critical to encourage many independent verifications that it be as fast as possible that someone can go from a clone of our tree of pure source code to the exact release hashes we publish.
Adding any binaries to that means someone that distrusts us must now go build those past releases as well, and if they rely on binaries, they must build those past past releases as well. This approach would make verification time go up dramatically every release.
Also, while you can use the latest prebuilt as a stage0, you could also use any other prebuilt from earlier down the chain as a stage0 and still get the same stage2 binaries. This is the next trust checkpoint you have, and it should be fine to have multiple ways to get there too, one for quick iterative releases and one that is reusing the minimal amount of bootstrapped packages.
Or you just try to only upgrade your stage0 once in a while when extremely necessary and it would build your new stage2.
So many ways to optimize the system, it's a choice to refuse to reuse what was trusted yesterday in order to build the next stage.
If the point is to let independent people verify the full build chain, then there is no "trust from yesterday" to reuse.
Even worse, if every release N depends on trusting release N-1, then you need to do a full rebuild of everything in the whole history, instead of just building the current release. This doesn't sound reasonable, or at least sounds incompatible with the project goals.
Do you have any reading material that you can share that might help me understand better? Docs for the distro, or an issue tracker I can search through?
In short, the entire distro is always built in one-shot at any given commit, and we can only rely on cached binaries from a past release if they or nothing in their supply chains changed. Given rustc depends on almost everything, mold3 would be built far too late to be useful for the most expensive build in the whole tree, which is rust.
Recursive dependencies would break our threat model, so we cannot use any rust tools to bootstrap rust. We bootstrap rust from llvm which we bootstrap from gcc which we bootstrap from tinycc which we bootstrap from M2Planet and so on back to 186 bytes of machine code.
Anyone getting a rust package from stagex must be able to build the entire tree up until that package and get the same hash, with no binary dependencies, thus removing any trust in maintainers.
If you can boostrap one version of stagex (so that its mold is trustworthy), I don't see why you can't use that to build another version of stagex.
As you say, you can rely on the cached binary because nothing in the old version's supply chain would change; it is pinned, and as long as you can build that, everything is fine.
In order for you to not have to trust us, you must clone our repo, of only source code, and build from zero to our released binary hashes. The shorter we can make the time for that to be possible, the more people we can convince to do it and ideally sign and publish their matching hashes. The more people we convince to do it, the less risk of us as maintainers being able add a backdoor without anyone noticing.
Mold was a tool to shave hours off the time most people have to spend doing a from-scratch verification. Forcing them to build a whole tree to get to rust to et to mold3, to then use that to build the whole tree a second time, would directly work against the goal of minimizing full tree verification time.
> rust from llvm which we bootstrap from gcc which we bootstrap from tinycc which we bootstrap from M2Planet and so on back to 186 bytes of machine code.
While the effort is laudable, the fact that these are built from source is not enabling me to verify them. Rust, LLVM, gcc, tinycc, all the way back to that 186 seed are prohibitively large to verify.
I'm not even in a position to verify the delta to these projects over a single day. I'm having to trust _someone_, many someones.
Our specific responsibility is to limit the number of people users have to trust to only the actual authors of the software they are installing.
> Rust is not actually the right tool for all problems.
Rust is certainly the right tool for this problem, your own decisions notwithstanding.
Most popular Linux distros take a position of hoping and praying supply chain attacks do not target them. I am not convinced this will go well for them in the post AI world, but hey, I also hope I am wrong.
Okay, but what if all the signers are people that a new user does not know? Who is to say all those signatures are not a bunch of made up AI identities?
This is why we make it trivial for anyone to clone our tree and build from zero and get the same result at any release with no required binaries of any kind anyone has to verify the provenance of. This is called full source bootstrapping. The cheaper we make that, the more people that will do it and the higher the chances users will see a signature from someone they personally trust.
I’ve been told that no one thinks one programming language makes everything safe but the tone of the comments suggest many do actually believe that.
Often having to make lots of small patches to preserve upstream functionality while fixing their obvious security and determinism bugs. Or we have to ignore autogenned code and figure out bootstrapping ourselves. Many upstreams just put binaries in their source code and call it reproducible.
For instance, XZ published a malicious hand-packed archive of their code that many distros use because it has all the auto-generated code already. We totally ignored that archive and pulled the code that was actually reviewed, and ran autogen ourselves. In doing so we were never impacted by the XZ attack.
I have sympathy for overburdened distro maintainers who feel that recent trends in upstream development practices are going in the wrong direction, as expressed in this blog post earlier this year: https://blogs.gentoo.org/mgorny/2026/03/07/money-isnt-going-...
Edit to add: And yet, I would be very reluctant to give up the safety of Rust to go back to C or C++. I don't know what the answer to that one is, but I don't think it's to tell the distro maintainers that their role (decided by whom?) is to adapt to what upstream developers are doing.
Distro maintainers ought _not_ tell upstreams whether they can or cannot use static linking, pin their dependencies, etc, and they certainly ought not call otherwise-successful software development methods "bad practices" just because they make Linux packagers' lives harder.
How about a rust-to-wasm process, you store the wasm binary, but have a human readable wasm interpreter that only implements enough for mold to work? Would that be trusty enough?
This is true for - as an example - the WUFFS GIF decoder. You can get C which decodes GIFs and was transpiled from WUFFS, but that's awful code and nobody wants to modify that code, whereas the WUFFS source code for the decoder is fine.
When we look at rui314's changes to mold today after 3.0 release, they just modify the mold source code in Rust, as you'd expect if this was in fact now written in Rust.
No, that's only sometimes true. If you like the transpiled version and start editing it directly then that's the new preferred form, but it was still a transpile to get from one language to the other.
Something I really look into because the crates situation (due to a too small stdlib à la R5RS Scheme) is Rust's Achilles' heel, since it directly affects its security claims.