> Why do none of these hijacks embed runtime attacks? It seems like worming the build machines is the goal, rather than compromising downstream users.
Developer machines are quite juicy targets. They tend to have all sorts of credentials lying around, so it often isn't too difficult to escalate from that to compromising AWS/GCP/etc.
On top of that there's very little preventing a compromise. Developers are inherently expected to run untrusted code, and the usual Linux / MacOS laptop probably isn't even running any kind of anti-virus protection. Want to compromise the downstream app? Now you also need to pass Play Protect & friends.
> It seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
Yes, we really should. It is frankly a miracle that it has taken so long for fetch-time / install-time code execution to start blowing up in our faces. If we are spending so much effort on run-time isolation, why are we completely ignoring all those practices during development?
Why are developers "inherently expected to run untrusted code"? Running untrusted code on developer machines seems like a terrible idea, given that it will compromise everything they build or deploy and (as you mentioned) all their credentials.
because cloning repos and installing dependencies is what developers do daily. 90% of more don't care or even know that doing so they are risking getting hacked.
Why this still happens? Why after many previous supply-chain attacks maintainers of package repositories still allow anyone uploading packages and pushing updates without security audit?
Who is funding this security audit? Are folks supposed to volunteer their free time? It's a difficult coordination problem. The best folks have come up is to delay adopting new releases by a few days and hope your dependency is popular enough that a security firm audits it for you in that timespan. If you have enough money I suppose you can start employing llms to audit things for yourself.
Well, it depends on the language: language which "capabilities" (both for the source code and for the building* ) could in theory really reduce a lot of the burden to identify supply chain attacks.
*: some research language have/had capabilities which would make supply chain attack "obvious" but for build systems I don't know if this exist.
Languages like Rust have sources of income to be able to finance such audit.
You don't need to audit all the crap is being uploaded right now. Only really necessary and widely-used packages should be managed in a centralized way, so, auditing all of them isn't that huge task.
Mozilla, Google and a couple of others are publishing their audits through cargo vet. There are also additional audits done by individuals you can use through cargo crev. Overall the number of audited crates is in the thousands and you will find audits for most of the popular crates.
In the end it is your decision to use unaudited or refuse unaudited crates.
> In the end it is your decision to use unaudited or refuse unaudited crates.
It should be the default behavior of the package manager to allow downloading only audited/trusted packages. Forcing end-users of the language to be responsible for audit of all dependencies is impractical.
Cargo (and PyPI) is undeniably better than NPM, which is just shockingly bad for cultural reasons. Yet it's not safe, and it's subject to the same class of exploit, as we're seeing.
Indeed, the solution is to get away from the wild soup of author-managed dependencies and go with something with an audited collection of software that is maintained by separate human beings from the known-vulnerable hackers writing the software.
And indeed Go and .NET and Java all qualify. But the gold standard here is Debian and all its downstreams.
There is a reason why software foundations like the Apache Software Foundation exist and this is one of the bigger ones.
Turns out, it just doesn't make sense to be an independent open source developer of a critical dependency anymore. You can write the software yourself, but you can't publish it yourself.
All the Rust library crate developers will have to get together and start their own software foundation.
I personally don't believe the standard library argument is very convincing, because even with Java the latest newly added HTTP client has some blatant problems that require you to go with a wrapper like Methanol.
Is Cargo really that much better, if at all? This is a genuine question.
You can't disable build.rs, Rust Analyzer executes proc macros as soon as you open project, minimum days since release has not yet made it into stable (though luckily it is planned for 1.100), and the number of project dependencies goes easily into several hundreds (still better than npm's few thousands, though, and I know that some deps are from Cargo workspaces), and on top of that, lots of packages stick to 0.x version for years.
Personally I see Rust community's packaging culture more akin to that of JS than to Java et al.
And better than npm are you fucking serious? The most widely used ecosystem on earth by far that actually gets used in prod vs your “theories” and ego??? LOL
I would like to know too, but the author's Github account seems to have been deleted, and the crates.io releases have been completely deleted too (not just yanked) so it seems impossible to view the file :/
Summary: it's cultural. Rust likely inherited the practice from Nodejs, who inherited it from Ruby. I think in Rust online spaces in particular there is also this undercurrent of "you're not smart enough to use certain parts of the language, so download libraries that handle that stuff for you."
i'd like to welcome you to the hell that is c/c++ dependency management. Make? cmake? qmake? conf? autoconf? configure? autotools? submodules??? AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
None of those (other than submodules which is just vendoring) does dependency management. They are more configuration management than anything. C, Python, Perl (and maybe ruby?) relies on flag switches and environment variables to find all the necessary files and modules for compiling/running a script. Cmake and autoconf just configure those.
With NPM and Rust's focus on project's level dependencies, there's no longer emphasis on API stability. Instead we have breakage every months, forcing everyone on the upgrade treadmill. It's easier to audit C library because they focus mostly on security updates instead of redesigning the API for the nth time.
All of those can (and some do) wget source code from the internet and run it if you kick off their build pipeline. People just tend to use these tools differently.
Nothing is preventing anyone from distributing Rust crates as standard system packages, like some distros do with Python.
I think Debian is actually shipping rust libs in their repositories: https://wiki.debian.org/Rust
Now Rust developers can also enjoy having to deal with popular packages that are years out of date/missing from popular repositories/getting deleted.
The language also takes backwards compatibility very seriously, anything that goes into std "must be maintained forever". It is also argued that a large stdlib means the maintainers have less time to work on the language itself.
I understand not every language can have Go's amazing stdlib, but I would much prefer Pyhton's approach where every now and then some package/function from the stdlib gets deprecated/removed. Rust's 3rd party ecosystem is the worst thing from the language, worse than the compile times.
The "npm-ness" of Cargo (centralized and standardized dependency management used at every opportunity) is generally pitched as one of the primary developer experience advantages over C++.
The thing is, just having a centralized standardized dependency manager doesn't in itself force your ecosystem into this dangerous "micro dependency" pattern. Java'd Maven is also a centralized standardized dependency manager, as is Python's PyPI. However, neither the Java nor the Python ecosystem have devolved into having dependency libraries with 4 methods in them becoming used and required by major libraries.
There are two reasons you might want to use a crate, The first being that you want to use a good solid implementation that you know someone has spent more time doing and works better than almost any solution you could integrate. The second is you don't want to spend time implementing that.
Writing macros in rust is a pretty horrible experience but it's not difficult
The horrible practice is not writing libraries, it's having individual small packages like this crate that are handled by individual small authors, and ending up with huge webs of small dependencies.
The better practice is for bigger projects to form collecting such small utilities into larger utility libraries that have some organization, security practices, code reviews, etc. Good examples are Java's Apache Commons or C++'s Boost.
I mean, yes, update your dependencies. But probably after a week or so after they've been release and tested by the first penguins willing to jump into the ocean.
Holding periods for new updates are a good idea, but I would also prefer having a tool that could provide diffs of the entire project state pre- and post-update; including transitive dependencies being added or removed. Git diffs won't show you that information, by design.
Why not? Cargo.lock includes the transitive dependencies, and Cargo itself reports everything that changes when you do an upgrade. Those are version changes, not code diffs, but it seems entirely tractable.
There's a reasonable solution to this. Tools like `cargo-audit` can tell you if a crate has security issues and guide you towards an update. You don't have to blindly update all dependencies this way.
I’m disappointed crates.io doesn’t have a stricter bar for serving a crate that has newly acquired a proc macro or build.rs. That seems like a trivial mitigation.
So? Runtime code requires actually executing the malicious code path which isn’t an immediate 100% hit rate for everyone that includes it in the dependency chain. For build.rs it’s a 100% compromise of everyone it’s in the dependency chain for. Additionally, at runtime you may not have access to secrets whereas at build time you most certainly do.
Doing software development outside of strict containerization, at the very least, looks increasingly prone to disaster.
Yes, we can argue about the culture of package management (as some of us have with especially npm from day one), but it's done, and your colleagues or AI sidekicks cannot be trusted not to download whatever and try to build and run it. All you can do is limit the effective blast radius.
> rust-rfcbot commented 2 days ago: This is now entering its final comment period
I wonder if this is not in part the same situation as we arguably had with the xz compromise: some imminent change would have made the attack harder (in xz's case, IIRC a change to systemd to dlopen the compression library instead of directly depending on it), and the attacker rushed before the opportunity window closed.
1. There isn’t a single universal standard for sandboxing across all the different platforms that are supported by Rust.
2. Even if there were, if you’re compiling untrusted code then why would you trust the built output?
If you’re building the create then I’d argue that any preventative steps afterwards is akin to closing the barn door after the horse has already bolted.
> Yeah, let's hold up the entire world of offensive cybersecurity capability while you work on that. Sure they will wait.
Nice sarcasm but you’re not actually addressing a solution to the problem I raised.
> You don't. You sandbox the hell out of it too.
So you’re now saying it’s ok to have exploits compiled into your application as long as it’s sandboxed?
I wonder how customers of your application feel about that? I’m certainly not going to be entering my bank details into your ecommerce platform (to give just one obvious example why your suggestion wouldn’t work).
This response seems disingenuous. It was never implied it was ok to have exploits in the compiled software, only that the blast radius is smaller if the compiled code is also sandboxed.
Smaller maybe. But it’s still an unacceptable risk. I refer you back to my example of compiled code being used by customers.
Arguing that compromised code is safer when sandboxed really misses the real problem: that you’re running compromised code in the first place. Hence my analogy of closing the barn door after the horse has already bolted.
What we need is to ensure we have stronger safeguards in place to prevent bad code from reaching build pipelines. If it’s in the build pipelines then we’ve already lost.
I get why people argue about sandboxing, it’s an easier problem to solve. But you still end up with a compromised artefact which you cannot ship. So the benefit is negligible.
> Nice sarcasm but you’re not actually addressing a solution to the problem I raised.
Are you not concerned about an asteroid impact rendering you extinct during your next Rust build? Why not? That's the same level of relevance as your supposed concern.
> So you’re now saying it’s ok to have exploits compiled into your application as long as it’s sandboxed?
You need to act like it is compromised in all cases, just like everything else.
> I wonder how customers of your application feel about that? I’m certainly not going to be entering my bank details into your ecommerce platform (to give just one obvious example why your suggestion wouldn’t work).
So your version is "we trust everyone and don't verify anything". Yeah, that's going to work. By your "logic" firewalls wouldn't be necessary.
> Are you not concerned about an asteroid impact rendering you extinct during your next Rust build? Why not? That's the same level of relevance as your supposed concern.
That’s a strawman argument and you’re still dodging the question.
> You need to act like it is compromised in all cases, just like everything else.
No. I act like compromised code is a legitimate risk regardless of how well your build pipeline is sandboxed.
I don’t understand why this is a hard concept for you to grasp.
> So your version is "we trust everyone and don't verify anything". Yeah, that's going to work. By your "logic" firewalls wouldn't be necessary.
That’s the literal opposite of my point (as well as another strawman).
That’s already easy to do. In fact most CI/CD pipelines already work this way. The problem is you still have a compromised executable at the end of the pipeline. So you haven’t actually solved much.
These very small dependencies that are then later causing issues either due to malicious nature or incompetence, have become pervasive in computing (for some reason). I think that these should be less of an issue now than ever. Outside of the largest, most critical dependencies, you really shouldn't be pulling in small libraries anymore. Just generate the code via AI. AI is not great at large scale programming I think, but its amazing at snippets of code. Something I ran into recently, I needed to use FFT2 on some matrices, and what I was using didn't have an existing solution. Converting some numpy fft2 tests to my target language, and having a full native implementation of fft2, and an accompanying test suite so it will behave exactly like numpy fft2. A few minutes and a few thousand lines of code later, I have a trusted implementation. Saves me an external dependency, some weird glue code, and an attack vector.
Sandboxing for build scripts can't work properly. If you sandbox too much, some necessary stuff can't be done. If you sandbox too little, it has no practical value.
So let each build script define its own level of sandboxing and then users can determine whether they are okay with that level or not, e.g. `cargo build --sandbox-level=...`
Rust, like C, C++, and every other systems programming language, is all about giving users the power to make mistakes. The philosophical difference when it comes to Rust is simply that it tries to force the user to flip off the safety on the gun before letting you shoot yourself in the foot. A Cargo config option letting people opt-out of sandboxing would be fully in line with Rust's philosophy.
Having a dangerous flag as a backwards compatibility flag is okay. I don’t think making users decide between multiple levels of sandboxing is constructive, they will just be trained to ignore it.
This is the kind of decision users likely don’t understand without looking at the source code of a crate and it’s bad UX to push it to be their responsibility.
No, the flag would not exist simply for backwards compatibility, it would exist because build scripts are occasionally necessary and there are plenty of legitimate uses for them, even if they should be opt-in.
As an easy start, how about letting build scripts read /usr, read and write a temporary build directory, have some /tmp scratch space, and be allowed to write its final output artifact. No network and otherwise isolated from the rest of the system.
I would argue that, if a build script doesn’t work in the setting, then it doesn’t deserve to be installable by a default cargo command.
Cargo is a cross-platform tool, so when it ships a sandboxing solution it will need to be a cross-platform solution, and because this is a security feature it needs to be bulletproof, so no half-measures like Docker. Something like a WASM runtime might fit the bill, though that will be much easier to get working for typical proc macros than for typical build scripts. If you only care about Unix, then you can do this yourself today by building code in your sandbox of choice.
> Cargo is a cross-platform tool, so when it ships a sandboxing solution it will need to be a cross-platform solution
This seems like an excuse, not an actual objection.
Linux can do seccomp or Landlock or gVisor or a combination. Seccomp and gVisor need no privileges. Windows has its internal weird mechanisms. Mac has sandbox-exec.
Cargo could easily pick an appropriate sandbox for each major platform and ship it by default.
> If you only care about Unix, then you can do this yourself today by building code in your sandbox of choice.
This is ridiculous. The sandbox should not have network access, but cargo needs network access to download the package in the first place.
There's no good reason a proc macro can't run in a no-IO sandbox by default. None. Doesn't require a language change. Doesn't require some microvmcapabilityeffect BS. It requires looking people straight in the eye and saying "no" when they complain about needing to prompt for privileges.
WASM is totally unnecessary. Vanilla seccomp is sufficient and runs at full performance. What is it with people trying to stick WASM in places it's not needed?
I'd argue that this kind of thing is easier to implement securely and cross-platform with wasm (of course, a performant wasm engine is its own source of complexity).
wasm has the additional benefits of allowing something like pre-compiling binaries once on a central server and thus skipping the CPU cycles for compiling all the macro crate dependencies on every crate compile. So even the people who don't care much about the security benefits have a reason. I'm kind of the "don't care about security" side because (as said elsewhere) it's trivial (https://docs.rs/ctor/latest/ctor/) for any macro to insert code that will run when compiled binaries/tests are run. You need the entire thing in a sandbox anyway, not just the macros, and that's something that can't be provided by cargo by default.
Sandboxing just build.rs would be
be a minor inconvenience for the attacker but a big pain for many that are in the unfortunate position to really need build scripts.
I imagine it would be sandboxed by default with an escape hatch to run build scripts outside of the sandbox with user verification. It makes people stop and think about what’s happening. Not perfect, but it does help. When working on JS ecosystem projects I manually approve build scripts and spend some time researching dependencies with build scripts to see if I can avoid running the build script. Some people will ignore it and run everything, but it’s a huge step in the right direction to make it operator-decided.
At the very least, it wouldn't be overly onerous when adding a dependency that requires a build script to require an opt-in via Cargo.toml, e.g. `build-script = true`. You'd make it viral so that any transitive dependency that requires a build script would affect its parent, then add the key as defaulting to `true` so as to not break backwards-compatibility, then switch the default to be more restrictive over a new edition. (This same key could be used to prevent proc-macros from having arbitrary system access as well, where by default proc macros could be compiled to WASM and run in a WASM sandbox and treated as simple tokens-in/tokens-out pipelines.)
100% agree. Wow I can’t believe how many think sandbox builds is an actually good idea, on a per language basis too. There are completely standard QoL issues in cargo that have been open for years and nobody is working on. Maintaining a sandbox for idk 3 operating systems minimum that have virtually no sandboxing support? For extremely diverse workloads that typically invoke commands? I mean.. good luck.
There are still elephants (or rather mammoths) in the room for supply chain security, such as having 1500 nested deps for a standard project. It’s like we never woke up from the nightmare of leftpad.
If your project had code from 100s of individuals, new versions can be pushed instantly, and nobody wants to review the code, then you have a time bomb. And also other problems.
cargo-deny can audit build scripts, but unfortunately not prevent execution of malicious build scripts exactly for the reason you gave. It could still help if you only ever use cargo add and update in a sandbox.
You audit the code, then you run cargo update and you are pwned. Asking the user to not make mistakes is the c++ approach to security - it doesn’t work.
Safe-ish recommendation: Only add well-known, trusted crates. Failing that, treat unknown crates as malware or containing malware dependencies until proven otherwise. Test untested tools/crates in a VM/dev container to be sure they work properly before trust them. Use Mark I eyeball and Mark III brain too. :)
Audit and limit crates (cargo-deny &| cargo-crev, && --offline) until tools exist to audit build.rs safety semi-automatically ($$ safeguard.sh maybe).
The root problem is two parts:
1. crates.io doesn't do mandatory curation. Lack of curation is fail. It's time-consuming and costly for reviewers without a doubt, but so is letting an ecosystem gain maximum entropy (go to shit) by Tragedy of the Commons depending entirely on the honor system. Name squatting, low-quality, unmaintained, typosquatting, and malware are the consequences of too much self-service / semi-self-service freedom.
2. Many, many cargo subcommands are over-eager to run build.rs because it assumes trusted crates:
In an untrusted/uncurated crates world and a build.rs exists or exists in a selected dependency, it shouldn't run at cargo-add time (unless it must). If it exists, on first run, it should be presented to the user in a viewer for manual review unless a magic CLI flag/env var is specified to accept it.
These 2 factors combined appear to create a Swiss cheese holes failure mode for running arbitrary crate `cargo add`. I have confidence a suitable add-in workflow &| standard command &| repository workflow will be adjusted to reduce the attack surface of the ecosystem.
build.rs by design can run absolutely anything. There tons of build.rs scripts that invoke a whole-ass C compiler toolchain to build and link C dependencies...
It isn't so much a question of sandboxing build.rs, as fundamentally changing the way that foreign dependencies are integrated into the rust toolchain (i.e. moving from a rust-centric system like Cargo to something more general like buck2)
The vast and overwhelming majority of build scripts are building C code, so the other solution is to move away from integrating with C dependencies to native Rust dependencies.
One of rust's strengths is it's ability to interface relatively easily with existing c code without having to rewrite absolutely everything in rust. I don't think that is something we want to give up.
build.rs changes nothing about how easy it is to integrate with C. What does simplify: figuring out how to supply library you need at build time.
Which is the result of how bad dependency managment is outside (i.e. DLL-hell).
Pretty much all other use cases of build.rs can be sandboxed. Well, there is sqlx that wants to connect to database at expansion time to compile check-queries (yew).
sqlx at least has the (optional) offline mode, where you "cargo sqlx prepare" once (which wants access to a db) and then you can build in offline mode which typechecks your queries against local files.
Although I suppose that's still doing a lot of shenanigans at compile time. It could be sandboxed pretty well (theoretically). I'd hate to give it up completely though, getting a compile time error when SELECT query params or return values have type mismatches is extremely nice.
sqlx offline mode being opt-in instead of default is what bothers me. You know what else can validate that your queries return what you expect? Integration tests. Shoutout to sqlx for #[sqlx::test] though.
I also wish that it would be the default. Integration tests are fine, but they aren't compile time. Elevating them to compile time and using an LSP enabled editor makes it just underline SQL errors before I've even had a chance to run a manual compile let alone a test...
LSP diagnostics is actually what made me switch from compile time check queries. Whole "is db up? are migrations applied?" dance tired me pretty quickly. Its fine if you use sqlite, but anything else gets annoying.
Just denying write access outside the build directory and denying network access would go a long way and won't break pretty much any well-behaved build systems.
> Just denying write access outside the build directory and denying network access
As the link you posted mentions, you need a tiny bit more than that: you also need write access to the temporary directory (/tmp and similar). Many build tools temporarily store files there; for instance, unless things have changed since I last looked, if you don't use the -pipe argument the C compiler stores its temporary intermediate files (preprocessor output, assembler input) there.
Worth noting that the JVM ecosystem doesn't have this issue because there is no notion of installing dependencies, and the package managers are just downloaders with nothing else. No build.rs equivalent.
To solve the problem of native/C dependencies, they just bundle pre-compiled libraries as data files.
It may or may not be a problem depending on whether it's actually used, which is still better than "gets you instantly the moment you compile the app". A lot of codepaths and even whole libraries aren't invoked just because a program depends on them.
It doesn't really matter anyway, because nobody is reading anything in their dependencies before it gets downloaded. Malware can also be hiding in plain sight in source code, as this attack and many others shows.
ABI stability is significantly different in a VM language to a native language, and reified generics necessarily require source compilation. Binary-only development for Rust would be exactly as 'sort of not really' as C++, for exactly the same reasons. (There is also no notion of 'installing' dependencies in Rust, and build-time code being malicious isn't much worse than runtime code being malicious.)
It's used to do all sorts of things. Rust version probing, bindings generation, precompiling bundled data, etc. The only reason it isn't used to do more is because proc macros do it instead, with an essentially identical execution model and sandboxing situation (ie build scripts in isolation are a red herring).
I mean sure, but anything the build script could do, the build artifact could also do, That is to say, if you don't trust your source why do you trust the thing it compiles into?
This is exactly what PMG is designed for ie. install/build time process level sandboxing. It currently doesn't support cargo, but I believe the challenges are same.
Here is my learning building PMG:
Sandboxing is good when the workload is predictable, and the goal of sandbox is to guard against exploitation of vulnerabilities, like sandbox protecting chrome tabs (renderers). But unfortunately build scripts are not predictable, at least not in npm/pypi world and I have seen build scripts doing weirdest of the things which is no different from malware. When popular packages do weird things, build breaks and users end up turning off the sandbox. This is a perpetual problem to deal with while building sandbox (or any least privilege solution) to protect unbounded workloads.
I think it would be a good start if crates at least had to opt in to a build script, and adding one later would require permission from crates that depends on it.
The vast majority of crates don't need build scripts, so it is vaguely feasible to audit the list of crates you use that might need them.
Sandboxing is a mitigation but a very limited one. The library itself can contain a malicious payload -- rust code ends up as native code executed on the host.
I think we may need to enter a world where there are fewer, heavily audited, libraries and dependency depth is limited. Allowing unvetted dependencies to be installed by default is not a good way forward. App stores have a similar problem and I think a lot of lessons have been learned there that can be applied.
Yes, or the compile time equivalents. Rust’s safety gets you most of the way there. We just need a capability model in the language, a more limited std and a way to ban untrusted 3rd party libraries from using unsafe code without explicit permission. I dream of a world where a function with the signature of add(u32, u32) -> u32 can’t burn my house down and steal my wife.
Functions should only have access to their arguments. Nothing more. We need to end ambient authority.
Are you sure about that? My understanding is that Haskell programs generally run much more slowly than their C counterparts because of all of Haskell’s magic. Like lazy evaluation and memoisation and however Haskell manages memory.
Rust certainly borrows from Haskell. Like all good languages. But I’m not aware of Haskell beating rust in program performance & memory usage benchmarks.
Yes, I'm sure about that. I said that it is a systems language, not that it is as fast as C! I don't mind if your terminology excludes Haskell from being a systems language, as long as it also excludes Go. They are both managed, garbage collected, fast languages.
> Like lazy evaluation and memoisation
Yes, that causes performance impact. If you don't want the performance impact then don't write code that uses those behaviors. Sure, that rules out large parts of the ecosystem, but I said Haskell was a systems language not that its ecosystem was generally suitable for systems programming.
> however Haskell manages memory
No, Haskell's memory manager is world class, with two (at least) tunable garbage collectors.
> But I’m not aware of Haskell beating rust in program performance & memory usage benchmarks
Nor am I!
> If Haskell were a viable systems language, I suspect the kernel would not have been converted to C.
I suspect they converted it to C because you can't write a kernel in a managed language with a garbage collector.
That world can’t exist unfortunately, because the minute someone wants a feature that isn’t in your core libraries, you just create a new library and we’re back to now.
Are there any plans to more seriously develop the standard library in Rust? Or is the plan to remain in this status quo where users of Rust import nonsense and the dependency tree explodes (or users are forced to invent their own wheel?).
Are there any comparisons between the state of the stdlib in C++ vs. Rust? I’d think that would serve as an excellent jumping off point to start chipping away.
The standard library is not versioned, so any API*/behaviour there must be maintained forever. When you put it in a separate crate, you can make better interfaces or be displaced by a better crate, while old code still compiles against the version it was written for. So even code effectively maintained within rust teams (hashbrown, rand, regex) may be in separate crates, std is primarily for OS abstraction and a shared type vocabulary.
That doesn't mean you have to import nonsense or thousands of dependencies. The top hundred are efficient, concise, well-maintained crates of the quality you'd get in a large std like Go or Swift, sometimes even higher as people can write cleaner/faster implementations that they otherwise wouldn't (or wouldn't be used over std). And those languages make breaking changes! A mostly stable language like C++ is full of junk, like <regex> or even unordered_map – c.f. Rust wholesale reimplemented HashMap 6–9 months after SwissTable released. Partly because it's easier to express the encapsulation you need, but there are hundreds of other design/interface decisions made deliberately to not constrain forwards compatibility, including a smaller std.
However, that's no excuse to have a worse developer experience in this area. We need better tools to vet and communicate the quality of a crate and its supply-chain, like community-curated or even additional org-maintained crates, and maybe a handful delivered precompiled in the default rustup distribution (but which could change over time). I don't think they need to be added to std itself though
Oh, so it's not only "JavaScript bad and npm bad". Apparently, if your language uses third party dependency registry, you are prone to malicious code, regardless if it's Javascript or not.
Use containers for development. And reduce the amount of third party deps you import into your projects. This is only going to get worse.
Pulling in lots of dependencies creates this kind of risk regardless of the ecosystem. That being said in the JS/NPM world you tend to have a LOT more dependencies (especially indirect ones) than other languages. I saw someone do a cursory analysis and JS/Node projects tend to have 5x the number rust or ruby projects.
This is really a cultural problem not a technical one.
We need effect based languages now. It's the only way to guarantee policies like no network, no file access, no unsafe code or FFI for a library before it even compiles. If anyone from epic is reading this please give us a timeline for open sorucing the Verse compiler.
In the mean time I think it's possible to hack Cargo and run all build scripts in a microVM. The blast radius will be limited to malicious code in the binary instead of uploading all your CI secrets and deleting the whole hard drive.
I think the only language that's used in production where malicious dependencies can't do arbitrary effects is https://roc-lang.org - but it's a pre-0.1.0 language and as such its production usage is extremely minimal for now.
> It's the only way to guarantee policies like no network, no file access, no unsafe code or FFI for a library before it even compiles.
It's not the only way. You can sandbox processes. I think sandboxing is the much more reasonable approach, because in the end of the day, there are still closed-source software products where you can't demand that the manufacturers use certain language safety features.
Sandboxing the process only works well when the malware requires more capabilities than the software itself.
So if your software needs to make HTTP requests and read the filesystem then the malware will be able to make HTTP requests and read the filesystem, which is enough for a ton of malware. Sure, maybe you can limit the directories it can access a bit and possibly some sort of network filtering, but it isn't a silver bullet.
Capability security in-language would make a huge difference, because the more granular you "sandbox" the less likely it is that the compromised component has the access it wants. For example if the HTTP client library is compromised maybe it can't access the filesystem so can't steal your cookies. Or maybe like in this case no permissions were needed at all and despite the process having enough capabilities for malware this malware can at worst return bad values and try to chain this to an exploit which is far more difficult than just running it itself.
(as I and others have mentioned in the thread): The attacker can just move the malicious code from build.rs to lib.rs (ie. build-time -> test/execution-time).
Then the problem is the language, as the grandparent observes.
Why does every build have the ability to run arbitrary code by default in the first place? That seems like something you should opt into only under very specific scenarios and even then it should probably be built around a WASM sandbox or microVM (as the parent suggested).
Because many many builds end up doing something just a little weird that the build system cannot handle by default. Ideally the build system would be fully property/dependency based and so it wouldn't have to run arbitrary code. In the real world everybody has something weird about their build that the build system cannot make work.
This is partially because for nearly every project the build system is something they need and don't care about. When they need something weird they hack just enough to make it work and never ask "how should the build system change so that this was a property instead of running code", and thus build systems are slow to improve. In a number of cases the build system did have a way to do that thing, but the person didn't know about it.
There are also a lot of code generators out there. I have yet to see a large project which didn't have their own code generator for something specific to their project (protobuf is an example from Google that has escaped and become useful elsewhere, but there are many others that are specific to one project. Yacc is from the 1970s, and stands for "yet another compiler compiler" - implying the idea was already common 50 years ago). You cannot have/use these useful tools without running arbitrary code.
I mean, I understand the need for an escape hatch, but it’s strange to me that it’s the default and that it’s unsandboxed. Feels like build scripts should be treated like CGo in Go—a rare escape hatch that people strongly try to avoid so that downstream builds are better. To that end, Go doesn’t even have macros or code generation and it gets by reasonably well without any build scripts should support—I understand that Rust’s emphasis on systems programming probably drives more scenarios where build scripts are genuinely necessary, but I strongly suspect that, as with C and C++, making it easy for people to do weird stuff probably drives a lot of people to think they need to do weird stuff.
Until a few years ago abuse of the escape hatch was seen as something theoretically possible but not realistic. Sadly attackers have abused trust and so even though most people are honest we can no longer trust anybody.
> Why does every build have the ability to run arbitrary code by default in the first place?
Nearly every time that you see a build script in use it's to support building C code, which has no standard build process and which often involves executing shell commands, and the shell itself is an arbitrary code execution environment with arbitrary access to the network.
Can you explain how an effect based language helps here? Also to note filesystem access can allow network access on many systems and also the other way around.
I think this is an attractive chimera, but will never be popular and will only be used much in specific domains.
The thing is, trust is a massive optimization. If you believe code from X is competently made and not malicious, you can accept that code without any additional overhead. (Building trust is a significant effort, but trust begets trust, so there's a resonable path from starting small to scaling.)
It's not that these mechanisms that require less trust (none of them eliminate it, BTW, when you examine them) can't work. It's that they'll inevitably be outcompeted by trust-based systems (in most domains).
E.g., compare the effort of creating a new, full featured, general purpose language, promoting it, building and maintaining all the tooling and documentation around it to support it, etc to the effort of creating and maintaining a curated package repository of trusted packages (and/or adding trusted code to extend the standard library) for existing language X.
(It's not clear to me you could provide enough value from a new language focused on effects that you could convince any but a few from adopting it at all, even if it dropped from the sky fully formed with all tools, docs, etc. in place from day one.)
I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
Rust has a sizable and well featured standard library at this point. I think it would be absurd to claim that the base language is "near-unusable" if you are using it for system programming, which is its intended use case.
Huge standard libraries make sense for Java, Go, Apple platform etc. because they are developed by giant enterprises that can manage the overhead. Thinner modular systems are a more natural fit for open source development.
It’s being used for a wide variety of other purposes however, and so perhaps it’s time to adapt to that reality.
While overhead is a real concern, I think it’s often blown out of proportion. Once a language achieves a certain baseline of stability and isn’t in constant flux and the standard library matures, changes become infrequent and maintenance load is low. The work is heavily front-loaded.
Not at all true, look at the large parts of Python's standard library that are effectively unusable or irrelevant. And they even allow some breaking changes over time (unlike Rust). This is especially true when it comes to anything related to internet protocols or file formats, but there are other modules that have far better replacements on PyPi too (re vs regex for example).
True. And the rust standard library is pretty good. But I really wish we had a good, fast, small futures executor in std. And accompanying async variants of File and Socket and so on.
Async rust is a jungle of weird compatibility questions. People treat async runtimes like sports teams. (I know I do). I wish it were more like Nodejs where async is just built in.
While admittedly async is a bit of a jungle for newcomers, I am glad that it is not built into the standard library. Where you see a bunch of needlessly competing runtimes, I see a healthy variety of different packages tailored to different users and technical specializations: tokio - reasonable default and server applications; smol - desktop applications or those who prefer the simplicity; embassy - embedded applications; futures crate - runtime agnostic utilities. I will gladly reach for any one of these depending on the kind of application I am developing.
Serde is not even part of the standard library. There’s no HTTP client (let alone server), logging, decent date/time…
That’s not what I call well featured.
Most programs should need close to zero dependencies.
Of course they’re free to do what they want, and there’s the question of maintenance overhead[0], but while https://blessed.rs/ needs to exist, Rust’s standard library is deficient in my eyes.
[0] One idea would be to treat a language more like an Linux distro and have library maintainers like package maintainers but a unified quality, audit, and release process
Thinner modular systems also require some good tooling and a good platform that ensures many security properties.
You will be trading a stable and vetted huge standard library for 200 small packages that may all be targets for supply-chain attack. Open-source projects don't all have the resources unfortunately to keep track of all their dependencies, so I'm not sure it's necessarily a more natural fit.
This is spot on. I am extremely uncomfortable with the large number of transitive dependencies that end up in pretty much any non-trivial rust application. No amount of memory safety will save us if a tiny, ubiquitous library that nobody scrutinizes because it’s nested eight layers down the dependency graph gets compromised.
I think Go apps tend to have better dependency hygiene because the language has a better standard library, which results in better culture around dependencies.
I also think it’s frustrating that both cargo and npm totally ignore the decades of prior art from Linux distributions that have figured out good ways to improve dependency management. We’d be in better shape if there was a community curated subset of known-good dependencies that are release-managed together that the broader ecosystem could build on, sort of like Ubuntu having “main” and “universe”.
I think Go ends up with fewer dependencies mostly because there is more friction in finding them. With rust (and npm) you just "cargo search xxx", then "cargo add xxx". Go makes you web search and poke around github looking for something. It's not onerous or anything, but just that extra step slows things down just a bit. C projects tend to have the least because it's even more annoying to add them and you have to create build commands that deal with slightly different distributions (and learn makefile+pkg-config or cmake, or autoconf).
Certainly having a larger stdlib helps Go projects keep their deps down, but I don't think it's the real reason.
Perhaps, but I think it depends on where the Go devs are coming from. In my experience the lack of proper dependency management in older versions of Go didn't really lessen dependencies, it just made teams have to deal with annoying $GOSRC issues. But then I worked at a place where the devs used PHP (w/composer) and node before Go was introduced.
Personally I came from a C background so I tended to use deps more sparingly.
In the end it's a bit of a balancing act—lots of dependencies is a larger opportunity for these kind of supply chain attacks to affect you. Copying stuff into your repo protects you from that, but it also makes it way more likely that you'll miss security fixes, unless you're actively looking for them. Re-implementing what you need is also viable but it slows down the dev process.
It absolutely is a culture thing. I've seen many threads asking about backend frameworks in Go, and every time there was a lot of answers akin to "screw framework dependencies, stdlib is more than enough".
> Certainly having a larger stdlib helps Go projects keep their deps down, but I don't think it's the real reason
I disagree: having a big stdlib that encompasses a huge chunk of common functionality (logging, a plethora of network server/client protocols, etc) means that for basic needs, the question of 3rd party libs never arise, whereas for other language, the question is not if you need a dep, but which one. Making gorilla easier to find won't undo the cultural reluctance of adding a 3rd party dep when using the (very pragmatic, IMO) stdlib server is adequate to the task.
I don't really agree. You reach for a dependency when you need to do something non-trivial and the stdlib doesn't have what you need (or the stdlib implementation is bad). You don't just sit around running `cargo search` for fun.
I think the ease of adding dependencies does drive the idea of "microdependencies", where you have many dependencies that do one little thing, and then those dependencies likely have their own small dependencies.
Whereas in the C world, you'll probably have a smaller number of dependencies, but it's likely that some of them are very big dependencies that offer a lot of functionality (like glib, for example).
I could maybe see the argument that ease of adding dependencies discourages people from writing small things themselves. Good ol' left-pad comes to mind. In an ecosystem like C, I'd just write that algorithm myself, over and over and over, because pulling in a dep just for that feels annoying.
I learned programming from C# which has an excellent standard library. Just recently started using Go which seems to have a good standard library. Been using python for years, can't say I ever had a complaint about the standard library.
Rust meanwhile seems to be following some "no standard library" philosophy.
I feel like there's an opportunity out there to become like an amoeba: fund and build a third party Rust standard library, absorb absolutely everyone who is so grateful to have a library, and then get so big absorb Rust itself.
> Rust meanwhile seems to be following some "no standard library" philosophy.
As an embedded dev, I personally find it very useful to compile Rust programs without a standard library. I would not use C# or Python for an embedded project for the reasons you probably prefer them for your projects.
The guide that NASA put out for spaceflight systems was on HN earlier this year and one of the rules is "no heap allocations; all memory must be declared and fixed size for the life of the app." Isn't Rust the ideal language for that kind of limitation? We are talking C, C++, Rust, Zig as options I would guess; Rust certainly seems the ideal from those options.
But you gotta admit people trying to build GUI libraries and using linked lists don't fit.
At upload time? Or by every client at download time? Or just adhoc by random users?
This does sound like an excellent way of improving automated package checks, even if it does result in some false positives and false negatives. For all sorts of packages, not just code dependencies.
By security scanners, the moment a crate is uploaded to crates.io. The way forward here is dependency cooldowns enabled by default for ordinary users (and crates.io changed to show the old version until the new one is a day or two old). Essentially, security scanners get to vet a package for a day or two before ordinary users see it in everyday development.
You are just shifting where the eyes are, the amount of eyeballs (developer time) is the same. Unless the proposal is to increase the amount of (other) developers time.
The proposal ends up amounting to either shifting stuff around, or asking to other people to put in more effort? Into a project that usually doesn't pay (programming languages and runtimes)
Which by the way, asking other people to do stuff for free IS the attack vector, if you are the kind of dev or company that gets hit by these, your ultimate root cause is that your business strategy is using code without paying for it. No such thing as a free lunch, you will pay for it, among other things, with an increased cybersecurity risk.
Packages can release updates arbitrarily, while standard libraries tend to have longer release cadences.
It's also more difficult to put arbitrary code into an stdlib, because stdlibs are scrutinized better.
Also, it's harder for some random anonymous developer to gain push access to stdlib repository.
One of the reasons so much Rust code is in libraries is not because there are no people to write it (duh), but because putting these in std commits maintainers to keeping backwards compatibility and slows down included package's release cycle.
I think one thing that has made Go have better dependency hygiene is not merely having "batteries included" in the standard lib, but the early focus on having production-ready implementations of a lot of stuff in the standard lib, as opposed to minimum viable implementations. I used to get a long way with one or two dependencies that would barely fan out at all. It's been a while since I worked on a Go project, so I'm not sure if it's still that way.
I think having production-grade implementations makes a huge difference also. Go’s net/http in the std lib feels like a good example of that, though I’m fairly new to go still. But it seems net/http still gets pretty big updates despite something like Chi being available as an external lib. Odin is doing this as well, even including raylib in the std lib. Odin is even more primed to keep you on the std lib as it doesn’t include a package manager at all.
So I went to have a look at Odin and was mind blown at the lispiness in the hellope example. A list of operators defined as "program"; that are applied to the accumulator then printed. It's a striking first impression for me and I want to see what the Odin people have cooked up.
I wasn't when I saw they made the exact same stupid error as Python: if and switch are statements, so that you need a stupid ternary operator to do the same.
Odin has been a masterpiece so far! Ginger Bill has a massive update coming in for it Jan 2027 as well. :) I think it's definitely going to take off in the next few years.
>There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
The big reason is money & time. Maybe less so today if you are happy with an AI generated stdlib, but I don't think it's fair to compare Rust (built by Mozilla, who's primary product is an open source web browser) to Swift/AppKit (built by Apple, 4.6T marketcap at the time of writing).
When Go was released it had an amazing stdlib, but that was because Google was funding it.
Python is batteries included, but all the batteries have corroded.
Look at C++'s long in the tooth STL.
You don't want to marry a language to fast aging libraries you have to support for eternity. Better libraries always naturally emerge.
Rust's decision here is fine.
The only thing I'd like to see is the ability to programmatically limit transitive dependency count or depth in Cargo. I'd also like crates to specify whether they limit their own deps and whether they have panick-y behavior or not.
I think maybe the right answer is to have a standardized, curated group of libraries pegged at some sort of LTS release that only backports security fixes. However, someone would need to pay for creating and maintaining this -- and then you wonder where the money would come from?
That or potentially fast tracking merging popular libraries into the standard library. Or at least concepts from popular libraries. Basically all the "most downloaded" crates on the Crates.io front page should be candidates for merging into the standard library.
Not everything should be eligible for this treatment. Certainly not an HTTP library or something without extremely widespread applicability.
Nearly all of my projects used once_cell, so it's good to see that pulled in. I wouldn't want to see anyhow, thiserror, anything opinionated, or anything domain specific in the std. That's a weight you have to bear forever.
Remember how long Python 2 -> 3 took, and look at the ancient and awful stuff in Python 3's standard library. Rust is not the right language for this.
Can we pick and choose the good without the bad? If we agree stability is a net-negative we could just say there will be a list of blessed trustworthy libraries with extra scrutiny and bureaucracy (make no mistake this is completely necessary to what people are asking for) to make sure they don't go rogue. But without a guarantee they will stick around and get updates forever.
I’m a big fan of how C# handles this. The standard library is versioned like any other library. Your program is explicit about the major version of the standard library you’re pulling in. And all major versions of the standard library continue to work.
For rust, this would mean something like adding std = 1.0.6 to your cargo.toml. The advantage is it means the standard library can deprecate and replace features. Upgrading std is an explicit step by the developer. Annoying, but in the age of LLMs it should be pretty easy.
The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.
> The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.
C++ went through that around C++11, where some types had to have ABI breaks (on some implementations, such as that of GCC's libstdc++). It has been a while, but as I remember it std::string was affected since it went from COW to SSO. Map might have been affected too (don't quite remember). It was a mess.
Rust can link multiple semver versions of the same dependency, but you can't pass data structures between those versions. For transitive dependencies that usually isn't a big deal if your direct dependencies use them internally rather than exposing the types in their own APIs. But std contain vocabulary types that everyone uses (Option, Result, String, etc) so that would really not work.
More than that. Rust requires that a program can be compiled from multiple crates, with those crates all targeting different editions. This limits a lot of breaking changes you might want to make. APIs can never really be removed.
There's probably to truth to this, but I would weight the design intention behind the language just as or more heavily than available resources.
Swift was intended from the start to not only replace Objective-C and all of its use cases, but also take on new use cases and bring a number of features from other languages that were newer and had different dominant paradigms.
Certainly having a juggernaut like Apple behind it has been instrumental in its success in achieving those goals. That said, a community-lead project with similar aims could probably achieve those aims as well, given enough time.
I think it's just rare for enthusiast-led projects to set out to do such things. My theory on why things tend to go that way is that the types of people to start programming language projects tend to heavily lean nuts-and-bolts and theory with a purist/idealist sort of mentality that's more concerned ideological purity than practical usability.
So to sum this all up, a community-run practical, multi-purpose, batteries-included language likely needs to at least partially designed and helmed by product engineer types so it doesn't end up anemic and stuck in an ideological rut.
I don't like the "bring your own" approach because unless one happens to be an absolute tour de force 10x engineer unstoppable god of a programmer (which most of us, myself included, are not), whatever you build is never going to be as well-rounded, fleshed out, and complete as something built by a larger organization, especially when it comes to UI libraries (which are monstrous projects if done right, e.g. meeting accessibility requirements).
I'm happy to contribute to a larger effort but anything I can build on my own is going to be a thin, flimsy happy meal toy compared to something with the backing of a company or well organized FOSS project.
I agree that building your own is hard and something that can be done by exceptional engineers, but software is famously a winner-take-all industry, even if the optimal strategy for lower percentile programmers and median programmers to import generic modules, as long as building your own results in the best product (which it does, whether importing frontend or backend modules, if you hand code something, it's going to be the optimal strategy for building a winner product. Even programmers that are median or below average probably have an incentive to aim for being exceptional, because the best value of median and below average programmers is not building median or low value software, but having a shot at building exceptional software.
Also the challenge of building a generic module is much larger than building your own. A generic module needs to have flexibility for many different options and integrations, when you build your own you build just what you need and tightly integrate it with the product
> Also the challenge of building a generic module is much larger than building your own. A generic module needs to have flexibility for many different options and integrations, when you build your own you build just what you need and tightly integrate it with the product
True but I believe overestimated. Usually rather than building what's needed, what gets built is what is thought to be needed (often a substantially smaller subset), and then over time you end up building a markedly poor version of a generic module.
>This package contains just four macros, which enable the creation of array references to portions of arrays or slices (or things that can be sliced).
I'm no rust programmer, but that doesn't sound like something that moves the needle. I talked about the ratio between value to risk being a relevant decision parameter, so getting infected by an http framework would be defensible, getting infected by adding this to a project to me is a PIP, and getting infected by installing leftpad or a custom cursor plugin in an IDE would be fireable.
The counterpoint is that whatever std invents is probably not going to be as good as what moviated people in the community make.
The rand crate is my favorite example of this. They have a whole bunch of different rngs. All rated by quality and performance. Some are csrngs and some aren’t. It’s a delight. “Batteries included” languages don’t come close.
Then there’s serde - which uses a clever technique with traits to allow compile time specialisation and optimisation of binary and json serialisers. Most people had no idea that was even possible in rust until someone in the community made it.
Maybe std should pick up the best crates from the community and bring them in house? This might be a good idea. But also maybe not. Since serde came out, several other crates have found ways to crush its performance numbers. Often by 2-5x if memory serves. What a relief we didn’t immortalise the slow version of serde, right?
I’m playing devils advocate here. I think std should be a bit bigger too. But there are real tradeoffs in doing so.
> Maybe std should pick up the best crates from the community and bring them in house? This might be a good idea. But also maybe not. Since serde came out, several other crates have found ways to crush its performance numbers. Often by 2-5x if memory serves. What a relief we didn’t immortalise the slow version of serde, right?
I think periodically rolling in the best crates is the way.
Don't immortalize anything, though. Always reserve the freedom to swap out integrated crates when something better rolls around. Negative impact can be reduced by limiting these kinds of changes to major new versions.
Another option is to swap internals while keeping the same public API. Apple has done this several times with Swift, resulting in massive speed improvements in things like string handling and JSON decoding without breaking anybody's code.
> Don't immortalize anything, though. Always reserve the freedom to swap out integrated crates when something better rolls around.
That's a non-starter, though. Once it's in std, it's not an "integrated crate", it's in std, just like any other part of it. Removing it is a breaking change, and I don't want a new incompatible Rust major version every year or two (or even five or ten, honestly). (Of course, Rust has editions to make breaking changes possible without affecting old code, but there are limits to what you can change in editions.)
Pretty much every language ecosystem fights really hard to maintain backward compatibility as they evolve, and every time they fail to do that, it causes headaches, sometimes for years. Python 2->3 is of course the canonical painful example of that, but there are others that are instructive if you only take the time to research.
(And before you mention something like Zig, which changes incompatibly frequently enough, remember that the author has made an active decision to remain pre-1.0 unstable. That's one reason why the Zig community isn't that large. That's not me looking down on them; that's an active choice they've made, and they're free to make that choice. But bigger/popular languages like Rust can't go back in time and decide to be like that.)
Yes, it’s annoying to have to make fixes around my codebases periodically, but to me that’s a hell of a lot less annoying than having to pull in a mile long list of dependencies to do anything of consequence. It’s also better than the bad parts of the language being frozen in time forever.
Apple breaks things with Swift not constantly but on occasion and while there’s some griping around it, it’s not a big deal. We make the needed changes and move on, knowing it’s enabling improvements that will be nice to have.
You seem to be under the mistaken impression that the Rust team is some huge organization. They're not. They are certainly larger than you or me sitting behind our keyboards at home, but they don't have endless resources, and they are always stretched pretty thin.
So sure, you want, say, a JSON library in Rust std? Who's going to shepherd it in and then maintain it? Well, you could hope that the people behind serde_json might want to do it, but what if they don't? Where are you going to get the people with the time to do it?
For something like Go, the answer is easy: tap into the money printer that is Google, and hire more people. But for Rust? Hell, the core team has probably only declined in number over the years (Mozilla layoffs, for one thing).
with LLMs it's becoming more common to just vibe up anything you need that might be missing. even if there is an available package you can pull in. the most secure option as well
> I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
> There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
I wholeheartedly agree. Most standard libraries out there (when I say most I easily mean of 99% languages that I know) have pitifully tiny implementations. They at most provide for string parsing, io, a bare networking layer (that's at best a wrapper around the POSIX syscalls), threading and that's it. If you want to do anything more complex, you either have to roll it yourself or use an assortment of the "blessed libs" for any given language, that may or may not work properly, and may or may not even work well with each other. I got pissed off by this at some point so I decided to go ahead and roll my own standard lib that would actually have everything I need to.. actually develop apps. Now I know most people probably aren't going to be using it (mainly because its functional API isn't what most C++ devs want) but for me it's actually been brilliant.
I don't see the problem with boost? Isn't that a perfect example of your well supported, community embraced option? I certainly feel much safer pulling something in from boost via the official debian repos than I do pulling a random package with npm or cargo or etc.
Primarily I think the underlying concern has to do with the pathway for authoring code. When you have contributors whose submissions are gated with a rigid third party process and where that third party is the one responsible for curating the code (as opposed to the author also being the curator as well as the publisher) then you have the possibility to catch a lot of wrongdoing before it succeeds.
There are many issues that I've seen taken up with Boost, but my personal peeve is how it can make building projects that incorporate it a pain, both because of its sheer size but also because it can sometimes be difficult to appease with its own dependencies.
> I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
> There’s no reason why this can’t be replicated elsewhere.
It is replicated elsewhere and is not limited to Apple; I can do the same using a tech stack of perhaps 2x techs and very limited (manually added) deps. I'm thinking Lazarus, which comes with most things I'd need for most apps.
> I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
Nobody can agree on what those common things are in a general purpose langusge though. It works for something like Go, because it is largely used for servers and command line tools from what I can tell.
For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.
And even if did put some of those things in std, that is stable forever. That is how C++ ended up with an unusably slow regex implementation in the standard library. And it isn't fixable. Everyone just ignores it and uses a third party library like PCRE, RE2, etc instead (depending on their specific needs). I would not expect web or GUI things in the standard library for this reason, those standards change too frequently for that to be stable forever.
And even something like Python, which is known for data science and machine learning, doesn't have core libraries for those activities in the standard library. Numpy, Pandas, Scipy, Pytorch etc are all third party projects.
Maybe that’s a good place to make the cut. One Rust crate for game dev, one for data science, etc. Then their deps are pinned and monitored by the crate maintainers.
C++ ended up in that situation because they don't want to break backwards compatibility (ABI, API) or change contracts, except in some rare cases, even on major releases. Quite often, it is a vendor issue to break the ABI and fix issues, and some have refused to do that.
It's a self-inflicted issue that most other languages with "batteries included" don't have since they will document breakage and upgrade paths when anything changes, or they are not impacted as much since they are using an intermediate language as interface and avoid most ABI issues.
The implementation is fixable though, you can break the ABI and address your issues. A lot of standard library defects are fixed in an unstable ABI mode they have, but that requires you to link it statically or dynamically and then ensure all your shared object dependencies are using the same exact version. It's not trivial for everyone, but many do this (usually the static version).
Rust has committed to a stable API (but the ABI is explicitly unstable). Which makes sense for a systems language. It isn't (and doesn't want to be) a kitchen sink language.
But even something like Python has lots of cruft in its standard library. And they are willing to make breaking changes. Nobody should use urllib.request for example, requests is a far better HTTP client library. To the point where the stdlib docs tell you so.
> Nobody can agree on what those common things are in a general purpose langusge though.
Well, then take the Lazarus / Free Pascal's approach and just put everything :-P. It isn't like code takes THAT much space.
If you install Lazarus you get a ton of stuff out of the box, aside from the crossplatform WYSIWYG RAD IDE and desktop application framework you also get 2D graphics libraries (with a bunch of image i/o), networking libraries, (de)compression libraries, a webapp framework, database clients, parsers for json, xml, markdown, javascript and a bunch of other stuff (even a parser for the language itself), a TUI framework, bindings for a bunch of external libraries and other stuff.
And all that stuff (or at least most of it) work on all the platforms the compiler supports. Yes, using FPC you can probably (didn't try) make a pretty markdown viewer with inline JPG and PNG display support that runs on Windows 3.1 :-P.
Unfortunately some people are trying to break this and introduce online package management, but so far at least FPC and Lazarus comes out of the box with not just batteries included but an entire carton of batteries just in case :-P.
> Nobody can agree on what those common things are in a general purpose langusge though. It works for something like Go
It, as you put it, works for Go because its designers took a hard stance and decided that it was a language for network systems and you're on your own if you want to make it do other things. Nobody needed to agree because it was already chosen for you.
> For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.
But does Rust need to be all things to all people? The Go ecosystem has a separate standard library under the Tinygo project for those who want to prioritize embedded development. Even if there was convergence on Rust as a language, it doesn't technically need one singular standard library. There can be a distribution for embedded developers, a distribution for game developers, etc.
> I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
Most of the things I wrote in Rust have less than 5 dependencies. Those 5 dependencies then depend on 20 other dependencies each. Those 20 dependencies have 2-3 other dependencies. And so on…
Counting the number of crates in the dependency graph in order to gauge exposure is a misunderstanding of how Rust works. In Rust the crate is the unit of compilation. Unlike in Javascript, the fact that a crate has dependencies on other crates does not mean the author pulled in random code written by someone else, it often just means that the author wanted to leverage crate-level compilation parallelism by splitting a large compilation unit out into several smaller ones.
> There’s no reason why this can’t be replicated elsewhere.
There are plenty reasons. Just not technical reasons.
At the end of the day, people like to feel useful. The ecosystem that attracted communities are those ones where everyone can feel they're contributing back to the communities. That's why the most successful languages, save those backed by big companies, are filled with very small libraries.
> I think we should be taking a more “batteries included” approach to language and library design. The entire reason we’re in this mess is because we’ve decided it’s ok or maybe even preferable if stdlibs are rail thin, rendering base languages near-unusable.
Rust is one of the better languages when it comes to batteries included in it's stdlib. Even then, it's impossible to have a 100% coverage batteries included language because nearly everyone on planet earth has some different/unique use case for their code that doesn't fit a stdlib.
A big selling point of Java also was a very strong built-in library, e.g. with red-black trees (TreeMap) and concurrent lock-free zero-mutex queues (ConcurrentLinkedQueue).
No wonder competitive coding champions sometimes preferred Java as they don't had to spend time coding it from scratch.
I really like the approach Julia is taking. The std libs are their own packages with their own versions. You can then build your own “standard library” by compiling a sysimage that freezes the versions (you can add third party packages as well).
Practically speaking the standard library is just meant as “what’s included in the default sysimage”.
This allows to have different standard libs for different use cases. Think of Linux distributions in a sense. There are already a couple of alternative “stdlibs” out there for different focuses.
> we should be taking a more “batteries included” approach to language and library design.
Okay, but how do we decide which approach to a problem should be standardised?
Rust's approach of "provide a minimal core to build libraries on top of" provides a fairly significant benefit - flexibility.
A prominent example is async: Rust has three fairly prominent async runtimes (being Tokio for general server-side use, smol for a simple non-work-stealing approach and embassy for embedded). Which one do you make the default? Tokio is very complex and work-stealing is arguably a very bad default in terms of development complexity. smol doesn't fit some workloads because of a simpler execution model. Embassy is way lower level and much more constrained than most applications need. Standardising over one implementation instead of the current (even if still messy due to early portability woes) standardisation around traits and general async semantics would make switching runtimes more problematic, even though the general case is solved with the same approach as Python's
Database access is also a complicated topic. Rust has multiple fairly popular DB libraries. The first two that come to mind are SQLx and Diesel. Both are too complex to be a part of the standard library, both required a fair bit of breaking changes, and if you wanted a raw core to build on top of - you can't really build SQLx on the type of core e.g. Go's database/sql has, you'd be basically forced to implement it all from scratch to handle the compile time type checks anyway
RNG sounds simple, except there are good reasons behind both rand and fastrand existing, and behind rand not being stabilised at 1.0.
DateTime handling is notoriously complicated, has multiple competing implementations (chrono, time, jiff) with various trade offs
Where do you draw the line? What do you standardise and which implementation do you standardise against?
I've seen golang exemplified a lot in adjacent threads, but golang's standard library both has fairly significant issues (json, no set type, very limited CLI flag parsing, UUID took until 1.28) and is standardised around a relatively specific goal (unix server usage)
Dunno, I'm not entirely sold on the whole "extend the standard library" idea for a language meant to run both high level applications and embedded stuff. If anything, recommending blessed.rs more is a better way IMO
You can never please everyone , and some stdlibs can age like milk (see ocaml).
Having libraries allow the community to explore rather than saddling it with a bad default. See the number of http server frameworks etc.
The real solution here is not for the stdlib to become a black hole, swallowing up every half-way popular library a but rather for “rustaceans” to adopt a more security-conscious approach to writing programs- C programs typically have few and well-chosen dependencies. Some languages, like Odin, completely eschew package managers because they see the very real issues stemming from writing a program with scores or hundreds of dependencies.
You seem to be ignoring that most people who want off the infernal npm/cargo hamster wheel are conscious that it implies a bit of spoilt milk or moldy bread in exchange for stability and that they're fine with it.
The gains are just too numerous and important to reject the few rust spots on a "good enough" stdlib.
But well, the kind of people who nod a bit too much while reading https://boringtechnology.club/ aren't using Rust. They're on C99, ANSI CL, Ada, maybe Python, Perl or Tcl.
Rust suffers from the same faults as the JS ecosystem. Any significant crate imports hundreds if not thousands of dependencies. The probability that one of the authors gets targeted by AI-assisted attacks is just too high.
Also most of these dependencies provide a breadth of features that the end package does probably not need.
My experience has been that it has a major advantage, in that freeze + offline actually work properly. You can collect the dependencies you need once, put them in version control and never ever talk to remote registry again
I guess that’s more of a problem if you use all-encompassing frameworks, but normally the things I’m using are very small components where the CVEs either don’t exist or are inconsequential/unexploitable for the programs I’m building
A language without a large stdlib pushes this that functionality into (transitive) dependencies. I hope more language will adopt batteries included approach.
No stdlib will ever include all the "batteries" you need. And large stdlibs have their own set of problems, like stagnation, tying the library version to the language version, backwards compatibility garantees preventing evolution, and either needing a wide range of domain experts to maintain it, or having developers maintain components they don't have a deep understanding of.
I think the sweet spot is having an ecosystem of "blessed" libraries that are reviewed for security and quality, but are versioned independently of the language, and maintained by subject matter experts.
This particular crate (arrayref) seems pretty niche, but also doable in std now.
It lets you take a slice `&[T]` and return a slice with a known number of elements`&[T; N]`. tiny-skia was probably using this to get some SIMD optimizations.
'bs58' jumped out at me as a random looking package name.
It implements the base58 encoding, which is used primarily by... Bitcoin.
I'd love to see the explanation of why "a TUI file manager" needs a crate closely associated with crypto coins and not much else.
It is precisely this kind of thing that makes Rust a no-go for most enterprises.
"I just want a CLI tool."
"Congratulations, your servers are mining crypto!"
PS: bs58 is maintained by one anonymous person and their package is used in just about every Rust crypto library and hence application. Compromise his account in the same manner as the 'xz tools' attack and you could still billions in crypto!
It doesn't actually use it, this is just referencing it in the dependency graph (Cargo.lock). bs58 is an optional (feature-gated) transitive dependency of another crate, which is never referenced at all in the top-level crate. The build.rs doesn't run or anything.
I believe there's (several) open issues about changing this behavior of the lock file, but I have moved on from Rust so don't know the status of things.
That said, this is confusing behavior. I remember I first noticed it when a bunch of crates were showing up in my lock file for every OS under the sun, even though I was specifically using a "wayland" (linux-only) feature.
Not to defend Rust's crazy culture of dependencies, mind you.
The lockfile confusion/bug is indeed a thing, but OP is noting that b58 was downloaded, which makes me think it's actually used.
I ran in to the lockfile bug myself a day ago and noticed that dependency-feature-flagged-crates show up in Cargo.lock, but their crates aren't downloaded.
I use nnn[0], a C tui file manager, and from the makefiles, you mostly need
readline
pcre
ncurses
pthread
and POSIX (and some unix thingies)
Which is why I love C projects, even with all the footguns. You have libraries with nice API that lets you write software without being on a treadmill to update the code every few months.
In my opinion, library authors should really minimize the amount of dependencies they have. Back to TypeScript, we are also the authors of https://github.com/okcontract/cells and we made a point of almost not having dependencies or even devDependencies.
I don’t know anything about this particular tool. But I’ll note that comparing package counts often doesn’t make sense between ecosystems, since some ecosystems (like Rust) lean heavily towards workspace patterns where several “physical” packages make up one “logical” unit.
(Rust even enforces for proc macros, which is why so many of the crates in your example end with “derive” or “macros.”)
Well I think the main important argument here is you shouldn't use the package management. A third party package manager would be even more concerning to use imo. I generally try to stay away from the typescript world so unfamiliar with nuances of NPM.
The author proposes to reinvent the wheel and depend on stale dependencies, which are EOL. Neither of that is an acceptable solution with LLM-based Agents being able to produce exploit(ExploitGym) chains in minutes from known bugs. The other issue is that Rust's forces the user to provide more information and APIs are usually kept generic for systems programmer, so standardizing things is not as straight-forward compared to Go where you can assume a memory-management, a virtual thread runtime and mostly ignore dynamic dispatching.
I don't think anyone is saying that you should depend on stale dependencies. You should not hand off your responsiblity to untrustworthy package managements services though. Evidence of the failure of automated package management is the near constant shai-hulud attacks as well as the originating post here.
The fact is that your dependencies are your responsibility. You should be staying up to date on what you depend on and their vulnerabilities. Most vulnerabilities affect specific code paths which may not be present in your code base and do not require mitigation. These are decisions you can make by understanding what your dependencies are instead of offloading to automated package management systems.
NPM has been the posterchild of these types of issues but the dependency stack I saw everytime I compiled even simple programs in Rust made me think Rust would be impacted next. Personally I think third party modules that are effectively universally used (axum, tokio, serde) should be integrated directly into the language.
[3] is no longer true. They're definitely not unprepared for an incident like this. It's not the first time they've done it and they published an update to their process in Feb:
> The bad package version has also just disappeared from crates.io with no indication its been yanked.
So, yanked crate versions do have an indication on crates.io. (Here's an example: [1]) The Rust blog post uses the word "deleted", and I'm guessing a bit here, but I think they mean that literally, and that the version here is deleted, not yanked. And I think that would be more appropriate: a yanked crate is still downloadable by cargo, if your lockfile is locked to it already; yanking only prevents lockfiles from newly automatically acquiring a lock on that version. That wouldn't be desirable in the case of a compromised crate: you don't want a download occurring at all. The tradeoff of "break those with locks on the crate" tips to being worth it.
That said, I agree with you, though: I think this state (if it is "deleted" and not yanked) should be plainly indicated on the crate's versions page. (Even better would be if it came with a link to, e.g., the blog post or a RUSTSEC so that you could find out why.) (& I think perhaps the docs for yank should point out whether or not it is appropriate in the "compromised crate" scenario, and if not, what to do instead.)
> That wouldn't be desirable in the case of a compromised crate: you don't want a download occurring at all.
Sure you do; you just don’t want the crate to be made available to people trying to make use of the crate as regular downstream consumers. You still want the crate available for download for analysis. You especially want to be able to deterministically reproduce the vulnerable version of your software that you already built.
I would posit that most package ecosystems should treat “contains an exploit” as its own package state; mostly ignore them; but still be aware of their existence.
In other words, if there’s any other non-exploited option under your version constraints, the dep should resolve to the nearest non-exploited version, even if older. But if it’s the only version, or if you have pinned the exploited version, then the packager should see it, but normally refuse to interact with it—i.e. refuse to lock to it if it’s the only version; or refuse to fetch it if it’s the locked version.
I say “normally” because you should be able to bypass this with a flag / env-var to the packager that basically means “I’m building this for forensic analysis, not for running.”
I think "deleted" is still always going to have to be a package state, realistically. A package could contain spam, abuse, copyright infringement and/or illegal content.
I don't mind malware being lumped in to that - the place to look up code for historical or analysis reasons ought to be version control, not crates.io
What about the place to look up packaged binary releases for historical or analysis reasons, esp. when the packaging system either builds such releases itself, or where it allows the submission of arbitrary build artifacts (where the build process that produced said artifact can be compromised in ways that have nothing to do with the source repo)?
I agree but think the problem is slightly more general. Consider that a package could be redacted not just due to exploit but also (among other things) for legal reasons, and that such reasons could be region specific. So a redacted package might or might not be available to manually download and some other metadata is desirable so that "audit" can notify you that you were compromised.
A package redacted for legal reasons should still be able to be built too! Presumably one or the other side of the relevant suit will need the package as evidence!
I would suggest that such packages should simply require authenticated access + manual “write an email to ask to have your account be put on a whitelist”-driven authorization. Such a package would still be there (in its redacted state); but attempts to download it would be a 401 (if unauthenticated) or 403 (if not on the whitelist) error response, which the packaging tooling would need to know how to handle.
(In any case, this “redacted” package state and my “exploited” package state could be two variants of the same logic, both following the same approach to locking/resolution, both requiring the flag to indicate that you understand what you’re doing and aren’t just going to run the resulting program. The only difference is that once you pass that flag, an “exploited” package would then just work, for anyone; while a “redacted” package would still have one more gate standing (server-side AAA) in the way.
> A package redacted for legal reasons should still be able to be built too! Presumably one or the other side of the relevant suit will need the package as evidence!
This doesn't make sense if the legal reason is accidental release of secrets or PI, in which case you absolutely do not want to leave it up.
I'm not talking about things like github tokens here, those can be revoked. But actual secrets or PI where that isn't possible.
I mean, unless the party you’re concerned about having your secret is the maintainers of the package store itself, what’s the functional difference between “it’s deleted from our servers” and “it’s marked private so only the author [or admins of the author’s org, insofar as user accounts are org-bound] can still download it”?
If I inadvertently committed (arbitrary example) my tax returns to a repo I'd hope that the operators of the package repository would promptly destroy all copies in their possession.
I think maybe you're missing the significance of legally mandated redaction - continuing to provide access could well constitute a criminal offense depending on the specifics. And there may be many other reasons for redaction - as a sibling comment notes you certainly don't want to continue serving a package that contains sensitive private information.
I see the "exploit" case as a strict subset of redaction in which you do want to continue serving downloads to parties that explicitly acknowledge that they understand the classification of the artifact in question.
Sure, I'm using "deleted" here to mean a third state more unvisible than "yanked"; I'm not trying to specify exact semantics about said third state, simply that it (a.) isn't yanked, and (b.) yanked's semantics are probably undesirable here and a third state has merit.
I mean, we have the crates available for analysis. We take snapshots before deleting them.
We would probably be open to sharing files with those who ask. Probably, I don't know what other team members would think about that. This roughly matches the workflow you propose below where people can ask for access to these things, just a bit more manual.
Any way of keeping it around on the server in a way that is fetchable by cargo is a complete non-starter. Bright red lines are useful in security: _the file is gone_ is way easier to keep secure than "the file is still there in a way that might be fetched if you have the right flags set" now means that that flags mechanism is now security sensitive.
If you'd like us to have a nice page that shows deleted-for-security crates, that's probably a proposal we'd welcome, but it is not really something anyone has time to work on themselves. Right now we just remove them from the db after taking a backup.
But I want to specifically signal a security concern to the requesting party. For example, a artifact proxy (like Artifactory or CodeArtifact) can use this response to warn developers.
Structured data in body or HTTP header is plenty for that, no need for a HTTP status.
For example, if we were talking about a human-readable /crates/* page, returning 410 Gone with a HTML page explaining the situation to a human, that page could have a JSON-LD snippet explaining the same to a computer.
A dedicated status code + Location header allows to decouple the way an advisory is presented completely. At that location, there can be any representation, HTML or structured data like JSON-LD. The RFC extension would be small and clean.
It is ok that you feel this way. As explicitly stated the report is _generated_ and id does require a different mindset to read AI generated reports. It contains quite a bit of information about attacker behavior and details of the timeline. It also linkes to a story that is now largely human narrated and I´ll keep improving on that as I find time. Having to juggle three separate issues yesterday did not leave much time for manual writeups.
The response was managed by the security-response team working with the infra team and the Rust Foundation's Security engineer. Tobias from the crates.io team was also involved but on vacation so it was not as active. This is normal.
What makes you say we were unprepared? We have been deleting malicious crates for a while. This is the first time those crates have wormed their way into an actual real crate that people use, but most of the playbook here was the usual one. We take a snapshot copy of the crate and then purge it from the system.
The security page thing is because the rustsec maintainers were not around or a part of the response as it was occurring. It's pretty normal for a security advisory on rustsec to take some time to merge. We should probably get everyone on the same page about when people not on the rustsec team can merge security advisories, because I do agree that getting them merged quickly is important sometimes.
The crates.io page and version history should have an entry, like red with an ! and a cross through with an advisory note explaining the security issue, accessible via the api as well so it is clear what happened.
The main crate entry should also contain a security advisory at top. I looked at the crate and it just looked normal; I had to dig to find the exact impact surface, and if I wasn't informed via secondary means (hackernews) I would not have known. This is unacceptable for a mature package management system.
cargo-audit is the automated mechanism you are looking for
The crate does have an advisories/"security" page on crates.io.
We could try and show the existence of a security-deleted crate on the page. This is not a priority for anyone, and I remain unconvinced that it needs to be (not that that is my decision anyway). File an issue and make your case to the crates.io team.
Jumping off the title and ignoring contents of post, as is customary:
look at this graph and guess which languages use each number of dependencies for their website: https://www.youtube.com/watch?v=E82ly38YEEQ&t=325s
I won't spoil the claim in the video about what the dependency number is correlated with, and I'm not even sure how true it is in general, but it's very interesting.
oof, not rust too. this has been brutal for the node ecosystem, hopefully cargo also gets the proper controls to help avoid this, like the in-flight min-publish-age
354 comments
[ 0.25 ms ] story [ 19.2 ms ] threadIt seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
Developer machines are quite juicy targets. They tend to have all sorts of credentials lying around, so it often isn't too difficult to escalate from that to compromising AWS/GCP/etc.
On top of that there's very little preventing a compromise. Developers are inherently expected to run untrusted code, and the usual Linux / MacOS laptop probably isn't even running any kind of anti-virus protection. Want to compromise the downstream app? Now you also need to pass Play Protect & friends.
> It seems like we should be building and testing everything in bubblewrap or some other sandbox going forward.
Yes, we really should. It is frankly a miracle that it has taken so long for fetch-time / install-time code execution to start blowing up in our faces. If we are spending so much effort on run-time isolation, why are we completely ignoring all those practices during development?
Well, it depends on the language: language which "capabilities" (both for the source code and for the building* ) could in theory really reduce a lot of the burden to identify supply chain attacks.
*: some research language have/had capabilities which would make supply chain attack "obvious" but for build systems I don't know if this exist.
You don't need to audit all the crap is being uploaded right now. Only really necessary and widely-used packages should be managed in a centralized way, so, auditing all of them isn't that huge task.
In the end it is your decision to use unaudited or refuse unaudited crates.
It should be the default behavior of the package manager to allow downloading only audited/trusted packages. Forcing end-users of the language to be responsible for audit of all dependencies is impractical.
Discussion: https://news.ycombinator.com/item?id=49372853
Direct post link: https://blog.rust-lang.org/2026/08/20/supply-chain-attack-on...
Other vendor posts:
* https://www.stepsecurity.io/blog/arrayref-rust-crate-supply-...
* https://research.jfrog.com/post/arrayref-proc-macro1-crates-...
* https://www.aikido.dev/blog/two-popular-rust-crates-arrayref...
Indeed, the solution is to get away from the wild soup of author-managed dependencies and go with something with an audited collection of software that is maintained by separate human beings from the known-vulnerable hackers writing the software.
And indeed Go and .NET and Java all qualify. But the gold standard here is Debian and all its downstreams.
Turns out, it just doesn't make sense to be an independent open source developer of a critical dependency anymore. You can write the software yourself, but you can't publish it yourself.
All the Rust library crate developers will have to get together and start their own software foundation.
I personally don't believe the standard library argument is very convincing, because even with Java the latest newly added HTTP client has some blatant problems that require you to go with a wrapper like Methanol.
You can't disable build.rs, Rust Analyzer executes proc macros as soon as you open project, minimum days since release has not yet made it into stable (though luckily it is planned for 1.100), and the number of project dependencies goes easily into several hundreds (still better than npm's few thousands, though, and I know that some deps are from Cargo workspaces), and on top of that, lots of packages stick to 0.x version for years.
Personally I see Rust community's packaging culture more akin to that of JS than to Java et al.
And better than npm are you fucking serious? The most widely used ecosystem on earth by far that actually gets used in prod vs your “theories” and ego??? LOL
Why do so many languages fall intp this horrible practice?
https://www.youtube.com/watch?v=E82ly38YEEQ
Summary: it's cultural. Rust likely inherited the practice from Nodejs, who inherited it from Ruby. I think in Rust online spaces in particular there is also this undercurrent of "you're not smart enough to use certain parts of the language, so download libraries that handle that stuff for you."
With NPM and Rust's focus on project's level dependencies, there's no longer emphasis on API stability. Instead we have breakage every months, forcing everyone on the upgrade treadmill. It's easier to audit C library because they focus mostly on security updates instead of redesigning the API for the nth time.
It never ends .·°՞(っ-ᯅ-ς)՞°·.
Nothing is preventing anyone from distributing Rust crates as standard system packages, like some distros do with Python.
I think Debian is actually shipping rust libs in their repositories: https://wiki.debian.org/Rust Now Rust developers can also enjoy having to deal with popular packages that are years out of date/missing from popular repositories/getting deleted.
I understand not every language can have Go's amazing stdlib, but I would much prefer Pyhton's approach where every now and then some package/function from the stdlib gets deprecated/removed. Rust's 3rd party ecosystem is the worst thing from the language, worse than the compile times.
Writing macros in rust is a pretty horrible experience but it's not difficult
The better practice is for bigger projects to form collecting such small utilities into larger utility libraries that have some organization, security practices, code reviews, etc. Good examples are Java's Apache Commons or C++'s Boost.
Yes, we can argue about the culture of package management (as some of us have with especially npm from day one), but it's done, and your colleagues or AI sidekicks cannot be trusted not to download whatever and try to build and run it. All you can do is limit the effective blast radius.
I wonder if this is not in part the same situation as we arguably had with the xz compromise: some imminent change would have made the attack harder (in xz's case, IIRC a change to systemd to dlopen the compression library instead of directly depending on it), and the attacker rushed before the opportunity window closed.
Granted some human will likely have to review it. Or packages flagged by Al could require users to explicitly allowlist them.
Many things I run I want to limit to r/w a single dir, and to have to request permission to make network calls.
1. There isn’t a single universal standard for sandboxing across all the different platforms that are supported by Rust.
2. Even if there were, if you’re compiling untrusted code then why would you trust the built output?
If you’re building the create then I’d argue that any preventative steps afterwards is akin to closing the barn door after the horse has already bolted.
Yeah, let's hold up the entire world of offensive cybersecurity capability while you work on that. Sure they will wait.
> 2. Even if there were, if you’re compiling untrusted code then why would you trust the built output?
You don't. You sandbox the hell out of it too.
Nice sarcasm but you’re not actually addressing a solution to the problem I raised.
> You don't. You sandbox the hell out of it too.
So you’re now saying it’s ok to have exploits compiled into your application as long as it’s sandboxed?
I wonder how customers of your application feel about that? I’m certainly not going to be entering my bank details into your ecommerce platform (to give just one obvious example why your suggestion wouldn’t work).
Arguing that compromised code is safer when sandboxed really misses the real problem: that you’re running compromised code in the first place. Hence my analogy of closing the barn door after the horse has already bolted.
What we need is to ensure we have stronger safeguards in place to prevent bad code from reaching build pipelines. If it’s in the build pipelines then we’ve already lost.
I get why people argue about sandboxing, it’s an easier problem to solve. But you still end up with a compromised artefact which you cannot ship. So the benefit is negligible.
Are you not concerned about an asteroid impact rendering you extinct during your next Rust build? Why not? That's the same level of relevance as your supposed concern.
> So you’re now saying it’s ok to have exploits compiled into your application as long as it’s sandboxed?
You need to act like it is compromised in all cases, just like everything else.
> I wonder how customers of your application feel about that? I’m certainly not going to be entering my bank details into your ecommerce platform (to give just one obvious example why your suggestion wouldn’t work).
So your version is "we trust everyone and don't verify anything". Yeah, that's going to work. By your "logic" firewalls wouldn't be necessary.
That’s a strawman argument and you’re still dodging the question.
> You need to act like it is compromised in all cases, just like everything else.
No. I act like compromised code is a legitimate risk regardless of how well your build pipeline is sandboxed.
I don’t understand why this is a hard concept for you to grasp.
> So your version is "we trust everyone and don't verify anything". Yeah, that's going to work. By your "logic" firewalls wouldn't be necessary.
That’s the literal opposite of my point (as well as another strawman).
I said OS level. It's something I should easily be able to do via the OS capabilities that would work for rust, npm, etc.
I would use it not only for rust builds but for nearly every app on my computer.
Because we shall all trust the Cargo/pip/npm repository that nobody cares about ?
¹ https://rust-lang.github.io/goals/2024h2/sandboxed-build-scr...
This is the kind of decision users likely don’t understand without looking at the source code of a crate and it’s bad UX to push it to be their responsibility.
I would argue that, if a build script doesn’t work in the setting, then it doesn’t deserve to be installable by a default cargo command.
This seems like an excuse, not an actual objection.
Linux can do seccomp or Landlock or gVisor or a combination. Seccomp and gVisor need no privileges. Windows has its internal weird mechanisms. Mac has sandbox-exec.
Cargo could easily pick an appropriate sandbox for each major platform and ship it by default.
> If you only care about Unix, then you can do this yourself today by building code in your sandbox of choice.
This is ridiculous. The sandbox should not have network access, but cargo needs network access to download the package in the first place.
Naturally. Use `cargo fetch` to download a package locally without invoking any build step: https://doc.rust-lang.org/cargo/commands/cargo-fetch.html
(Oh and btw, proc macros also run arbitrary code.)
(It mostly boils down to "somebody needs to do it". I'd really like proc macros precompiled to wasm by crates.io…)
wasm has the additional benefits of allowing something like pre-compiling binaries once on a central server and thus skipping the CPU cycles for compiling all the macro crate dependencies on every crate compile. So even the people who don't care much about the security benefits have a reason. I'm kind of the "don't care about security" side because (as said elsewhere) it's trivial (https://docs.rs/ctor/latest/ctor/) for any macro to insert code that will run when compiled binaries/tests are run. You need the entire thing in a sandbox anyway, not just the macros, and that's something that can't be provided by cargo by default.
There are still elephants (or rather mammoths) in the room for supply chain security, such as having 1500 nested deps for a standard project. It’s like we never woke up from the nightmare of leftpad.
If your project had code from 100s of individuals, new versions can be pushed instantly, and nobody wants to review the code, then you have a time bomb. And also other problems.
Cargo, please give me a way to whitelist build.rs.
See https://embarkstudios.github.io/cargo-deny/checks/bans/cfg.h...
> Asking the user to not make mistakes is the c++ approach to security
Then demand crates.io do better by actually curating every version of every published crate.
Audit and limit crates (cargo-deny &| cargo-crev, && --offline) until tools exist to audit build.rs safety semi-automatically ($$ safeguard.sh maybe).
The root problem is two parts:
1. crates.io doesn't do mandatory curation. Lack of curation is fail. It's time-consuming and costly for reviewers without a doubt, but so is letting an ecosystem gain maximum entropy (go to shit) by Tragedy of the Commons depending entirely on the honor system. Name squatting, low-quality, unmaintained, typosquatting, and malware are the consequences of too much self-service / semi-self-service freedom.
2. Many, many cargo subcommands are over-eager to run build.rs because it assumes trusted crates:
In an untrusted/uncurated crates world and a build.rs exists or exists in a selected dependency, it shouldn't run at cargo-add time (unless it must). If it exists, on first run, it should be presented to the user in a viewer for manual review unless a magic CLI flag/env var is specified to accept it.
These 2 factors combined appear to create a Swiss cheese holes failure mode for running arbitrary crate `cargo add`. I have confidence a suitable add-in workflow &| standard command &| repository workflow will be adjusted to reduce the attack surface of the ecosystem.
It isn't so much a question of sandboxing build.rs, as fundamentally changing the way that foreign dependencies are integrated into the rust toolchain (i.e. moving from a rust-centric system like Cargo to something more general like buck2)
Which is the result of how bad dependency managment is outside (i.e. DLL-hell).
Pretty much all other use cases of build.rs can be sandboxed. Well, there is sqlx that wants to connect to database at expansion time to compile check-queries (yew).
Although I suppose that's still doing a lot of shenanigans at compile time. It could be sandboxed pretty well (theoretically). I'd hate to give it up completely though, getting a compile time error when SELECT query params or return values have type mismatches is extremely nice.
Any C library that's also packaged by debian supports being built under these conditions because it's required for everything except non-free packages: https://www.debian.org/doc/debian-policy/ch-source.html#main...
As the link you posted mentions, you need a tiny bit more than that: you also need write access to the temporary directory (/tmp and similar). Many build tools temporarily store files there; for instance, unless things have changed since I last looked, if you don't use the -pipe argument the C compiler stores its temporary intermediate files (preprocessor output, assembler input) there.
To solve the problem of native/C dependencies, they just bundle pre-compiled libraries as data files.
Rust could do the same thing.
It doesn't really matter anyway, because nobody is reading anything in their dependencies before it gets downloaded. Malware can also be hiding in plain sight in source code, as this attack and many others shows.
Here is my learning building PMG:
Sandboxing is good when the workload is predictable, and the goal of sandbox is to guard against exploitation of vulnerabilities, like sandbox protecting chrome tabs (renderers). But unfortunately build scripts are not predictable, at least not in npm/pypi world and I have seen build scripts doing weirdest of the things which is no different from malware. When popular packages do weird things, build breaks and users end up turning off the sandbox. This is a perpetual problem to deal with while building sandbox (or any least privilege solution) to protect unbounded workloads.
https://github.com/safedep/pmg
Id like to add project provided runtime capabilities/permissions (eg. apparmor profiles) to the list to.
Maybe the day will come, where projects not providing these things will be considered broken, like nix does.
The vast majority of crates don't need build scripts, so it is vaguely feasible to audit the list of crates you use that might need them.
I think we may need to enter a world where there are fewer, heavily audited, libraries and dependency depth is limited. Allowing unvetted dependencies to be installed by default is not a good way forward. App stores have a similar problem and I think a lot of lessons have been learned there that can be applied.
Functions should only have access to their arguments. Nothing more. We need to end ambient authority.
It guarantees that pure functions are pure.
https://www.microsoft.com/en-us/research/publication/safe-ha...
https://downloads.haskell.org/ghc/latest/docs/users_guide/ex...
Yes, I want this but in a fast, compiled systems language like rust.
Rust certainly borrows from Haskell. Like all good languages. But I’m not aware of Haskell beating rust in program performance & memory usage benchmarks.
> Like lazy evaluation and memoisation
Yes, that causes performance impact. If you don't want the performance impact then don't write code that uses those behaviors. Sure, that rules out large parts of the ecosystem, but I said Haskell was a systems language not that its ecosystem was generally suitable for systems programming.
> however Haskell manages memory
No, Haskell's memory manager is world class, with two (at least) tunable garbage collectors.
> But I’m not aware of Haskell beating rust in program performance & memory usage benchmarks
Nor am I!
> If Haskell were a viable systems language, I suspect the kernel would not have been converted to C.
I suspect they converted it to C because you can't write a kernel in a managed language with a garbage collector.
Often they will just start attacking other languages.
Egomaniacs working on Rust. Arrogant idiots
Are there any comparisons between the state of the stdlib in C++ vs. Rust? I’d think that would serve as an excellent jumping off point to start chipping away.
That doesn't mean you have to import nonsense or thousands of dependencies. The top hundred are efficient, concise, well-maintained crates of the quality you'd get in a large std like Go or Swift, sometimes even higher as people can write cleaner/faster implementations that they otherwise wouldn't (or wouldn't be used over std). And those languages make breaking changes! A mostly stable language like C++ is full of junk, like <regex> or even unordered_map – c.f. Rust wholesale reimplemented HashMap 6–9 months after SwissTable released. Partly because it's easier to express the encapsulation you need, but there are hundreds of other design/interface decisions made deliberately to not constrain forwards compatibility, including a smaller std.
However, that's no excuse to have a worse developer experience in this area. We need better tools to vet and communicate the quality of a crate and its supply-chain, like community-curated or even additional org-maintained crates, and maybe a handful delivered precompiled in the default rustup distribution (but which could change over time). I don't think they need to be added to std itself though
Use containers for development. And reduce the amount of third party deps you import into your projects. This is only going to get worse.
This is really a cultural problem not a technical one.
In the mean time I think it's possible to hack Cargo and run all build scripts in a microVM. The blast radius will be limited to malicious code in the binary instead of uploading all your CI secrets and deleting the whole hard drive.
It's not the only way. You can sandbox processes. I think sandboxing is the much more reasonable approach, because in the end of the day, there are still closed-source software products where you can't demand that the manufacturers use certain language safety features.
So if your software needs to make HTTP requests and read the filesystem then the malware will be able to make HTTP requests and read the filesystem, which is enough for a ton of malware. Sure, maybe you can limit the directories it can access a bit and possibly some sort of network filtering, but it isn't a silver bullet.
Capability security in-language would make a huge difference, because the more granular you "sandbox" the less likely it is that the compromised component has the access it wants. For example if the HTTP client library is compromised maybe it can't access the filesystem so can't steal your cookies. Or maybe like in this case no permissions were needed at all and despite the process having enough capabilities for malware this malware can at worst return bad values and try to chain this to an exploit which is far more difficult than just running it itself.
Then the problem is the language, as the grandparent observes.
It's like if `git clone` ran random stuff from the cloned repo.
This is partially because for nearly every project the build system is something they need and don't care about. When they need something weird they hack just enough to make it work and never ask "how should the build system change so that this was a property instead of running code", and thus build systems are slow to improve. In a number of cases the build system did have a way to do that thing, but the person didn't know about it.
There are also a lot of code generators out there. I have yet to see a large project which didn't have their own code generator for something specific to their project (protobuf is an example from Google that has escaped and become useful elsewhere, but there are many others that are specific to one project. Yacc is from the 1970s, and stands for "yet another compiler compiler" - implying the idea was already common 50 years ago). You cannot have/use these useful tools without running arbitrary code.
Nearly every time that you see a build script in use it's to support building C code, which has no standard build process and which often involves executing shell commands, and the shell itself is an arbitrary code execution environment with arbitrary access to the network.
https://github.com/brianv0/formwork
https://hale-lang.org/proof/
The thing is, trust is a massive optimization. If you believe code from X is competently made and not malicious, you can accept that code without any additional overhead. (Building trust is a significant effort, but trust begets trust, so there's a resonable path from starting small to scaling.)
It's not that these mechanisms that require less trust (none of them eliminate it, BTW, when you examine them) can't work. It's that they'll inevitably be outcompeted by trust-based systems (in most domains).
E.g., compare the effort of creating a new, full featured, general purpose language, promoting it, building and maintaining all the tooling and documentation around it to support it, etc to the effort of creating and maintaining a curated package repository of trusted packages (and/or adding trusted code to extend the standard library) for existing language X.
(It's not clear to me you could provide enough value from a new language focused on effects that you could convince any but a few from adopting it at all, even if it dropped from the sky fully formed with all tools, docs, etc. in place from day one.)
I can very easily build a highly functional, pleasant to use Apple platform app with 5 or fewer top level dependencies. In many cases, I reach for between 0-2 total.
There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
That would make it unnecessary to pull in foreign dependencies in the overwhelming majority of projects. What few do get pulled in becomes lightweight, easily verifiable syntactic sugar or libraries with purposes too niche to be worth targeting.
Of course this approach can go wrong too. You could easily end up with a monster like Boost, but that comes down to project administration keeping creep under control and proper modular design.
Huge standard libraries make sense for Java, Go, Apple platform etc. because they are developed by giant enterprises that can manage the overhead. Thinner modular systems are a more natural fit for open source development.
While overhead is a real concern, I think it’s often blown out of proportion. Once a language achieves a certain baseline of stability and isn’t in constant flux and the standard library matures, changes become infrequent and maintenance load is low. The work is heavily front-loaded.
Async rust is a jungle of weird compatibility questions. People treat async runtimes like sports teams. (I know I do). I wish it were more like Nodejs where async is just built in.
That’s not what I call well featured.
Most programs should need close to zero dependencies.
Of course they’re free to do what they want, and there’s the question of maintenance overhead[0], but while https://blessed.rs/ needs to exist, Rust’s standard library is deficient in my eyes.
[0] One idea would be to treat a language more like an Linux distro and have library maintainers like package maintainers but a unified quality, audit, and release process
You will be trading a stable and vetted huge standard library for 200 small packages that may all be targets for supply-chain attack. Open-source projects don't all have the resources unfortunately to keep track of all their dependencies, so I'm not sure it's necessarily a more natural fit.
I think Go apps tend to have better dependency hygiene because the language has a better standard library, which results in better culture around dependencies.
I also think it’s frustrating that both cargo and npm totally ignore the decades of prior art from Linux distributions that have figured out good ways to improve dependency management. We’d be in better shape if there was a community curated subset of known-good dependencies that are release-managed together that the broader ecosystem could build on, sort of like Ubuntu having “main” and “universe”.
Certainly having a larger stdlib helps Go projects keep their deps down, but I don't think it's the real reason.
One of the core tenets of early Go was the maxim "a little copying is better than a little dependency".
Probably because of this stance, they didn't even HAVE a dependency-management solution for years
I strongly agree the fewer dependencies the better, on average.
Personally I came from a C background so I tended to use deps more sparingly.
In the end it's a bit of a balancing act—lots of dependencies is a larger opportunity for these kind of supply chain attacks to affect you. Copying stuff into your repo protects you from that, but it also makes it way more likely that you'll miss security fixes, unless you're actively looking for them. Re-implementing what you need is also viable but it slows down the dev process.
I disagree: having a big stdlib that encompasses a huge chunk of common functionality (logging, a plethora of network server/client protocols, etc) means that for basic needs, the question of 3rd party libs never arise, whereas for other language, the question is not if you need a dep, but which one. Making gorilla easier to find won't undo the cultural reluctance of adding a 3rd party dep when using the (very pragmatic, IMO) stdlib server is adequate to the task.
There’s no poking, unless you are into that sort of thing
I think the ease of adding dependencies does drive the idea of "microdependencies", where you have many dependencies that do one little thing, and then those dependencies likely have their own small dependencies.
Whereas in the C world, you'll probably have a smaller number of dependencies, but it's likely that some of them are very big dependencies that offer a lot of functionality (like glib, for example).
I could maybe see the argument that ease of adding dependencies discourages people from writing small things themselves. Good ol' left-pad comes to mind. In an ecosystem like C, I'd just write that algorithm myself, over and over and over, because pulling in a dep just for that feels annoying.
I'm not sure that's a bad thing. I suspect with increased friction, users are more likely to scrutinise dependencies before importing them.
Rust meanwhile seems to be following some "no standard library" philosophy.
I feel like there's an opportunity out there to become like an amoeba: fund and build a third party Rust standard library, absorb absolutely everyone who is so grateful to have a library, and then get so big absorb Rust itself.
As an embedded dev, I personally find it very useful to compile Rust programs without a standard library. I would not use C# or Python for an embedded project for the reasons you probably prefer them for your projects.
But you gotta admit people trying to build GUI libraries and using linked lists don't fit.
This does sound like an excellent way of improving automated package checks, even if it does result in some false positives and false negatives. For all sorts of packages, not just code dependencies.
The proposal ends up amounting to either shifting stuff around, or asking to other people to put in more effort? Into a project that usually doesn't pay (programming languages and runtimes)
Which by the way, asking other people to do stuff for free IS the attack vector, if you are the kind of dev or company that gets hit by these, your ultimate root cause is that your business strategy is using code without paying for it. No such thing as a free lunch, you will pay for it, among other things, with an increased cybersecurity risk.
Packages can release updates arbitrarily, while standard libraries tend to have longer release cadences.
It's also more difficult to put arbitrary code into an stdlib, because stdlibs are scrutinized better.
Also, it's harder for some random anonymous developer to gain push access to stdlib repository.
One of the reasons so much Rust code is in libraries is not because there are no people to write it (duh), but because putting these in std commits maintainers to keeping backwards compatibility and slows down included package's release cycle.
The big reason is money & time. Maybe less so today if you are happy with an AI generated stdlib, but I don't think it's fair to compare Rust (built by Mozilla, who's primary product is an open source web browser) to Swift/AppKit (built by Apple, 4.6T marketcap at the time of writing).
When Go was released it had an amazing stdlib, but that was because Google was funding it.
Look at C++'s long in the tooth STL.
You don't want to marry a language to fast aging libraries you have to support for eternity. Better libraries always naturally emerge.
Rust's decision here is fine.
The only thing I'd like to see is the ability to programmatically limit transitive dependency count or depth in Cargo. I'd also like crates to specify whether they limit their own deps and whether they have panick-y behavior or not.
https://crates.io/
https://crates.io/crates/once_cell
https://doc.rust-lang.org/std/cell/struct.OnceCell.html
Not everything should be eligible for this treatment. Certainly not an HTTP library or something without extremely widespread applicability.
Nearly all of my projects used once_cell, so it's good to see that pulled in. I wouldn't want to see anyhow, thiserror, anything opinionated, or anything domain specific in the std. That's a weight you have to bear forever.
Remember how long Python 2 -> 3 took, and look at the ancient and awful stuff in Python 3's standard library. Rust is not the right language for this.
https://github.com/rust-stdx/stdx
https://news.ycombinator.com/item?id=48571266
Note that there are baby versions of this that are uncontentious, for example
https://blessed.rs/crates
this is missing the LTS release. but it is a curated group of libraries that are relatively uncontentious to recommend.
One can very simply....make breaking changes. Yes it will require some work to update but that's already the case when you upgrade library versions.
The dependency hell doesn't help anyone here, the downside is much worse - lots of bloat, overgeneric libraries and awful supplychain security.
For rust, this would mean something like adding std = 1.0.6 to your cargo.toml. The advantage is it means the standard library can deprecate and replace features. Upgrading std is an explicit step by the developer. Annoying, but in the age of LLMs it should be pretty easy.
The one big question I can’t answer is what happens if your dependencies use a different version of std? Are std v1 Strings compatible with std v2 strings? Oof.
C++ went through that around C++11, where some types had to have ABI breaks (on some implementations, such as that of GCC's libstdc++). It has been a while, but as I remember it std::string was affected since it went from COW to SSO. Map might have been affected too (don't quite remember). It was a mess.
Rust can link multiple semver versions of the same dependency, but you can't pass data structures between those versions. For transitive dependencies that usually isn't a big deal if your direct dependencies use them internally rather than exposing the types in their own APIs. But std contain vocabulary types that everyone uses (Option, Result, String, etc) so that would really not work.
Swift was intended from the start to not only replace Objective-C and all of its use cases, but also take on new use cases and bring a number of features from other languages that were newer and had different dominant paradigms.
Certainly having a juggernaut like Apple behind it has been instrumental in its success in achieving those goals. That said, a community-lead project with similar aims could probably achieve those aims as well, given enough time.
I think it's just rare for enthusiast-led projects to set out to do such things. My theory on why things tend to go that way is that the types of people to start programming language projects tend to heavily lean nuts-and-bolts and theory with a purist/idealist sort of mentality that's more concerned ideological purity than practical usability.
So to sum this all up, a community-run practical, multi-purpose, batteries-included language likely needs to at least partially designed and helmed by product engineer types so it doesn't end up anemic and stuck in an ideological rut.
https://www.youtube.com/watch?v=GZOuz-SG7-g
Funny how you skipped "I should build my own batteries" and when straight to "increasing and centralizing the duties of your main gratis 'vendor'".
I'm happy to contribute to a larger effort but anything I can build on my own is going to be a thin, flimsy happy meal toy compared to something with the backing of a company or well organized FOSS project.
Also the challenge of building a generic module is much larger than building your own. A generic module needs to have flexibility for many different options and integrations, when you build your own you build just what you need and tightly integrate it with the product
True but I believe overestimated. Usually rather than building what's needed, what gets built is what is thought to be needed (often a substantially smaller subset), and then over time you end up building a markedly poor version of a generic module.
let's look at the actual package from OP
>https://docs.rs/arrayref/latest/arrayref/
>This package contains just four macros, which enable the creation of array references to portions of arrays or slices (or things that can be sliced).
I'm no rust programmer, but that doesn't sound like something that moves the needle. I talked about the ratio between value to risk being a relevant decision parameter, so getting infected by an http framework would be defensible, getting infected by adding this to a project to me is a PIP, and getting infected by installing leftpad or a custom cursor plugin in an IDE would be fireable.
>This package contains just four macros, which enable the creation of array references to portions of arrays or slices (or things that can be sliced).
The package in the OP is definitely not in the category of decades of development, it's closer to a leftpadism.
The rand crate is my favorite example of this. They have a whole bunch of different rngs. All rated by quality and performance. Some are csrngs and some aren’t. It’s a delight. “Batteries included” languages don’t come close.
Then there’s serde - which uses a clever technique with traits to allow compile time specialisation and optimisation of binary and json serialisers. Most people had no idea that was even possible in rust until someone in the community made it.
Maybe std should pick up the best crates from the community and bring them in house? This might be a good idea. But also maybe not. Since serde came out, several other crates have found ways to crush its performance numbers. Often by 2-5x if memory serves. What a relief we didn’t immortalise the slow version of serde, right?
I’m playing devils advocate here. I think std should be a bit bigger too. But there are real tradeoffs in doing so.
I think periodically rolling in the best crates is the way.
Don't immortalize anything, though. Always reserve the freedom to swap out integrated crates when something better rolls around. Negative impact can be reduced by limiting these kinds of changes to major new versions.
Another option is to swap internals while keeping the same public API. Apple has done this several times with Swift, resulting in massive speed improvements in things like string handling and JSON decoding without breaking anybody's code.
That's a non-starter, though. Once it's in std, it's not an "integrated crate", it's in std, just like any other part of it. Removing it is a breaking change, and I don't want a new incompatible Rust major version every year or two (or even five or ten, honestly). (Of course, Rust has editions to make breaking changes possible without affecting old code, but there are limits to what you can change in editions.)
Pretty much every language ecosystem fights really hard to maintain backward compatibility as they evolve, and every time they fail to do that, it causes headaches, sometimes for years. Python 2->3 is of course the canonical painful example of that, but there are others that are instructive if you only take the time to research.
(And before you mention something like Zig, which changes incompatibly frequently enough, remember that the author has made an active decision to remain pre-1.0 unstable. That's one reason why the Zig community isn't that large. That's not me looking down on them; that's an active choice they've made, and they're free to make that choice. But bigger/popular languages like Rust can't go back in time and decide to be like that.)
Yes, it’s annoying to have to make fixes around my codebases periodically, but to me that’s a hell of a lot less annoying than having to pull in a mile long list of dependencies to do anything of consequence. It’s also better than the bad parts of the language being frozen in time forever.
Apple breaks things with Swift not constantly but on occasion and while there’s some griping around it, it’s not a big deal. We make the needed changes and move on, knowing it’s enabling improvements that will be nice to have.
So sure, you want, say, a JSON library in Rust std? Who's going to shepherd it in and then maintain it? Well, you could hope that the people behind serde_json might want to do it, but what if they don't? Where are you going to get the people with the time to do it?
For something like Go, the answer is easy: tap into the money printer that is Google, and hire more people. But for Rust? Hell, the core team has probably only declined in number over the years (Mozilla layoffs, for one thing).
> There’s no reason why this can’t be replicated elsewhere. The key is to make the programming language reasonably robust with at least 80% of common non-UI dev needs built in and put the remaining 20% and UI bits into a small family of well-supported, community-embraced, preferably first party libraries.
I wholeheartedly agree. Most standard libraries out there (when I say most I easily mean of 99% languages that I know) have pitifully tiny implementations. They at most provide for string parsing, io, a bare networking layer (that's at best a wrapper around the POSIX syscalls), threading and that's it. If you want to do anything more complex, you either have to roll it yourself or use an assortment of the "blessed libs" for any given language, that may or may not work properly, and may or may not even work well with each other. I got pissed off by this at some point so I decided to go ahead and roll my own standard lib that would actually have everything I need to.. actually develop apps. Now I know most people probably aren't going to be using it (mainly because its functional API isn't what most C++ devs want) but for me it's actually been brilliant.
Primarily I think the underlying concern has to do with the pathway for authoring code. When you have contributors whose submissions are gated with a rigid third party process and where that third party is the one responsible for curating the code (as opposed to the author also being the curator as well as the publisher) then you have the possibility to catch a lot of wrongdoing before it succeeds.
There are many issues that I've seen taken up with Boost, but my personal peeve is how it can make building projects that incorporate it a pain, both because of its sheer size but also because it can sometimes be difficult to appease with its own dependencies.
> There’s no reason why this can’t be replicated elsewhere.
It is replicated elsewhere and is not limited to Apple; I can do the same using a tech stack of perhaps 2x techs and very limited (manually added) deps. I'm thinking Lazarus, which comes with most things I'd need for most apps.
Nobody can agree on what those common things are in a general purpose langusge though. It works for something like Go, because it is largely used for servers and command line tools from what I can tell.
For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.
And even if did put some of those things in std, that is stable forever. That is how C++ ended up with an unusably slow regex implementation in the standard library. And it isn't fixable. Everyone just ignores it and uses a third party library like PCRE, RE2, etc instead (depending on their specific needs). I would not expect web or GUI things in the standard library for this reason, those standards change too frequently for that to be stable forever.
And even something like Python, which is known for data science and machine learning, doesn't have core libraries for those activities in the standard library. Numpy, Pandas, Scipy, Pytorch etc are all third party projects.
- not obviously/generally useful (i.e. useless or too specific/should be a program not a library)
- obviously trivial (code it yourself when you need it/compose from other primitives)
- already available (open source) elsewhere by a credible team that supports and maintains it
Anything else… put it in the stdlib
It's a self-inflicted issue that most other languages with "batteries included" don't have since they will document breakage and upgrade paths when anything changes, or they are not impacted as much since they are using an intermediate language as interface and avoid most ABI issues.
The implementation is fixable though, you can break the ABI and address your issues. A lot of standard library defects are fixed in an unstable ABI mode they have, but that requires you to link it statically or dynamically and then ensure all your shared object dependencies are using the same exact version. It's not trivial for everyone, but many do this (usually the static version).
But even something like Python has lots of cruft in its standard library. And they are willing to make breaking changes. Nobody should use urllib.request for example, requests is a far better HTTP client library. To the point where the stdlib docs tell you so.
Well, then take the Lazarus / Free Pascal's approach and just put everything :-P. It isn't like code takes THAT much space.
If you install Lazarus you get a ton of stuff out of the box, aside from the crossplatform WYSIWYG RAD IDE and desktop application framework you also get 2D graphics libraries (with a bunch of image i/o), networking libraries, (de)compression libraries, a webapp framework, database clients, parsers for json, xml, markdown, javascript and a bunch of other stuff (even a parser for the language itself), a TUI framework, bindings for a bunch of external libraries and other stuff.
And all that stuff (or at least most of it) work on all the platforms the compiler supports. Yes, using FPC you can probably (didn't try) make a pretty markdown viewer with inline JPG and PNG display support that runs on Windows 3.1 :-P.
Unfortunately some people are trying to break this and introduce online package management, but so far at least FPC and Lazarus comes out of the box with not just batteries included but an entire carton of batteries just in case :-P.
It, as you put it, works for Go because its designers took a hard stance and decided that it was a language for network systems and you're on your own if you want to make it do other things. Nobody needed to agree because it was already chosen for you.
> For Rust, well I would like to prioritise features for embedded development, someone else might want game dev features (geometry and physics stuff), yet another person wants data science things. And so on.
But does Rust need to be all things to all people? The Go ecosystem has a separate standard library under the Tinygo project for those who want to prioritize embedded development. Even if there was convergence on Rust as a language, it doesn't technically need one singular standard library. There can be a distribution for embedded developers, a distribution for game developers, etc.
Most of the things I wrote in Rust have less than 5 dependencies. Those 5 dependencies then depend on 20 other dependencies each. Those 20 dependencies have 2-3 other dependencies. And so on…
Arrayref seems somewhat niche, but its kind of in the standard library as of Jan 2026 as std::slice::as_array https://doc.rust-lang.org/std/primitive.slice.html#method.as...
There are plenty reasons. Just not technical reasons.
At the end of the day, people like to feel useful. The ecosystem that attracted communities are those ones where everyone can feel they're contributing back to the communities. That's why the most successful languages, save those backed by big companies, are filled with very small libraries.
No body gets excited making C#/nuget packages.
Rust is one of the better languages when it comes to batteries included in it's stdlib. Even then, it's impossible to have a 100% coverage batteries included language because nearly everyone on planet earth has some different/unique use case for their code that doesn't fit a stdlib.
No wonder competitive coding champions sometimes preferred Java as they don't had to spend time coding it from scratch.
Practically speaking the standard library is just meant as “what’s included in the default sysimage”.
This allows to have different standard libs for different use cases. Think of Linux distributions in a sense. There are already a couple of alternative “stdlibs” out there for different focuses.
Okay, but how do we decide which approach to a problem should be standardised?
Rust's approach of "provide a minimal core to build libraries on top of" provides a fairly significant benefit - flexibility.
A prominent example is async: Rust has three fairly prominent async runtimes (being Tokio for general server-side use, smol for a simple non-work-stealing approach and embassy for embedded). Which one do you make the default? Tokio is very complex and work-stealing is arguably a very bad default in terms of development complexity. smol doesn't fit some workloads because of a simpler execution model. Embassy is way lower level and much more constrained than most applications need. Standardising over one implementation instead of the current (even if still messy due to early portability woes) standardisation around traits and general async semantics would make switching runtimes more problematic, even though the general case is solved with the same approach as Python's
Database access is also a complicated topic. Rust has multiple fairly popular DB libraries. The first two that come to mind are SQLx and Diesel. Both are too complex to be a part of the standard library, both required a fair bit of breaking changes, and if you wanted a raw core to build on top of - you can't really build SQLx on the type of core e.g. Go's database/sql has, you'd be basically forced to implement it all from scratch to handle the compile time type checks anyway
RNG sounds simple, except there are good reasons behind both rand and fastrand existing, and behind rand not being stabilised at 1.0.
DateTime handling is notoriously complicated, has multiple competing implementations (chrono, time, jiff) with various trade offs
Where do you draw the line? What do you standardise and which implementation do you standardise against?
I've seen golang exemplified a lot in adjacent threads, but golang's standard library both has fairly significant issues (json, no set type, very limited CLI flag parsing, UUID took until 1.28) and is standardised around a relatively specific goal (unix server usage)
Dunno, I'm not entirely sold on the whole "extend the standard library" idea for a language meant to run both high level applications and embedded stuff. If anything, recommending blessed.rs more is a better way IMO
Having libraries allow the community to explore rather than saddling it with a bad default. See the number of http server frameworks etc.
The real solution here is not for the stdlib to become a black hole, swallowing up every half-way popular library a but rather for “rustaceans” to adopt a more security-conscious approach to writing programs- C programs typically have few and well-chosen dependencies. Some languages, like Odin, completely eschew package managers because they see the very real issues stemming from writing a program with scores or hundreds of dependencies.
The gains are just too numerous and important to reject the few rust spots on a "good enough" stdlib.
But well, the kind of people who nod a bit too much while reading https://boringtechnology.club/ aren't using Rust. They're on C99, ANSI CL, Ada, maybe Python, Perl or Tcl.
Also most of these dependencies provide a breadth of features that the end package does probably not need.
I think the sweet spot is having an ecosystem of "blessed" libraries that are reviewed for security and quality, but are versioned independently of the language, and maintained by subject matter experts.
It lets you take a slice `&[T]` and return a slice with a known number of elements`&[T; N]`. tiny-skia was probably using this to get some SIMD optimizations.
As of Jan 2026, this seems to be in the standard library under slice::as_array https://doc.rust-lang.org/std/primitive.slice.html#method.as...
It implements the base58 encoding, which is used primarily by... Bitcoin.
I'd love to see the explanation of why "a TUI file manager" needs a crate closely associated with crypto coins and not much else.
It is precisely this kind of thing that makes Rust a no-go for most enterprises.
"I just want a CLI tool."
"Congratulations, your servers are mining crypto!"
PS: bs58 is maintained by one anonymous person and their package is used in just about every Rust crypto library and hence application. Compromise his account in the same manner as the 'xz tools' attack and you could still billions in crypto!
I believe there's (several) open issues about changing this behavior of the lock file, but I have moved on from Rust so don't know the status of things.
That said, this is confusing behavior. I remember I first noticed it when a bunch of crates were showing up in my lock file for every OS under the sun, even though I was specifically using a "wayland" (linux-only) feature.
Not to defend Rust's crazy culture of dependencies, mind you.
I ran in to the lockfile bug myself a day ago and noticed that dependency-feature-flagged-crates show up in Cargo.lock, but their crates aren't downloaded.
[0] https://github.com/jarun/nnn/
https://github.com/hbbio/rc
In my opinion, library authors should really minimize the amount of dependencies they have. Back to TypeScript, we are also the authors of https://github.com/okcontract/cells and we made a point of almost not having dependencies or even devDependencies.
(Rust even enforces for proc macros, which is why so many of the crates in your example end with “derive” or “macros.”)
Just because you don't develop a package manager for your language, doesn't mean someone else won't. See NPM.
Just because you expect people to behave like X doesn't mean they will. In this case people will automate packaging of artifacts.
NPM was a third party manager (it's not part of EcmaScript nor was it part of Node.js at start) and so is Maven. Reputation matters more than origin.
Those who do not know history are doomed to rediscover it.
The fact is that your dependencies are your responsibility. You should be staying up to date on what you depend on and their vulnerabilities. Most vulnerabilities affect specific code paths which may not be present in your code base and do not require mitigation. These are decisions you can make by understanding what your dependencies are instead of offloading to automated package management systems.
NPM has been the posterchild of these types of issues but the dependency stack I saw everytime I compiled even simple programs in Rust made me think Rust would be impacted next. Personally I think third party modules that are effectively universally used (axum, tokio, serde) should be integrated directly into the language.
The bad package version also just disappears from crates.io [2] and there's no security advisory there either [3]
[1]: https://web.archive.org/web/20260820145918/https://github.co...
[2]: https://crates.io/crates/arrayref/versions
[3]: https://crates.io/crates/arrayref/security (I'd give an Wayback link but they've broken that https://web.archive.org/web/20260820150747/https://crates.io...)
Google should read this too. They simply remove Android apps and Chrome extensions without a single word. No page explaining why they removed it.
https://blog.rust-lang.org/2026/02/13/crates.io-malicious-cr...
Not having an advisory INSTANTLY available isn't a sign of a decaying org. Chill.
So, yanked crate versions do have an indication on crates.io. (Here's an example: [1]) The Rust blog post uses the word "deleted", and I'm guessing a bit here, but I think they mean that literally, and that the version here is deleted, not yanked. And I think that would be more appropriate: a yanked crate is still downloadable by cargo, if your lockfile is locked to it already; yanking only prevents lockfiles from newly automatically acquiring a lock on that version. That wouldn't be desirable in the case of a compromised crate: you don't want a download occurring at all. The tradeoff of "break those with locks on the crate" tips to being worth it.
That said, I agree with you, though: I think this state (if it is "deleted" and not yanked) should be plainly indicated on the crate's versions page. (Even better would be if it came with a link to, e.g., the blog post or a RUSTSEC so that you could find out why.) (& I think perhaps the docs for yank should point out whether or not it is appropriate in the "compromised crate" scenario, and if not, what to do instead.)
[1]: https://crates.io/crates/aes/versions
Sure you do; you just don’t want the crate to be made available to people trying to make use of the crate as regular downstream consumers. You still want the crate available for download for analysis. You especially want to be able to deterministically reproduce the vulnerable version of your software that you already built.
I would posit that most package ecosystems should treat “contains an exploit” as its own package state; mostly ignore them; but still be aware of their existence.
In other words, if there’s any other non-exploited option under your version constraints, the dep should resolve to the nearest non-exploited version, even if older. But if it’s the only version, or if you have pinned the exploited version, then the packager should see it, but normally refuse to interact with it—i.e. refuse to lock to it if it’s the only version; or refuse to fetch it if it’s the locked version.
I say “normally” because you should be able to bypass this with a flag / env-var to the packager that basically means “I’m building this for forensic analysis, not for running.”
I don't mind malware being lumped in to that - the place to look up code for historical or analysis reasons ought to be version control, not crates.io
I would suggest that such packages should simply require authenticated access + manual “write an email to ask to have your account be put on a whitelist”-driven authorization. Such a package would still be there (in its redacted state); but attempts to download it would be a 401 (if unauthenticated) or 403 (if not on the whitelist) error response, which the packaging tooling would need to know how to handle.
(In any case, this “redacted” package state and my “exploited” package state could be two variants of the same logic, both following the same approach to locking/resolution, both requiring the flag to indicate that you understand what you’re doing and aren’t just going to run the resulting program. The only difference is that once you pass that flag, an “exploited” package would then just work, for anyone; while a “redacted” package would still have one more gate standing (server-side AAA) in the way.
This doesn't make sense if the legal reason is accidental release of secrets or PI, in which case you absolutely do not want to leave it up.
I'm not talking about things like github tokens here, those can be revoked. But actual secrets or PI where that isn't possible.
I see the "exploit" case as a strict subset of redaction in which you do want to continue serving downloads to parties that explicitly acknowledge that they understand the classification of the artifact in question.
Yes, I agree with you that analysis is good.
We would probably be open to sharing files with those who ask. Probably, I don't know what other team members would think about that. This roughly matches the workflow you propose below where people can ask for access to these things, just a bit more manual.
Any way of keeping it around on the server in a way that is fetchable by cargo is a complete non-starter. Bright red lines are useful in security: _the file is gone_ is way easier to keep secure than "the file is still there in a way that might be fetched if you have the right flags set" now means that that flags mechanism is now security sensitive.
If you'd like us to have a nice page that shows deleted-for-security crates, that's probably a proposal we'd welcome, but it is not really something anyone has time to work on themselves. Right now we just remove them from the db after taking a backup.
For example, if we were talking about a human-readable /crates/* page, returning 410 Gone with a HTML page explaining the situation to a human, that page could have a JSON-LD snippet explaining the same to a computer.
What makes you say we were unprepared? We have been deleting malicious crates for a while. This is the first time those crates have wormed their way into an actual real crate that people use, but most of the playbook here was the usual one. We take a snapshot copy of the crate and then purge it from the system.
The security page thing is because the rustsec maintainers were not around or a part of the response as it was occurring. It's pretty normal for a security advisory on rustsec to take some time to merge. We should probably get everyone on the same page about when people not on the rustsec team can merge security advisories, because I do agree that getting them merged quickly is important sometimes.
The main crate entry should also contain a security advisory at top. I looked at the crate and it just looked normal; I had to dig to find the exact impact surface, and if I wasn't informed via secondary means (hackernews) I would not have known. This is unacceptable for a mature package management system.
Luckily I was unaffected in this case.
The crate does have an advisories/"security" page on crates.io.
We could try and show the existence of a security-deleted crate on the page. This is not a priority for anyone, and I remain unconvinced that it needs to be (not that that is my decision anyway). File an issue and make your case to the crates.io team.
I won't spoil the claim in the video about what the dependency number is correlated with, and I'm not even sure how true it is in general, but it's very interesting.
we put together a Scroll on how this manifested with node, if anyone finds it helpful to understand that attack vectors & mitigation steps: https://endash.us/toolkit/items/mini-shai-halud