With the recent incidents affecting Trivy and litellm, I find it extremely useful to have a guide on what to do to secure your release process.
The advices here are really solid and actionable, and I would suggest any team to read them, and implement them if possible.
The scary part with supply chain security is that we are only as secure as our dependencies, and if the platform you’re using has non secure defaults, the efforts to secure the full chain are that much higher.
One (amongst other) big problem with current software supply chain is that a lot of tools and dependencies are downloaded (eg from GitHub releases) without any validation that it was published by the expected author. That's why I'm working on an open source, auditable, accountless, self hostable, multi sig file authentication solution. The multi sig approach can protect against axios-like breaches. If this is of interest to you, take a look at https://asfaload.com/
The open source ecosystem has come very far and proven to be resilient. And while trust will remain a crucial part of any ecosystem, we urgently need to improve our tools and practices when it comes to sandboxing 3rd party code.
Almost every time I bump into uv in project work, the touted benefit is that it makes it easier to run projects with different python versions and avoiding clashes of 3rd dependencies - basically pyenv + venv + speed.
That sends a cold shiver down my spine, because it tells me that people are running all these different tools on their host machine with zero sandboxing.
If anyone from Astral sees this: at this level of effort, how do you deal with the enormous dependence on Github itself? You maintain social connections with upstream, and with PyPA... what if Github is compromised/buggy and changes the effect of some setting you depend on?
What if? GitHub has is extremely buggy! I'm getting increasingly frustrated with the paper cuts that have become endemic across the entire platform. For example its not uncommon for one of our workflows to fail when cloning a branches of the repo they are running in.
We talk to GitHub as well! You're right that they are an enormous and critical dependency, and we pay close attention to the changes they make to their platform.
The only binaries of uv in the world you can get that were full source bootstrapped from signed package commits to signed reviews to multi-signed deterministic artifacts are the ones from my teammates and I at stagex.
All keys on geodistributed smartcards held by maintainers tied to a web of trust going back 25 years with over 5000 keys.
Though thankful for clients that let individual maintainers work on stagex part time once in a while, we have had one donation ever for $50 as a project. (thanks)
Why is it a bunch of mostly unpaid volunteer hackers are putting more effort into supply chain security than OpenAI.
>Why is it a bunch of mostly unpaid volunteer hackers are putting more effort into supply chain security than OpenAI.
Unpaid volunteer hackers provide their work for free under licenses designed for the purpose of allowing companies like OpenAI to use their work without paying or contributing in any form. OpenAI wants to make the most money. Why would they spend any time or money on something they can get for free?
> Why is it a bunch of mostly unpaid volunteer hackers are putting more effort into supply chain security than OpenAI.
Didn't the acquisition only happen a few weeks ago? Wouldn't it be more alarming if OpenAI had gone in and forced them to change their build process? Unless you're claiming that the article is lying about this being a description of what they've already been doing for a while (which seems a bit outlandish without more evidence), it's not clear to me why you're attributing this process to the parent company.
Don't get me wrong; there's plenty you can criticize OpenAI over, and I'm not taking a stance on your technical claims, but it seems somewhat disingenuous to phrase it like this.
>Why is it a bunch of mostly unpaid volunteer hackers are putting more effort into supply chain security than OpenAI.
To be frank. Because more effort doesn't actually mean that something is more secure. Just because you check extra things or take extra steps that doesn't mean it actually results in tangibly better security.
> All keys on geodistributed smartcards held by maintainers tied to a web of trust going back 25 years with over 5000 keys.
Neither the age nor the cardinality of the key graph tells me anything if I don’t trust the maintainers themselves; given that you’re fundamentally providing third-party builds, what’s the threat model you’re addressing?
It’s worth nothing that all builds of uv come from a locked resolution and, as mentioned in TFA, you can get signed artifacts from us. So I’m very murky on the value of signed package commits that come from a different set of identities than the ones actually building the software.
I don't think you are annoyed. You have done this to produce a reproducible linux distribution which your partners sell support for.
I wouldn't find this annoying at all - I would expect to have to do this for hundreds of packages.
Without unpaid volunteers things like Debian do not exist. Don't malign the situation and circumstances of other projects, especially if they are your competitors.
Compete by being better, not by complaining louder.
OpenClaw has been an outstanding success, it is providing people the ability to leak their keys, secrets, and personal data, and allowing people to be subject to an incredible number of supply chain attacks when its users have felt their attack surface was just too low.
Your efforts have been on increasing security and reducing supply chain attacks, when the market is strongly signaling to you that people want reduced security and more supply chain attacks!
The entire paragraph about version pinning using hashes (and using a map lookup for in-workflow binary deps) reminds me that software engineers are forever doomed to reinvent worse versions of nixpkgs and flakes.
I don't even love Nix, it's full of pitfalls and weirdnesses, but it provides so much by-default immutability and reproducibility that I sometimes forget how others need to rediscover this stuff from first principles every time a supply chain attack makes the news.
I don't have much experience with GitHub's CI offering. But if this is an accurate description of the steps you need to take to use it securely ... then I don't think it can, in fact, ever be used securely.
Even if you trust Microsoft's cloud engineering on the backend, this is a system that does not appear to follow even the most basic principles of privilege and isolation? I'm not sure why you would even try to build "supply-chain security" on top of this.
Pinning github actions by commit SHA does not solve the supply chain problem if the pinned action itself is pulling in other dependencies which themselves could be compromised. An action can pull in a docker image as a dependency for example. It is effectively security theatre. The real fix is owning the code that runs in your CI pipelines. Or fork the action itself and maintain it as part of your infrastructure.
I maintain `repomatic`, a Python CLI + reusable workflows. It bakes most of the practices from this post into a drop-in setup for Python projects (uv-based, but works for others too). The goal is to make the secure default the easy default for maintainers who just want to ship packages. Also addresses a lot of GitHub Actions own shortcomings.
But thanks to the article I added a new check for the fork PR workflow approval policy.
43 comments
[ 2.9 ms ] story [ 57.1 ms ] threadThe advices here are really solid and actionable, and I would suggest any team to read them, and implement them if possible.
The scary part with supply chain security is that we are only as secure as our dependencies, and if the platform you’re using has non secure defaults, the efforts to secure the full chain are that much higher.
SPOF. I'd suggest use automatic tools to audit every line of code no matter who the author is.
https://github.com/backnotprop/oss-security-audit
Almost every time I bump into uv in project work, the touted benefit is that it makes it easier to run projects with different python versions and avoiding clashes of 3rd dependencies - basically pyenv + venv + speed.
That sends a cold shiver down my spine, because it tells me that people are running all these different tools on their host machine with zero sandboxing.
What if? GitHub has is extremely buggy! I'm getting increasingly frustrated with the paper cuts that have become endemic across the entire platform. For example its not uncommon for one of our workflows to fail when cloning a branches of the repo they are running in.
All keys on geodistributed smartcards held by maintainers tied to a web of trust going back 25 years with over 5000 keys.
https://stagex.tools/packages/core/uv/
Though thankful for clients that let individual maintainers work on stagex part time once in a while, we have had one donation ever for $50 as a project. (thanks)
Why is it a bunch of mostly unpaid volunteer hackers are putting more effort into supply chain security than OpenAI.
I am annoyed.
Unpaid volunteer hackers provide their work for free under licenses designed for the purpose of allowing companies like OpenAI to use their work without paying or contributing in any form. OpenAI wants to make the most money. Why would they spend any time or money on something they can get for free?
Didn't the acquisition only happen a few weeks ago? Wouldn't it be more alarming if OpenAI had gone in and forced them to change their build process? Unless you're claiming that the article is lying about this being a description of what they've already been doing for a while (which seems a bit outlandish without more evidence), it's not clear to me why you're attributing this process to the parent company.
Don't get me wrong; there's plenty you can criticize OpenAI over, and I'm not taking a stance on your technical claims, but it seems somewhat disingenuous to phrase it like this.
To be frank. Because more effort doesn't actually mean that something is more secure. Just because you check extra things or take extra steps that doesn't mean it actually results in tangibly better security.
> All keys on geodistributed smartcards held by maintainers tied to a web of trust going back 25 years with over 5000 keys.
Neither the age nor the cardinality of the key graph tells me anything if I don’t trust the maintainers themselves; given that you’re fundamentally providing third-party builds, what’s the threat model you’re addressing?
It’s worth nothing that all builds of uv come from a locked resolution and, as mentioned in TFA, you can get signed artifacts from us. So I’m very murky on the value of signed package commits that come from a different set of identities than the ones actually building the software.
I wouldn't find this annoying at all - I would expect to have to do this for hundreds of packages.
Without unpaid volunteers things like Debian do not exist. Don't malign the situation and circumstances of other projects, especially if they are your competitors.
Compete by being better, not by complaining louder.
OpenClaw has been an outstanding success, it is providing people the ability to leak their keys, secrets, and personal data, and allowing people to be subject to an incredible number of supply chain attacks when its users have felt their attack surface was just too low.
Your efforts have been on increasing security and reducing supply chain attacks, when the market is strongly signaling to you that people want reduced security and more supply chain attacks!
I don't even love Nix, it's full of pitfalls and weirdnesses, but it provides so much by-default immutability and reproducibility that I sometimes forget how others need to rediscover this stuff from first principles every time a supply chain attack makes the news.
Even if you trust Microsoft's cloud engineering on the backend, this is a system that does not appear to follow even the most basic principles of privilege and isolation? I'm not sure why you would even try to build "supply-chain security" on top of this.
But thanks to the article I added a new check for the fork PR workflow approval policy.
More at: https://github.com/kdeldycke/repomatic