This makes sense to me. I guess I'll start hunting for the equivalent of `govulncheck` for Rust/Cargo.
Separately, I love the idea of the `geomys/sandboxed-step` action, but I've got such an aversion to use anyone else's actions, besides the first-party `actions/*` ones. I'll give sandboxed-step a look, sounds like it would be a nice thing to keep in my toolbox.
I was actually working on this last week, funnily enough. I've been working on a capability analysis tool for Rust, and if you're already generating a call graph via static analysis, taking that and matching it against the function-level vulnerability data that exists in RustSec isn't that hard.
The go ecosystem is pretty good about being backwards compatible. Dependabot regular update prs once a week seems like a good option in addition to govulncheck.
I automate updates with a cooldown, security scanning, and the usual tests. If it passes all that I don't worry about merging it. When something breaks, it is usually because the tests were not good enough, so I fix them. The next step up would be to deploy the update into a canary cluster and observe it for a while.
Better that than accrue tech debt. When you update on "your schedule" you still should do all the above, so why not just make it robust enough to automate? Works for me.
The number of ReDoS vulnerabilities we see in Dependabot alerts for NPM packages we’re only using in client code is absurd. I’d love a fix for this that was aware of whether the package is running on our backend or not. Client side ReDoS is not relevant to us at all.
Is there an equivalent for the JS ecosystem? If not, having Dependabot update dependencies automatically after a cooldown still seems like a better alernative, since you are likely to never update dependencies at all if it's not automatic.
We’ve built a modern dependabot (or works with it) agent: fossabot analyzes your app code to know how you use your dependencies then delivers a custom safe/needs review verdict per upgrade or packages groups of safe upgrades together to make more strategic jumps. We can also fix breaking changes because the agents context is so complete.
We have some of the best JS/TS analysis out there based on a custom static analysis engine designed for this use-case. You get free credits each month and we’d love feedback on which ecosystems are next…Java, Python?
Totally agree with the author that static analysis like govulncheck is the secret weapon to success with this problem! Dynamic languages are just much harder.
We have a really cool eval framework as well that we’ve blogged about.
I kind of wish Dependabot was just another tab you can see when you have contributor access for a repository. The emails are annoying and I mostly filter, but I also don't want a bunch of stale PRs sitting around either... I mean it's useful, but would prefer if it was limited to just the instances where I want to work on these kinds of issues for a couple hours across a few repositories.
Coming from someone with an almost ascetic dependency discipline, I look at some meta-dependencies as an outsider (dependabot, pnpm/yarn, poetry/venv/pipenv, snap/flatpak), a solution to too many dependencies that is yet another dependency, it feels like trying to get out of a hole by digging.
I think that for FOSS the F as in Gratis is always going to be the root cause of security conflicts, if developers are not paid, security is always going to be a problem, you are trying to get something out of nothing otherwise, the accounting equation will not balance, exploiting someone else is precisely the act that leaves you open to exploitation (only according to Nash Game Theory). "158 projects need funding" IS the vector! I'm not saying that JohnDoe/react-openai-redux-widget is going to go rogue, but with what budget are they going to be able to secure their own systems?
My advice is, if it ever comes the point where you need to install dependencies to control your growing dependency graph? consider deleting some dependencies instead.
The govulncheck approach (tracing actual code paths to verify vulnerable functions are called) should be the default for every ecosystem, not just Go.
The fundamental problem with Dependabot is that it treats dependency management as a security problem when it's actually a maintenance problem. A vulnerability in a function you never call is not a security issue — it's noise. But Dependabot can't distinguish the two because it operates at the version level, not the call graph level.
For Python projects I've found pip-audit with the --desc flag more useful than Dependabot. It's still version-based, but at least it doesn't create PRs that break your CI at 3am. The real solution is better static analysis that understands reachability, but until that exists for every ecosystem, turning off the noisy tools and doing manual quarterly audits might actually be more secure in practice — because you'll actually read the results instead of auto-merging them.
I think this is pretty good advice. I find Dependabot useful for managing scheduled dependency bumps (which in turn is useful for sussing out API changes, including unintended semver breakages from upstreams), but Dependabot’s built-in vulnerability scanning is strictly worse than just about every ecosystem’s own built-in solution.
> Dependencies should be updated according to your development cycle, not the cycle of each of your dependencies. For example you might want to update dependencies all at once when you begin a release development cycle, as opposed to when each dependency completes theirs.
We're in this space and our approach was to supplement Dependabot rather than replace it. Our app (https://www.infield.ai) focuses more on the project management and team coordination aspect of dependency management. We break upgrade work down into three swim lanes: a) individual upgrades that are required in order to address a known security vulnerability (reactive, most addressed by Dependabot) b) medium-priority upgrades due to staleness or abandonedness, and c) framework upgrades that may take several months to complete, like upgrading Rails or Django. Our software helps you prioritize the work in each of these buckets, record what work has been done, and track your libyear over time so you can manage your maintenance rotation.
The custom Github Actions approach is very customisable and flexible. In theory you could make and even auto approve bumps.
If you want something more structured, I’ve been playing with and can recommend Renovate (no affiliation). Renovate supports far more ecosystems, has a better community and customisation.
Having tried it I can’t believe how relatively poor Dependabot, the default tool is something we put up with by default. Take something simple like multi layer dockerfiles. This has been a docker features for a while now, yet it’s still silently unsupported by dependabot!
At this point your steps are so simple id skip GitHub actions security tyre fire altogether. Just run the go commands whilst listening on GitHub webhooks and updating checks with the GitHub checks API.
GitHub actions is the biggest security risk in this whole setup.
The core problem is that Dependabot treats dependency graphs as flat lists. It knows you depend on package X, and X has a CVE, so it alerts you. But it has no idea whether you actually call the vulnerable code path.
Go's tooling is exceptional here because the language was designed with this in mind - static analysis can trace exactly which symbols you import and call. govulncheck exploits this to give you meaningful alerts.
The npm ecosystem is even worse because dynamic requires and monkey-patching make static analysis much harder. You end up with dependency scanners that can't distinguish between "this package could theoretically be vulnerable" and "your code calls the vulnerable function."
The irony is that Dependabot's noise makes teams less secure, not more. When every PR has 12 security alerts, people stop reading them. Alert fatigue is a real attack surface.
> These PRs were accompanied by a security alert with a nonsensical, made up CVSS v4 score and by a worrying 73% compatibility score, allegedly based on the breakage the update is causing in the ecosystem.
Where did the CVSS score come from exactly? Does dependabot generate CVEs automatically?
44 comments
[ 3.0 ms ] story [ 55.9 ms ] threadSeparately, I love the idea of the `geomys/sandboxed-step` action, but I've got such an aversion to use anyone else's actions, besides the first-party `actions/*` ones. I'll give sandboxed-step a look, sounds like it would be a nice thing to keep in my toolbox.
Hopefully I'll have something out next week.
search revealed Sonatype Scan Gradle plugin. how is it?
I made a GitHub action that alerts if a PR adds a vulnerable call, which I think pairs nicely with the advice to only actually fix vulnerable calls.
https://github.com/imjasonh/govulncheck-action
You can also just run the stock tool in your GHA, but I liked being able to get annotations and comments in the PR.
Incidentally, the repo has dependabot enabled with auto-merge for those PRs, which is IMO the best you can do for JS codebases.
https://fossa.com/products/fossabot/
We have some of the best JS/TS analysis out there based on a custom static analysis engine designed for this use-case. You get free credits each month and we’d love feedback on which ecosystems are next…Java, Python?
Totally agree with the author that static analysis like govulncheck is the secret weapon to success with this problem! Dynamic languages are just much harder.
We have a really cool eval framework as well that we’ve blogged about.
I think that for FOSS the F as in Gratis is always going to be the root cause of security conflicts, if developers are not paid, security is always going to be a problem, you are trying to get something out of nothing otherwise, the accounting equation will not balance, exploiting someone else is precisely the act that leaves you open to exploitation (only according to Nash Game Theory). "158 projects need funding" IS the vector! I'm not saying that JohnDoe/react-openai-redux-widget is going to go rogue, but with what budget are they going to be able to secure their own systems?
My advice is, if it ever comes the point where you need to install dependencies to control your growing dependency graph? consider deleting some dependencies instead.
The fundamental problem with Dependabot is that it treats dependency management as a security problem when it's actually a maintenance problem. A vulnerability in a function you never call is not a security issue — it's noise. But Dependabot can't distinguish the two because it operates at the version level, not the call graph level.
For Python projects I've found pip-audit with the --desc flag more useful than Dependabot. It's still version-based, but at least it doesn't create PRs that break your CI at 3am. The real solution is better static analysis that understands reachability, but until that exists for every ecosystem, turning off the noisy tools and doing manual quarterly audits might actually be more secure in practice — because you'll actually read the results instead of auto-merging them.
We also let renovate[bot] (similar to dependabot) merge non-major dep updates if tests pass. I hardly notice when deps have small updates.
https://github.com/search?q=org%3Amoov-io+is%3Apr+is%3Amerge...
We're in this space and our approach was to supplement Dependabot rather than replace it. Our app (https://www.infield.ai) focuses more on the project management and team coordination aspect of dependency management. We break upgrade work down into three swim lanes: a) individual upgrades that are required in order to address a known security vulnerability (reactive, most addressed by Dependabot) b) medium-priority upgrades due to staleness or abandonedness, and c) framework upgrades that may take several months to complete, like upgrading Rails or Django. Our software helps you prioritize the work in each of these buckets, record what work has been done, and track your libyear over time so you can manage your maintenance rotation.
If you want something more structured, I’ve been playing with and can recommend Renovate (no affiliation). Renovate supports far more ecosystems, has a better community and customisation.
Having tried it I can’t believe how relatively poor Dependabot, the default tool is something we put up with by default. Take something simple like multi layer dockerfiles. This has been a docker features for a while now, yet it’s still silently unsupported by dependabot!
GitHub actions is the biggest security risk in this whole setup.
Honestly not that complicated.
Go's tooling is exceptional here because the language was designed with this in mind - static analysis can trace exactly which symbols you import and call. govulncheck exploits this to give you meaningful alerts.
The npm ecosystem is even worse because dynamic requires and monkey-patching make static analysis much harder. You end up with dependency scanners that can't distinguish between "this package could theoretically be vulnerable" and "your code calls the vulnerable function."
The irony is that Dependabot's noise makes teams less secure, not more. When every PR has 12 security alerts, people stop reading them. Alert fatigue is a real attack surface.
> These PRs were accompanied by a security alert with a nonsensical, made up CVSS v4 score and by a worrying 73% compatibility score, allegedly based on the breakage the update is causing in the ecosystem.
Where did the CVSS score come from exactly? Does dependabot generate CVEs automatically?