is that entropy not self inflicted by swes constantly reinventing the wheel with new languages, frameworks, and libraries? they may be better but constantly having new adoption means nothing ever really lasts long
100% - documentation will always go stale. the source of truth is the code however, and making sure the code is kept shiny is the primary task of engineering. documentation is non-load-baring - you can tear it all down and the project will still work correctly. Legacy tooling can literally cause breakages and security concerns.
> Legacy tooling can literally cause breakages and security concerns.
Absolutely. My domain is a little different, but I've taken to keeping the installers of every version of a specific CAD program I've used, due to compatibility breaking updates that it has had, in the event that I might need to go back and open a design from a few years ago.
Everyone wants to be Tony Stark, nobody wants to be the janitor. Thus things fall apart and get constantly rebuild, in a "good-enough-fashion", which is very reminiscent of biologic life.
We are codes transport mechanism, without us, its not alive. And only good code gets copied, reused, to life on till heat death.
I love the Iron Man reference. Everyone is always chasing the shiny new thing. Since my company build devtools the shiny thing is the thing that usually gets rusty - which is kind of the whole point I guess.
Depends what you mean by works. Of course installing the same version of the tool will in most cases keep doing the same thing it's always done. But it's possible the tool always had bugs that were never noticed, has significant improvements that could benefit the project, etc.
I would settle for knowing I could still build it. It is amazing how fragile build chains are and the amount of maintenance they require. The build tools themselves, let alone pulling in dependencies.
This is the main reason I'm so happy when I see builds run in docker containers. I don't have to worry as much about tracking down the exact dependencies that we're used to build a specific release.
I think docker is a false sense of security. Depending on the language, there can be many implicit system dependencies which you do not pin. If you try to run a container that references ubuntu:13.04, will it still work today? Maybe. Still leagues better than where we were.
I am still hoping for Nix or a version 2 of the project to sweep the world and fix this mess.
> Once you’ve confirmed the tools are running as expected
Kind of glossing over what's often the hardest part. Sure, ESLint (the author's example) might be easy enough, but who hasn't seen a new compiler or key library cause problems? Bugs, including security bugs, can be introduced as well as fixed. Often these problems are hard to notice (until they bite you hard) and/or work around. While the solution to the easiest part is not unwelcome, software supply chain issues are much more complex and difficult than "once you've confirmed" would suggest.
there are bots that can resolve this one angle of the problem. they have drawbacks that are great opportunities to improve upon - dependabot loves spamming projects with update notifications and doesn't understand what its actually doing - it is kind of a semi-smart regex parser. valuable but we can do better.
I have several intranet applications that I need to port to another web framework because of this specific issue. I guess it servers me right for not choosing the most popular framework for my applications.
Sometimes I wonder if we're optimizing for the wrong things in the dependency management space: we used to have a distinction between "bleeding edge" and "stable" distribution schemes, with programmers (and users) picking the appropriate channel based on their stability and feature requires.
That seems to have been mostly discarded: we now assume that a development dependency is broken and needs to be updated if it's even slightly stale, despite being stable and working just fine. I can understand part of why that is (e.g. the overhead required to support multiple versions), but I think it's shortsighted of us to treat non-adherence to the "bleeding edge" as a security or code smell.
I think it's also tied up with security - in the past, you could have one release cycle with a stable release coming out every n releases, but now, because of vulnerabilities, larger projects usually need to have two release cycles: one for the bleeding edge, and an LTS that gets security updates ported in. That's a lot of work to maintain, so a lot of projects just nix the LTS.
Definitely, and I do not blame projects whatsoever for nixing the LTS -- it's a lot of work!
Still, I wonder about the general propriety: not every bug fix is a security fix, and treating them as such produces noise and fatigue that reduces the overall effectiveness of vulnerability reporting (since engineers become desensitized to reports).
Much of Rust's ecosystem embodies this: there are crates (like serde and clap) that have extremely regular bugfix releases, but those fixes do not (to my knowledge) surface as security concerns; instead, they're usually small invariant checks or missing trait implementations.
The ultimate reason is that security bug fixes are an absolute requirement, and unless someone is paying, most OSS providers aren't interested in doing multiple sets of bug fixes. That is the promise- and entire business model, honestly, of Red Hat, they will backport security bugfixes to your stable version.
Otherwise, since we rely so much on free software development, it kinda has to be that way.
I didn't see anything about them filtering out repos which have been abandoned, were only for testing, or other such considerations. The title is technically correct, but I feel that most people would interpret it as "82% of active Open-Source projects".
Is it? Where did the 82% figure even come from? Are 82% of all Open Source projects even written in languages with tooling that promotes projects to self-manage their dependencies?
The world doesn't end at JavaScript, Python, or even Rust.
The practical consequences of running an outdated eslint version are, for most projects, zero. The cost of upgrading is non zero. So upgrading only happens when there's a compelling reason to update like:
- A real security issue - things like the reDoS example cited are part of the crying wolf that causes people to treat security scanners less seriously
- A new feature (lint rule in eslint's case, or maybe new JS syntax support) is desired. The new upgrade can be deferred until that time
- You're doing updates _anyway_ and update eslint as part of that housekeeping.
1. How often are security issues found that are already fixed compared to security issues that are still in the latest version and require an even newer version to fix?
2. What are the security issues that can be found in a local tool which has as its only input your own code and it's output is only printed in tooltips for human review? Even if you had a way for specially formed code to run other code in eslint, it's a bit like saying bash has an RCE because you can type a command into it.
- There's a bug in one of the linters that is giving false positives (passing good code or missing bad code). This may not be a security issue, it may just result in poor code quality.
> Software projects have very short shelf lives. The moment a commit lands on main the software is spoiling; new versions of runtimes and compilers are shipped, dependencies cut new releases with bug fixes and exciting new features, and connected APIs are deprecated.
Does this ring true for anyone? This sounds like hell. I am so glad to be in a field that minimizes third party dependencies.
The C code I write today would run the same 20 years ago and will run the same 20 years from now.
This is just another example of someone with one experience in software development thinking that their experience can be applied universally. It really depends on the kind of software you're writing and the environment you're working in.
The C++ code I wrote in 2008 doesn't compile today, despite depending only on libSDL. Lots of small things that have broken, mostly GCC behaving differently.
Still miles better than the JS and cloud spaces where code has such a short half-life you need to handle it with tongs and keep it in a lead box when you aren't using it, but still. All code rots.
The proliferation of ReDoS "vulnerabilities" has been extremely frustrating (as someone working on accurate and high-signal vulnerability reporting tools), and belies completely perverse incentives in the bug bounty/vulnerability reporting space (reporters are incentivized to obtain CVEs to pressure companies into accepting low-quality reports).
I think part of the problem is that people aren't distinguishing trusted inputs from untrusted inputs. ReDoS is only a possible problem if untrusted inputs can cause it. Some inputs are trusted, and while it's not crazy to check them (people make mistakes), that's a different kind of concern. ReDoS is a potential security vulnerability, but only in certain special cases.
If arbitrary people can insert arbitrary code into what your service sends, and other people will then run it, then ReDoS isn't even the beginning of your security problems.
Well written, honestly I find this hard, e.g. not using eslint that often, I have made no clear decision yet in which kind code input from npm for example falls.
eslint doesn't scan your mode modules. If it did, it would be untrusted, as they could include as .eslintrc.js file and run arbitrary code. However, they could also include a npm post install script and run arbitrary code, or run arbitrary code when your code imports them, so it's not like eslint changes your risk profile in regards to npm dependencies.
Thanks for taking the time to comment extensively.
> so it's not like eslint changes your risk profile in regards to npm dependencies.
Depends. If it is the only dependency from npm, it changes all to that regard. This is perhaps the hard case for me.
> they could also include a npm post install script and run arbitrary code
Is there a cheap flag or global parameter to disable that so that the build does not run wild?
In composer there is --no-scripts --no-plugins which effectively can prevent things (and scripts / plugins are bound to root projects so they aren't that far-reaching as in npm by default, but the field of problem is technically the same).
At the end of the day I find clamping such features away as necessary as pinning the tool versions for a stable build. Me is just weak with npm.
ReDoS vulnerabilities in tooling - that's the key point. The "risk" is that if you do something weird in your code, then your linter might take a long time or crash (i.e. by running out memory). The same way that doing crazy things with types can make your compiler take a long time or crash (- type system).
I didn't consider TypeScript tooling. I guess that makes sense since it's usually converted to JavaScript instead of being run directly where errors would affect end users.
I do a lot of projects that are once and done. Once it's out there, there might be a maintenance budget, but it's minimal. Stuff just runs, until someone notices that there's an issue. (that's not so bad with sites that are essentially static SPAs, for ones where there are servers it's more risky)
It's a constant battle to build projects that shipped 18 months ago. One thing wasn't pinned. Or the build system used node 12 and doesn't run on CI anymore. So Babel needs updated. CDK did a major rev and now we have to force pin the old version.
We have run into this with the HL7 standards development organization. It is no longer possible to build new releases of artifacts from some of the older standards because the build process depends on obsolete custom tools which no longer run on modern operating system versions. In theory it might be possible to update the tools, or construct a virtualized build environment, but in practice they don't have enough resources to fight through all the issues.
This seems to equate linters with other kinds of tools, and I think that's mistaken.
Linters attempt to look for "less than best practices". When you update a linter, you often also need to update code for the new style rules (unless you just don't care about the results). Those reports can be helpful, because those style rules can also warn about real problems like security vulnerabilities. But most are minor issues, so it may make sense to delay them.
I would generally prioritize outputs of tools that are designed to find serious problems (like security vulnerabilities), and then update things like linters when I can (but consider those lower priority). We should be unsurprised to find developers prioritizing some kinds of reports over others.
I think this definition is outdated - linters find real bugs now. The boundaries between linters, static analyzers, and security tools in real life are basically nonexistent, it's entirely tool by tool. Also, autoformatters should style code, not linters. Used to be that linters told you about code style issues, and many still do, but ppl should turn all rules off and use autoformatters instead. Much easier/better.
I disagree that the definition is "outdated". Yes, linters find real bugs, I never said they couldn't. But tools typically called "linters" routinely also report things that aren't serious defects, while other tools that focus on specific issues (e.g., security vulnerabilities) tend to find those issues. A "static analyzer" is anything that analyzes code without running it, so linters and Static Application Security Testing (SAST) tools are both kinds of static analysis tools. Linters and SAST tools (for example) tend to focus on different kinds of rules. Under the hood they may use the same underlying technology, but they're applying them differently, so it makes sense to have different words for them.
You might find real bugs with these linting rules, sure. But if I want to prioritize finding and fixing security defects, I'll prioritize fixing the security defects found by a tool designed to find the most likely kinds of security defects (e.g., look for XSS, SQL injection, etc.). Not these rules. You can have code that violates many ESLint rules and still works correctly for end-users.
That doesn't mean linters are useless. Quite the contrary, I think linters are really helpful & I strongly encourage their use. Fixing issues found by linters can not only fix real bugs, but they can sometimes encourage creation of simpler code that is easier to analyze by both humans and machines. But I wouldn't prioritize updating a linter over updating other tools and fixes. Yes, update linters, but when you have limited resources, that's not where you spend your first hour.
I think there's a lot of legitimate debate about autoformatters, that's a different topic.
1. key system dependencies -- this includes things like Android APIs, IDE APIs, etc. ~ to work on newer versions of those platforms, you need to upgrade and make changes to the code
2. other runtime dependencies -- these are dependencies used to make the system run, but that don't need to be updated in sync with the target platform ~ this can be things like the version of Spring, a HTTP library, etc.
3. development dependencies -- these are dependencies used to build the project
The system dependencies result in a treadmill, where if you don't keep up with the changes then your application, plugin, etc. will no longer work on the latest version. Just keeping up with these can be a lot of work, especially if you need/want to support multiple versions.
The other dependencies can cause headaches when upgrading major (or even minor) versions. For example, making sure that all the libraries you use are compatible when upgrading another library or language (Scala, Python, Java, PHP, etc.). This can be a lot of work to make sure that nothing is broken.
Development dependencies are typically low on the priority of things to upgrade. If the version of ESLint you are using works and there are no key benefits of performing an upgrade, then it is less likely to be upgraded.
Bar charts are not very useful. The second bar chart here, especially, is hard to learn from. Plotting something like (days since last project release) vs (days since release of tool version used) seems like it might be be much more insightful.
The open-source community did this to themselves. When the most popular frameworks and foundation tools (e.g. Python) disregard backward compatibility again and again, we have taken broken compatibility for granted and formed habit of pinning specific versions. It was not like this a decade ago. At that time, we more often assumed a minor version bump in dependencies would naturally keep everything running without hassles.
Alternative title: 82% of projects on GitHub have an outdated code formatting library as a dependency.
I wouldn’t call this tool rot. This to me seems like fear-mongering to drive conversions of yet another tool for managing dependencies from the company behind this blog post.
It would be interesting to maybe analyse usage of React, Django, Laravel or such popular projects which you might be certain are key to a GitHub project and might lead to some interesting observations.
Instead i would summarise this article as a graph of a few searches on github that give you as much data as relevant stats page from npm or PyPI - not much.
What conclusions can we draw that some projects on github, which we have no idea how big or how used they are, run an outdated code formatter? Maybe 82% of them are archived, abandoned or were one offs to learn the language?
60 comments
[ 3.4 ms ] story [ 108 ms ] threadDocumentation is a constant fight against entropy.
Working safely is a constant fight against entropy/complacency.
And so on
Absolutely. My domain is a little different, but I've taken to keeping the installers of every version of a specific CAD program I've used, due to compatibility breaking updates that it has had, in the event that I might need to go back and open a design from a few years ago.
We are codes transport mechanism, without us, its not alive. And only good code gets copied, reused, to life on till heat death.
I am still hoping for Nix or a version 2 of the project to sweep the world and fix this mess.
- configuration changes in the new versions
- undocumented changes in the new versions
- subtle changes in the build result that lead to flaws in the artefacts
- some of them leading to security flaws (those in the real camp)
- some of them leading to data-loss when running on production systems.
But sure, if you have kept the linter up-to-date ... .
Oh my god, your vscode/text editor will freeze or crash if your code contain a specially crafted string. What a very import thing to call out.
Kind of glossing over what's often the hardest part. Sure, ESLint (the author's example) might be easy enough, but who hasn't seen a new compiler or key library cause problems? Bugs, including security bugs, can be introduced as well as fixed. Often these problems are hard to notice (until they bite you hard) and/or work around. While the solution to the easiest part is not unwelcome, software supply chain issues are much more complex and difficult than "once you've confirmed" would suggest.
Also: stealth advertising. :-P
That seems to have been mostly discarded: we now assume that a development dependency is broken and needs to be updated if it's even slightly stale, despite being stable and working just fine. I can understand part of why that is (e.g. the overhead required to support multiple versions), but I think it's shortsighted of us to treat non-adherence to the "bleeding edge" as a security or code smell.
Still, I wonder about the general propriety: not every bug fix is a security fix, and treating them as such produces noise and fatigue that reduces the overall effectiveness of vulnerability reporting (since engineers become desensitized to reports).
Much of Rust's ecosystem embodies this: there are crates (like serde and clap) that have extremely regular bugfix releases, but those fixes do not (to my knowledge) surface as security concerns; instead, they're usually small invariant checks or missing trait implementations.
Otherwise, since we rely so much on free software development, it kinda has to be that way.
TLDR; there are a lot of active projects running rusty / rotten tools
Similarly, it's a strong claim to equate "non-latest release" with "rusty/rotten tooling".
Is it? Where did the 82% figure even come from? Are 82% of all Open Source projects even written in languages with tooling that promotes projects to self-manage their dependencies?
The world doesn't end at JavaScript, Python, or even Rust.
- A real security issue - things like the reDoS example cited are part of the crying wolf that causes people to treat security scanners less seriously
- A new feature (lint rule in eslint's case, or maybe new JS syntax support) is desired. The new upgrade can be deferred until that time
- You're doing updates _anyway_ and update eslint as part of that housekeeping.
2. What are the security issues that can be found in a local tool which has as its only input your own code and it's output is only printed in tooltips for human review? Even if you had a way for specially formed code to run other code in eslint, it's a bit like saying bash has an RCE because you can type a command into it.
Does this ring true for anyone? This sounds like hell. I am so glad to be in a field that minimizes third party dependencies.
The C code I write today would run the same 20 years ago and will run the same 20 years from now.
When Java came out, we went through the tool hell described in this article.
Still miles better than the JS and cloud spaces where code has such a short half-life you need to handle it with tongs and keep it in a lead box when you aren't using it, but still. All code rots.
If one points out ReDos 'vulnerabilities' in tooling as a reason to upgrade it, you've instantly lost the argument.
By the same logic we should ban TypeScript because its type system is turing complete. This reasoning isn't even wrong.
The proliferation of ReDoS "vulnerabilities" has been extremely frustrating (as someone working on accurate and high-signal vulnerability reporting tools), and belies completely perverse incentives in the bug bounty/vulnerability reporting space (reporters are incentivized to obtain CVEs to pressure companies into accepting low-quality reports).
If arbitrary people can insert arbitrary code into what your service sends, and other people will then run it, then ReDoS isn't even the beginning of your security problems.
> so it's not like eslint changes your risk profile in regards to npm dependencies.
Depends. If it is the only dependency from npm, it changes all to that regard. This is perhaps the hard case for me.
> they could also include a npm post install script and run arbitrary code
Is there a cheap flag or global parameter to disable that so that the build does not run wild?
In composer there is --no-scripts --no-plugins which effectively can prevent things (and scripts / plugins are bound to root projects so they aren't that far-reaching as in npm by default, but the field of problem is technically the same).
At the end of the day I find clamping such features away as necessary as pinning the tool versions for a stable build. Me is just weak with npm.
It's a constant battle to build projects that shipped 18 months ago. One thing wasn't pinned. Or the build system used node 12 and doesn't run on CI anymore. So Babel needs updated. CDK did a major rev and now we have to force pin the old version.
You don't need always the newest "sht".
Linters attempt to look for "less than best practices". When you update a linter, you often also need to update code for the new style rules (unless you just don't care about the results). Those reports can be helpful, because those style rules can also warn about real problems like security vulnerabilities. But most are minor issues, so it may make sense to delay them.
I would generally prioritize outputs of tools that are designed to find serious problems (like security vulnerabilities), and then update things like linters when I can (but consider those lower priority). We should be unsurprised to find developers prioritizing some kinds of reports over others.
So let's look at ESLint's list of rules: https://eslint.org/docs/latest/rules/ Here are some examples:
* getter-return: Enforce `return` statements in getters
* no-constant-condition: Disallow constant expressions in conditions
* no-fallthrough: Disallow fallthrough of `case` statements
* no-irregular-whitespace: Disallow irregular whitespace
You might find real bugs with these linting rules, sure. But if I want to prioritize finding and fixing security defects, I'll prioritize fixing the security defects found by a tool designed to find the most likely kinds of security defects (e.g., look for XSS, SQL injection, etc.). Not these rules. You can have code that violates many ESLint rules and still works correctly for end-users.
That doesn't mean linters are useless. Quite the contrary, I think linters are really helpful & I strongly encourage their use. Fixing issues found by linters can not only fix real bugs, but they can sometimes encourage creation of simpler code that is easier to analyze by both humans and machines. But I wouldn't prioritize updating a linter over updating other tools and fixes. Yes, update linters, but when you have limited resources, that's not where you spend your first hour.
I think there's a lot of legitimate debate about autoformatters, that's a different topic.
1. key system dependencies -- this includes things like Android APIs, IDE APIs, etc. ~ to work on newer versions of those platforms, you need to upgrade and make changes to the code
2. other runtime dependencies -- these are dependencies used to make the system run, but that don't need to be updated in sync with the target platform ~ this can be things like the version of Spring, a HTTP library, etc.
3. development dependencies -- these are dependencies used to build the project
The system dependencies result in a treadmill, where if you don't keep up with the changes then your application, plugin, etc. will no longer work on the latest version. Just keeping up with these can be a lot of work, especially if you need/want to support multiple versions.
The other dependencies can cause headaches when upgrading major (or even minor) versions. For example, making sure that all the libraries you use are compatible when upgrading another library or language (Scala, Python, Java, PHP, etc.). This can be a lot of work to make sure that nothing is broken.
Development dependencies are typically low on the priority of things to upgrade. If the version of ESLint you are using works and there are no key benefits of performing an upgrade, then it is less likely to be upgraded.
I wouldn’t call this tool rot. This to me seems like fear-mongering to drive conversions of yet another tool for managing dependencies from the company behind this blog post.
It would be interesting to maybe analyse usage of React, Django, Laravel or such popular projects which you might be certain are key to a GitHub project and might lead to some interesting observations.
Instead i would summarise this article as a graph of a few searches on github that give you as much data as relevant stats page from npm or PyPI - not much.
What conclusions can we draw that some projects on github, which we have no idea how big or how used they are, run an outdated code formatter? Maybe 82% of them are archived, abandoned or were one offs to learn the language?