This is such a good idea. Another one I would like to have is a single number repressing legacy code. I'm not sure exactly what but is there a way to represent the code which people are writing workarounds for.
There are tools[0] that show you which users touched which files/modules and how many of those users (if any!) are still in the company.
Doesn't necessarily tell you what code is legacy – perhaps a function is just so solid, that there was no reason to touch it in years. But I've found such analysis helpful and it can give you warning signs about what knowledge is being lost in the team and which parts of your own codebase became unknown territory.
[0] I know of CodeScene but suppose there are others
Can't see this as being particularly useful. Libraries under active development will have larger more frequent releases (but small number of libyears out of date) whereas mature software with only occasional bug fixes may be 10s of libyears out of date.
Well, it's up to you to interpret the libyears how you like. It doesn't have to be that more libyears is worse, but it will mean that you're missing X libyears of security updates (and also X libyears of potential new bugs).
That really doesn't follow. A barely maintained mess of a personal project running for 10 years is not more mature than a 7 year project heavily developed and released by a high quality team, with 3 years of stability in the main API and in use around the world.
Anecdotally the Python tool seemed to indicate for me a 0 for a dependency which was up to date (but hadn't been updated in a good year at the very least).
A more accurate (but more unwieldy to measure) metric would be to count the lines of code that have been changed since the version used and the most recent stable version. (I think this is what commenter amelius implied?) It wouldn't quite capture the nature changes made, but it would very much uncouple from the quite unwieldy assumption that libraries are all developed at the exact same pace.
I don't think that lines of code is a good metric here. A few lines of code can fix a major security issue in parts of a dependency that you actually use, while thousands of lines of code can just add new features that you are not using anyway, otherwise you would have upgraded already.
My immediate thought looking at this number was not that it should be minimized but that there ought to be a sweet spot range and a number below which it probably shouldn't go and a number above which it shouldn't go.
It's always context-dependent. Take Lisp languages. For Common Lisp, when I see a library that was last released or updated 10 years ago, I'm thinking it's probably as feature-complete as it's ever going to be, and otherwise perfectly fine. Same in case of Emacs Lisp? I'm thinking generations out of date, and has a solid chance of not working anymore. Here, it's a difference between a battle-tested, standardized (ANSI/ISO) platform (CL), vs. fast-evolving one (Emacs).
If you're on a 3 year old version of the library because the library introduced a change which you will never be able to adopt so you're forever stuck on the 3 year old version, you're in a much worse position than if you're just 3 versions behind because you haven't taken the time to upgrade yet. As such, libyears become an optimistic measure of badness in that situation.
Nor did I claim it would. If you are auditing your dependencies like that then you don't need it, I said, as in it's not going to give you any extra information.
If you're not, and very many people are not, then total age of dependencies is a decent low-effort approximation for the probability of bug fixes affecting parts of dependencies that you're using.
That depends on the changes to the library since, and how and where the library is used.
Suppose I regularly generate a CSV file, all ASCII, where all the rows are integers or fixed precision numbers. I have a ten year old CSV library that processes that file, and has worked without any problem for ten years.
I have no interest in updating the library. Updates can introduce downtime, but provide no improvement. In fact, they introduce a slight performance hit because of new features and that I don't need. There is also the risk that the updates will introduce bugs, and then I'll have to spend time diagnosing the bug, and coming up with a fix.
Now let me reverse this: suppose there are two libraries to do the same task, A and B. They don't have the same features, but for your use case, they are both easy to use and do exactly what you need.
A was first released in the 1980s and was last updated five years ago. It's still maintained and is available in most Linux distributions.
B was first released three years ago and has had 20 updates since, 18 of which included fixes for security issues that don't affect A. (The website for A is regularly updated to indicate that it has been tested and these issues do not affect t.)
I see the overall point as not seeing every dependencies as things that need upgrade.
Any library that is effectively a dataset could fall into this as well: if you want to freeze your environment at a specific reference point and only update the actual moving parts, the libyear measurement won't be for you.
This reminds me of interface softwares that keep old version of some libraries to emulate the original behavior, butnin a controlled and isolated way.
On the other hand, it was only somewhat recently that CLDR acknowledged that languages with noun inflection exist, so it’s kind of a wash. E.g. in Russian, Ukrainian, and Belarusian (at least) you use the nominative of the month’s name in May 2024, but the genitive in 9 May 2024, etc., rendering most older allegedly-localized software that used a generic list of month names ungrammatical.
Agreed. If the dependency is under active development then it should be only counted as being behind if there is a newer version released for that dependency. The libyear should be calculated as "latest version's release date" - "currently used version's release date".
What complicates this is deciding whether the dependency is under active development or not. If its EOL'd then you still want libyear to accumulate, even if you use the latest released version. I guess comparing to an end-of-life date then would make sense, but it's probably harder keep track of.
I've tried out some of the libraries, and it looks like they do calculate the difference between the installed version and the last (stable) release. If a dependency hasn't seen a release in ten years, those ten years don't count against the dependency drift. This is exactly what I would want.
However, they only check openly accessible (i.e. OSS) dependencies. If one of those hasn't seen a release in ten years, I would look for an alternative.
For some applications it might be of great use but for a vast and complex applications architecture, the libyear metric might only oversimplify the complexity of dependency management,compatibility issues, updates and security patches, etc
I noticed that it focuses only on the age of dependencies without considering other factors like the how critical is the update, and how stable it is, and the improvements in newer versions, etc.
At a previous company we had this big web-app (filterable) matrix which listed each project dependency, but the neat thing was that you can tag dependencies to add weight and importance.
Initially i thought it would need to be more complex, but but it was more than enough.
Right, but that means it's not a decent starting measure.
To me, "decent starting measure if there is no appetite for something more in-depth" sounds like "just drop it in, it's enough to get started, we'll figure out the rest later", but that could be temporarily harmful.
Exercising judgment is the opposite of that, no? Then you're going into depth.
> If your system has a one year old dependency and a three year old dependency, then your whole system is four libyears old.
which don't explain much.
I suspect what they meant to say is that Rails 5.0.0 (June 2016) is 1 year (not libyear) behind 5.1.2 (June 2017) and that the "libyear age" of a project is the sum of how old each of its dependencies is. But if so, they should say so clearly somewhere on their page.
What confused me was saying that some version of Rails is some number of libyears behind some other version, when in order to know that you'd need to expect the dependencies of Rails itself...
It's nice to have a name for this, and for it to be quantifiable. I could see this on some kind of product dashboard - maybe automatically generated.
But, while I appreciate the need for simplicity, I also wonder if it would be wise to scale dependencies by how prevalent they are in the codebase. For example, if I'm using a five year old version of react but the library I use to convert temperature units is up-to-date, then thats bad. But if I'm using the latest react and the conversion lib is old then thats less bad.
The features that you described are somewhat close to what I'm trying to build with depshub.com. Dependency visibility is still a major problem in any engineering team that cares about dependencies, and it's often very hard to say if a project is moving in the right direction in terms of updates. Some teams just completely ignore the fact that they need to update dependencies, but this usually comes with the consequence of "updating ASAP because we need X feature or Y bugfix."
All the major tools (dependabot, renovate) to keep dependencies up to date treat all the dependencies equally when in reality there are always core libraries (e.g., react) and everything else. While trying to keep *everything* up to date is extremely challenging, what I'm trying to do is to find a balance between what and when needs to be updated (using code static analysis, different data sources, AI etc) and automate it in a simple manner.
libyear assumes that constant development happens all the time in a library and software has to change and grow constantly. There are some libraries that are just mature and doesn't need to change. 5.0.1 could be released 5 years ago, and 5.0.1 today with just changes to docs. It doesn't mean that it took 5 libyears to develop and release the new version. This is the type of thinking that attackers used in xz trying to pressure original author to add more official maintainers, because he wasn't merging and updating code fast enough. "You're not merging and releasing our code fast enough, therefore you're doing bad job".
What would be other measures that could be similarly useful? Lines of code or story points? Maybe even a number of tests added?
That is an argument for libyears, not against it. If they only updated the docs, why didn't you upgrade already? If anything, projects using stable libraries like this could very easily reach 0 libyears, because there are no breaking changes when upgrading.
It's for and against at the same time. Depends on your software upgrade roadmap. I think that's just a silly number, that us correct and incorrect at the same time, but probably good enough on a large scale, just like story points.
It's worth also reading https://chaoss.community/kb/metric-libyears/ - as noted elsewhere, regularly updating libraries, or a library that infrequently pushes out large breaking changes will not reflect as easily in Libyear, but it's worth having it as something to gauge out-of-date-ness
I've been using that alongside some other metrics for providing insights into how behind teams are on updates
> I've been using that alongside some other metrics for providing insights into how behind teams are on updates
What are some other metrics that you are using? I am working on a product that is helping to keep dependencies up to date and would love to integrate some of these things in the product.
I agree with other comments that it's not a perfect measure but it's a solid step in the right direction from having no metrics at all.
The libs we're measuring up to could have their own libyears to upgrade, but we can only control what's in our hands.
Sometimes a small security patch is worth more than a major version bump of features, so I consider measuring the time instead of major versions a benefit.
I guess that having to rely on metrics to begin with means the battle is lost and your have no control on the code you are using.
Maybe we should stop boilerplating everything and write the actual code we need. For the most part softwares usually use a tiny fraction of capabilities of any given library.
Maybe before trying to limit our lag in dependencies update of unlimited levels of libraries we should focus first on having a maximum level of dependencies. Like one project would use a maximum of 2 level of libraries dependencies and you would have to rewrite those that have too many levels.
The javascript ecosystem for instance is totally unmanageable as I see it. We just pretend we have a bit of control but in reality nobody knows what code is executed really and this is sad.
That is better, I agree. I'd take a lean set of dependencies any day, but it becomes increasingly more difficult the more velocity the project gets.
Suddenly less and less is considered core and it's easier than ever to 'outsource' to external libs to save time. Or is it rather that the project gets more velocity because of that?
> We just pretend we have a bit of control but in reality nobody knows what code is executed really and this is sad.
True, this is also slowly starting to be the case with other languages.
With Python it can be so bad that even attempting to 'build' and run the same project a year later may well fail.
Much to what I'm used to with JavaScript projects by now.
As a worthless junior dev, thank you for the post. I am seeing a general sentiment of reluctancy towards an introduction of yet another dubious metric in an environment where "software quality" is hijacked to mean something else.
This lead me questioning how good is it to judge a project by its age + last commit (+ project size/complexity + funding/community), as this is what I do in practice. I agree that SemVer isn't really designed to be human-readable and is a rather meaningless / deceiving metric due to divergent practices of different developers.
Matters a lot what kind of applications the library is for. Being a couple of years behind in mobile application development usually means you have to spend a week piecing together a development environment to get the crap to build so you can start the library upgrading process.
A couple of years hither or dither with grey-haired Java libraries matters very little. There might be some vulnerabilities but you probably know about them and have workarounds, and sometime next year it's likely you'll be allowed a month or two to do 'life cycle management' in the dependency stack.
As a person who works on automated dependency updates (depshub.com), the libyear indicator is often not very useful. There are several other indicators to consider, such as release frequency, update type (major/minor/patch), the dependency's criticality for your project, etc.
Instead of solely focusing on reducing the libyear for your projects, a better approach is to minimize the steps needed to keep your project reasonably up to date. For instance, think about managing 20 PRs weekly to update various package.json packages versus 1 PR for critical dependencies when necessary.
It's important to note that updating dependencies is not a consistent task that can be done at the same pace all the time. Expect varying update volumes and complexities that may need attention at different times. Setting a fixed configuration for, let's say, 10 updates per week may not be effective, as it could lead to dealing with unnecessary updates regularly (e.g., aws-cli, which has almost daily releases).
Finding the right balance between keeping your project up to date and spending too much time on dealing with dependencies is the hardest part here that doesn't have a 100% right answer yet.
I've been doing this work for a while and been pushing my org to move from updating the "base layer" every 2-4 years (Ubuntu LTS versions) down to every six months (each NixOS release).
I think one thing I've increasingly found is that it's important to set up the infrastructure for parallel building— it's not realistic to have a flag day twice a year, and it's not realistic to try to test everything first "on a branch". If you can have a transitional period of a few weeks where the product outputs (containers, dev environments, whatever) are consistently available in both the old and new flavour, then you can invite people to try the updated thing, while still having an escape lane to keep using the older thing if stuff turns out to be broken in a way that's beyond their capacity to correct in the moment.
I'm interested in knowing more about how companies typically gauge the freshness of their codebase dependencies. putting all the nuances/details aside, i think we can all agree that having a codebase with most dependencies that are over 8 years old is a pretty clear indication that it's way overdue for an update, right?
i was thinking, would it be helpful to keep track of how far behind each dependency is in terms of minor, patch, and major updates? but this seems a bit too complex to explain to the management. i'm trying to figure out the best way to explain this to management so they understand why it's important to stay current. any ideas on how we can measure improvements? maybe we should agree on a few key factors to track our progress and see if we're getting better or worse.
> I was thinking, would it be helpful to keep track of how far behind each dependency is in terms of minor, patch, and major updates?
This is exactly what I've added for depshub.com, and people seem to like it a lot. It just gives you better visibility across all of your connected repositories about what the current status of each dependency is and how the major vs. minor vs. patch ratio changes over time. While it's still a naive metric, it's the easiest to understand and visualize - and as a result, the one that is used the most.
> Any ideas on how we can measure improvements?
- Quantitative: Spend as little amount of time as possible on trying to keep everything relatively up to date (hours/month)
- Qualitative: not having any CVE issues, not having major updates for core libraries and tools.
The explanation could be better, but I really like the idea.
It punishes you for not updating your dependencies and for having too many direct dependencies. But it doesn't punish you for indirect dependencies (that you have little control of), or libraries that are "done" (since it compares to the newest stable release, not the current year). A sensible balance.
Maybe one could write a browser extension to display the libyear of GitHub pages?
Isn't "we are 5 major releases behind" more obvious that "two libyears".
> If your system has a one year old dependency and a three year old dependency, then your whole system is four libyears old.
How far down the tree do we go? Either fully, which means that one project with 365 one day old dependencies is 1 libyear old. Or not at all, as the rails example suggests, in which case if I have a wrapper around rails that I bump to an old rails version, anyone using my wrapper would have older rails but a fewer libyears?
There is no single answer to all of this, because it's too complex to boil down to a single number. But I think it's a bit odd to introduce a whole new thing that doesn't measure at all what's changed.
So that means that the way to game this would be to create a wrapper project (e.g. called "external_deps") for all of my dependencies, and then have my actual project depend on that one. So I'm externalizing all of my tech debt onto external_deps and as long as I never make a new release of that, my main project will always be 0 libyears old.
> So that means that the way to game this would be to create a wrapper project (e.g. called "external_deps") for all of my dependencies, and then have my actual project depend on that one
No, you can't resort to chicanery to manipulate your metric.
In this case, the libraries you want to use become transitive dependencies, but if your code uses those transitive dependencies, then your project still depends on them.
People already do that with vendorizing, too. It was noted the other week on HN of someone "collateralizing" their tech debt by vendorizing a dependency to remove CVE/deprecation/obsolescence warnings for a Rust dependency with no update path: https://news.ycombinator.com/item?id=39827645
A metric like this can't keep you honest (just about no matter how you design it, people will find loopholes), but it can help honest people document their needs.
> There is no single answer to all of this, because it's too complex to boil down to a single number.
Having a single number that is a rough measure is still useful, though perhaps a (weighted?) average would be more useful.
Anyway, I just downloaded the Python tool (called "libyear" on PyPI) and ran it to quite quickly find three dependencies on my project that were over 2 years behind. That was helpful and I would use it again.
I had the same immediate reaction that you did, but the key here is that the age of dependencies is not absolute but calculated against the latest release. So a one-day-old dependency with no succeeding version is 0 libyears old, not 1/365.24. Likewise for a decade-old release, if it is in fact the latest release.
It’s about the time delta between the used version and the latest available version.
Adding transitive dependencies might be tricker, because then you can't tell what of your direct dependencies to upgrade.
It might be more useful to assign each direct dependency's transitives to it, and then you can do maths to figure out whether updating a direct dependency actually has an effect (e.g. 2 directs both with a 50 lib year dependency aren't worth upgrading unless you upgrade both of them.)
They're tracking different related things. I run a startup in this space and we track: aggregate libyear of your direct dependencies; total # of direct dependencies with libyear > 2; # of direct dependencies at least one major version behind; dependencies that have been abandoned by the maintainer.
I think the top-line aggregate libyear number is helpful to monitor over time to get a general sense of the slope of your technical debt. If the number is trending upwards then your situation is getting worse and you're increasing the chance you find yourself in an emergency (i.e., a CVE comes out and you're on an unsupported release line and need to go up major versions to take the patch).
Tracking total # of major versions behind gets at the same thing but it's less informative. If you're on v1 of some package that has a v2 but is actively releasing patches for the v1 line that should be a lower priority upgrade than some other package where your v1 line hasn't had a release in 5 years.
It feels like it just has so many weird edge cases. A stable 2.3 branch that hasn't changed while the 1.2 branch has major security issues punishes you for not using the 1.x version.
A regularly updated 1.x branch for docs/security looks like you're doing fine even though the project is on 3.x and deprecating soon.
Perhaps as a vague guide to point to potential issues, sure.
While granting that libyear is clear in what is measures, I still think it measures the wrong thing. What should we be measuring?
I have some ideas for my projects, but I don't have the answer for your project.
Semantic versioning ain't the only game in town for sure, and I'm not anchoring on it as the best or only way.
But I will say this: when one has figured out what is important to measure, build metrics for that. You almost certainly will need to factor in supply chain security. And probably some metrics for recency about the hardware platforms you deploy to. This could look like a weighted score, perhaps. But it is unreasonable to hope that libyear or semver to do that for you.
Time flows faster in periods of high volatility and slower in periods of low volatility. Instead of measuring time directly it should be adjusted by things like changes committed, LOC added/removed, CVEs opened/closed, etc.
If your team really wants more dependencies than they have the capacity to review, my team and I are happy to help greatly reduce this risk.
My company (https://distrust.co) has reviewed hundreds of dependencies on behalf of our clients, and some even pay for this as a monthly retainer service.
We are not an cyber insurance firm, so we don't offer any guarantees or compensation if you get compromised. We operate the same as any security auditing firm.
One of our clients reviews all their dependencies in-house with their own team and has us be the second pass for extra assurance. They have also tested our team in the past by not revealing vulnerabilities they have already identified to be sure we are indeed reviewing the way we say we do.
Security bugs are everywhere because very few internet rando library authors have infosec experinece, so you are unlikely to get a clean report on any given dependency tree, but /usually/ the items are low to medium risk and can be chased upstream or easily patched out.
Sometimes however we find really problematic ones like CVE-2018-9057 that impact you if you have ever used it even once ever.
Also there is an economy of scale here in that if one of our other clients asked for review of a react dependency last week, we can copy over our opinion and consume fewer hours on the clock.
It's just an absurd idea that your typical, average "modern JS" devs using React + Redux/Next.js (with a truckload of other deps), for example (as is pretty common these days), will review (or wait for a review of or pay for reviews of) the dep tree. People have enough on their plate, I'm willing to put my life on a bet that 90% of JS devs don't review their deps. It's preposterous to even suggest that. Just look into your node modules of these projects. This is done in maybe the top 10% most rich AND security minded businesses out there, mostly in the FAANG or the banking/govt sector. Individual devs or small teams? Forget it. Never worked at a place that did so or even heard of one.
It may be different with other languages and frameworks that may be moving more slowly or the dep tree is much flatter, but JS moves fast, browsers change often, and the dep trees are massive. Bugfixes and small feature upgrades could easily change a truckload of dep code.
Then, as if it wasn't enough, I'm not even sure how would you review something like an Electron-based app... what, you're gonna review the entire Electron-Chromium-Node chain on top of all the mess mentioned above? Sure :D
I would double down on your claim and say 99.9% of JS devs don't review dependencies. Also 100 years ago almost zero doctors washed their hands. The majority of people in a profession doing something does not make it any less negligent.
My customer base is almost exclusively companies in industries like Fintech where giving strangers the ability to push any code they want to production is undesirable.
In one case, the customer has their own internal team review all dependencies -and- has my team review them, and only adds a dependency to the allow list that get a thumbs up from both.
libyear is an opinionated metric that prioritises less well tested software. Meanwhile, companies pay a lot of money for RHEL and other products that promise a stable environment that freezes specific (major) releases of software for years - and also promises backports of any necessary security fixes, without those pesky new features and breaking changes that come with using bleeding-edge releases.
Different people, projects, organisations, all have different risk appetites. We need all of them working together; late adopters wouldn't have the stability they crave if early adopters didn't exist to test the crazy broken fresh software.
While everyone needs to manage dependencies, there's no one right way to do it, so everyone does it their own way. They only thing we can probably agree on is doing _no_ maintenance on dependencies is a bad thing.
Without those pesky new features and breaking changes that come with using bleeding-edge releases.
This is usually a popular counterargument when people are talking about keeping everything up to date. What people should consider though is to try to keep everything *relatively* up to date, without always being on the latest version but still not very far away from the latest release.
GitHub, Stack Overflow, etc., are full of data about potential issues when updating to library X to version Y, and usually, you're able to find this when it's too late - either you've got an error in production or you're in the middle of an update and you discover that there are some issues with the version that you want to use.
Exploring these data points is still a pretty much untapped area, and this is something that I'm trying to explore with my product that updates dependencies automatically in a more "smarter" and autonomous way at depshub.com.
I would be happy to see more people working in this area since it's clear that there is a problem that needs to be solved and unfortunately the current status quo is "while everyone needs to manage dependencies, there's no one right way to do it, so everyone does it their own way."
116 comments
[ 3.0 ms ] story [ 160 ms ] threadDoesn't necessarily tell you what code is legacy – perhaps a function is just so solid, that there was no reason to touch it in years. But I've found such analysis helpful and it can give you warning signs about what knowledge is being lost in the team and which parts of your own codebase became unknown territory.
[0] I know of CodeScene but suppose there are others
Additionally, excluding 'imports', namespacing, and other boilerplate helps too.
If you value some library years more, and some less, then weight the sum.
It's like saying there is no point of natural numbers, because when you count apples, some apples might be rotten.
A more accurate (but more unwieldy to measure) metric would be to count the lines of code that have been changed since the version used and the most recent stable version. (I think this is what commenter amelius implied?) It wouldn't quite capture the nature changes made, but it would very much uncouple from the quite unwieldy assumption that libraries are all developed at the exact same pace.
For example, CLDR changed the UK abbreviation for September from "Sep" to "Sept" and broke a lot of code as libraries used newer versions of the data https://unicode-org.atlassian.net/browse/CLDR-14412
My immediate thought looking at this number was not that it should be minimized but that there ought to be a sweet spot range and a number below which it probably shouldn't go and a number above which it shouldn't go.
The total age of dependencies tell you nothing useful.
If you're not, and very many people are not, then total age of dependencies is a decent low-effort approximation for the probability of bug fixes affecting parts of dependencies that you're using.
I don't care about CVEs that only affect functions my app do not use.
That depends on the changes to the library since, and how and where the library is used.
Suppose I regularly generate a CSV file, all ASCII, where all the rows are integers or fixed precision numbers. I have a ten year old CSV library that processes that file, and has worked without any problem for ten years.
I have no interest in updating the library. Updates can introduce downtime, but provide no improvement. In fact, they introduce a slight performance hit because of new features and that I don't need. There is also the risk that the updates will introduce bugs, and then I'll have to spend time diagnosing the bug, and coming up with a fix.
Now let me reverse this: suppose there are two libraries to do the same task, A and B. They don't have the same features, but for your use case, they are both easy to use and do exactly what you need.
A was first released in the 1980s and was last updated five years ago. It's still maintained and is available in most Linux distributions.
B was first released three years ago and has had 20 updates since, 18 of which included fixes for security issues that don't affect A. (The website for A is regularly updated to indicate that it has been tested and these issues do not affect t.)
Are you better off using A or B?
Because, in general, as you drift behind, the friction of upgrading will increase.
You might not need to update today, but you're not in control of external events that may force your hand (sudden critical security vulns).
> Are you better off using A or B?
In this contrived example, it depends.
Any library that is effectively a dataset could fall into this as well: if you want to freeze your environment at a specific reference point and only update the actual moving parts, the libyear measurement won't be for you.
This reminds me of interface softwares that keep old version of some libraries to emulate the original behavior, butnin a controlled and isolated way.
What complicates this is deciding whether the dependency is under active development or not. If its EOL'd then you still want libyear to accumulate, even if you use the latest released version. I guess comparing to an end-of-life date then would make sense, but it's probably harder keep track of.
However, they only check openly accessible (i.e. OSS) dependencies. If one of those hasn't seen a release in ten years, I would look for an alternative.
For some applications it might be of great use but for a vast and complex applications architecture, the libyear metric might only oversimplify the complexity of dependency management,compatibility issues, updates and security patches, etc
I noticed that it focuses only on the age of dependencies without considering other factors like the how critical is the update, and how stable it is, and the improvements in newer versions, etc.
Initially i thought it would need to be more complex, but but it was more than enough.
Libyear seems like a decent starting measure if there is no appetite for something more in-depth, IMO and YMMV.
Maybe, but couldn't measuring, and thus reacting to, a bad measure be worse than doing nothing?
To me, "decent starting measure if there is no appetite for something more in-depth" sounds like "just drop it in, it's enough to get started, we'll figure out the rest later", but that could be temporarily harmful.
Exercising judgment is the opposite of that, no? Then you're going into depth.
> Rails 5.0.0 (June 2016) is 1 libyear behind 5.1.2 (June 2017).
and
> If your system has a one year old dependency and a three year old dependency, then your whole system is four libyears old.
which don't explain much.
I suspect what they meant to say is that Rails 5.0.0 (June 2016) is 1 year (not libyear) behind 5.1.2 (June 2017) and that the "libyear age" of a project is the sum of how old each of its dependencies is. But if so, they should say so clearly somewhere on their page.
The concept really does seem obvious, especially since it sounds like man year, but it needs better documentation.
But, while I appreciate the need for simplicity, I also wonder if it would be wise to scale dependencies by how prevalent they are in the codebase. For example, if I'm using a five year old version of react but the library I use to convert temperature units is up-to-date, then thats bad. But if I'm using the latest react and the conversion lib is old then thats less bad.
Probably feature creep though...
All the major tools (dependabot, renovate) to keep dependencies up to date treat all the dependencies equally when in reality there are always core libraries (e.g., react) and everything else. While trying to keep *everything* up to date is extremely challenging, what I'm trying to do is to find a balance between what and when needs to be updated (using code static analysis, different data sources, AI etc) and automate it in a simple manner.
Libyears are meaningless. A library either has known vulnerabilities or it doesn't. When it doesn't, old is often better than new one.
Libyear – a simple measure of software dependency freshness - https://news.ycombinator.com/item?id=24975339 - Nov 2020 (16 comments)
What would be other measures that could be similarly useful? Lines of code or story points? Maybe even a number of tests added?
I've been using that alongside some other metrics for providing insights into how behind teams are on updates
What are some other metrics that you are using? I am working on a product that is helping to keep dependencies up to date and would love to integrate some of these things in the product.
The libs we're measuring up to could have their own libyears to upgrade, but we can only control what's in our hands.
Sometimes a small security patch is worth more than a major version bump of features, so I consider measuring the time instead of major versions a benefit.
Maybe we should stop boilerplating everything and write the actual code we need. For the most part softwares usually use a tiny fraction of capabilities of any given library.
Maybe before trying to limit our lag in dependencies update of unlimited levels of libraries we should focus first on having a maximum level of dependencies. Like one project would use a maximum of 2 level of libraries dependencies and you would have to rewrite those that have too many levels.
The javascript ecosystem for instance is totally unmanageable as I see it. We just pretend we have a bit of control but in reality nobody knows what code is executed really and this is sad.
Suddenly less and less is considered core and it's easier than ever to 'outsource' to external libs to save time. Or is it rather that the project gets more velocity because of that?
> We just pretend we have a bit of control but in reality nobody knows what code is executed really and this is sad.
True, this is also slowly starting to be the case with other languages. With Python it can be so bad that even attempting to 'build' and run the same project a year later may well fail. Much to what I'm used to with JavaScript projects by now.
This lead me questioning how good is it to judge a project by its age + last commit (+ project size/complexity + funding/community), as this is what I do in practice. I agree that SemVer isn't really designed to be human-readable and is a rather meaningless / deceiving metric due to divergent practices of different developers.
A couple of years hither or dither with grey-haired Java libraries matters very little. There might be some vulnerabilities but you probably know about them and have workarounds, and sometime next year it's likely you'll be allowed a month or two to do 'life cycle management' in the dependency stack.
Instead of solely focusing on reducing the libyear for your projects, a better approach is to minimize the steps needed to keep your project reasonably up to date. For instance, think about managing 20 PRs weekly to update various package.json packages versus 1 PR for critical dependencies when necessary.
It's important to note that updating dependencies is not a consistent task that can be done at the same pace all the time. Expect varying update volumes and complexities that may need attention at different times. Setting a fixed configuration for, let's say, 10 updates per week may not be effective, as it could lead to dealing with unnecessary updates regularly (e.g., aws-cli, which has almost daily releases).
Finding the right balance between keeping your project up to date and spending too much time on dealing with dependencies is the hardest part here that doesn't have a 100% right answer yet.
I think one thing I've increasingly found is that it's important to set up the infrastructure for parallel building— it's not realistic to have a flag day twice a year, and it's not realistic to try to test everything first "on a branch". If you can have a transitional period of a few weeks where the product outputs (containers, dev environments, whatever) are consistently available in both the old and new flavour, then you can invite people to try the updated thing, while still having an escape lane to keep using the older thing if stuff turns out to be broken in a way that's beyond their capacity to correct in the moment.
i was thinking, would it be helpful to keep track of how far behind each dependency is in terms of minor, patch, and major updates? but this seems a bit too complex to explain to the management. i'm trying to figure out the best way to explain this to management so they understand why it's important to stay current. any ideas on how we can measure improvements? maybe we should agree on a few key factors to track our progress and see if we're getting better or worse.
This is exactly what I've added for depshub.com, and people seem to like it a lot. It just gives you better visibility across all of your connected repositories about what the current status of each dependency is and how the major vs. minor vs. patch ratio changes over time. While it's still a naive metric, it's the easiest to understand and visualize - and as a result, the one that is used the most.
> Any ideas on how we can measure improvements?
- Quantitative: Spend as little amount of time as possible on trying to keep everything relatively up to date (hours/month) - Qualitative: not having any CVE issues, not having major updates for core libraries and tools.
It punishes you for not updating your dependencies and for having too many direct dependencies. But it doesn't punish you for indirect dependencies (that you have little control of), or libraries that are "done" (since it compares to the newest stable release, not the current year). A sensible balance.
Maybe one could write a browser extension to display the libyear of GitHub pages?
> If your system has a one year old dependency and a three year old dependency, then your whole system is four libyears old.
How far down the tree do we go? Either fully, which means that one project with 365 one day old dependencies is 1 libyear old. Or not at all, as the rails example suggests, in which case if I have a wrapper around rails that I bump to an old rails version, anyone using my wrapper would have older rails but a fewer libyears?
There is no single answer to all of this, because it's too complex to boil down to a single number. But I think it's a bit odd to introduce a whole new thing that doesn't measure at all what's changed.
You are the one that has an advantage to know you're behind, not someone else.
No, you can't resort to chicanery to manipulate your metric.
In this case, the libraries you want to use become transitive dependencies, but if your code uses those transitive dependencies, then your project still depends on them.
A metric like this can't keep you honest (just about no matter how you design it, people will find loopholes), but it can help honest people document their needs.
Having a single number that is a rough measure is still useful, though perhaps a (weighted?) average would be more useful.
Anyway, I just downloaded the Python tool (called "libyear" on PyPI) and ran it to quite quickly find three dependencies on my project that were over 2 years behind. That was helpful and I would use it again.
It’s about the time delta between the used version and the latest available version.
It might be more useful to assign each direct dependency's transitives to it, and then you can do maths to figure out whether updating a direct dependency actually has an effect (e.g. 2 directs both with a 50 lib year dependency aren't worth upgrading unless you upgrade both of them.)
I think the top-line aggregate libyear number is helpful to monitor over time to get a general sense of the slope of your technical debt. If the number is trending upwards then your situation is getting worse and you're increasing the chance you find yourself in an emergency (i.e., a CVE comes out and you're on an unsupported release line and need to go up major versions to take the patch).
Tracking total # of major versions behind gets at the same thing but it's less informative. If you're on v1 of some package that has a v2 but is actively releasing patches for the v1 line that should be a lower priority upgrade than some other package where your v1 line hasn't had a release in 5 years.
A regularly updated 1.x branch for docs/security looks like you're doing fine even though the project is on 3.x and deprecating soon.
Perhaps as a vague guide to point to potential issues, sure.
IE, it discourages someone from sucking in a library just to use one tiny function that they could recreate in 10-15 minutes.
I have some ideas for my projects, but I don't have the answer for your project.
Semantic versioning ain't the only game in town for sure, and I'm not anchoring on it as the best or only way.
But I will say this: when one has figured out what is important to measure, build metrics for that. You almost certainly will need to factor in supply chain security. And probably some metrics for recency about the hardware platforms you deploy to. This could look like a weighted score, perhaps. But it is unreasonable to hope that libyear or semver to do that for you.
Time flows faster in periods of high volatility and slower in periods of low volatility. Instead of measuring time directly it should be adjusted by things like changes committed, LOC added/removed, CVEs opened/closed, etc.
Blindly upgrading is worse than never upgrading unless you are addressing a specific CVE that impacts you.
Public open source code is code you did not have to write which can be a time saver, but you do not get to skip code review.
If you do not have time to review 2000 dependencies, then you should drop them favoring simple functions that only do what you need.
If your team really wants more dependencies than they have the capacity to review, my team and I are happy to help greatly reduce this risk.
My company (https://distrust.co) has reviewed hundreds of dependencies on behalf of our clients, and some even pay for this as a monthly retainer service.
JS dependency debt is good for my business.
One of our clients reviews all their dependencies in-house with their own team and has us be the second pass for extra assurance. They have also tested our team in the past by not revealing vulnerabilities they have already identified to be sure we are indeed reviewing the way we say we do.
Security bugs are everywhere because very few internet rando library authors have infosec experinece, so you are unlikely to get a clean report on any given dependency tree, but /usually/ the items are low to medium risk and can be chased upstream or easily patched out.
Sometimes however we find really problematic ones like CVE-2018-9057 that impact you if you have ever used it even once ever.
Also there is an economy of scale here in that if one of our other clients asked for review of a react dependency last week, we can copy over our opinion and consume fewer hours on the clock.
It may be different with other languages and frameworks that may be moving more slowly or the dep tree is much flatter, but JS moves fast, browsers change often, and the dep trees are massive. Bugfixes and small feature upgrades could easily change a truckload of dep code.
Then, as if it wasn't enough, I'm not even sure how would you review something like an Electron-based app... what, you're gonna review the entire Electron-Chromium-Node chain on top of all the mess mentioned above? Sure :D
My customer base is almost exclusively companies in industries like Fintech where giving strangers the ability to push any code they want to production is undesirable.
In one case, the customer has their own internal team review all dependencies -and- has my team review them, and only adds a dependency to the allow list that get a thumbs up from both.
https://en.wikipedia.org/wiki/Diffusion_of_innovations
libyear is an opinionated metric that prioritises less well tested software. Meanwhile, companies pay a lot of money for RHEL and other products that promise a stable environment that freezes specific (major) releases of software for years - and also promises backports of any necessary security fixes, without those pesky new features and breaking changes that come with using bleeding-edge releases.
Different people, projects, organisations, all have different risk appetites. We need all of them working together; late adopters wouldn't have the stability they crave if early adopters didn't exist to test the crazy broken fresh software.
While everyone needs to manage dependencies, there's no one right way to do it, so everyone does it their own way. They only thing we can probably agree on is doing _no_ maintenance on dependencies is a bad thing.
This is usually a popular counterargument when people are talking about keeping everything up to date. What people should consider though is to try to keep everything *relatively* up to date, without always being on the latest version but still not very far away from the latest release.
GitHub, Stack Overflow, etc., are full of data about potential issues when updating to library X to version Y, and usually, you're able to find this when it's too late - either you've got an error in production or you're in the middle of an update and you discover that there are some issues with the version that you want to use.
Exploring these data points is still a pretty much untapped area, and this is something that I'm trying to explore with my product that updates dependencies automatically in a more "smarter" and autonomous way at depshub.com.
I would be happy to see more people working in this area since it's clear that there is a problem that needs to be solved and unfortunately the current status quo is "while everyone needs to manage dependencies, there's no one right way to do it, so everyone does it their own way."