Won’t this be solved fairly soon when package managers have automatic scanning of updates by AIs that are superhumanly good at spotting malicious code?
> No one will learn their lesson. This has been happening for decades and no one has learned anything from it yet. This is the defining hubris of this generation of [X].
> Perhaps Google and Mozilla, leaders in JavaScript standards and implementations, will start developing a real standard library for JavaScript, which makes micro-dependencies like left-pad a thing of the past. This could be combined with a consolidation of efforts, merging micro-libraries into larger packages with a more coherent and holistic scope and purpose, which prune their own dependency trees in turn.
This is as big as a strawman as I can imagine. Both of these "solutions that won't happen" are already happening:
- The ECMAScript standard already defines a `Strong.padStart()` as part of the "real" standard library of Javascript [1]
- There is a very well known larger package that combines many micro-utilities like this into one, lodash [2]
> No one will learn their lesson. This has been happening for decades and no one has learned anything from it yet. This is the defining hubris of this generation of software development.
Really seems like the author wants to hate on the ecosystem for the sake of hating on it.
> Perhaps Google and Mozilla, leaders in JavaScript standards and implementations, will start developing a real standard library for JavaScript, which makes micro-dependencies like left-pad a thing of the past.
It's not wrong, but this take is kind of tired and well out of date. For about a decade or so left-pad's functionality has been standard in all browsers or runtimes. Plenty of other micropackages have been obsoleted as well and the current zeitgeist is to avoid publishing or using any sort of micropackage.
"Zero dependencies" is now a top marketing term in the frontend world. Unfortunately, their removal is an ongoing process and it's taken way too long already to fully purge the ecosystem of these packages. However, it's not because the JavaScript community has never thought of this issue before. "Add more features to the JS standards and don't use is-number" is not a particularly new idea or valuable insight.
But beyond that, there were plenty of not-tiny packages impacted as well. Continuing to beat this dead horse may be fun, but it distracts from the actual issue here.
Oh, this old chestnut. "Just do what the distros do".
OK, sure, let's pencil this out.
Debian has ~1k volunteers overseeing ~20k packages. Say the ratio is 20:1.
npm alone -- not counting other ecosystems, just npm -- has 3 million packages.
So you'd need 150k volunteers. One hundred and fifty thousand unpaid individuals, not counting original authors.
For one repo.
"Nonsense", you riposte. "Only maybe 100k of these packages are worth it!"
Cool, cool. Then you'd need "only" 5 thousand volunteers. Debian maxed out at 1k and it is probably the source of the most-used software in history. But sure, we'll find 5 thousand qualified people willing to do it for free.
Oh, but how do you identify those 100k packages? OK, let's use download count. Or maybe reference count. Network centrality perhaps? Great, great. But some of them will be evicted from this paradise of rigorous repackaging. What replaces them? Oh, shoot, we need humans to go over up to 3 million packages to find the ones we want to keep.
What I need distro boosters to understand is that the universe of what is basically a package manager for large C libraries is at least two orders of magnitude smaller than everything else, bordering on three if you roll all the biggest repos together. The dynamics at language ecosystem scale are simply different. Yelling at the cloud that it should actually be a breeze isn't going to change things.
Have been seeing these rants since the incident. Yet no concrete suggestions. Just high level hand wavy stuff like "better package management". What does that mean? We already have mandatory 2 factor, private npm registries.
Ultimately the reason the ecosystem is so fragile is because a ton of packages are maintained by solo devs. So it only takes one hack to impact a ton of code bases.
The only thing I can think of to prevent this is automated LLM scanning of every npm package when any dependency or subdependency (and that's its own gnarly tree) is updated.
10 comments
[ 1.5 ms ] story [ 32.6 ms ] threadhttps://news.ycombinator.com/item?id=45286526
yarn feat: implement npmMinimumReleaseAge and npmMinimumReleaseAgeExclude config options
https://github.com/yarnpkg/berry/pull/6901
Now define "unintended side effect"
Now add "no one is maintaining it anymore"[0]
-------
[0] https://xkcd.com/2347/
[X]
"software development"
"climate change"
"healthcare reform"
"political polarization"
...
This is as big as a strawman as I can imagine. Both of these "solutions that won't happen" are already happening:
- The ECMAScript standard already defines a `Strong.padStart()` as part of the "real" standard library of Javascript [1]
- There is a very well known larger package that combines many micro-utilities like this into one, lodash [2]
> No one will learn their lesson. This has been happening for decades and no one has learned anything from it yet. This is the defining hubris of this generation of software development.
Really seems like the author wants to hate on the ecosystem for the sake of hating on it.
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
[2] https://lodash.com/
It's not wrong, but this take is kind of tired and well out of date. For about a decade or so left-pad's functionality has been standard in all browsers or runtimes. Plenty of other micropackages have been obsoleted as well and the current zeitgeist is to avoid publishing or using any sort of micropackage.
"Zero dependencies" is now a top marketing term in the frontend world. Unfortunately, their removal is an ongoing process and it's taken way too long already to fully purge the ecosystem of these packages. However, it's not because the JavaScript community has never thought of this issue before. "Add more features to the JS standards and don't use is-number" is not a particularly new idea or valuable insight.
But beyond that, there were plenty of not-tiny packages impacted as well. Continuing to beat this dead horse may be fun, but it distracts from the actual issue here.
OK, sure, let's pencil this out.
Debian has ~1k volunteers overseeing ~20k packages. Say the ratio is 20:1.
npm alone -- not counting other ecosystems, just npm -- has 3 million packages.
So you'd need 150k volunteers. One hundred and fifty thousand unpaid individuals, not counting original authors.
For one repo.
"Nonsense", you riposte. "Only maybe 100k of these packages are worth it!"
Cool, cool. Then you'd need "only" 5 thousand volunteers. Debian maxed out at 1k and it is probably the source of the most-used software in history. But sure, we'll find 5 thousand qualified people willing to do it for free.
Oh, but how do you identify those 100k packages? OK, let's use download count. Or maybe reference count. Network centrality perhaps? Great, great. But some of them will be evicted from this paradise of rigorous repackaging. What replaces them? Oh, shoot, we need humans to go over up to 3 million packages to find the ones we want to keep.
What I need distro boosters to understand is that the universe of what is basically a package manager for large C libraries is at least two orders of magnitude smaller than everything else, bordering on three if you roll all the biggest repos together. The dynamics at language ecosystem scale are simply different. Yelling at the cloud that it should actually be a breeze isn't going to change things.
Ultimately the reason the ecosystem is so fragile is because a ton of packages are maintained by solo devs. So it only takes one hack to impact a ton of code bases.
The only thing I can think of to prevent this is automated LLM scanning of every npm package when any dependency or subdependency (and that's its own gnarly tree) is updated.