As the article says, I'm not convinced code signing would have made this not happen - given the distribution model you could trivially just have node library updates check to see whether the owner/maintainer had the same user name.
Similarly it is absolutely not the fault of the old maintainer - no one should have to commit to permanently supporting a library they wrote years in the past. And if you weren't maintaining a library why would you necessarily be opposed to transferring ownership over to someone who is at least apparently working on/with it?
Because they could have and should have just forked it.
I wouldn't want my name being forever attributed to something I stopped working on years ago. If I make a lib with my name on it it's my lib and my name's gotta stay on it forever. Feel free to clone it and do whatever you want with it, but this one should stay mine for no other reason than the one we're seeing today. Then link to the latest maintained fork in your README.
The problem is not the github repo, it’s the npm directory - specifically all those libraries that depend on libX resolve the library by that name. If you ever want to migrate ownership/maintainer of a library host through npm you have to take over the name.
The alternative is changing the name (should libfoo.js become libfoo2.js? Libfoo-maintainer_name.js) etc.
Then all the libraries that depend on it have to change their deps and their actual code (require(x) or import x are both based on library name).
If they don’t then any libraries that transitively include the old version of the library could be compromised by any bugs that happen to be included in the old version, so you also can’t say “everyone who cares about the library should update” because library A may depend on library-B which isn’t updated.
If anything the problem here is that npm doesn’t have any real “the maintainer has changed” notification, and that’s couple with the average developer not having any way to know if that change is legitimately ok without auditing the published code.
Alternatively, they could add a freeze option that locks to the last major version, and all newer releases would be a full release version when you switch maintainers.
We don't need a standard library, they said. Letting basic functionality be spread out through a hundred thousand separate packages is fine, they said...
what would you put in the standard library? Specifically remembering that if it’s part of the standard library it has to be present in all JS environments.
There is a standard library... what comes in Node, or what comes in the Browser... anything beyond that is non-standard. I'm not sure where your problem is here. It's true of any other platform on the planet, some things are in the box, others are not.
The complain is that Node/the browser's "box" is too small. OP wants a "batteries included" standard library.
If size/resource consumption is a concern, the library could be broken up and `import`ed as necessary now that modules are a thing. Whatever standards body that chooses to implement this wouldn't even need tot worry about breaking old code and name collisions, as it could all be opt-in.
The attack was present in the minified JS, but not the source JS.
(I'm ignorant about NPM.) Does this mean on NPM, the publisher gets to publish the source AND the minified version? Is there no validation that the minified code is sourced from the true source code?
That's right, what gets published is published from a dev's machine (or perhaps CI), which can (and often does!) include files (perhaps build artifacts) which aren't committed to source control. Similar to how perhaps a Java package might have the source in source control, but you push a built JAR to your package manager.
Publishing a package on npm is not much more than uploading a .tgz. You could put _anything_ in it as long as it has a package.json at its root. Nothing is done on Npm's side to ensure things are what they claim they are upon publishing: they only check (superficially) if you are not by any chance trying to republish the latest version. Even that is done locally, not at the registry level.
This is so superficial. How can you talk about ecosystem fragility when NPM is one of the best open source companies and which actually cope with problems (unlike most package registries who are completely passive)
The ecosystem is more than just NPM the company (or tool, or registry...). And I think the problem is far from superficial, if it was superficial it'd be trivial to fix.
In theory the same applies to PyPI or Rubygems or whatever else you use too, but it's the fact that JavaScript packaging culture has grown to have many, small "does one thing well" packages that makes it harder to be across those which you are giving trust to.
Aside from validating your dependencies, when you run the code ensure it can't phone home.
On AWS, lock-down outgoing traffic. The Node instance (heaven help you if it's in a browser) on the back-end shouldn't be able to get out to anything, with the necessary exceptions of your codes back-end data store, and, perhaps temporarily, your deployment store.
The trust issue is a fundamental issue with volunteer / anonymously-contributed open source.
At least with paid software, in theory if not in practice, the organization publishing the software is responsible for verifying it, and for keeping track of whom it is employing to contribute to the code, so you can figure out who to lock up if they intentionally push malicious code.
Any thoughts on how these situations could be automatically detected by a third party given the current realities of the JS ecosystem? Here are some quick ideas:
1. Detect commits that are radically different from previous ones in the same repo by some measure (tricky)
2. Detect npm publish events whose contents are different than source repo
3. Detect author/ownership changes in the context of npm packages
Require all public packages published from accessible source code built/compiled by the package repository project directly...
This would have the side effect of requiring likely paid exceptions for popular binary packages or those that have more complex build requirements. However, the vast majority of npm packages could probably be built via some form of template.
npm test - must have a configured script in package.json and must not error (this is easy enough to work around, but should be required)
npm run build - should be added as a required step, where the output to be packaged is outputted process.env.BUID_OUTPUT as a directory.
This would at the very least minimize some of the risks... signed commits could be an additional step, but coordination to make tooling for this easier to use would have to happen.
Also, ownership changes should have a FREEZE only allowing a new Major release after transfer of a package.
The problem is that if you import 1000 packages by 500 different authors, you must trust 500 authors.
You must trust them not only to not be malicious, but not to have been compromised. How can there be an automated solution to this problem? The only solution is to develop some discipline around dependencies. Unfortunately the current culture around Node/NPM has gone in the opposite direction.
Another guy blaming a programming language because a third party independent company dont check out every single piece of line for malicious code. Kids these days think injecting 400 dependency libraries is a better idea than writing their own code. P.S Always remember, it's your responsibility to check out what code library you adding to your project.
sorry if it came across as blaming, I was trying to establish the structural issues with the ecosystem and reflect on ways we can improve on it. The classic "reflect, iterate & improve" cycle. Thanks for the feedback that that message was lost, I'll try better next time.
Two reasons. Either you clickbaiting with some catchy titles or you really believe it's reasonable next time someone push malicious code to Nuget I will start writing an article with a title "Why C# sucks..."
If "clickbait" is writing enticing headlines, sure, I'm proud to be "guilty" of that, but the usual definition of clickbait is if the article contains no substance. I hoped that timeline and reflections would add value though. If you want to write a blog post called "Why C# sucks" if you feel the language and ecosystem specifically enables that go ahead. I don't really understand your hostility here, and I'm sorry if you feel I wasted your time with the summary. I'll choose not to engage further with you at this point as I'm not finding your feedback constructive.
It seems reasonably constructive to me. If I were to simplify what they're saying, with a nicer tone, it'd be "if you mean what you say, say what you mean".
I like to think of academic journal articles in that regard: the title neatly summarises the abstract (which itself summarises the article), so there's as little ambiguity as possible.
I have no opinion on the article either way, might just be something to sit at the back of one's head for next time. I get the feeling HN has two crowds: those who prefer newspaper-style articles and those who prefer academic journal-style articles. Difficult to find the happy medium (sic).
Maybe it could come down to requiring npm modules be accessible on a publicly accessible git repository with enough instructions in the package to build that package's output. Exceptions would be the more custom/large packages than the rule. Binary packages would of course also need some exceptions.
There could be a concept of "trusted" packages. That have had at least a review step. So either your package is trivial and easily automated or difficult with manual review. Where the costs are absorbed is the thing.
In the end, I do feel that anything that's being published publicly should have publicly available source, and that the build should be repeatable at the least... that wouldn't prevent such a thing (if it was checked into git), but would at least make it harder to hide.
What I don't understand, how come there are no automatic red flags/or permissions when a deep dependency attempts to write on desk or make network calls?
I mean in theory any npm package could have a payload that would hijack the machine or override other system functions, where is the security in any of that?!
27 comments
[ 4.3 ms ] story [ 79.5 ms ] threadAs the article says, I'm not convinced code signing would have made this not happen - given the distribution model you could trivially just have node library updates check to see whether the owner/maintainer had the same user name.
Similarly it is absolutely not the fault of the old maintainer - no one should have to commit to permanently supporting a library they wrote years in the past. And if you weren't maintaining a library why would you necessarily be opposed to transferring ownership over to someone who is at least apparently working on/with it?
I wouldn't want my name being forever attributed to something I stopped working on years ago. If I make a lib with my name on it it's my lib and my name's gotta stay on it forever. Feel free to clone it and do whatever you want with it, but this one should stay mine for no other reason than the one we're seeing today. Then link to the latest maintained fork in your README.
The alternative is changing the name (should libfoo.js become libfoo2.js? Libfoo-maintainer_name.js) etc.
Then all the libraries that depend on it have to change their deps and their actual code (require(x) or import x are both based on library name).
If they don’t then any libraries that transitively include the old version of the library could be compromised by any bugs that happen to be included in the old version, so you also can’t say “everyone who cares about the library should update” because library A may depend on library-B which isn’t updated.
If anything the problem here is that npm doesn’t have any real “the maintainer has changed” notification, and that’s couple with the average developer not having any way to know if that change is legitimately ok without auditing the published code.
If size/resource consumption is a concern, the library could be broken up and `import`ed as necessary now that modules are a thing. Whatever standards body that chooses to implement this wouldn't even need tot worry about breaking old code and name collisions, as it could all be opt-in.
The attack was present in the minified JS, but not the source JS.
(I'm ignorant about NPM.) Does this mean on NPM, the publisher gets to publish the source AND the minified version? Is there no validation that the minified code is sourced from the true source code?
In theory the same applies to PyPI or Rubygems or whatever else you use too, but it's the fact that JavaScript packaging culture has grown to have many, small "does one thing well" packages that makes it harder to be across those which you are giving trust to.
On AWS, lock-down outgoing traffic. The Node instance (heaven help you if it's in a browser) on the back-end shouldn't be able to get out to anything, with the necessary exceptions of your codes back-end data store, and, perhaps temporarily, your deployment store.
At least with paid software, in theory if not in practice, the organization publishing the software is responsible for verifying it, and for keeping track of whom it is employing to contribute to the code, so you can figure out who to lock up if they intentionally push malicious code.
1. Detect commits that are radically different from previous ones in the same repo by some measure (tricky)
2. Detect npm publish events whose contents are different than source repo
3. Detect author/ownership changes in the context of npm packages
What else?
This would have the side effect of requiring likely paid exceptions for popular binary packages or those that have more complex build requirements. However, the vast majority of npm packages could probably be built via some form of template.
npm test - must have a configured script in package.json and must not error (this is easy enough to work around, but should be required)
npm run build - should be added as a required step, where the output to be packaged is outputted process.env.BUID_OUTPUT as a directory.
This would at the very least minimize some of the risks... signed commits could be an additional step, but coordination to make tooling for this easier to use would have to happen.
Also, ownership changes should have a FREEZE only allowing a new Major release after transfer of a package.
You must trust them not only to not be malicious, but not to have been compromised. How can there be an automated solution to this problem? The only solution is to develop some discipline around dependencies. Unfortunately the current culture around Node/NPM has gone in the opposite direction.
I like to think of academic journal articles in that regard: the title neatly summarises the abstract (which itself summarises the article), so there's as little ambiguity as possible.
I have no opinion on the article either way, might just be something to sit at the back of one's head for next time. I get the feeling HN has two crowds: those who prefer newspaper-style articles and those who prefer academic journal-style articles. Difficult to find the happy medium (sic).
There could be a concept of "trusted" packages. That have had at least a review step. So either your package is trivial and easily automated or difficult with manual review. Where the costs are absorbed is the thing.
In the end, I do feel that anything that's being published publicly should have publicly available source, and that the build should be repeatable at the least... that wouldn't prevent such a thing (if it was checked into git), but would at least make it harder to hide.
I mean in theory any npm package could have a payload that would hijack the machine or override other system functions, where is the security in any of that?!