GitHub suspended access to their account for a commit to their own software, because it caused a problem for all these companies. For one it shouldn't have, like pinning a dependency and auditing all changes should be done ideally. These libraries are always licensed in a way that excludes warranty of any kind.
But I honestly don't care if companies "exploit" open-source software by making money using them and not donating to the developer. That may be unhealthy for the ecosystem, but neither side is entitled to anything. I would donate, but not expect a donation, and poisoning the well the way these developers did is not going to help any of us.
That seems like a bit of a shaky ground to stand on for GH.
If someone publishes code for themselves, and at no time asks anyone to take it as a dependency, then at a later date they change that code in a way that breaks other people's use of it, do GH then take over the account?
Sure, trying to find exactly where the malicious line gets crossed is pretty hard and subjective, and maybe that will bite GH one day. But this specific case is not anywhere near that line, the sole intent of those commits was to break others, and he admitted so himself.
This is like arguing about whether the james webb telescope really is in space since we don't have a precise consensus about what altitude is considered the frontier with space.
I'm sure this case is clear, my point was around the wider principal that by going down this line GH set themselves up as arbiter of "malice"
To take a trickier example, say a GH user has a lib, then decides to re-architect it, breaks the API and for their own purposes pushes it to an existing version, breaking all other use of it. Now that's a nasty thing to do, but is it malice?
Another, real-world, example is I know of a user who publishes "honey PoCs" for security issues, where the repo. appears to be a exploit code but actually isn't. He's been accused of malice in doing this, but his intent is research for a talk on how people use code blindly without testing.
Is that malice, should GH take his account down?
By stepping into this area GH are going to have to find answers to this and also the problem of who maintains the repos of accounts they nuke?
> By stepping into this area GH are going to have to find answers to this a
I think they already have, those 2 examples you mentioned already happened and were dealt with.
I think intent is important to take into consideration, since after all that is the definition of malicious: intent to cause harm.
Your first example clearly has no intent to cause harm. That case probably happened thousands of time already since not everyone is willing/able to follow semver cleanly and strictly. Never heard about GH taking any measure against that. And I would definitely not expect them to as a user/maintainer.
For the second case, I think GH policy is that you can host that kind of PoCs, but the repo has to be clearly documented as doing such (e.g. you can't just add some vuln into some unrelated code "for research"), and the vulnerability cannot be an active one: "We understand that the publication and distribution of proof of concept exploit code has educational and research value to the security community, and our goal is to balance that benefit with keeping the broader ecosystem safe. In accordance with our Acceptable Use Policies, GitHub disabled the gist following reports that it contains proof of concept code for a recently disclosed vulnerability that is being actively exploited." - GitHub." [1]
Back to Marak's case, my opinion is that GH did the right thing: If he just had his code in a repo, with no semver, no other contributors/maintainers, and such, and decides to nuke it, then I hope GH would not have done anything.
But when you are using all the tools and trust of open source: Other people contributing to your repo, other people being active maintainers/admins and spending times out of their days to fix bugs on it, when you leverage NPM to make it easier for you to distribute your package to others widly etc, you give up the privilege being able to act unilaterally like an asshole without consequences.
The thing is, intent isn't always clear and oftentimes Github are unlikely to have all the context needed. It's easy to determine with simple examples, the real world is often messier.
For example in that first case, say all they had was a wave of people saying "x broke my application" it would look a lot like the case in the article, and they'd have to dig in to find out it was just a bad API change without semver being followed.
Also requires Github to have a staffed department to deal with this, now they've established themselves as the arbiters.
For me, there's a split between a repository (NPM) and a hosting company (Github). For this case I'd have forked the repo, rolled back the malicious change in the fork, and hooked the fork up to NPM, and leave the original GH account alone. That solves the problem of the breakage, without getting in to banning whole GH accounts.
I happened to create an infinite loop in some of my programs but not intentionally like the author did. Furthermore he intentionally pushed the infinite loop with the result to DoS everybody knowingly or unknowingly using his software. This is malicious behavior IMHO.
Why would the developer of any software that comes explicitly without warranty be hold responsible for downstream breakages? It's not as if one could force people to upgrade to newer versions and they can always keep depending on the old releases.
I'm really struggling to see how either of those two sentences fits into this conversation. "Code is speech" seems to be pointing in the direction of a spurious First Amendment argument, "stop listening" seems counterproductive, and the example of someone defacing their own Facebook page is at the very least incomplete without you saying whether or not you think Facebook would be obliged to continue hosting the defaced page.
In this case, the developer's behavior was malicious: they intentionally caused damage. This is very different than some good faith change that breaks stuff downstream. Sure, the license says "no warranty". But github can decide that they won't tolerate vandals on their platform. It would be within their right to revert the bad change from the git database they hold, go back to the last good change and lock the developer out.
Hey you're the one stating they caused damage. They printed some zaglo strings. Hard to see how that damages anything other than making a few CI jobs fail.
Thanks for stating the obvious. It isn't silly at all to publish malware and vaporize your reputation, right?; maybe it was good after all, people will become careful.
Maybe he wanted libraries that printed blather in an infinite loop. Then it can't be "malware" to put that in his own repositories.
If other people don't want that, then they shouldn't pull from his repositories. If they do that anyway, then that's their own fault. Nobody forced them to.
> the author themselves said in this case, that the reason to submit the malware was to give a "fuck you" to the big corps.
Yeah, so obviously he did want libraries that give a "fuck you" to the big corps (by printing blather in an infinite loop). Then it still can't be "malware" to put that in his own repositories.
And my point still stands: If other people -- you, big corps, whoever -- don't want that, then they shouldn't pull from his repositories. If they do that anyway, then that's still just as much their own fault. Because, still, nobody forced them to.
"Kind of" is doing some pretty heavy lifting there. No, you don't "have to"; you're perfectly free to write your own software in stead. Or even just use a prior version of his code that does what you want it to, in stead of blindly updating to one that doesn't. He didn't force you (or the writers of whatever software you're using) to update, now did he?
Was the author aware that "hoobs and its security camera plugins" were going to break from this push? Or any prod servers, for that matter?
I see no code in there that checks if it is running in production. In fact, it is a reasonable expectation that people don't throw code into production blindly, but rather test any changes out first.
malware is malware. You don't have a right to change ur software to malware. "wElL yOu ShOuLd HaVe Tested" no you shouldn't push software in bad faith designed to crash apps that use it.
> You don't have a right to change ur software to malware.
Yes, I do. I may not have a right to push malware onto unwilling victims, but I absolutely have a right to change _my_ software however I want.
> "wElL yOu ShOuLd HaVe Tested"
Please, no need to be childish here. I have not taken that tone, nor will I respond to it in kind here.
> no you shouldn't push software ... designed to crash apps that use it.
Show me where a `git push` == "push[ing] software ... to ... apps that use it". When the `git push` is to my own repository, mind you, not someone else's app.
> ... in bad faith ...
Finally, I agree with you on something.
Of course this was in bad faith! That was clearly the point. When I write software and put it out there, and somebody comes and uses it, and I break my software to spite them, I am obviously acting in bad faith towards my users.
But that does not make it malice, or my software malware. I did not reach down into other people's computers/apps and change what they run.
Broke cli tools (firebase for me), computer crashed because of some weird infinite loop out of memory error that I wasn't able to recover from. Anyway good wake up call not relaying on million of deps for npm works hopefully.
No, you broke your CLI tools. By unthinkingly pulling in stuff from someone else's repository without vetting it. Or using other tools which did, which amounts to the same thing.
He didn't force anyone to update to the new version, right? So how is it his problem? Some other entity had to go and update the version they depend on.
And if you now say "well, that happens automatically", I say; suites them right. They should have tested the stuff.
The warranty issue is a red herring. A warranty is an affirmative guarantee of quality: you are (in essential concept of not in precise detail) agreeing to be held to a "strict liability" standard. If I buy real estate and I am granted a warranty deed, and the title to the property comes into question, the seller can be brought to account to make me whole or indemnify me, regardless of who is at fault for the title defect.
Without a warranty, you're not held to strict liability, but you can probably be held liable under the default legal regime. If I buy real estate and get a quit-claim deed, there is no promise that the seller has unencumbered rights to the property. However, if I can show the seller intentionally defrauded me, they can still be held civilly and criminally liable for the fraud.
I don't think he pushed malware, did he? He just broke his own project and published the broken version. That's not pushing malware.
I don't get why people don't just pin versions, honestly.
I'm not saying he did a good thing. But neither did he push malware nor has he any obligation to publish unbroken packages. If you're using FOSS projects without a service contract, don't whine if something breaks.
Let's say I set up a lemonade stand in my neighborhood every weekend, where I pour a bunch of cups for people to take, put up a sign that says it's free, and I set out a tip jar.
After a few weeks, I get upset that people have been taking the lemonade without
leaving tips, so the next time I set up the stand I add a toxin that I know will cause immediate damage to anyone who ingests it. To protect myself, I have of course been posting a sign every weekend that says the lemonade is provided as-is.
So – did I do something wrong, or not? Will a court look at this situation and say, "gee, he just poisoned his _own_ lemonade and set it out for public use, it's not like he forced anybody to drink it"?
This feels like 100% black-and-white criminal conduct, and I would hope anyone who pulls a malicious stunt like this would be held liable for it.
I don’t understand the mindset of open source developers who dedicate significant time energy and life to free software, unless there’s a tangible, quantifiable advantage to doing so.
That advantage may well be indirect such as reputational or learning. I just don’t grasp why people do it for nothing, to the advantage of large companies.
It's quite simple: there IS a "tangible, quantifiable advantage to doing so". The problem is that you imply "...to the person writing the code". That's where your confusion lies.
I am getting huge value from the people who built stuff before me. When I build stuff I can (hopefully) make the world better in the future. That's a "tangible, quantifiable advantage" to doing open source. It's just not an advantage to me personally. But lift your gaze an inch off the ground and you'll see we don't need to be ego centric sociopaths. We can build together. For the species. Everyone wins.
> But lift your gaze an inch off the ground and you'll see we don't need to be ego centric sociopaths. We can build together. For the species. Everyone wins.
I don't know in what fairy tale you live in but the ego-centric billionaire sociopaths that exploit this system wins.
> I don’t understand the mindset of open source developers who dedicate significant time energy and life to free software, unless there’s a tangible, quantifiable advantage to doing so.
They get: meaning, status, influence, connections, reputation, and opportunities
The free and open nature of their contribution makes it much easier to get all these benefits than they would with a paid and proprietary solution.
Because it's fun ya mook. That's it. That's the reason.
It's fun to tinker. It's fun to put things out there into the ether. It's fun to exercise the brain and try new things and learn new ways to do things and publish things. The second it stops being fun, we stop.
I’ve realized the idea that the “Hacker” part of “Hacker News” is no longer here, and just a nod to some ancient, possibly apocryphal, past.
Discussions now are about how you shouldn’t run your own server, and you should use popular stuff so you can speed up development and get your startup going.
I mean, I know about ycombinator and all. But it doesn’t seem to truly encompass the hacker spirit, if you ask me.
Actually, in my experience, people that either run their own servers and/or encourage to do so are vastly overrepresented. Interesting self-hosted projects regularly make it to the front page, too. There are, of course, a lot of people and opinions on here, but the overall hacker spirit seems to be alive and well.
I was just recently starting a blog series and attempting to pair it with a YouTube channel for a new project. If you want to monetize a project, I'd assume that would be a way to do it.
They can and they did, but it still feels malicious because they intentionally reverted the maintainer's latest version, which is the author's will on their creation.
It's a bit like me going to my bank to close the account and instead they throw me out and keep my money.
Honestly I don't understand why microsoft did anything at all. Can't people just pin a version?
doesn't npm have policies for packages to follow semvar? I could see why they would have policies to rollback broken minor versions that are distributed via npm.
Packages are literally remote code exec vulns in the hands of package authors. At the very least, it takes them under a minute to break your app, simply by deleting their package. Read the article. This is not the first time it's happened, and it's not going to be the last. [0]
I write backends (mostly in PHP, although not exclusively), and I release a lot of my code under libre licenses. But I don't do packages. I don't want that level of control over other people's projects, it's scary as fuck. I have enough responsibilities as is.
I have a mailing list for people who use my code, when an update is out they can download the .php files, 'require' them and test them before deployment, but never will I do packages.
IMO, re-inventing the wheel sometimes is not the worst thing. Including code written by strangers that you haven't inspected and that they can remotely modify is. Stop using packages that are essentially wrappers around three-line Stack Overflow answers.
In this case, the old-fashioned way is the better way, and you'll have a hard time convincing me otherwise.
> I don't want that level of control over other people's projects, it's scary
How far do you take this though? The average GNU Linux distro ships with a whole pile of packages already installed, from a multitude of different authors.
I know it's bad practice, but I just checkin vendor files/libs to source control. Makes auditing new releases of libraries a bit easier. Assuming they aren't binaries of course.
You're committing something that is not a part of your source code into your version control system, assuming your source code is git, this is irrevocable without rewriting history.
It's mainly a nuisance. It takes up unnecessary space. Introduces possible annoying merge conflicts etc etc and it's not trivial to remove it.
As reference, I migrated repositories from TFVC to git. One team relies on checking in packages into source control, another one does so far less. One repo is significantly nimbler.
Checking packages into source control is making your VCS a package manager. Presumably you have one. Don't hammer nails with your screwdriver
The benefit of having your dependencies vendored is that you have everything needed to build your application without having to download stuff from the internet. You get to ensure what exactly makes it into your application. Yes, it will increase the repository size, but I don't see why merge conflicts would be a problem since you are just replacing a file with a new version.
You would have one anyways for changing the lock file or whatever. Changing your dependencies is something that you may want to be able to undo. It's useful to be able to go back to a known working version of your program with versions of your dependencies that you know work.
> But this cache is usually not easily transferable to someone compared to them just cloning a repo
right, so they need to download "stuff from the internet". it Doesn't matter much if that stuff is from a remote repo or hosted by a package repository. Except if it's architecture dependent, in which case you definitely don't want to share across architectures. Not to mention they may already have a viable copy in a proxy or cache
> You have the source code to all of the dependencies in your application
I'm afraid I still don't follow
> How many forks of a dependency do you use? Just using the master branch and upgrading along that should be good enough for 99% of your dependency
Well, if I was expecting things to not break I'd never follow upstream master for a dependency.
But the question pertained to merge conflicts. If several people track the same remote and check in dependencies into VCS I'd expect annoying merge conflicts
Or are we perhaps misunderstanding each other? I'm not sure I follow what you mean by forks. Releases are typically on different branches or tags
You’re free to update as you like. It’s entirely possible to audit your local packages.
Letting maintainers update your projects is a convenient feature. If it is a liability in your use case you can work around it. Yes it will take more effort, but your use case justifies it.
Yes, but AFAIK, those are heavily tested or audited in some manner. That's different from including code written by randos in your app that they can remotely change at any time.
Can you really say, with a straight face, that you inspect the diffs of your entire dependency closure every time you deploy an update? With the level of scrutiny required to detect a maliciously-obfuscated security exploit?
If you can, you're an infinitely more diligent developer than I am, that's for sure.
Fortunately the problem could become more tractable if something like SES / Endo takes off:
"Endo protects program integrity both in-process and in distributed systems. SES protects local integrity, defending an application against supply chain attacks: hacks that enter through upgrades to third-party dependencies. Endo does this by encouraging the Principle of Least Authority. ... Endo uses LavaMoat to automatically generate reviewable policies that determine what capabilities will be distributed to third party dependencies."
I actually do this, on occasion. (Not 100%, and not to a degree I'd say "yeah, I'd've caught this colors/fakers thing." But enough to say that I've seen a decent sample.)
There is literally, on average, no difference in average quality between commits on FOSS projects, and commits on projects we pay external entities for. Some paid projects are just crap code, and some FOSS code is extremely high quality.
I've had to roll-back / hard-pin dependencies from both low-quality FOSS & low-quality paid projects because of commits that — once you find & read them — are just bananas.
(I have no idea how to solve the root problem here, honestly.)
> I have no idea how to solve the root problem here
I'm not even sure what the problem is. If it's "updating dependencies introduces severe side effects" wich, I think, should be accounted for in the process
> Yes, but AFAIK, those are heavily tested or audited in some manner. That's different from including code written by randos in your app that they can remotely change at any time.
It seems like the problem here is more cultural than technical, specifically that the JavaScript community has fully embraced packages that are "written by randos" that are "wrappers around three-line Stack Overflow answers."
I use packages, but I wouldn't use any that are developed by a rando with no reputation or "institutional oversight." An important part of choosing to use one is evaluating the maintainers.
Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should.
Not just that they've fully embraced the "written by randos" but even worse: "as soon as the rando publishes an update or change, use it!" They seem to fully automate updates because packages are so poorly written (and frankly, it probably helps with revenue stream, if their client's websites occasionally break and need them to fix it.)
...and meanwhile NPM's idea of vetting packages is basically "YOLO, BRO!"
Javascript is used on the front end. Front end devs obsess (or at least used to obsess) over download sized. So you'd have crazy stuff like custom builds of Underscore (https://underscorejs.org/) with just the functions you wanted. Think manual sandboxing, if that makes any sense. You could get a package of Underscore with just map, filter and reduceRight, if you wanted to.
Now, when Node came around, people wanted as much as possible to have the same libraries available on the front end, so the same obsession with size was carried over.
Ergo the micro-milli-nano-packages they make.
Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure).
It's a linker. Tech from the 1950s.
Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't.
Java's largely to blame for this, Sun REALLY, REALLY hated stuff that could be hooked into any OS and wasn't portable, so they didn't provide a linker. Everything was supposed to be on their JVM and you were going to install their JVM everywhere (2 billion devices!!!) and to hell with small stuff or heaven forbid, including native libraries. Javascript followed (on top of the Java restrictions they added: dynamic, poorly defined language, that would have made linking with tree shaking really hard, anyway). .Net also followed.
Almost 3 decades later we're trying to undo that damage.
- JavaScript is a very dynamic language with dynamic property access and a few other features that make it hard to guarantee that the linker won't accidentally remove too much
- historically there was no standardized "module" format until ESM (ES modules) came up (with some time in between with few competing non-standardized proposals), so statically analyzing exports/imports was difficult; in frontend you'd long rely on just creating and reading global variables (i.e. side-effects).
Hence it's been "safer"/easier to create small packages.
But it's not only this. Once you put a mega-package in your repo, it's easy to gradually start relying more and more on the things it gives you. Even if it supported perfect tree shaking, you'd call one method here, one method there, and with each build your bundle size would balloon (which is not good if you could write one line of code while lib method's code is 1000 lines because it supports IE4 and 17 parameters).
Whereas when you rely on small packages, you need to make a conscious choice each time to pick another dependency.
You probably don't care about this on servers written in C++ or Java that much; but on frontend it's a big deal; hell, even when building native apps for Android/iOS you have size limits for the stores submission / limits for the number of methods (tech limitation in Android). Big companies invest crazy money to shrink their native bundle sizes (https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/).
I think that dynamic languages played an important role in pushing for the development of mainstream static typing that didn't suck. ML's been around for a very long time, but there was seemingly little interest in pervasive type inference in languages actually used in industry until they had to compete with the concision of dynamic typing.
Actually building large systems in dynamic languages? Probably going to turn out to be a mistake though.
I really don’t think it is fair to blame it on Java. Having very little native dependency is a huge plus to an ecosystem (just look at what Java will be able to do with Loom thanks to the almost all-Java dependencies). Also, Java was particularly keen on downloading class files at runtime, so linking everything was not even possible.
And it is not even a difficult thing to fix without going the linker way: java’s modules essentially solve it (as well as javascript modules could/can) — just specify what is visible outside a package and both ecosystems can “tree-shake” non-used code (though I dislike this nonstandard term)
> Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should.
This isn't a Javascript problem, this is a node problem. Node is just a Javascript platform among many. The fact that the node community decided to go with all these "nano packages" has absolutely nothing to do with Javascript. Nothing forced Node, the distribution, to come with such a barebone standard library. Absolutely nothing... but the idea of being dependent of NPM which was orchestrated by NPM founders, that's how NPM, a private business made money and eventually sold to Microsoft.
Packing is most of the point of a distro. They are specifically taking on that responsibility. They also have a better perspective to handle overall compatibility.
In a sense you've pointed out the alternative to having the programmer handling the packaging -- having some third party package and distribute. And this separation of responsibility turns out be almost always be a better solution. Distribution and coding are, after all, two full jobs (without 100% skill overlap). Plus, hopefully it indicates that at least two sets of eyeballs have at least glanced at the code (not the full desired many-eyeball outcome, but as good as we can expect sometimes).
Given that Debian (and its descendants...) packages a shitload of npm packages, it's a wide stretch to say there is more QA for these packages from the Debian side than there is from the npm side.
The one thing that Debian provides is that in the case there is a security issue, admins worldwide only need to do "apt update && apt upgrade" and they are safe, without having to check all of the software that runs on their servers (as long as said software comes from Debian, that is!).
I do think people would be served, generally, by being more aware of the fact that distros are not some doing some hardcore security vetting. But the alternative is just to use whatever was pushed up to NPM, right? In that case, Debian packager+NPM push > NPM push by definition, unless the Debian packager somehow provides negative QA, which seems unlikely. (Also, on the incredibly unlikely offchance that some Debian packager reads this comment -- your work is incredibly useful and I very much appreciate it, just trying to be realistic about what exactly is provided by your group!)
This is true for npm. After the incident with leftpad, you can't unpublish anymore. You can, however, publish a new patch update that completely breaks everything.
> This is true for npm. After the incident with leftpad, you can't unpublish anymore. You can, however, publish a new patch update that completely breaks everything.
You absolutely can unpublish, it just requires more steps. If NPM gets a DMCA takedown request they will absolutely have to fulfill it.
It is indeed all, even if you ignore the "popular" qualifier. If a license could be unilaterally revoked, it would fail to meet the Open Source Definition for that reason.
The differences between the two are extremely minimal, basically only relating to patent rights relating to the software. Go read https://www.gnu.org/philosophy/free-sw.en.html - the FSF's Free Software definition, and https://opensource.org/osd - the Open Source Definition (both by the respective parties that coined the terms and maintain them to this day) and see what the actual differences are. They're not many.
First you agreed to ToS when you uploaded things to npm. I haven't read the terms but it should be enough for npm to publish on npm no matter the license.
Secondly and as important if you publish something under an Open Source license(1) then you _cannot
unpublish it_. You granted copyright to
_everyone_ for and existing both now and in the future to distribute and use it(2) (legally it's a bit more complex but that's what it boils down to).
(1): Assuming you had the legal right to do so, but if not you are liable for any fall out, not npm (because ToS, they still need to take it down reasonable fast, but they might be able to sue you).
Your Content belongs to you. You decide whether and how to license it. But at a minimum, you license npm to provide Your Content to users of npm Services when you share Your Content. That special license allows npm to copy, publish, and analyze Your Content, and to share its analyses with others. npm may run computer code in Your Content to analyze it, but npm's special license alone does not give npm the right to run code for its functionality in npm products or services.
You are technically correct. The best kind of correct! In practical terms, it depends on the license used. Since most licenses used in open source will prevent you from making these kind of requests, this consequence isn't likely to have any practical implications.
You are assuming that the true rights holders of all the code in the package actually agreed to the given license. Someone unrelated to the package development can still claim it includes an illegally-copied, unlicensed version of their code.
> If NPM gets a DMCA takedown request they will absolutely have to fulfill it.
Assuming the package is released under a Free Software licence, what grounds would there be for a DMCA takedown?
I suppose a developer could include the lyrics to a pop song in their code (possibly encrypted), and then tell the copyright holder about it (since I don't think you can make a DMCA request on behalf of a copyright holder without their permission), but I would hope that such a poison-pill would be caught long before the package became widely depended on.
Perhaps you're thinking someone would risk perjury(?) charges for making a false DMCA request against their package, and NPM would act on the request without questioning it; but remember that NPM is owned by Microsoft and they have previously stood up to frivolous DMCA requests (after a fashion)[0]. That article has the lede: "Software warehouse also pledges to review claims better, $1m defense fund for open-source coders".
The people trolling YouTube over copyright are either making false Content ID claims[0] (not DMCA takedown requests), or claiming infringement based on an incorrect match of something they genuinely hold the copyright on.[1]
You're probably right, though, that there is enough imprecision in the system for someone to claim that someone else's code snippet infringes on the copyright of a code snippet the claimant had previously published.
> I don't think you can make a DMCA request on behalf of a copyright holder without their permission
In theory, you're right. In practice, there's never any actual consequences for filing a false DMCA claim. Worst case is that the thing doesn't get taken down, but that's no worse than if they didn't file it at all.
Corps don’t care about DMCA takedowns from natural persons. I sent a takedown once, the CEO replied that he was sorry it had come to that, but they still distributed it for years under a license I did not grant. This CEO is licensed to practice law in California, btw.
Some parties that are distributing other peoples' stuff lose a safe-harbor protection from liability themselves if they ignore it.
This means intermediaries who don't benefit much directly from distributing a given bit of content will immediately comply with the DMCA takedown process. But this does nothing if you send the notice to someone who is actually using it.
The correct move is to send DMCA to the infringer's ISP/host. Then the ISP has to take it down unless counter-notified that they say they're not infringing. In turn, that counter-notification improves your position for any litigation that may ensue.
Funny, my company couldn't use Webpack 1 because a dependency of a dependency... depended on an ancient package from the days when it was common to not bother with attaching a license.
Legally, that meant that noone could use it. In practice, nobody but our legal department cared, so we had to wait for version 2 when the dependency chain was updated to remove it.
> Assuming the package is released under a Free Software licence, what grounds would there be for a DMCA takedown?
Noncompliance with the license, e.g. by removing required copyright notices/attribution in the code (this has happened in the past). Or straight-up uploading someone else's non-free code.
I didn't remember that particular legal complication, so thanks for prompting me to look it up. It seems that his argument was that Bukkit couldn't be distributed because it contained Mojang's proprietary code, but the fact that it also contained some of his code meant that he was a copyright holder for the purposes of the DMCA.[0]
This seems like an edge case that wasn't anticipated by the DMCA, but I can see the argument that mixing GPL code with proprietary code is creating and distributing a derivative work, in violation of the GPL. Without proprietary code being present, though, I don't think a developer can DMCA takedown their own GPL software.
> If NPM gets a DMCA takedown request they will absolutely have to fulfill it.
No, they don't. Honoring DMCA takedowns allow benefit from an additional safe harbor from any existing infringement liability for the alleged infringing content, but are not mandatory in their own.
Because npm install has the insane default behavior of adding a fuzzy qualifier to your package.json, for example ^6.0.2 means all of the following versions are accepted: 6.0.2, 6.0.9, 6.7.84
It’s not particularly insane. package.json and package-lock.json have different purposes, namely package.json specified intent e.g. I want a version that satisfies >=5.2.3 && < 6.0.0 and package-lock.json records the exact resolved version.
Off the top of my head Bundler, CocoaPods, Cargo, SPM, Pipfile(and various other Python dependency managers), and composer also all work like this.
Cargo even makes it implicit that a version like “1” means “^1.0.0” in Cargo.toml.
The first line of NPM install's documentation[0] says(emphasis mine):
> This command installs a package, and any packages that it depends on. If the package has a *package-lock or shrinkwrap file, the installation of dependencies will be driven by that*, with an npm-shrinkwrap.json taking precedence if both files exist. See package-lock.json and npm shrinkwrap.
What does happen is: if you have added a new package in package.json it will be installed based on the semver pattern specified there, or if you run npm install some-package@^x.y.z the same thing happens. Further, if you modify package.json by changing the semver pattern for an existing package that will also cause this behaviour.
Running `npm install` in a package that already has a package-lock.json will simply install what's in package-lock.json. `npm install` only changes the lock file to add/remove/update dependencies when it detects that package.json and package-lock.json disagrees about the specified dependenices and their semver patterns e.g. having foo@^2.3.1 in package.json and foo@1.8.3 in package-lock.json will cause foo to be update when running `npm install`.
Because of version locks. Normally you install “^X.Y.Z” which means any version at major X with at least minor Y and revision Z. For more conservative codebases you install “~X.Y.Z” which also locks the minor.
npm install will traditionally install the most recent packages that match your constraints. You need “npm ci” to use true version locks
Very often, package installation is automated as part of a build pipeline. So if you want to build and deploy a new version of your software, you'll kick off the pipeline and that could potentially download a newer version of a package than was previously being used.
Incidents like this highlight that this may not be the best idea.
If you're using NPM without lockfiles, you're gonna have a bad time with discrepancies between trying things on your dev machine and building things in CI machines.
When you have a package-lock.json NPM will install exactly the same version of everything in your dependency tree, making the CI builds much more like what's on your dev machine (modulo architecture/environment changes)
You're never going to be able to prevent that at a technical level. You can prevent it with workflow, though: 1) sync packages locally and build from those versions; 2) peg to a specific version and don't auto-update; 3) deploy to a test environment and not directly to production.
That's another JS ecosystem widespread malpractice.
Autobumping versions, or version ranges as they're called in Maven land.
Dependencies should only use fixed versions and all updates should be manual.
You should only use auto-upgradable versions during development, and the package manager should warn you that you're using them (or your dependencies are).
If package A depends on package C at version 1.0 but package B depends on C at version 1.1, what version of C will be pulled in?
Dependency management is not as simple as only upgrading one direct dependency at a time after careful review.
The NPM ecosystem is particularly difficult to work with as it has deep and broad transitive dependency trees, many small packages, and a very high rate of change.
You either freeze everything and hope you don't have an unpatched vulnerability somewhere or update everything and hope you don't introduce a vulnerability somewhere.
There are package exclusions, package forcing and of course, full dependency tree checks where you review what everything pulls in.
The JS ecosystem will probably have to change but because it's so decentralized, that change will be orders of magnitude harder than, for example, PHPs transition from 3 (4, 5) to 7.
> Dependency management is not as simple as only upgrading one direct dependency at a time after careful review.
Most package managers won't allow these stunts and conflicts have to be resolved UPSTREAM. NPM chose to go the "YOLO" way and will fetch every single version of a package that meets the dependency demands. Terrible design, but the purpose of that was growth for NPM, the company, not the best interest of the ecosystem.
I'm sorry but this is completely wrong. NPM has lock files which explicitly lockdown the version you have downloaded after your first install. These are commited to source control, so all subsequent installs will use the exact same version of dependencies, and nested dependencies too.
You need to ask npm to upgrade or delete your lock file and node modules to run into this issue.
Immutability feels like the best approach here. Go's module system is pretty good in this respect: "proxy" is just a proxy that serves module code, and "sum" is an append-only transparency log of the hashes of all published versions. You can't "unpublish" from the log, but you can get code hosted on proxy removed for various reasons... which users can protect themselves against by running their own proxy. Go's module version resolution strategy means that the chosen module version never changes without explicit input from the user so no "publish a new version that breaks everyone's CI" issue.
All together I don't see how GP's "email php files around" is as any better than this system in any way.
A key difference with Maven projects is that you specify exact dependency versions instead of “always use latest” or some variant of that, as is pretty common in the Node world.
Admittedly, I don't think it has nearly as wide a usage as it has in the NPM world. Dependabot (I know I'm not the first to mention it, here, today) is probably more of a factor.
Still, it strikes me that this sort of "attack" (or mishap) is exceedingly rare in the Java ecosystem, while it's pretty common in the NPM world, and I don't immediately understand why that would be so.
I was not aware of that feature. To call it rare would be an understatement I think.
> while it's pretty common in the NPM world, and I don't immediately understand why that would be so.
I think it boils down to Node projects typically specifying dependencies in the form “any version >= X”, effectively “always use the latest.” Dependencies can therefore get bumped silently just by rebuilding, essentially. Whereas in the Java world updating dependencies is a deliberate process.
Tread carefully with all the supply chain attacks out there, it might not even be the authors doing these. We are entering a dependency attack massive war.
Dependencies are a balance but also a sign of weakness of a system in the modern day. There at least needs to be delayed, dependency bot like analysis before you integrate. Even then, they just leave your systems open to worse than DLL hell, telemetry tracking/data, and attack vectors that can take down or target many, many systems.
How about using dependencies but pinning the version and only updating if you know what the update contains?
I'm still continually baffled that we ended up in a world where automatically accepting updates from every dev and their dog is not just the norm but recommended practice.
You also have to rely on all of your dependencies doing that for their dependencies and so on.
It’s really a mindset/vigilance you need for the whole ecosystem.
Transitive dependencies are also your dependencies, even if you didn't consciously include them. So in an ideal world, you should vet all changes to dependencies of your codebase, including transitive dependencies.
Whether or not this would be compatible with the way dependencies are used today is another question.
I'm not even sure it's not a fool's errand with the current software ecosystem.
I think at some point it will have to be a language level feature. The ability to sandbox or provide permissions to packages/functions. Just like our OS had to, just like browsers had to, just like phones had to.
Our code is the platform, the packages the apps. It's a similar use case.
If I could download a module, and tell the compiler this module, and everything it uses (including packages that I also use, but through a different call tree) will never access the network or write to disk, it'd help grant some small peace of mind in terms of security at least.
It might, I'm not familiar but after a quick look it seems to operate on a vetted trust model i.e. you can use these because we checked and they are compatible. So you could miss out on a lot of the ecosystem.
I was leaning more towards the web approach where we assume everyone is out to get us, but they can't unless we give them that one permission they need. If it's a statically typed language then it'd even allow dependency walking to see what permissions are used at a granular level and we can decide not to bring in anything that's too loose. This of course won't solve cases like logic bugs, but it'd help mitigate the impact.
> I think at some point it will have to be a language level feature. The ability to sandbox or provide permissions to packages/functions.
> If I could download a module, and tell the compiler this module, and everything it uses (including packages that I also use, but through a different call tree)
Javascript's prototype based inheritance looks like it can help facilitate such conditional submodule invocation. But, and partially for performance reasons, static compiling would be necessary. So Javascript and its dominant NPM package ecosystem can never go in a direction like this.
If only C++ or Python (dynamically typed, I know) had prototypes instead of class based inheritance.
Edit:
Looks like another commenter referenced what we're probably talking about:
> Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure).
> It's a linker. Tech from the 1950s.
> Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't.
Can we create an open source linker for JavaScript and NPM packages?
The problem of new versions of dependencies breaking old code happens all the time in a variety of ecosystems. It's not exactly "solved," it's a continual problem similar to picking what you will eat for dinner. There are pros and cons to each approach, in this case the con is that every now and then you have to manually pin a version. If we did it the other way, we would have to manually upgrade versions to get obvious and easy improvements. As with most things people happily call "turds," it's actually a tradeoff and not as simple as just being bad.
As for reinventing the wheel, are wheels really settled science? As far as I know, new kinds of wheels are being created all the time. It's not just that new people are creating them, there are new things wheels need to do every day and new sets of requirements that the old existing wheel designs don't fulfill. Look at wheels from 50 years ago and they are nothing like the wheels of today. The wheels on cars are nothing like the wheels on aircraft, which in turn are nothing like the wheels on trains.
> only updating if you know what the update contains
People suck at this. What this actually tends to do is mean "no updates, ever" unless you have a particularly rigorous culture of dependency management.
Or we get a culture where upstream writes in more detail what an upstream is supposed to contain and downstream verifies that the update indeed does what they write. If this leads to fewer updates overall, I have no problem with that.
If you change "contain" to "do", then this is the MAC security model as implemented by SELinux.
a culture where upstream writes in more detail what their code is supposed to do and downstream enforces that the software indeed does (not do anything beyond) what they specified
It didn't lead to fewer updates, it led to less usage of SELinux.
I think anyone who thinks they're doing this is fooling themselves. You can review code for accidental vulnerabilities but if someone is trying to slip in a backdoor it shouldn't be hard to do so in a stealthy manner.
The reality is that the entire dependency concept is just broken. There is an implicit trust that all dependencies are equally trusted. Your logging package is just as capable of performing file and network operations as your http package, even if you assume it won't.
That's silly.
It is up to programming languages and package managers to solve these problems. They're also not that hard to solve, in my opinion. "Run arbitrary code on a computer" is a model we've been securing for decades with web browsers, both in terms of web pages and extensions, and now too with mobile.
Solving "this code can do X but that code shouldn't be able to" is similarly easy to solve with languages that support effects or capabilities.
Pony's object capabilities are one example of an existing implementation. I don't think there's any "inventing" To do here, it's all just implementation work.
It's actually trivially easy once you remove ambient authority, which is the real source of these security problems. Consider how a program could modify your files if it cannot willy-nilly turn any old string into a file handle.
Adding permissions is a reasonable step, but I don't think it solves the problem. We know, it's very hard to get granularity right with permission systems and there is a strong temptation to just give everything all permissions.
Dependencies with dangerous but necessary permissions can still abuse them: Your network library will still be able to add a bitcoin miner.
What happened if an update requests a new permission?
Also, how would that have prevented the current situation? Infinite loops are famously hard to detect and prevent automatically.
> Adding permissions is a reasonable step, but I don't think it solves the problem. We know, it's very hard to get granularity right with permission systems and there is a strong temptation to just give everything all permissions.
A lot of that stems from permissions systems being implemented outside of the code they constrain. In theory a compiler knows every reachable system call and all points of data input that could reach them, and as such it could constrain the program's capabilities accordingly.
In fact, compilers already do this for control flow integrity - it would just be a more advanced system.
> What happened if an update requests a new permission?
It's going to depend on the system. For browser extensions the new permission means a new prompt, so you'd get a CI failure until a human updated a lockfile.
> Also, how would that have prevented the current situation? Infinite loops are famously hard to detect and prevent automatically.
It really depends on the system. You could have a CPU capability that restricts cycles or forces preemption, etc.
I'm not saying you can solve literally all security problems but you can reduce risk considerably. If "infinite loop" is the scariest thing a dependency can do we're in a pretty good position. An unconditional infinite loop should break your CI tests.
> In theory a compiler knows every reachable system call and all points of data input that could reach them
Sorta yes, sorta no.
Imagine I'm making a chat client, and I want users to be able to drag and drop images to share. But the OS doesn't have an "open drag-and-dropped file, extension .png or .jpg" function call, it only has "open file" which lets me open ~/.ssh/id_rsa too.
Or if I'm making a web browser and I want to support U2F tokens. But there's no OS "talk to U2F token" call - the browser needs access to the system calls for "talk to arbitrary USB devices".
> But the OS doesn't have an "open drag-and-dropped file, extension .png or .jpg" function call, it only has "open file" which lets me open ~/.ssh/id_rsa too.
A programming language doesn't have to expose system calls directly. It arguably it shouldn't, in fact, for exactly this reason.
If you end up with "X can open any file" that's something that's worth noting to a consumer. The sandbox capabilities don't have to be perfect in order to expose a scary situation.
Further, you can restrict a process to only open specific files in a number of ways on Linux, including based on path. There's room for improvement, though.
> I'm still continually baffled that we ended up in a world where automatically accepting updates from every dev and their dog is not just the norm but recommended practice.
I think it follows from two things:
1. We use open source software for everything. This is also true of our dependencies, so we get hundreds or thousands of transitive dependencies. Many of which are presumably written by dogs, because (i) no one can tell if you're a dog on the internet, and (ii) OSS maintainers are so overworked they ask their dogs for help.
2. Languages and libraries are full of footguns, software is full of bugs and therefore vulnerabilities, and no one cares enough to go through the enormous effort to fix things. And this is true through the whole stack. So the only practical way to stay secure-ish is to reactively patch software as vulnerabilities are publicly discovered. And also defence in depth. (I would distinguish between the publicly known time that a vulnerability is discovered, and the first time it was discovered. You hope the two are the same but for many vulnerabilities, if a clever adversary found them first we'd never know.)
With these two things together, you have a ton of questionable dependencies, and you need to update them all the time for security reasons.
Yeah I guess then we can just shrug and go on because there is nothing we can do to stop our app from randomly breaking tomorrow.
> So the only practical way to stay secure-ish is to reactively patch software as vulnerabilities are publicly discovered.
But this is different from just blindly accepting any update that upstream gives you.
> And also defence in depth.
This sounds increasingly like security theater. You can always more layers obstacles to make things harder for malware that is already on your system, but it's not clear to me how much this actually reduces your atrack surface.
Each layer (e.g. firewall rules that require that all internet access go through a proxy), adds non-trivial amount of work for the hacker to get anything useful done.
1. best case - hacker will give up.
2. good case - you have more time to notice and react.
How much layer cost you, how much does it cost for hacker to overcome it.
Things to remember:
1. Not all hackers are nation states. Most are not.
2. We must accept that no security measure is absolute even against script kidies. Given enough time and luck/misfortune js sandbox will do "rm /sensitive/file".
Recent Log4shell example shows that one can follow all best practices and still get bit in unexpected way.
I see systems with various system-level vulnerabilities all the time for work, and try to assist my clients (internal project teams) in prioritizing fixes. Besides the usual CVSS scores, I try to focus first on what is being used or exposed. Network services, file-input processes come to mind. Vulns not in this list should also be fixed when found, but my thought is centered on what might be primarily exploitable.
This leads to some thoughts on statically-compiled applications; while they might have some vulnerable dependency, I suspect that it's harder when the attacker is limited to the app's "baked-in" functionality that defines how those dependencies get used.
Edit: Also, I should note that while I would greatly prefer, and do advise, that they base their environments on minimal OS distributions, this seems rare. The base system patching would be much easier to manage if it started from some BSD-like minimal state, or Alpine Linux, and included only what it needs. Instead, any infrastructure vulnerability assessment leads the teams to chasing down numerous patches in things they have, but never use.
Crazy what happens when you decide to freeload off a stranger’s code who you have no contract or agreement with whatsoever, beyond a license you must accept to use the software which disclaims any warranty whatsoever, even fitness for any purpose.
I have zero sympathy for anyone complaining they were hurt by this. I think Marak is teaching an important and principled lesson here.
> these trillion dollar corporations just take and take and never give,
Which trillion dollar corporations are these?
A quick google says "Apple, Microsoft, Alphabet, Amazon, Telsa, Meta, NVidia, and Berkshire Hathaway" are the the only trillion dollar companies
Except for the last one all of those companies give back vast amounts of open source support. All you using VSCode for free, that's Microsoft's payback. Oh, and Microsoft pays for NPM hosting and github, also Typescript, C#, F#, .NET. Hardly not giving anything back. Apple gives Swift, Clang, LLVM, Webkit to name a few. Alphabet, Go, Chrome (which also means Electron on which VSCode is running), Android, and plenty of others. Meta provides React, Redux. Not sure what open source Tesla gives back but they have given their patents (https://www.tesla.com/blog/all-our-patent-are-belong-you). Nvidia gives away tons of open source as well (https://developer.nvidia.com/open-source)
The author should first change his LICENSE before he does crap like this. Make sure the terms of the license upfront state that "billion/trillion dollar companies" are not permitted to use this library. Stop using the MIT license for your hactivist project.
Most of what you wrote has merit, but I dont believe the author was trying to impart much of what you wrote. I let my comments inline with yours.
> a) blindly 'updating' and deploying code without testing is a horrible idea
This is important risk factor each organization should be aware of. Though he wasn't trying to convey this, it was merely a byproduct.
> b) these trillion dollar corporations just take and take and never give, and boy do they whine and cry when the devs don't 'hold up their end of the deal' and keep turning out perfect, fully tested software for free
This was the intent, but anecdotally, Google, Amazon, Apple, Microsoft, Twitter, Meta, Stripe, Netflix, all contribute to open source.
> c) 'the source is open so anyone can look at it and therefore it's bug free' is and always has been a mentally retarded philosophy
Since the source is open, it must be bug free is a flawed conclusion. Again, not the lesson Marak was imparting.
> d) the software deployment process as it's currently practiced is horribly flawed
I don't think that is accurate.
> e) these people ought to count their blessings that the code is flawed in such an obvious and immediately detectable rather than subtle and devious and much more destructive way
I can't imagine a more destructive way that would result in the catastrophe one is expecting. Should Marak have been more malicious, having a myriad of well funded corporations targeting him would not be fun.
and last but not least:
> f) giving control over one's code to evil microsoft via github is an incredibly stupid idea, as such authority WILL be abused by the evil scumbags.
The large Fortune 500 company probably isn’t even paying the lowly developer enough who took the time to find a terminal colors package anyway. You really think this is the person that’s going to be able to lobby for dev budget? They are literally trying to keep their own god damn job.
It might be time we have a package marketplace like Steam that companies can subscribe to and independent developers can make some money via the marketplace.
freeload is a fairly loaded term. "disclaims any warranty" isn't the same as malicious action; I don't have to pay you if your house burns down if you don't have an insurance policy (contract) with me but I'm still liable if I commit arson.
Also, AFAIK, Marak is not the original author; Is he also a freeloader for attempting to commercialise this code?
> teaching an important and principled lesson here
The history of this issue speaks differently to their intentions, but even so, there is a way to "teach lessons" and that's by doing something alarming but harmless. AFAIK Marak wanted to cause harm, and acted in a way to do so.
Are these really RCE vulnerabilities? Looking at it systematically I only see this as an RCE vector if you're doing one or more things very wrong. This assumes that packages are immutable and an author can't update a version that's already there. This is how NuGet works, and IMO is how any remotely sane package manager will work. There's no reason for a version to be mutable in this context.
Pegging to a specific version limits exposure. Syncing these packages to your own on-prem/isolated environment limits it further. Deploying all changes to a test/staging environment where they're reviewed first limits it even more.
I mean yeah if your build process takes @latest of all your packages and then pushes it right into production, that opens you up to a lot of risk. It's also incredibly stupid for anything beyond a personal project (and probably even those).
This doesn't strike me as a weakness in package management, it strikes me as a weakness in doing package management wrong.
The tool should take some blame here. I agree that it’s ultimately the developers fault for allowing code to be automatically injected from not fully trusted sources on minor updates, but the package manager makes it way too easy to do.
For example, when I npm install a package, it defaults to specifying a semver compatible version in package.json, rather than doing the secure thing and pinning a version.
But whether this default behaviour should change is not is also a security tradeoff. Pinning versions means that you will keep using an insecure version of a dependency until you update, whereas using a semver compatible version allows you to “automatically” pick up a fixed and compatible version. In practice however with lock files and local caches, the developer always needs to update for security patches anyway.
However, given the current NPM landscape (with packages having numerous small dependencies from a large variety of authors), going towards the former instead of the latter is definitely makes a lot more sense.
To boost this: it's worth reading about the difference between "npm install" and "npm clean-install". The "ignore-scripts" flag/configuration setting can also be valuable.
Go does this well. It chooses the minimum viable version that satisfies the constraints for each package.
The minor security updates are solved well by periodically running security linters and scanners. There's even a recent GitHub feature for it. That will alert you that you need to update a package.
> it defaults to specifying a semver compatible version in package.json, rather than doing the secure thing and pinning a version
Note that if you have a package-lock.json (which you will by default), it will prevent any surprise updates even within the semver range specified. You have to manually run `npm update` to get the latest versions that match your semver. Personally I think this is the best middle-ground.
This is, unfortunately, not true by default. I had a case where I did `yarn install` and there were updates installed.
To make this work correctly, you need to do `yarn install --frozen-lock-file` or `npm ci`.
It’s absolutely _insane_ that this is the case. Gemfile.lock, Cargo.lock, and every other lock file format that I have used in packaging does this correctly.
It used to be true. npm install used to do what npm ci does. It was super annoying to learn that the hard way.
One of the core issues of NPM style package management is package bloat means you absolutely can't review all release notes for every module in your tree. So you just trust the top level packages, and pray they would mention something if their dependencies change how they themselves work. Practically I rarely see anyone read release updates for even those top level packages, they just update everything and test then send it up to prod is very typical.
If you are cool with that, rad, but it's the pinnacle of the fast food tech ethos literring software right now. Everyone is moving so fast that you barely get to learn something properly or maintain it well enough before it's defunct and we are on to the next thing. I might have a slightly bias view of it, working mostly for agencies I see a lot of projects.
Some orgs are much more in line with GP’s suggestion. Marketing sites may feel low risk and in my view the iteration speed required justifyies having a trusty stack with known good versions to start from. Personally my method of construction is very conservative and I thrive in B2B SaaS environments, where in Consumer front-end orgs I can be seen as a dinosaur at times. I love new and shiny things as much as the next dev, and enough incidents will hopefully create a more conservative culture of using free lunch-looking stuff more cautiously. Race to the bottom dynamics in a sense, lacking any regulation. The expectation is move fast and break things, I get that, because of the first to market/time is money bias/truth. Inexperienced devs won’t have the scars to push back if there are upstream changes to review while their boss expects the feature updates to be live ASAP. I imagine that with decades regulation will force certain processes—not that I want it more than the next dev who loves shiny stuff and delivering results fast/delighting my boss.
Nah, open source software is "use at your own risk" and there's 0 guarantee for anything. All responsibility lies with the user. If you don't like that responsibility, don't use open source software without reviewing it first.
I have a medium-sized data science project in Python. Nothing crazy. It's 180 packages, apparently, and 2.9M lines of code (whitespace, comments and all). Charitably let's call it 1m SLOC.
Seriously, you expect anyone to audit all this? It's basically impossible for any solo dev / small org, and as I say, it's not even a big project. A vulnerability is like half a line, or sometimes a typo.
Clearly, very different proposal for a large org, but even then, no small task.
> This is ultimately the fault of the person deliberately updating their package to break other people's software.
Ultimately the 2017 Equifax data breach was the fault of the people who hacked into Equifax's website.
We need systems in place to defend against people doing malicious things, but yes ideally individual developers shouldn't be the ones tasked with reviewing all of their dependencies' code.
Operating System provided packages, for example, are generally reviewed by someone other than the author, which can lead to a more secure supply chain.
Rust's cargo-crev review system also seems like a possible solution the problem.
You can pin the direct dependency, but what if the packages you depend on don't pin their own dependencies? The standard (default behavior) is to use ^, which will automatically install new minor versions. Package.lock helps, but there's no sane way manage upgrades. Just running "npm audit fix" could result in pulling down a bad package.
Pretty sure pinning only pins the direct dependency. And most libraries do not "pin" their own dependencies, because it's more work to maintain. Security & bugs fixes that would otherwise be resolved via minor patches must be manually addressed. It also helps with resolving shared dependencies.
NPM is highly optimized to make sharing code as easily as possible, but that comes at a heavy price.
If you are not updating daily any typical webapp tree will accumulate known vulns.
Holding back updates is not a sane strategy either. You must review all the new patched versions of all dependencies daily or, failing that, do the work once to drop all the deps you can not afford to maintain reviews for.
I’m a self taught Python programmer. I haven’t don’t much front end.
Why do some JS devs import tiny packages to do simple things? I don’t feel like I’ve seen this behavior in Python. Is it because browsers are an awful environment?
And if you're optimizing for the browser, you can't count on the improvements being there. So you still want to use third-party libraries for their polyfills.
I prefer to look at the code, decide if A. it's worth it, B. it wouldn't be funner/better to just clone it as a 'plugin' in my own code, and C. it looks like it has a good team/support around it.
In browser land, the less code you ship, the better. Removing dead js code is hard, because of its dynamic nature and using common js imports making it harder for treeshaking algorithms.
So, people had incentive to write and use smaller packages.
Now, the situation has improved. If you use esmodules all the way, and only import what you need, then your bundler can remove unused modules from final build
> In browser land, the less code you ship, the better
That's kinda funny. These node tools I run into these days usually take forever to install, waste a lot of space due to creating their own package mirror and generally are prone to break because of dependencies.
There is usually nothing tiny about them, even thought they only have a few lines of code.
As someone who does front-end JS stuff and uses a bunch of packages here is why I do it:
I got tired of copying and pasting the same classes between projects. The worse part was I'd add new features to the newer projects and when I would have to go back to work on something from a year or two ago I'd have to spend time backporting all the new code. I also don't like how bloated a bunch of the "popular" packages are. Why do something in 40kB of JS when you can do it in 3kB. Smaller is faster which is important to me because one of my main selling points is that I build modern-looking marketing websites that load and render under 5 seconds on a slow 3G connection.
> I got tired of copying and pasting the same classes between projects. The worse part was I'd add new features to the newer projects and when I would have to go back to work on something from a year or two ago I'd have to spend time backporting all the new code.
Why not create your own common library and publish it to a private repo? There's a lot of options between using a stranger's package and what you're describing.
> Why not create your own common library and publish it to a private repo? There's a lot of options between using a stranger's package and what you're describing.
Exactly what I thought. Fascinating how they can have missed this obvious solution.
In my opinion, golang does this correctly. You can just git submodule the source code of your dependencies. That way, you're always in control over what gets updated and when.
I often find myself... ripping out a lot of what I 'need' into something that maybe isn't always well-maintainable, but it's my fuckfest of code, and if something breaks it's because I choose to eff it up myself. Esp, when it's something API related, most php api sdk's are poorly maintained anyways and need updating as I go, plus I usually learn the api pretty well as I rebuild and test the new classes.
Ironically, I'm working on a laravel package myself that i'm hoping to maintain (and turn into a viable side project) that's basically jetstream with SaaS components, and UI elements... (think ui component libraries + laravel jetstream + extra SaaS/ERP things like tenancy beyond just teams but..like Org which can have teams, projects, employees, and each user can belong to multiple orgs, teams, projects, and have attached profiles to each.
For a lot of the UI stuff, I've basically repacked MIT stuff for tailwindcss components, and laravel/livewire added some extra configurations and options, and made it so you don't need Jetstream, just this thing... so a lot of it is actually other's packaged code pulled into one package so, ideally there's one dependency that could even be easily forked and repurposed for a team's needs but cover a lot of boilerplate possibilities.
I’m surprised the AWS SDK doesn’t pin its dependencies and put new versions through its paces before letting end users possibly use a compromised utility with catastrophic results.
> At the very least, it takes them under a minute to break your app, simply by deleting their package. Read the article. This is not the first time it's happened, and it's not going to be the last. [0]
That hasn’t been true for 7 years now, it was changed after the left-pad incident and that article everyone keeps quoting is from 2016. Deleting a GitHub repo or a package does not remove it from npm as part of their policy.
Published versions are immutable, you can only submit a new patch with a new version number. It's common for dependencies to be pinned to a minor version (getting patches automatically), however if you use a package-lock.json, as is the default/best-practice, I believe you should be guarded from any surprise patches. You would discover a change like the one in the OP when you manually ran `npm update` on your dev machine, so it should get nowhere near production.
I mean... that's true if you ever use any code that you haven't read through line-by-line. That's not specific to package managers in general, much less NPM, so I think it's out of scope for this discussion.
Not really. I can be reasonably sure that end-user applications I download for a desktop are limited in the damage they can do (even more so for iOS or Android). This isn't something that happens often with programming libraries, but there's no inherent reason they can't be built in a way that they run in a rights-limited environment.
Moreover, anyone who either has malice intentions (or depend on other packages, of whom authors do) can make the whole process much less noticeable with relying on variables from URLs that get executed, which may themselves be linked to other dynamic dependencies, creating all sorts of logic/time bomb or RCE attacks.
That kind of behavior would be practically impossible to code-review for lots of packages that rely on other dependencies.
Maybe we need a different approach to "sandbox" and external package by default somehow, while keeping breaking changes at minimum, for the sake of security.
This is what the folks working on WASM/WASI and related projects are trying to achieve.
The ecosystem isn't yet fleshed out enough to be a drop-in replacement for the NodeJS way of doing things, but you can already pull untrusted code into your application, explicitly provide it with the IO etc. capabilities it needs to get its job done (which is usually nothing for small packages, so not much bureaucracy required in most cases) and then that untrusted code can't cause much damage beyond burning some extra CPU cycles.
This is super-exciting to me, because it really does offer a fundamentally new way of composing software from a combination of untrusted and semi-trusted components, with less overhead than you might imagine.
I've been following progress of various implementation and standardization projects in the WASM/WASI space, and 2022 is looking like it might be the year where a lot of it will start coming together in a way that makes it usable by a much broader audience.
Which GraalVM can do much better as well, even optimizing through language boundaries (it can effectively inline a C FFI call into whatever language made that)
This is on a different level than pledge. pledge applies to the whole process. This sandboxing, as far as I understand, would restrict syscall access to individual functions and modules inside a process.
Nothing could be further than the truth. Capability-secure Java code just looks like Java with no surprises. The only difference is that ambient authority has been removed, which means that no code can just call new File("some_file.txt") and amplify a string that conveys no permissions into a file object conveying loads of permissions, you have to be explicitly given a Directory object that already conveys permission to a specific directory, and on which you call directory.createFile("some_file.txt")
Just remove the rights amplification anti-pattern and programs instantly become more secure.
Java's security manager blocks access to existing APIs that are already linked. The new approach relies on explicitly making only specific APIs available.
I think this will be the way of going. It resembles me of having all server components all over the place creating a mess, and now we have Docker and Kubernetes. From what I see this would be a more lightweight version of containerization: not for VMs/services but for each JS package.
Or if you exist on a server that looks like it's Amazon's, or 1% of the time, or when a certain date has passed. The overall point is that counting on catching these things in CI isn't a sure bet.
A fine-grained permissions system could fix this by disallowing raw shell execs, or at least bringing immediate attention to the places (in the code) they are used.
Interesting. Do those reviews apply to packages as a whole, or different versions of a specific package? Edit: Yes, the reviews can apply to specific versions.
I'm personally a fan of using Debian/Ubuntu packages, because generally code goes through a human before it gets published. That human has already been trusted by the Debian or Ubuntu organization.
This aims to explicitly solve the problem of "okay, but most maintainers just skim the code at best and spend time on packaging", plus it aims to parallelize it.
And while some packages have been distroized (eg. a lot of old perl packages, a lot of python packages, some java/node packages) I have no idea if any rust package is distro packaged separately. (Since rust is static linked there's no real reason to package source code. Maybe as source package. But crates.io is already immutable.)
Even if they did they're not 'breaking your app in minutes', as if all live apps which use that package are suddenly going to poll npm for deleted packages. That's absurd.
Of course that's absurd, that's not really the core of the argument though. I would still consider it breaking my app if I now need to go replace that package somehow, or pull it from some archive, before I can re-deploy my application.
I wholeheartedly agree with this commentary. Any insight into why this is so much the case with npm but not seemingly as bad in other ecosystems (dependency trees in npm are huge).
I feel like the implicit trust makes even using popular packages such as react seem a bit sketchy. I’m betting react devs audit upstream packages, but I don’t know if any formal statements that they do. Multiply that by all the other common projects and you have a huge auditability issue.
>> Any insight into why this is so much the case with npm but not seemingly as bad in other ecosystems (dependency trees in npm are huge).
I would think that the sheer popularity of the JavaScript (and therefore Node) ecosystems contributes partially to it - there's a massive industry out there about skilling new developers up in JavaScript, Node, and some front-end frameworks. But it definitely doesn't explain all of it.
I actually attribute it more to the micro package architecture, but maybe I don’t know it well enough. I don’t know any other ecosystem with a left-padding package for instance.
I noticed a few years ago that my bank didn't use much in the way of dependencies for their website (possibly just jQuery) - clearly they agree that depending on React opens you up to depending on... who knows what.
- some organisations have the funding to review packages such as react
In the future I think security bureaucracy will prevent security conscious organisations from having nice new things. This happens in places like the military (who were known to use WinXP long after public EOL).
a peer-viewed standard library is key, just like what glibc or libstdcpp for c/c++ that covers 80% of the normal use cases, the rest you're in charge for its quality check.
with javascript/node, you can write 100 lines of code then pulling 100 modules, it's quite different and hard to assure its quality and safety over long period of time.
I heard rust also has a very small stdlib, that gave me concerns, but I don't code rust, I do hope though all languages can have a stdlib that is 20% the size covers 80% of normal needs.
True enough overall, but I'm surprised no one has let you know that NPM actually doesn't allow you to delete packages anymore (after the left-pad controversy). You have to email them and then you are judged by how many downloads you have. If you have users relying on your package, they do not let you delete it.
Yes ! Re-inventing the wheel is literally perfectly fine. You take a round piece of wood, make it roll, and BAM you re-invented the wheel. No need to go to Toyota, buy their wheel making machinery for millions and run that to make a little thing roll.
It's the same with packages, it's FINE to have to redo a bit thousand separator logic, do you truly need a transitive dependency hell with ^1.1.1 in the package list that auto upgrade at random !!? I've had several cases where the whole company is all hands on deck because some dep somewhere moved up and all subsequent builds fail - what are people doing in node, we never had these issues in Java.
> Packages are literally remote code exec vulns in the hands of package authors
Something mentioned in this article caught my eye:
> While searching for Marak’s libraries, I found this npm-test-access library. This library seems to be used for what the name describes: to test access to NPM. Marak seems like a very capable software engineer, and it’s unclear to me why he’d need a package like this. So, this make me personally doubt a little bit if Marak is really behind all of this, or if maybe his account got compromised, or if something else it at play.
If you wanted to take over other people’s NPM packages by pushing a compromised update to one of your own widely used packages, the first thing it would do is check to see if the victim had access to publish to NPM.
Marek, who has just started publishing malicious updates to his own widely used packages, has just created a package to check for access to NPM.
> Packages are literally remote code exec vulns in the hands of package authors
There are 20m+ weekly downloads of the colors package alone. He has what amounts to remote execution privileges to people using that package. When the subject of compromised packages comes up and he’s demonstrated that he’s willing to publish malicious updates, it’s completely fair to wonder what else he’s willing to do with that level of access to that many systems. It’s irresponsible not to consider what his packages can do to your systems.
> This seems like irresponsible speculation and insinuating that Marek is about to commit a felony?
No, it's speculation that Marek didn't do any of this, but that instead his account was hacked. You either responded to the wrong comment by mistake, or totally misunderstood the one you replied to.
Extending your logic to extreme you'll not be able to use any OS, compiler, processor unless you build it. We can't build practically anything of use without any 3rd party library in any language/platform.
The problem here is not packages but lack of stdlib and tendency of package writers to have shit ton of further dependencies.
I wish there was a package manager for node.js that is made for "static" or offline usage, and is able to compare headers of libraries before upgrading them.
But here we are, 10 years in, with nobody giving a damn about semantic versioning.
Life could be so much easier with an actual package manager that isn't just some git clone replacement.
Every time I see a client importing unsigned code with no evidence anyone they trust has reviewed it, I flag it as a supply chain attack vector in their audit and recommend mitigations.
Some roll their eyes, but I will continue to defend it is a serious issue almost every company has, particularly since I have exploited this multiple times to prove a point by buying a lapsed domain name that mirrors JS many companies import ;)
Not really, individual package developers don't have as much inmediate control over the repository's state as they do with NPM. Packages go through a review by one of the trusted developers and sometimes automated QA and testing (including as of late reproducibility testing, i.e. does the source match the binary?), before being uploaded to the repository.
If you can't trust the team behind the distro, then sure, your supply chain is compromised, but it's significantly less likely for a single package developer to cause any damage, as all the big distros have rather extensive policy and procedures to prevent such things.
I use Gentoo which uses portage the package manager and the way portage works is it pulls source then compiles. Source is rarely checked by everyone. Small packages exist as well. Many Linux distro simply barrow binaries from "trusted" sources. The entire eco system is really a deck of cards.
This is a false equivalence brought up every time anyone mentions how vulnerable the npm/gems/pip ecosystems are to supply chain attacks.
Linux code is always reviewed before deployment, goes through many eyeballs, people are careful about this. The same is not true of npm, or any of the other services (as this event clearly shows).
Parts of the Linux stack equivalent to colors and faker are carefully audited and reviewed by multiple people? That sounds to me like elevating them to important bits in a false equivalence.
When it comes to security (among other things), one simply cannot say that all the important bits are in the kernel. If that were the case, there would not be an issue to discuss here.
If projects are importing tens or hundreds of third party libs without any kind of validation or review the process is fatally flawed.
Whatever the language or repository system reusing libraries like React, Requests, Apache commons, or lodash make sense after reviewing the pros and cons (functionality, security, size, performance etc). But blindly adding small repositories to your packages file without understanding the implications is only increasing the risk of trouble.
Node and npm for some reason seems to have encouraged this - remember leftpad.
Pay the maintainers of the libraries you use, and have a contract with them that states their obligation to maintain and support your use of their code
Again, it's still under a FLOSS license. you can use it for free as in beer, under the terms of the license. If you want further expectations satisfied (i.e. ongoing maintenance of the software), that's what money is paid for. You're not paying for the software, you're paying for services around it. (and the nice thing with Open Source is that if the original creator isn't available for whatever reason, you can pay somebody else for them, which is a lot harder with not-open software)
Because you haven't solved the problem for FLOSS, you've solved the problem for non-FLOSS that might also happen to also be FLOSS aka contribute somehow to a FLOSS version of the project - but the solution doesn't help those under the FLOSS licence, and complicates incentives to contribute to FLOSS/"community" versions.
Great for corporates who can buy the support contract, but is also suspiciously similar to the "freemium" model were FLOSS devs are suddenly incentivised to make the FLOSS offering insecure in comparison to the paid licence.
In this case, Marak is his own bad-actor/saboteur, how would the support contract help? It would be far more likely to make free-users 2nd class users, and as such it might be better to simply keep the products managerially separate due to that conflict of interest.
And lets be honest here - when does something stop being a reasonable "maintenance fee", and start to become rent-seeking / extortion? I think if you want to get paid, you simply don't work with MIT/GPL, or fork to a different licence; Changing your mind halfway through isn't reasonable IMHO.
MIT basically means "anyone working on this codebase agrees to MIT terms for their code, and as such authorship isn't so important". If you change your mind, you broke your agreement. If suddenly your authorship matters, what about every other author who stuck to their MIT agreement?
A meta repository that lists versions reviewed by a trusted group of people? It would ad latency to bug fixes and limit the amount of available libraries, but would prevent single developers from taking down the ecosystem on a whim.
That is more or less what Arch Linux does. There are oficial repos (core and extra) maintained by Arch Linux developers, an unsupported packages collection (AUR) where anyone can upload a package recipe and an intermediary between those two called community repository that is mantained by trusted users.
This is what `npm audit` and GitHub "DependendaBot" are both doing (originally in parallel with their own meta-databases, though now that GitHub owns npm things are lot tighter integrated, it sounds like).
Admittedly:
A) Both of these meta-repository tools are reactive rather than proactive: they flag bad versions rather than known good versions.
B) It doesn't take too many HN searches to find people don't trust `npm audit` or DependaBot either because both have provided a lot of false positives and false negatives over the years.
C) If someone does trust one or both, often the easiest course of action is to automate the acceptance of their recommendations and just blindly accept them leaving us about where we started and just blurring the lines between what is repository and what is "meta-repository". (Even the "Bot" in DependaBot's name implies this acceptance automation is its natural state, and the bot's primary "interface" is automated Pull Requests).
Maybe limit the capabilities of software e.g. dictate what permissions are reasonable. Maybe require certain "standard libs" for things like console output that limit what can be output.
You don't need to dictate a standard set of permissions, you just need to remove a single very common anti-pattern called "rights amplification".
Why is a program able to concoct a random string that conveys no authority, into a file handle that conveys monstrous authority potentially over an entire operating system, ie. file_open : string -> File.
That's just crazy if you think about it: a program that only has access to a string can amplify its own permissions into access to your passwords file. This anti-pattern is unfortunately quite pervasive, but it's a library design issue that can be tackled in most existing languages by using better object oriented-design: don't use primitive types, use more domain specific types, and don't expose stdlib functions whereby code can convert an object that conveys few permissions into one that conveys more permissions.
This means deeper parts of a program necessarily have fewer permissions, and the top-level/entry point typically has the most permissions. It makes maintenance and auditing easier to boot.
Do you have advice for projects that use Maven? I know every package on Maven Central has a PGP signature, but as far as I know, Maven doesn't verify them.
You don't have to always pull the latest release on utility packages if you want to evade such problems. Sure, you would need to audit a lot of packages in certain languages...
But yes, I prefer to "vendor" my dependencies too, especially on large projects.
Security vulns are fixed daily in most webapp dependency trees.
If you do not update you are vulnerable to piles of issues anyone can look up.
If you update blindly you may import new obvious supply chain attacks.
The solution is actually doing code review. If you can not afford to review 2000 dependencies then you can not afford 2000 dependencies. The extra effort to use a minimal framework and some cherry picked functions may be worth it for most orgs.
> I have a mailing list for people who use my code, when an update is out they can download the .php files, 'require' them and test them before deployment, but never will I do packages.
This offers no benefit in terms of security, over a package dependency locked at a specific version.
The end result is the same: the user ends up downloading the .php files, and testing them in deployment, but through composer instead of curl.
It doesn't contribute to security at all, it just makes it awkward for other people to use your code.
Composer (PHP dependency manager) does not force you into doing automatic upgrades.
You can keep a dependency fixed at a particular version indefinitely. You can also point composer at a private vendored repository of the dependency if you don't trust the upstream server.
I do this solely because I don't like packages, I don't use them, and I don't want to maintain them for other people.
To the people who want to use my code, it is recommended prominently in multiple places that they not blindly trust the code and actually inspect it before using it. The friction in this process is intended.
The code I write is primarily for me. Other people can use it if they want to, and I hope it helps them, but I don't care much about how many chose to use it or not. If they do, they have to work with my preferred way of distributing code.
There have been times where third parties have included my code in their packages, but I'm explicitly not the package author in those cases, so it (the package) is not my responsibility.
> it is recommended prominently in multiple places that they not blindly trust the code and actually inspect it before using it. The friction in this process is intended.
There is nothing inherent in using packages that means you have to blindly trust the code, neither does providing a package mean you have to accept any more responsibility over providing a .php file (packages are just .php files with a few metadata files that allow them to be downloaded using composer rather than curl).
Fair enough if someone doesn't want to add metadata to allow their code to be downloaded by composer, but I disagree that that offers any security benefit.
> There is nothing inherent in using packages that means you have to blindly trust the code
Agreed, but packages are an additional layer of abstraction, and you and I both know that the vast, vast majority of devs will not "look under the hood".
Packages are often seen as a one-step plug-and-play solution. I don't want people to see my code that way. They should dive in and inspect it before using it (it is always written with this in mind - with extensive commenting and documentation).
> neither does providing a package mean you have to accept any more responsibility
Honestly, this is a personal thing for me. If people are using my code, I will feel responsible to some extent. IMO, the advantage of my method is that (at least a few) more people will test/audit my code as opposed to if it was available as a package. Which increases the likelihood of any possible bugs in the code getting caught.
> Packages are often seen as a one-step plug-and-play solution. I don't want people to see my code that way. They should dive in and inspect it before using it (it is always written with this in mind - with extensive commenting and documentation).
> IMO, the advantage of my method is that (at least a few) more people will test/audit my code as opposed to if it was available as a package. Which increases the likelihood of any possible bugs in the code getting caught.
The person who unthinkingly installs a package will also unthinkingly include your script using 'require'.
The only thing that happens is anyone who is interested in auditing your code and uses composer is inconvenienced with busywork, that would otherwise be handled by composer, e.g. autoloading the library.
> Honestly, this is a personal thing for me. If people are using my code, I will feel responsible to some extent.
The point you made was that you would feel more responsibility for a package rather than a PHP file. There's no reason why this should be the case. Both methods result in your code being run by 3rd parties.
You have misunderstood, the latter refers to "download the .php files, 'require' them", which is the situation you say exists right now.
I'm going to leave this by saying that I think the idea that you can make developers more conscientious by increasing busywork, is false. All it achieves is creating more busywork. Unconscientious developers will do the busywork and not scrutinise the library anyway, conscientious developers will just have to do extra busywork.
A better solution would be to provide a composer metadata file and to publish each new release using a new major release number each time, which is arguably the proper way to signal to consumers of the library that each version needs careful scrutiny and testing, as major release numbers signal breaking changes.
If I understand notRobot correctly, each instance of this 'busywork' is initiated by an email, which is an opportunity for pointing out the importance of testing. It is a matter of fact that most people are susceptible (in some degree) to such influences, so if notRobot is making this point with each announcement, it may have some effect (though probably small) - and regardless, if the process turns away some people who consider this busywork too onerous, and some of those also take the same attitude with regard to testing, then so much the better, from notRobot's point of view! NotRobot is under no obligation to do anything any differently, or give any justification at all.
> The person who unthinkingly installs a package will also unthinkingly include your script using 'require'
The npm stories show that most people do this with npm though. This color thing shows many people will just install whatever without checking: manually or automatically.
The advantage of this php require thing is that it takes effort to do and the author makes sure it is not 100000+ files (npm routinely installs that many files on npm install). Package management is great; it works well with NuGet for instance. But those are a sane community; no one used leftpad and such, so the tree of source to audit is not so large, not counting MS, but then again, you are not auditing nodejs are you? Npm is worse than gems, nuget, whatever php has etc simply because the community is pretty broken in as much that everything has to be a package and, even though you can type the functionality faster than you can search for it (yeah yeah whine tests whine docs: for leftpad, nobody cares about those things; it's trivial functionality), people use those.
Now faker (don't know colors) is non trivial: question is, what makes this to happen here and not in, say, nuget popular packages? Is it still/again the community or something else...
Passing around PHP files via email is functionally equivalent to passing out mix-tapes on street corners. Not a good tactic when a record label right around the corner will give you world-wide distribution for free. The only string attached is you'll have to rely on others of which you know very little, if anything.
I do not recommend being consistent with that position in other areas of your life otherwise you might quickly find yourself in a jungle, starving and naked. Given that relying on others for shelter, food, or clothing is clearly out of the question!
You've misunderstood, the email only contains a notification that a new release is out, along with a notice about inspecting+testing code and a changelog. Similar to how many FOSS mailing lists work.
The actual code is downloaded from either a git or http server, not via attachments to the emails themselves.
> There is nothing inherent in using packages that means you have to blindly trust the code
I use about a dozen different package managers and I have no idea how to check the code they download before they install/deploy it. I often check the source on Github if I need to look something up, but I have no idea how I'd go about verifying that the code on Github is the same as whatever the package managers install.
That sounds like a personal problem. .deb and .rpm packages are nothing more than tar archives with a specific file structure. dpkg and rpm both have options to extract the package locally. dpkg -L NAME will show you all the files the installed package has placed on your file system (not generated ones by the code obviously but ones that came with the archive). pip has similar options.
More broadly, and I am sorry if I am wrong here, but what do you expect to glean from reading that code if you don’t bother reading the man page for your package manager?
With node_modules, the amount of required code becomes unmanageable to review very, very quickly (sometimes with the installation of a single package).
This just seems like willful ignorance and has very little to do with package managers. If you were interested in looking at the code, a quick google search or running `--help` would go pretty far.
At the very least, distributing in this way (presumably with some license clause that it can't be later placed in a package repository) prevents other libraries using this library as a dependency. Expecting developers to review what's happening with libraries is mostly unrealistic, but expecting them to review changes in the dependencies of the libraries you use is completely hopeless.
> At the very least, it takes them under a minute to break your app, simply by deleting their package.
Not really, if you pin your versions exactly and don't do auto-updates. This also means that you have to update your packages manually and inspect what the latest versions are doing -- which is good practive anyway. NPM packages can not be depublished any longer as far as I know.
IMO this is what forking is for. You don't have to rewrite it, you simply copy it somewhere that it isn't going to change unless you make the change happen, because some third party screwing around with your production code is just Bad News even if they have the best of intentions.
You should still look over it and make sure it's not obviously malicious, but simply using forks or local repos of open source packages would probably save 98% of these kinds of headaches (with a 2% allowance for insecure/malicious open source code).
I have worked with several banks as a devops consultant and have helped implement self-hosted proxy repositories for NPM, NuGet, etc. These proxies save a copy of every package downloaded and store it locally for as long as the bank wants. Developers are then blocked from downloading from any other repository than these proxies. This solves the problem of packages being taken offline, but it does very little against malicious code on new versions of the package. However it also provides the ability to reproduce code X years into the past, as can be required of banks by financial regulators in various countries.
As always it is a cost/benefit trade-off: what is the benefit to auditing every package vs. the cost of auditing every package?
Totally agree with you, i think it's time to think carefully and immediately start using services like Vulert(https://bit.ly/336DZub) that tracks your open-source softwares for free and notifies you in real-time if any seccurity issue is found within your applciation. it's free.
atleast in this way we can secure ourselves from supply chain attacks
The library inserts ANSI escape sequences [1] between the text you want to colorize in order to, well, colorize it ¯\_(ツ)_/¯
Many people are obsessed with colors in the Terminal, and so, they reach out to libraries like this. They exist in every major programming language ecosystem, even though colorizing text is as simple as writing this \x1b[48;5;011<TEXT>\x1[0m . One of the disadvantages of this rudimentary colorization technique is that if you have short-term memory, you will quickly forget the meaning of these numbers, but you can solve that problem with constants, there is no reason to install a third-party library with potentially malicious code to add this type of functionality to a high-profile project like AWS-CDK [2].
I wish these libraries would support NO_COLOR [3] more consistently.
I have seen many “modern” CLI tools (the ones people like to build using Rust or Go) overuse colors with no option to disable them.
Honestly? They probably don't know how, or that's just not part of the JS culture. I've met a lot of JS-only developers, and most probably don't even know what ANSI escape sequences are, let alone how to work with them. The lack of basic computer knowledge from the JS ecosystem is shocking. I don't expect this to poll well in Peoria, as it were, but this has been my consistent observation.
My sense is that it’s time to evolve licensing such that wealthy major consumers of packages that have become somewhat essential are naturally paying a licence fee.
The problem is not in what the code does it’s a problem with the agreement for use.
Actually in attempting to answer my own question, on other platforms like YouTube and Medium, popular content receives monetary support by virtue of being popular.
What if this was addressed at the “platform” level, I’m thinking the package manager here, NPM.
If npm had paid plans that would essentially mop up larger corporations they could then auto-distribute funds Spotify style based on “number of listens”.
I’d personally want to see this work mainly as enterprise plans.
> If npm had paid plans that would essentially mop up larger corporations they could then auto-distribute funds Spotify style based on “number of listens”.
Until there's enough money in the pot that making your packages seem very important happens to be a productive use of one's time. At that point, you have to start dealing with fake downloads, dependencies added for no reason to somewhat more popular packages that aren't paying much attention... and suddenly you need to take money from the pool to pay for your fraud prevention team.
Perhaps download's value could be weighted by associated domains? For example if Apple.com is relying on it the author will get paid more than random.example
Except the Spotify model is also rife with issues. Artists generally hate Spotify and hardly make a living off of “pay per stream”. Most of them still very much depend on tours, merch, and, at the higher level, brand deals to make any money off of their craft.
Spotify isn't a replacement for tours. It's a replacement for cds and/or radio, both of which make artists similar amounts of miniscule amounts of money.
For programmers, you'd be correct. That only really be a replacement for patreons, tips, and donations, which would typically be a miniscule amount. It just redistribute it instead. (Your $x subscription just automatically gets allotted instead of manually allotted).
It's not the original gift that is the problem, all maintainers start very happy early on but keeping software up while adding more features is costly, someone needs to pay an it's almost always paid by the maintainer in terms of free time.
If you intend to keep it as the original gift, it will be called abandoned.
Free as in freedom is not the same as free as in beer.
This model where someone develops something for free and then those that benefit the most don't contribute back isn't sustainable. I don't know if the packages owner was conscious about it but this was a political act and hopefully the impact will be positive.
From where we are we have two options:
(1) companies find a way to make open source financially rewarding;
(2) companies use their own crap instead.
Is there a license that is like MIT but with special clauses for people making big bucket?
I don't think I can just put an extra clause on it that says something on the line of "if you are using this to earn more than X big macs by year then you have to pay me or be subject to a fine".
This radically changes things as it may void the liability clause and also make code less fungible. And there's the issue of fairness to contributors.
Unless someone find a way to replace devs with a machine there will be demand. And for the good and the bad there are efforts and partial success on doing that like tools to develop sites using only a GUI.
> My sense is that it’s time to evolve licensing such that wealthy major consumers of packages that have become somewhat essential are naturally paying a licence fee.
Yes, it's called paid software, but don't worry, it's going to be trendy again soon.
The days of free software contribution are almost over.
Devs want to be paid for their work. too many corporations made billions from open source projects while maintainers live in quasi poverty.
What is needed is a proper market place for paid open source software. Github isn't one.
CVE are the proper marketing. The perspective of having a package with a vulnerability and no-one to provide a fix is frightening to any software maintainer.
I think I could be extorted dozens of thousands for a single upgrade. Heck, when the electrician auditor says our office is not certified for 2022, I pay $700 for a professional to fix it. Software will be the same very soon.
Obviously he has this right to do what he wants and he did. But the consequences are that his reputation is in tatters. Probably doesn't bother him but equally the repos will be forked and utilization continues of the predecessor. Might not be a maintainer mind you so the dependent repos will quickly find alternates.
Doesn't it seem strange that Snyk is creating a vulnerability report for this + labeling it a DoS? DoS is something someone executes against a target, in this case a package had it's functionality (purposefully) altered. That's like calling changing the API of a popular library DoS, because now application authors need to change their code/use a different library...
Fittingly enough, all four solutions for this particular issue all goes back to Snyk, as seen in the bottom of the blog post. Seems like they are labeling this a DoS to justify being able to publish something in their database and blog.
But if the API offered a function called .countBy but then renamed that function to be .countAllBy, now I can't run my application anymore, causing my service to go down if I upgrade the version without testing it, is that a DoS now?
no. is it really that complex of a concept that intent of a change matters too, and introducing an endless loop to cause trouble to users is different from a legitimate API change that does a useful thing?
NPM expects packages to follow semantic versioning. If a package contained a breaking change like that, there would be a major version bump, and you'd have to upgrade manually.
If the maintainers wasn't acting maliciously, they could change this new version to count as a major release, and then it wouldn't be a DoS.
This change introduced an infinite loop upon import. It is nothing like changing an identifier which would've provided an error message about where the issue occurred.
> Doesn't it seem strange that Snyk is creating a vulnerability report for this + labeling it a DoS?
I think it most definitely falls into "malicious code" that certainly is not done in good faith, and if not handled properly by downstream users, can cause a lot of unexpected problematic failures.
Whatever labels are used to characterize this, whether to call it a vulnerability/DoS or not, are just a matter of arguing over semantic meaning.
I guess most customers of snyk if not all, are going to be happy that they tagged this as a vulnerability. I'm not sure why they would have done anything differently, seems like exactly why they are paid for (I don't use them and have no relationship with them)
DoS is just Denial of Service - he published a patch update that removes all functionality with specific intent to break the application that uses his code. His action, and this version of the library, is literally an attack on your application and "Denial of Service" is the only goal.
The truth is that it become a race to the bottom with a lot of open source projects, specially the ones that are not hard to replicate.
If the author made the project GPL, someone else would create a similar library with a more permissive license, which would end up taking the market and making the original library irrelevant.
He's a nutjob that blew up his own house while trying to make a bomb or something. There's even an article somewhere. It's not the first time he's being ..weird.
The author of this package was caught with 50lbs of Potassium Nitrate (in the middle of NYC) and a bunch of materials on making bombs and booby traps when his apartment caught fire:
> When investigators entered Squires' apartment to look further, they found more bomb making items including potassium nitrate.
Magnesium powder, sulfur powder, copper powder, aluminum powder, hobby fuse and mixing cups were also discovered in the home.
"The chemicals separately are what they are, but taken together they can assemble an explosive device," Deputy Commissioner of Intelligence and Counterterrorism John Miller said. "There were books about military explosives, booby traps and other things...What we're looking at here is the totality of the circumstances that raised our concern to a level where we're going to need more investigation."
Does that sound to you like he wanted to make a candy rocket?
You can do chemistry all your want, but attempting to build a bomb, even of the attempt doesn't succeed, is illegal.
At the time of the article, the investigation was still ongoing. That's likely why they were no charges yet.
> You can do chemistry all your want, but attempting to build a bomb, even of the attempt doesn't succeed, is illegal.
I will also say, that as a native New Yorker, doing this type of "kitchen chemistry" (if that was he was doing) is _extremely_ reckless in a dense residential neighborhood.
He was either was just a hobbyist who liked experimenting with explosives and he was fine with recklessly endangering an entire community.... or he was planning to commit a bombing.
A different article:
> On Thursday, law enforcement sources told News 4 the fire started because he had a box next to his stove that caught fire. He tossed it, trying to douse the flames, and it landed in his living room, which then also caught fire.
So he went to the hospital with severe burns on his hands. We don't know yet exactly what he was doing, but I don't think he deserves anybody's sympathy. That has its limits.
You guys are quoting all these "scary" lists of chemicals not realizing you're only proving my point. Those aren't chemicals for making explosives. They're for making fireworks at best. Non-detonating things that could burn fast and have pretty colors.
I suppose we should charge everyone who starts a fire while cooking with reckless endangerment too? I get that there are different standards of liberty in dense urban areas but I don't think this is beyond them. It's just cops and feds talking up their non-bust.
They'll have had the local fire marshall come in to determine legality and when that failed to create a crime they'd fall back on the BS charge that was dismissed to justify their violence on the scene.
He’s also going on about a wild conspiracy theory about Aaron Swartz getting assassinated because he was on to Ghislaine Maxwell, or something like that. And linking it to his open source comments in a way that doesn’t seem to make sense.
He’s almost certainly going through major mental issues, along the lines of schizophrenia or something similar. He needs help.
as is usual with a lot of recent conspiracy theories, seems analogous to apophenia[0] to me, or something similar.
the non-conspiracy "fact" seems to be what most people here think about Swartz: that he killed himself after a overzealous prosecution. Nothing to do with Epstein or Swartz's role at Reddit.
What’s the concrete evidence making this likely to be true? If there’s none, just wild speculation, then I’d consider it a wild conspiracy theory.
The link seems to be “Swartz downloaded millions of scholarly articles using an MIT network, and Epstein/Maxwell donated money to
MIT.” That seems to be about it? Not exactly a logical reason to conclude that Swartz was assassinated as part of an Epstein/Maxwell coverup.
The only way to prevent this is to pin the actual commit. Because the meaning of the semantic version numbers is up to the package maintainers in most packaging systems. And even then you need a way to source exactly that version without relying on the original author's cooperation.
Pinning all dependencies to this extent is extremely inconvenient. More inconvenient, arguably, than to deal with this shit once in a while...
Not sure if that is even enough, at least in NPM the .lock files work on semantic versions, not commits. I'm not sure if NPM enforces you to change the semantic version with each commit.
And even if all of that works, you still run head first into the issue once you inevitably upgrade the dependencies.
There is no way around the issue because "dependency hell" is indeed a thing.
Dependencies change. Dependencies of dependencies change. You can't not update any of your dependencies for too long or there will be other problems. Something is gotta give.
Pin all you want, if the repo/vendor/maintainer pulls the release then you're not getting access to your dependencies at all.
If anything, this is the reason you use pull-through proxies. Your proxy will hold the version you depend on, regardless of upstream drama. Keep your proxy backed up and you'll be able to use those dependencies until the end of time, or you finally decide to migrate to an alternative.
I'd say the likelihood is about 50% you have a NPM package in your dependencies right now that pulls some binary or whatever from a random S3 bucket during installation.
And their PRs aren't merged until they do, because we'd have talked about it before hand and achieved consent around the idea that pinning dependencies is a practice our team will start doing, or some other specific practice that solves this problem.
The little `^` in version numbers in NPM's `package.json` file is such a bizarre choice. The fact that it by default installs all new dependencies with that means that builds on different machines at different times could result in _completely_ different artifacts.
This helps with CI and deploys, but on developer machines running `npm i` will install different things at different times. The amount of churn a `package-lock.json` file undergoes when all of the dependencies have a `^` is crazy.
Sure, save this to `.npmrc` right next to your `package.json`. It doesn't retroactively change versions, so any existing ~ or ^ ranges need to have those characters removed. But further `npm i` invocations will save the versions without range characters.
At some point people need to stop pulling in random unsigned libraries from the internet and deploying them without any review or testing. This chaos seems like it would be entirely preventable with just a small sprinkling of best practices.
Signing would not have helped at all here - the author decided to nuke their project (and likely their last reputation), they could have signed that commit/package.
I'm seriously downvoted for this? We have just had an incident where a maintainer acted maliciously and has demonstrated that a single point of trust is insufficient. If we really care about avoiding issues with open source software, clearly it is necessary to get multiple maintainers to sign off on changes to widely used open source projects. We have had all the technology components needed to implement this for decades, we just need the will to implement a system that is better. If we don't use this incident to improve, then it's going to happen again, and maybe the consequences will be worse.
Most Linux distributions are using a single key for the signing of packages, and this might be an easier place to start changing over to a multiple signature model.
In this particular case is was a single jilted developer, but determined actor could easily attack someone with access to the keys to sign compromised packages, as per the obligatory xkcd on the matter https://xkcd.com/538/ .
Multiple signatures would indeed be a good mitigation against coercion, especially if the signatories were in different jurisdictions.
Ideally you'd want a system which separates reputation from meatspace identity, so that well-trusted reviewers couldn't be easily targeted offline. Unfortunately that would require a lot of good opsec, and go against the financial incentive for someone to disclose their online identity as part of a salary negotiation, for example.
Time and time again I'll keep saying this: This problem is only solved with package repositories that require review by a maintainer to publish. Linux distributions solved this ages ago.
Change that to multiple maintainers. Best practices should mean that any single point of failure is mitigated. I'm shocked to say it, but the blockchain might actually be a useful model for trust here.
In essence, It seems to be the case of a developer getting screwed, being disillusioned, becoming political, making bombs?, attacking the ecosystem etc.
Many years ago, I recall another developer of popular NPM packages(Azer Koçulu) pulling a similar thing[0].
We followed each other on Twitter, I recall him being disillusioned with SV and angry to Wikipedia for some reason(I think he believed on some greater plan or agenda pushed by SV companies, including Wikipedia). I disagreed and got unfollowed and blocked. Later, if I recall correctly, he got married and was touring the world.
Programmers have all the power to hurt things but they rarely think about using that power. Sometimes it's not how much value you can create that wins the day, but how much pain you can strike at other people that counts. Politics is like that. Ugly, but necessary.
The sad thing is that some(most?) of them only want to think about the code and do not want to do deal with the real world implications of the software that they are involved in.
It's a bit wild that the sum total money spent on salaries for engineers handling potential problems stemming from this or defending against the possibility in the future could probably have covered paying the maintainer a living wage many times over.
In America, most unskilled software developers make somewhere in the $80k to $140k range. A living wage is around $20k for absolute bare minimum essentials. Skilled devs still get around $200k.
Point is, maybe 10 people. And that’s if you like ramen.
Moderately skilled it engineers other of backgrounds and devs can make much more than $200k, just go check levels.fyi
To the parent comments point
> It's a bit wild that the sum total money spent on salaries for engineers handling potential problems stemming from this or defending against the possibility in the future could probably have covered paying the maintainer a living wage many times over.
The collective effort across numerous companies is much more than just $200k and you can bet your butt on that.
I don't know where people get these crazy numbers -
Even in America outside of the coasts and outside of FAANG, making $140-$150+ as a senior developer is very good (and compared to almost all other industries is absurd) - salary.com which doesn't just rely on self-reported info as levels does reports the median salary + bonus for senior software engineers as $120k
Outside of the US, even in more expensive places in the EU, even the equivalent of $100k for a super senior lead architect would be Very Good - I don't know a single SWE in the midwest in the US - including senior embedded systems engineers working on medical devices, senior firmware devs working on networking equipment, or any web engineer that makes more than $175k and I know plenty of Very Good senior full-stack web devs that make $125-$150
$125k a year is still a top of the top salary in the US, so don't cry for them tho
I just don’t think HN is interested in this anymore. There was a time. It’s gone now.
Dumb comments saying software engineers make 100x a living wage (as if this would be a bad thing) are the flavor du jour. It’s hard not to respond in kind.
But thank you, for what it’s worth. I remember you from 2010. It was quite a time.
For what it's worth, I didn't read thefourthchime's comment in a way that would suggest that software engineers are making 100x a living wage. The way I understood it, they meant that the total cost of all engineer hours spent on rectifying the problems caused by the 'colors' and 'faker' issue would easily cover 100 living wages.
Tragedy of the commons, shortsightedness and misaligned individual incentives.
Individual contributors in large companies, especially, would want their companies to fund FOSS projects they use. But approval processes are generally extremely complicated and there's nothing to gain internally by doing it. And we're talking about money that these corporations spend each millisecond. They barely need approvals for many other activities costing 10x, 100x in other domains.
Most of the companies that I’ve worked for have funded the FOSS that we used. By allowing me and my colleagues to contribute features we needed, or fix bugs that were affecting us. The core maintainers probably never knew these PRs were funded at an hourly rate paid for by some big bank, and sadly quite a few of the projects that I’ve contributed to have rug-pulled into some sort of non-FOSS enterprise product. We all benefit from FOSS, including all these disgruntled maintainers. The FOSS way should be to pay it forward, to contribute to projects where you can. If you’re expecting to get paid for it, it’s not FOSS. Deciding you can’t maintain a project anymore is fine, but pulling it out from under the people that are using it is incredibly anti-FOSS.
> The FOSS way should be to pay it forward, to contribute to projects where you can.
In theory, this was enforced by copyleft requiring derivative works to also be free software. In practice, companies use software with permissible licenses instead because then they can reap the benefits without any requirement to pay it forward.
> If you’re expecting to get paid for it, it’s not FOSS.
Being paid for your time has nothing to do with whether your source code is public or what freedoms users have when using your software. Conflating free software with volunteer labor is exactly what leads to situations like this one, where the author's business based on faker got copied wholesale by a competitor who simply ignored their attempts to reach out.
> Deciding you can’t maintain a project anymore is fine, but pulling it out from under the people that are using it is incredibly anti-FOSS.
The mechanisms that allow a rug-pull are entirely choices made by the users of the libraries for their own convenience; the author did everything needed for you to download a working copy and use it in perpetuity. It's your fault for choosing to rely on NPM, choosing to not cache your dependencies, and choosing not to pin your dependencies.
> In theory, this was enforced by copyleft requiring derivative works to also be free software. In practice, companies use software with permissible licenses instead because then they can reap the benefits without any requirement to pay it forward.
If you want to fix this, stop contributing to permissively-licensed software. If you have a change you want to make, make or find a GPL fork of it and contribute it to that instead.
Individual action is not gonna solve anything, you have to understand why people choose permissive licenses in the first place:
- They're contributing while at work and work only allows permissive licenses
- They're familiar with permissive libraries because of the previous point
- Permissive licenses are perceived as simpler
- They've been pushed away from the free software movement by the FSF/Stallman/Linus
- They don't think copyleft is the right form of enforcement
Fair! I didn't finish the thought, which was to address those shortcomings by either making free software work better for those needs (e.g. work with tech unions to negotiate guaranteed funding of projects used by companies) or by making it less attractive to use permissive software (e.g. via regulation).
Copyleft software isn’t free, it comes with a very hefty price tag. You don’t pay it forward by handing over all your IP. You pay it forward by contributing back. I have no moral qualms about using OSS in any project I’m working on, commercial or otherwise. Because I have published my own libraries for anybody to use, and contributed a huge amount of PRs to the software I use. When you publish something with a permissive licence, it stops being yours, but you benefit from having a huge number of people improve it for you. That’s how it works, that’s how it gets paid forward.
The OP is also attempting to use a proven failure of a business model, and then throwing a tantrum when it fails. Sure he’s within his rights to do so, but he has no moral high ground here, and I don’t think he’s entitled to any sympathy for adopting a business model that everybody knows for sure doesn’t work.
Working on someone's software that they make no money from isn't "paying" it forward or in any direction.
> it stops being yours, but you benefit from having a huge number of people improve it for you
It stops being yours, but somehow everyone who works on it can say they're helping you. This isn't fair. You don't have to pay for it, but fixing and adding features to the software that you use to make a living can't be counted as charity work.
FOSS has never been about getting paid to write software. It’s not a charity, it’s a contribution to a community. I contribute because I benefit from being part of a system that has contributors in it. The people who expect direct compensation for their contributions do not uphold those values, and are deteriorating the integrity of the FOSS system itself.
Your conception of free software is not how free software is generally understood; free software is about the rights of the users, not the expectation that people contribute to it. Sure, if you _define_ free software as being about a lack of ownership by one person and expected contributions, then you can criticize this.
But what happened here is that free/open source software doesn't have a consistent stance on paying maintainers or contributors, and this author feels that it's unfair and (potentially in the midst of other personal issues, it seems?) took advantage of a problem with how the ecosystem pulls in dependencies to complain about it.
As a matter of practicality, commercial entities using a maintainers' work should donate to maintainers to incentive them to, well, at least not go rogue, or to be on their good side when they rogue. Companies pay their employees to incentive them to function in the interests of the company. While this isn't fool-proof (principal-agent problem), it lowers the odds of a pissed off employee having the will/self-righteous fury to pursue something more aggressive than resigning in a huff.
For clarification, you're not referring to "contributing features [you] needed" or fixing bugs on the clock as funding? That's probably a nice thing, unless nobody needs a particular feature except the people who fund you at an hourly rate, but it's not "funding the FOSS" you use. That's when you give someone money. You can't eat features and bugfixes - without the rug-pulling you're decrying here.
FOSS is licensing, not religion. Rug-pulling a project from people who are enjoying using it isn't "anti-FOSS" IMO. This may not apply to you, but for all the contrast that OSS people project between their pragmatism and Free Software people being insane religious zealots on a jihad against money, OSS advocates seem to imbue a lot of flaky new agey spiritism into what FOSS is or isn't.
The author didn’t write all of the code, though. The code has a long history (including in other languages) and many contributors.
Why should this one developer collect payment but not everyone else who contributed it?
Regardless, it’s ridiculous to give something away openly under a permissive license and then later get angry when people use it exactly as you license it.
> Regardless, it’s ridiculous to give something away openly under a permissive license and then later get angry when people use it exactly as you license it.
Doing your best to live in a bad system does not invalidate the complaints you have about that system.
> That would involve exchanging his labor for currency.
That's the goal. Or at least one goal. But you can't just press a button and do that.
Being in charge of and an expert on open source software can be a way get people to buy your labor, but it's much harder than it should be. Instead many companies will demand you work for free, because it's open source!
Also trying to do something good for the world shouldn't make it so hard to make money. The companies get value but don't want to pay even a pittance.
> Being in charge of and an expert on open source software can be a way get people to buy your labor, but it's much harder than it should be. Instead many companies will demand you work for free, because it's open source!
It's hard to get paid when you decide to give your work away. If only there was some way a person could enter into a contract in order to guarantee payment in exchange for their work. What a radical idea...
> Why should this one developer collect payment but not everyone else who contributed it?
Exactly, no one said _only_ the lead maintainer should be compensated. _All_ of the labor, not just the labor that happens to have a day job that benefits from it, should be compensated. That includes non-coding labor like support or community management, too.
That’s just a a software development business. Open source exchanges labor for conditions the use of the product of that labor. If you want to exchange labor for money then exchange labor for money
The maintainer gave their work away for free. By definition - and by explicit license it isn't worth any wage, much less a "living wage many times over."
The maintainer wants to have their cake and eat it too - they likely believe in FOSS for moral reasons yet consider it immoral when companies take their software and use it freely under the terms offered.
If you want people to pay you for your work, don't give it away for free. If you give it away for free, don't have a temper tantrum if someone gets rich off of your work without compensating you, because those were the rules you chose to play under.
Imagine they introduced something worse. Could any developer explain to a manager why you needed to import this package? "Why do we need colors there?", "Why can't we make that colored ourself?"
An easy answer would be "we import thousands of packages either directly or recursively, so while we may be able to replicate the work of any one of those (which is unlikely to be true in the first place), it would take thousands+ of engineer hours to replicate all of them, and there was no way of knowing that this one among thousands would be sabotaged."
At my last job we (InfoSec) had the devs fill out "ownership" forms for when they want to include something third-party into the product. Other than forcing the team to do due diligence on the third-party it also made them responsible for keeping it secure and them the people "at fault" if something went wrong due to it.
While it was seen as an unnecessary hurdle set up by us I hope it started some meaningful conversations in the teams and maybe even end up with them "reinventing" the wheel for the better.
These sorts of "security" measures kill productivity and ultimately accrue (along with others) to the point where your organization moves so slowly that its lunch gets eaten by upstart competitors who aren't burdened by self-imposed make-work.
Yes. A myriad of methods falling into two main categories:
1. Robust build and deployment processes. Locked-down build servers, proxied/cached package registries, locked dependencies, automated dependency upgrades, tests, rollbacks, etc. Pretty much exactly what you need to mitigate unexpected breaking changes in dependencies, regardless of whether they're security risks or not.
2. Comprehensive dependency inventory. List of all your dependencies, where they're used, what vulnerabilities they're affected by, various other metadata, automated threat-hunting, manual review and annotation.
Trust but verify. No need for developers to fill out forms, wait on your (context-free) approval, resort to implementing worse versions of things themselves because they don't want to jump through hoops, etc.
Everyone decloud and only use the standard libraries compilers provide. What a wonderful world! Everyone is forced to do some system programming. Going to be pain in the beginning but then whoever really passionate about programming (not shipping products but programming) is going to be happy.
OK just joke :/ Although I do secretly wish to wake up one morning and find out we have to do things like in the early 90s.
This wasn't AWS CDK, it was a package to fake data and a package with some ANSI escape sequence constants. The comparison doesn't make sense. The problem is that developers apparently can't even differentiate between when you should use a library and when you shouldn't; they just pull in the first result from an NPM search. You can probably trust AWS, which is good because CDK is complicated. You can't necessarily trust random NPM package authors, which is good because rewriting `colors` is not a Herculean task.
As the article states, AWS CDK depends on colors, if I want to use AWS CDK, I have to use colors too. I don't get the choice to re-implement that myself unless I want to stop using the official CDK library
Well, my comment (in this case) was directed at the people at AWS who decided to use the "colors" package instead of just spending a half hour adding their own ANSI escape sequences.
i would pay for a service that runs an NPM mirror of a "last known good" version of packages to avoid this kind of thing. just keep all my dependencies a few weeks behind NPM to give things like this a chance to get caught, and let me continue blindly updating.
every time something like this happens, the reaction in the comments is the same: well you should test your dependencies. and yeah, i do that before release, but running an npm update on my dev branch and finding one of my dependencies has broken something is a bunch of work i could do without, and seems like work that is being duplicated by a ton of developers.
The replies here (and on twitter) really are making me realize a lot of the hurt that will possibly stem from marak's actions really were ripe to happen given the sheer number of javascript developers who don't understand at all what they're doing.
IT people for some reason tend not to be great at introspection which tells me this won't be fixed because this is more than just one asshole dude, this is a systemic issue with js and web development as a whole. This sort of thing would be really hard to accomplish in Linux for example because of the mentality they adopted early on when Linus had his first burn out moment, but the fact that even after leftpad nothing has changed really shows how systemically broken web dev is.
Anecdotal but everywhere I've worked on JS codebases has used lockfiles. It might be this isn't affecting that many but those it is affecting are loud.
Yeah but in this case you might not be directly dependent on colors. You might be dependent on http-server, which is in turn dependent on colors. You can only roll back http-server, and unless http-server rolls back colors, you are stuck.
you mean roll back package-json.lock or package.json? yeah, but thats not really what i want. maybe my process is crazy here, but this is how i update dependencies:
on a somewhat regular basis, as time allows, i run npm update to bring all my dependencies to the lastest version. then i test (including thorough manual testing / qa), commit, and release with updated dependencies. if some update is broken and i don't have time to fix it then yeah, i can roll back. but the point is to be on relatively up-to-date version of as much as possible, so if something is broken then it turns into a game of trying to figure out which library it was that broke things. i don't want to just not update any dependencies because something is broken.
I'm confused. You're looking for a way to go back to the last good version. By your methodology wouldn't that be before you last updated? Or are you looking for a curated service?
A reasonable maintainer semvers their packages.
You shouldn't update to the latest version if you want to minimize the likelihood of issues. That's what stable releases are for
Maybe everyone should include a clause in the license to force commercial use to pay certain "tribute" e.g. 0.01% of gross revenue.
Worst case everyone is going to invent their own wheels, which is beneficiary to all lower-echelon programmers (but not so for managers/tech leads as they have responsibility to ship things) because I as one definitely want to invent as many wheels as possible.
1,103 comments
[ 4.3 ms ] story [ 405 ms ] threadBut I honestly don't care if companies "exploit" open-source software by making money using them and not donating to the developer. That may be unhealthy for the ecosystem, but neither side is entitled to anything. I would donate, but not expect a donation, and poisoning the well the way these developers did is not going to help any of us.
If someone publishes code for themselves, and at no time asks anyone to take it as a dependency, then at a later date they change that code in a way that breaks other people's use of it, do GH then take over the account?
I'm (honestly) trying to understand why GitHub can't just go ahead and remove any account as they see fit.
Edit: Am I being downvoted for asking a question?
If the intent of the push was to damage downstream users of the software then it is malicious towards them.
This is like arguing about whether the james webb telescope really is in space since we don't have a precise consensus about what altitude is considered the frontier with space.
To take a trickier example, say a GH user has a lib, then decides to re-architect it, breaks the API and for their own purposes pushes it to an existing version, breaking all other use of it. Now that's a nasty thing to do, but is it malice?
Another, real-world, example is I know of a user who publishes "honey PoCs" for security issues, where the repo. appears to be a exploit code but actually isn't. He's been accused of malice in doing this, but his intent is research for a talk on how people use code blindly without testing.
Is that malice, should GH take his account down?
By stepping into this area GH are going to have to find answers to this and also the problem of who maintains the repos of accounts they nuke?
I think they already have, those 2 examples you mentioned already happened and were dealt with.
I think intent is important to take into consideration, since after all that is the definition of malicious: intent to cause harm.
Your first example clearly has no intent to cause harm. That case probably happened thousands of time already since not everyone is willing/able to follow semver cleanly and strictly. Never heard about GH taking any measure against that. And I would definitely not expect them to as a user/maintainer.
For the second case, I think GH policy is that you can host that kind of PoCs, but the repo has to be clearly documented as doing such (e.g. you can't just add some vuln into some unrelated code "for research"), and the vulnerability cannot be an active one: "We understand that the publication and distribution of proof of concept exploit code has educational and research value to the security community, and our goal is to balance that benefit with keeping the broader ecosystem safe. In accordance with our Acceptable Use Policies, GitHub disabled the gist following reports that it contains proof of concept code for a recently disclosed vulnerability that is being actively exploited." - GitHub." [1]
Back to Marak's case, my opinion is that GH did the right thing: If he just had his code in a repo, with no semver, no other contributors/maintainers, and such, and decides to nuke it, then I hope GH would not have done anything.
But when you are using all the tools and trust of open source: Other people contributing to your repo, other people being active maintainers/admins and spending times out of their days to fix bugs on it, when you leverage NPM to make it easier for you to distribute your package to others widly etc, you give up the privilege being able to act unilaterally like an asshole without consequences.
[1]: https://www.bleepingcomputer.com/news/security/githubs-new-p...
For example in that first case, say all they had was a wave of people saying "x broke my application" it would look a lot like the case in the article, and they'd have to dig in to find out it was just a bad API change without semver being followed.
Also requires Github to have a staffed department to deal with this, now they've established themselves as the arbiters.
For me, there's a split between a repository (NPM) and a hosting company (Github). For this case I'd have forked the repo, rolled back the malicious change in the fork, and hooked the fork up to NPM, and leave the original GH account alone. That solves the problem of the breakage, without getting in to banning whole GH accounts.
Play silly games, win silly prizes.
If other people don't want that, then they shouldn't pull from his repositories. If they do that anyway, then that's their own fault. Nobody forced them to.
That's because the author themselves said in this case, that the reason to submit the malware was to give a "fuck you" to the big corps.
Yeah, so obviously he did want libraries that give a "fuck you" to the big corps (by printing blather in an infinite loop). Then it still can't be "malware" to put that in his own repositories.
And my point still stands: If other people -- you, big corps, whoever -- don't want that, then they shouldn't pull from his repositories. If they do that anyway, then that's still just as much their own fault. Because, still, nobody forced them to.
I see no code in there that checks if it is running in production. In fact, it is a reasonable expectation that people don't throw code into production blindly, but rather test any changes out first.
Yes, I do. I may not have a right to push malware onto unwilling victims, but I absolutely have a right to change _my_ software however I want.
> "wElL yOu ShOuLd HaVe Tested"
Please, no need to be childish here. I have not taken that tone, nor will I respond to it in kind here.
> no you shouldn't push software ... designed to crash apps that use it.
Show me where a `git push` == "push[ing] software ... to ... apps that use it". When the `git push` is to my own repository, mind you, not someone else's app.
> ... in bad faith ...
Finally, I agree with you on something.
Of course this was in bad faith! That was clearly the point. When I write software and put it out there, and somebody comes and uses it, and I break my software to spite them, I am obviously acting in bad faith towards my users.
But that does not make it malice, or my software malware. I did not reach down into other people's computers/apps and change what they run.
Good that you woke up to that.
He didn't force anyone to update to the new version, right? So how is it his problem? Some other entity had to go and update the version they depend on.
And if you now say "well, that happens automatically", I say; suites them right. They should have tested the stuff.
Not his problem.
What one usually gets in trouble for is causing destruction.
Regarding this case; wherever they ran in to problems, they probably should thank him for exposing that serious flaw in their release process.
Since it is his code; can you vandalize your own property?
> go back to the last good change and lock the developer out.
That's one reason for not using GitLab as source management tool. It gives them way too much power.
Surely, you mean GitHub.
Without a warranty, you're not held to strict liability, but you can probably be held liable under the default legal regime. If I buy real estate and get a quit-claim deed, there is no promise that the seller has unencumbered rights to the property. However, if I can show the seller intentionally defrauded me, they can still be held civilly and criminally liable for the fraud.
he's not, He's being held responsible for intentionally pushing malware
I don't get why people don't just pin versions, honestly.
I'm not saying he did a good thing. But neither did he push malware nor has he any obligation to publish unbroken packages. If you're using FOSS projects without a service contract, don't whine if something breaks.
Let's say I set up a lemonade stand in my neighborhood every weekend, where I pour a bunch of cups for people to take, put up a sign that says it's free, and I set out a tip jar.
After a few weeks, I get upset that people have been taking the lemonade without leaving tips, so the next time I set up the stand I add a toxin that I know will cause immediate damage to anyone who ingests it. To protect myself, I have of course been posting a sign every weekend that says the lemonade is provided as-is.
So – did I do something wrong, or not? Will a court look at this situation and say, "gee, he just poisoned his _own_ lemonade and set it out for public use, it's not like he forced anybody to drink it"?
This feels like 100% black-and-white criminal conduct, and I would hope anyone who pulls a malicious stunt like this would be held liable for it.
That advantage may well be indirect such as reputational or learning. I just don’t grasp why people do it for nothing, to the advantage of large companies.
I am getting huge value from the people who built stuff before me. When I build stuff I can (hopefully) make the world better in the future. That's a "tangible, quantifiable advantage" to doing open source. It's just not an advantage to me personally. But lift your gaze an inch off the ground and you'll see we don't need to be ego centric sociopaths. We can build together. For the species. Everyone wins.
I don't know in what fairy tale you live in but the ego-centric billionaire sociopaths that exploit this system wins.
They get: meaning, status, influence, connections, reputation, and opportunities
The free and open nature of their contribution makes it much easier to get all these benefits than they would with a paid and proprietary solution.
It's fun to tinker. It's fun to put things out there into the ether. It's fun to exercise the brain and try new things and learn new ways to do things and publish things. The second it stops being fun, we stop.
Discussions now are about how you shouldn’t run your own server, and you should use popular stuff so you can speed up development and get your startup going.
I mean, I know about ycombinator and all. But it doesn’t seem to truly encompass the hacker spirit, if you ask me.
Necessity is the mother of invention after all.
And you know what? I support this kind of thinking.
I mean, if people can monetize videos on Youtube, shouldn't developers monetize their software too?
What it should be training you to do is test updates first rather than blindly applying them in a production environment...
I'm not convinced that GitHub has any business suspending his account.
It's a bit like me going to my bank to close the account and instead they throw me out and keep my money.
Honestly I don't understand why microsoft did anything at all. Can't people just pin a version?
Just a proxy that delays new version availability for a week would protect you from this.
But yarn add directly from the account of a madman who made your dependency is so much easier.
Pin dependendency versions, don't allow auto-upgrade. And cache your dependencies locally for as long as you depend on them.
Yes.
Or vet them thoroughly.
Packages are literally remote code exec vulns in the hands of package authors. At the very least, it takes them under a minute to break your app, simply by deleting their package. Read the article. This is not the first time it's happened, and it's not going to be the last. [0]
I write backends (mostly in PHP, although not exclusively), and I release a lot of my code under libre licenses. But I don't do packages. I don't want that level of control over other people's projects, it's scary as fuck. I have enough responsibilities as is.
I have a mailing list for people who use my code, when an update is out they can download the .php files, 'require' them and test them before deployment, but never will I do packages.
IMO, re-inventing the wheel sometimes is not the worst thing. Including code written by strangers that you haven't inspected and that they can remotely modify is. Stop using packages that are essentially wrappers around three-line Stack Overflow answers.
In this case, the old-fashioned way is the better way, and you'll have a hard time convincing me otherwise.
[0]: https://qz.com/646467/how-one-programmer-broke-the-internet-...
How far do you take this though? The average GNU Linux distro ships with a whole pile of packages already installed, from a multitude of different authors.
It's mainly a nuisance. It takes up unnecessary space. Introduces possible annoying merge conflicts etc etc and it's not trivial to remove it.
As reference, I migrated repositories from TFVC to git. One team relies on checking in packages into source control, another one does so far less. One repo is significantly nimbler.
Checking packages into source control is making your VCS a package manager. Presumably you have one. Don't hammer nails with your screwdriver
that's using version control to act as a proxy. AFAIK, a lot of package managers already cache local copies
> You get to ensure what exactly makes it into your application
sorry but i don't follow
> I don't see why merge conflicts would be a problem since you are just replacing a file with a new version
Are you working alone?
But this cache is usually not easily transferable to someone compared to them just cloning a repo.
>sorry but i don't follow
You have the source code to all of the dependencies in your application.
>Are you working alone?
How many forks of a dependency do you use? Just using the master branch and upgrading along that should be good enough for 99% of your dependency.
right, so they need to download "stuff from the internet". it Doesn't matter much if that stuff is from a remote repo or hosted by a package repository. Except if it's architecture dependent, in which case you definitely don't want to share across architectures. Not to mention they may already have a viable copy in a proxy or cache
> You have the source code to all of the dependencies in your application
I'm afraid I still don't follow
> How many forks of a dependency do you use? Just using the master branch and upgrading along that should be good enough for 99% of your dependency
Well, if I was expecting things to not break I'd never follow upstream master for a dependency.
But the question pertained to merge conflicts. If several people track the same remote and check in dependencies into VCS I'd expect annoying merge conflicts
Or are we perhaps misunderstanding each other? I'm not sure I follow what you mean by forks. Releases are typically on different branches or tags
[1] https://git-scm.com/book/en/v2/Git-Tools-Submodules#:~:text=....
You checkout a project and start it. No downloading required, and no 10000s of files.
Letting maintainers update your projects is a convenient feature. If it is a liability in your use case you can work around it. Yes it will take more effort, but your use case justifies it.
This is at least obvious DoS, I’m sure it’s easy to slip in an innocuous line that, dunno, ships your ssh keys to some rando server.
If you can, you're an infinitely more diligent developer than I am, that's for sure.
"Endo protects program integrity both in-process and in distributed systems. SES protects local integrity, defending an application against supply chain attacks: hacks that enter through upgrades to third-party dependencies. Endo does this by encouraging the Principle of Least Authority. ... Endo uses LavaMoat to automatically generate reviewable policies that determine what capabilities will be distributed to third party dependencies."
https://github.com/endojs/endo
> With the level of scrutiny required to detect a maliciously-obfuscated security exploit
Nope. Not paid to do that and I have not been given any such responsibility.
That said, I think the attack vector on this is very low.
Packages are rarely updated to the latest version.
We don't use a lot of packages.
We mostly use packages from trusted sources.
We use packages that are open source.
There is literally, on average, no difference in average quality between commits on FOSS projects, and commits on projects we pay external entities for. Some paid projects are just crap code, and some FOSS code is extremely high quality.
I've had to roll-back / hard-pin dependencies from both low-quality FOSS & low-quality paid projects because of commits that — once you find & read them — are just bananas.
(I have no idea how to solve the root problem here, honestly.)
I'm not even sure what the problem is. If it's "updating dependencies introduces severe side effects" wich, I think, should be accounted for in the process
It seems like the problem here is more cultural than technical, specifically that the JavaScript community has fully embraced packages that are "written by randos" that are "wrappers around three-line Stack Overflow answers."
I use packages, but I wouldn't use any that are developed by a rando with no reputation or "institutional oversight." An important part of choosing to use one is evaluating the maintainers.
Does the JavaScript ecosystem have anything like Apache Commons? I'm guessing not, but it probably should.
...and meanwhile NPM's idea of vetting packages is basically "YOLO, BRO!"
Javascript is used on the front end. Front end devs obsess (or at least used to obsess) over download sized. So you'd have crazy stuff like custom builds of Underscore (https://underscorejs.org/) with just the functions you wanted. Think manual sandboxing, if that makes any sense. You could get a package of Underscore with just map, filter and reduceRight, if you wanted to.
Now, when Node came around, people wanted as much as possible to have the same libraries available on the front end, so the same obsession with size was carried over.
Ergo the micro-milli-nano-packages they make.
Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure).
It's a linker. Tech from the 1950s.
Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't.
https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h...
Java's largely to blame for this, Sun REALLY, REALLY hated stuff that could be hooked into any OS and wasn't portable, so they didn't provide a linker. Everything was supposed to be on their JVM and you were going to install their JVM everywhere (2 billion devices!!!) and to hell with small stuff or heaven forbid, including native libraries. Javascript followed (on top of the Java restrictions they added: dynamic, poorly defined language, that would have made linking with tree shaking really hard, anyway). .Net also followed.
Almost 3 decades later we're trying to undo that damage.
- JavaScript is a very dynamic language with dynamic property access and a few other features that make it hard to guarantee that the linker won't accidentally remove too much
- historically there was no standardized "module" format until ESM (ES modules) came up (with some time in between with few competing non-standardized proposals), so statically analyzing exports/imports was difficult; in frontend you'd long rely on just creating and reading global variables (i.e. side-effects).
Hence it's been "safer"/easier to create small packages.
But it's not only this. Once you put a mega-package in your repo, it's easy to gradually start relying more and more on the things it gives you. Even if it supported perfect tree shaking, you'd call one method here, one method there, and with each build your bundle size would balloon (which is not good if you could write one line of code while lib method's code is 1000 lines because it supports IE4 and 17 parameters).
Whereas when you rely on small packages, you need to make a conscious choice each time to pick another dependency.
You probably don't care about this on servers written in C++ or Java that much; but on frontend it's a big deal; hell, even when building native apps for Android/iOS you have size limits for the stores submission / limits for the number of methods (tech limitation in Android). Big companies invest crazy money to shrink their native bundle sizes (https://blog.pragmaticengineer.com/uber-app-rewrite-yolo/).
The maintenance burdens these languages are creating will make Cobol look like a kiddie bike with training wheels next to monster trucks.
Actually building large systems in dynamic languages? Probably going to turn out to be a mistake though.
And it is not even a difficult thing to fix without going the linker way: java’s modules essentially solve it (as well as javascript modules could/can) — just specify what is visible outside a package and both ecosystems can “tree-shake” non-used code (though I dislike this nonstandard term)
This isn't a Javascript problem, this is a node problem. Node is just a Javascript platform among many. The fact that the node community decided to go with all these "nano packages" has absolutely nothing to do with Javascript. Nothing forced Node, the distribution, to come with such a barebone standard library. Absolutely nothing... but the idea of being dependent of NPM which was orchestrated by NPM founders, that's how NPM, a private business made money and eventually sold to Microsoft.
In a sense you've pointed out the alternative to having the programmer handling the packaging -- having some third party package and distribute. And this separation of responsibility turns out be almost always be a better solution. Distribution and coding are, after all, two full jobs (without 100% skill overlap). Plus, hopefully it indicates that at least two sets of eyeballs have at least glanced at the code (not the full desired many-eyeball outcome, but as good as we can expect sometimes).
The one thing that Debian provides is that in the case there is a security issue, admins worldwide only need to do "apt update && apt upgrade" and they are safe, without having to check all of the software that runs on their servers (as long as said software comes from Debian, that is!).
It has happened before. Last time there was anything major was over a decade ago though.
https://lists.debian.org/debian-security-announce/2008/msg00...
You also can't unpublish once a single person has downloaded the package, I believe.
You absolutely can unpublish, it just requires more steps. If NPM gets a DMCA takedown request they will absolutely have to fulfill it.
The they have gotten the right for npm to distribute the source code in context of npm.
There is absolutely no copyright or publishing right transfer that takes place when one "publishes" a package on NPM (or on Github). None.
The original author is absolutely entitled to a DMCA takedown notice and NPM would have to oblige him.
That's the first mistake you are making.
Secondly and as important if you publish something under an Open Source license(1) then you _cannot unpublish it_. You granted copyright to _everyone_ for and existing both now and in the future to distribute and use it(2) (legally it's a bit more complex but that's what it boils down to).
(1): Assuming you had the legal right to do so, but if not you are liable for any fall out, not npm (because ToS, they still need to take it down reasonable fast, but they might be able to sue you).
(2): Within the constraints of the license.
* no other packages in the npm Public Registry depend on
* had less than 300 downloads over the last week
* has a single owner/maintainer
So while your point is taken that unpublishing is possible under some circumstances, it is not for popular packages that are in use today.
[1] https://docs.npmjs.com/policies/unpublish
https://news.ycombinator.com/item?id=29868199
neither do NPM TOS, or whatever Microsoft thinks they are entitled to, since NPM is owned by Microsoft.
Assuming the package is released under a Free Software licence, what grounds would there be for a DMCA takedown?
I suppose a developer could include the lyrics to a pop song in their code (possibly encrypted), and then tell the copyright holder about it (since I don't think you can make a DMCA request on behalf of a copyright holder without their permission), but I would hope that such a poison-pill would be caught long before the package became widely depended on.
Perhaps you're thinking someone would risk perjury(?) charges for making a false DMCA request against their package, and NPM would act on the request without questioning it; but remember that NPM is owned by Microsoft and they have previously stood up to frivolous DMCA requests (after a fashion)[0]. That article has the lede: "Software warehouse also pledges to review claims better, $1m defense fund for open-source coders".
[0] https://www.theregister.com/2020/11/16/github_restores_youtu...
Tell that you Youtube's copyright trolls
You're probably right, though, that there is enough imprecision in the system for someone to claim that someone else's code snippet infringes on the copyright of a code snippet the claimant had previously published.
[0] https://torrentfreak.com/u-s-indicts-two-men-for-running-a-2...
[1] https://freebeacon.com/culture/google-youtube-algorithm-copy...
In theory, you're right. In practice, there's never any actual consequences for filing a false DMCA claim. Worst case is that the thing doesn't get taken down, but that's no worse than if they didn't file it at all.
Some parties that are distributing other peoples' stuff lose a safe-harbor protection from liability themselves if they ignore it.
This means intermediaries who don't benefit much directly from distributing a given bit of content will immediately comply with the DMCA takedown process. But this does nothing if you send the notice to someone who is actually using it.
The correct move is to send DMCA to the infringer's ISP/host. Then the ISP has to take it down unless counter-notified that they say they're not infringing. In turn, that counter-notification improves your position for any litigation that may ensue.
I'm not sure what about the current open source ecosystem makes you think anyone would catch something like this.
Legally, that meant that noone could use it. In practice, nobody but our legal department cared, so we had to wait for version 2 when the dependency chain was updated to remove it.
Noncompliance with the license, e.g. by removing required copyright notices/attribution in the code (this has happened in the past). Or straight-up uploading someone else's non-free code.
This seems like an edge case that wasn't anticipated by the DMCA, but I can see the argument that mixing GPL code with proprietary code is creating and distributing a derivative work, in violation of the GPL. Without proprietary code being present, though, I don't think a developer can DMCA takedown their own GPL software.
[0] "As the Minecraft Server software is included in CraftBukkit, and the original code has not been provided or its use authorized, this is a violation of my copyright." https://github.com/github/dmca/blob/master/2014/2014-09-05-C...
No, they don't. Honoring DMCA takedowns allow benefit from an additional safe harbor from any existing infringement liability for the alleged infringing content, but are not mandatory in their own.
Off the top of my head Bundler, CocoaPods, Cargo, SPM, Pipfile(and various other Python dependency managers), and composer also all work like this.
Cargo even makes it implicit that a version like “1” means “^1.0.0” in Cargo.toml.
The first line of NPM install's documentation[0] says(emphasis mine):
> This command installs a package, and any packages that it depends on. If the package has a *package-lock or shrinkwrap file, the installation of dependencies will be driven by that*, with an npm-shrinkwrap.json taking precedence if both files exist. See package-lock.json and npm shrinkwrap.
What does happen is: if you have added a new package in package.json it will be installed based on the semver pattern specified there, or if you run npm install some-package@^x.y.z the same thing happens. Further, if you modify package.json by changing the semver pattern for an existing package that will also cause this behaviour.
Running `npm install` in a package that already has a package-lock.json will simply install what's in package-lock.json. `npm install` only changes the lock file to add/remove/update dependencies when it detects that package.json and package-lock.json disagrees about the specified dependenices and their semver patterns e.g. having foo@^2.3.1 in package.json and foo@1.8.3 in package-lock.json will cause foo to be update when running `npm install`.
0: https://docs.npmjs.com/cli/v6/commands/npm-install
npm install will traditionally install the most recent packages that match your constraints. You need “npm ci” to use true version locks
Incidents like this highlight that this may not be the best idea.
When you have a package-lock.json NPM will install exactly the same version of everything in your dependency tree, making the CI builds much more like what's on your dev machine (modulo architecture/environment changes)
Or at least day 1000? Npm was launched in 2010, 11 years ago, and I'm quite sure immutable packages were implemented about 3 years ago.
Again, this is not rocket science, we knew the attack angles.
Thought it would be clear enough.
Autobumping versions, or version ranges as they're called in Maven land.
Dependencies should only use fixed versions and all updates should be manual.
You should only use auto-upgradable versions during development, and the package manager should warn you that you're using them (or your dependencies are).
Dependency management is not as simple as only upgrading one direct dependency at a time after careful review.
The NPM ecosystem is particularly difficult to work with as it has deep and broad transitive dependency trees, many small packages, and a very high rate of change.
You either freeze everything and hope you don't have an unpatched vulnerability somewhere or update everything and hope you don't introduce a vulnerability somewhere.
The JS ecosystem will probably have to change but because it's so decentralized, that change will be orders of magnitude harder than, for example, PHPs transition from 3 (4, 5) to 7.
Is it? Everybody is pulling from Microsoft owned servers now, as Microsoft owns both Github and NPM.
I don't think you're right in the builder/building practices sense.
Most package managers won't allow these stunts and conflicts have to be resolved UPSTREAM. NPM chose to go the "YOLO" way and will fetch every single version of a package that meets the dependency demands. Terrible design, but the purpose of that was growth for NPM, the company, not the best interest of the ecosystem.
You need to ask npm to upgrade or delete your lock file and node modules to run into this issue.
All together I don't see how GP's "email php files around" is as any better than this system in any way.
Admittedly, I don't think it has nearly as wide a usage as it has in the NPM world. Dependabot (I know I'm not the first to mention it, here, today) is probably more of a factor.
Still, it strikes me that this sort of "attack" (or mishap) is exceedingly rare in the Java ecosystem, while it's pretty common in the NPM world, and I don't immediately understand why that would be so.
> while it's pretty common in the NPM world, and I don't immediately understand why that would be so.
I think it boils down to Node projects typically specifying dependencies in the form “any version >= X”, effectively “always use the latest.” Dependencies can therefore get bumped silently just by rebuilding, essentially. Whereas in the Java world updating dependencies is a deliberate process.
Tread carefully with all the supply chain attacks out there, it might not even be the authors doing these. We are entering a dependency attack massive war.
Dependencies are a balance but also a sign of weakness of a system in the modern day. There at least needs to be delayed, dependency bot like analysis before you integrate. Even then, they just leave your systems open to worse than DLL hell, telemetry tracking/data, and attack vectors that can take down or target many, many systems.
I'm still continually baffled that we ended up in a world where automatically accepting updates from every dev and their dog is not just the norm but recommended practice.
Whether or not this would be compatible with the way dependencies are used today is another question.
I think at some point it will have to be a language level feature. The ability to sandbox or provide permissions to packages/functions. Just like our OS had to, just like browsers had to, just like phones had to.
Our code is the platform, the packages the apps. It's a similar use case.
If I could download a module, and tell the compiler this module, and everything it uses (including packages that I also use, but through a different call tree) will never access the network or write to disk, it'd help grant some small peace of mind in terms of security at least.
I was leaning more towards the web approach where we assume everyone is out to get us, but they can't unless we give them that one permission they need. If it's a statically typed language then it'd even allow dependency walking to see what permissions are used at a granular level and we can decide not to bring in anything that's too loose. This of course won't solve cases like logic bugs, but it'd help mitigate the impact.
I'm just not sure if it's even feasible?
Not sure if you can scope down permissions as part of an module import or if it only works when you initialize the interpreter
Deno runs code in a sandbox where you need to give permissions to scripts/modules for them to access local files, the network, etc:
https://deno.land/manual@v1.17.2/getting_started/permissions
> If I could download a module, and tell the compiler this module, and everything it uses (including packages that I also use, but through a different call tree)
Javascript's prototype based inheritance looks like it can help facilitate such conditional submodule invocation. But, and partially for performance reasons, static compiling would be necessary. So Javascript and its dominant NPM package ecosystem can never go in a direction like this.
If only C++ or Python (dynamically typed, I know) had prototypes instead of class based inheritance.
Edit:
Looks like another commenter referenced what we're probably talking about:
> Now, about the technical solution to this. We have this, for well defined programming languages (read: statically typed ones, or dynamically typed ones with a clear structure).
> It's a linker. Tech from the 1950s.
> Link (include) just the stuff you want, "tree shake"/"remote dead code" whatever you don't.
Can we create an open source linker for JavaScript and NPM packages?
Tech cycles with people who reinvent the wheel and keep making the same mistakes
All these problems have long been solved
So invariably, people discovered that if they threw out the complexities of the solutions, they could make faster progress.
Then they eventually ran into the corner cases.
That's the time loop that keeps happening.
As for reinventing the wheel, are wheels really settled science? As far as I know, new kinds of wheels are being created all the time. It's not just that new people are creating them, there are new things wheels need to do every day and new sets of requirements that the old existing wheel designs don't fulfill. Look at wheels from 50 years ago and they are nothing like the wheels of today. The wheels on cars are nothing like the wheels on aircraft, which in turn are nothing like the wheels on trains.
People suck at this. What this actually tends to do is mean "no updates, ever" unless you have a particularly rigorous culture of dependency management.
a culture where upstream writes in more detail what their code is supposed to do and downstream enforces that the software indeed does (not do anything beyond) what they specified
It didn't lead to fewer updates, it led to less usage of SELinux.
I think anyone who thinks they're doing this is fooling themselves. You can review code for accidental vulnerabilities but if someone is trying to slip in a backdoor it shouldn't be hard to do so in a stealthy manner.
The reality is that the entire dependency concept is just broken. There is an implicit trust that all dependencies are equally trusted. Your logging package is just as capable of performing file and network operations as your http package, even if you assume it won't.
That's silly.
It is up to programming languages and package managers to solve these problems. They're also not that hard to solve, in my opinion. "Run arbitrary code on a computer" is a model we've been securing for decades with web browsers, both in terms of web pages and extensions, and now too with mobile.
Solving "this code can do X but that code shouldn't be able to" is similarly easy to solve with languages that support effects or capabilities.
It just hasn't been done yet.
Dependencies with dangerous but necessary permissions can still abuse them: Your network library will still be able to add a bitcoin miner.
What happened if an update requests a new permission?
Also, how would that have prevented the current situation? Infinite loops are famously hard to detect and prevent automatically.
A lot of that stems from permissions systems being implemented outside of the code they constrain. In theory a compiler knows every reachable system call and all points of data input that could reach them, and as such it could constrain the program's capabilities accordingly.
In fact, compilers already do this for control flow integrity - it would just be a more advanced system.
> What happened if an update requests a new permission?
It's going to depend on the system. For browser extensions the new permission means a new prompt, so you'd get a CI failure until a human updated a lockfile.
> Also, how would that have prevented the current situation? Infinite loops are famously hard to detect and prevent automatically.
It really depends on the system. You could have a CPU capability that restricts cycles or forces preemption, etc.
I'm not saying you can solve literally all security problems but you can reduce risk considerably. If "infinite loop" is the scariest thing a dependency can do we're in a pretty good position. An unconditional infinite loop should break your CI tests.
Sorta yes, sorta no.
Imagine I'm making a chat client, and I want users to be able to drag and drop images to share. But the OS doesn't have an "open drag-and-dropped file, extension .png or .jpg" function call, it only has "open file" which lets me open ~/.ssh/id_rsa too.
Or if I'm making a web browser and I want to support U2F tokens. But there's no OS "talk to U2F token" call - the browser needs access to the system calls for "talk to arbitrary USB devices".
Sandboxing PC software is tough.
A programming language doesn't have to expose system calls directly. It arguably it shouldn't, in fact, for exactly this reason.
Further, you can restrict a process to only open specific files in a number of ways on Linux, including based on path. There's room for improvement, though.
I think it follows from two things:
1. We use open source software for everything. This is also true of our dependencies, so we get hundreds or thousands of transitive dependencies. Many of which are presumably written by dogs, because (i) no one can tell if you're a dog on the internet, and (ii) OSS maintainers are so overworked they ask their dogs for help.
2. Languages and libraries are full of footguns, software is full of bugs and therefore vulnerabilities, and no one cares enough to go through the enormous effort to fix things. And this is true through the whole stack. So the only practical way to stay secure-ish is to reactively patch software as vulnerabilities are publicly discovered. And also defence in depth. (I would distinguish between the publicly known time that a vulnerability is discovered, and the first time it was discovered. You hope the two are the same but for many vulnerabilities, if a clever adversary found them first we'd never know.)
With these two things together, you have a ton of questionable dependencies, and you need to update them all the time for security reasons.
> So the only practical way to stay secure-ish is to reactively patch software as vulnerabilities are publicly discovered.
But this is different from just blindly accepting any update that upstream gives you.
> And also defence in depth.
This sounds increasingly like security theater. You can always more layers obstacles to make things harder for malware that is already on your system, but it's not clear to me how much this actually reduces your atrack surface.
1. Reduce blast radius: assuming component X is compromised, how far and wide can it be felt?
2: Principle of least privilege: once compromised, what can X do or access? Extend to the credentials X carries or has access to.
3: Detection: how early and how well can you detect the compromise in previous two steps?
You can never prevent a compromise, but you can make it easier to notice when it has happened, and you can limit what the attackers can do afterwards.
It does help. Quite a bit.
Each layer (e.g. firewall rules that require that all internet access go through a proxy), adds non-trivial amount of work for the hacker to get anything useful done.
1. best case - hacker will give up.
2. good case - you have more time to notice and react.
How much layer cost you, how much does it cost for hacker to overcome it.
Things to remember:
1. Not all hackers are nation states. Most are not.
2. We must accept that no security measure is absolute even against script kidies. Given enough time and luck/misfortune js sandbox will do "rm /sensitive/file".
Recent Log4shell example shows that one can follow all best practices and still get bit in unexpected way.
This leads to some thoughts on statically-compiled applications; while they might have some vulnerable dependency, I suspect that it's harder when the attacker is limited to the app's "baked-in" functionality that defines how those dependencies get used.
Edit: Also, I should note that while I would greatly prefer, and do advise, that they base their environments on minimal OS distributions, this seems rare. The base system patching would be much easier to manage if it started from some BSD-like minimal state, or Alpine Linux, and included only what it needs. Instead, any infrastructure vulnerability assessment leads the teams to chasing down numerous patches in things they have, but never use.
Manually checking before updating does not scale.
I have zero sympathy for anyone complaining they were hurt by this. I think Marak is teaching an important and principled lesson here.
What lesson is that?
Which trillion dollar corporations are these?
A quick google says "Apple, Microsoft, Alphabet, Amazon, Telsa, Meta, NVidia, and Berkshire Hathaway" are the the only trillion dollar companies
Except for the last one all of those companies give back vast amounts of open source support. All you using VSCode for free, that's Microsoft's payback. Oh, and Microsoft pays for NPM hosting and github, also Typescript, C#, F#, .NET. Hardly not giving anything back. Apple gives Swift, Clang, LLVM, Webkit to name a few. Alphabet, Go, Chrome (which also means Electron on which VSCode is running), Android, and plenty of others. Meta provides React, Redux. Not sure what open source Tesla gives back but they have given their patents (https://www.tesla.com/blog/all-our-patent-are-belong-you). Nvidia gives away tons of open source as well (https://developer.nvidia.com/open-source)
> a) blindly 'updating' and deploying code without testing is a horrible idea
This is important risk factor each organization should be aware of. Though he wasn't trying to convey this, it was merely a byproduct.
> b) these trillion dollar corporations just take and take and never give, and boy do they whine and cry when the devs don't 'hold up their end of the deal' and keep turning out perfect, fully tested software for free
This was the intent, but anecdotally, Google, Amazon, Apple, Microsoft, Twitter, Meta, Stripe, Netflix, all contribute to open source.
> c) 'the source is open so anyone can look at it and therefore it's bug free' is and always has been a mentally retarded philosophy
Since the source is open, it must be bug free is a flawed conclusion. Again, not the lesson Marak was imparting.
> d) the software deployment process as it's currently practiced is horribly flawed
I don't think that is accurate.
> e) these people ought to count their blessings that the code is flawed in such an obvious and immediately detectable rather than subtle and devious and much more destructive way
I can't imagine a more destructive way that would result in the catastrophe one is expecting. Should Marak have been more malicious, having a myriad of well funded corporations targeting him would not be fun.
and last but not least:
> f) giving control over one's code to evil microsoft via github is an incredibly stupid idea, as such authority WILL be abused by the evil scumbags.
Another tradeoff organizations should understand.
It might be time we have a package marketplace like Steam that companies can subscribe to and independent developers can make some money via the marketplace.
- Don't rely on open source software. It's all FUD. Always use software from companys you can sue by a contract.
- If you need an an open source library to color your console output, pay them 6 figures per year.
SCNR
Also, AFAIK, Marak is not the original author; Is he also a freeloader for attempting to commercialise this code?
> teaching an important and principled lesson here
The history of this issue speaks differently to their intentions, but even so, there is a way to "teach lessons" and that's by doing something alarming but harmless. AFAIK Marak wanted to cause harm, and acted in a way to do so.
Pegging to a specific version limits exposure. Syncing these packages to your own on-prem/isolated environment limits it further. Deploying all changes to a test/staging environment where they're reviewed first limits it even more.
I mean yeah if your build process takes @latest of all your packages and then pushes it right into production, that opens you up to a lot of risk. It's also incredibly stupid for anything beyond a personal project (and probably even those).
This doesn't strike me as a weakness in package management, it strikes me as a weakness in doing package management wrong.
For example, when I npm install a package, it defaults to specifying a semver compatible version in package.json, rather than doing the secure thing and pinning a version.
But whether this default behaviour should change is not is also a security tradeoff. Pinning versions means that you will keep using an insecure version of a dependency until you update, whereas using a semver compatible version allows you to “automatically” pick up a fixed and compatible version. In practice however with lock files and local caches, the developer always needs to update for security patches anyway.
However, given the current NPM landscape (with packages having numerous small dependencies from a large variety of authors), going towards the former instead of the latter is definitely makes a lot more sense.
The minor security updates are solved well by periodically running security linters and scanners. There's even a recent GitHub feature for it. That will alert you that you need to update a package.
Note that if you have a package-lock.json (which you will by default), it will prevent any surprise updates even within the semver range specified. You have to manually run `npm update` to get the latest versions that match your semver. Personally I think this is the best middle-ground.
To make this work correctly, you need to do `yarn install --frozen-lock-file` or `npm ci`.
It’s absolutely _insane_ that this is the case. Gemfile.lock, Cargo.lock, and every other lock file format that I have used in packaging does this correctly.
One of the core issues of NPM style package management is package bloat means you absolutely can't review all release notes for every module in your tree. So you just trust the top level packages, and pray they would mention something if their dependencies change how they themselves work. Practically I rarely see anyone read release updates for even those top level packages, they just update everything and test then send it up to prod is very typical.
If you are cool with that, rad, but it's the pinnacle of the fast food tech ethos literring software right now. Everyone is moving so fast that you barely get to learn something properly or maintain it well enough before it's defunct and we are on to the next thing. I might have a slightly bias view of it, working mostly for agencies I see a lot of projects.
Let's not do victim blaming here.
This is ultimately the fault of the person deliberately updating their package to break other people's software.
The other is, to do anything at all of practical use in 98% of jobs, day 1 is installing a tonne of OS stuff.
It’s not practical to expect pretty much every dev to inspect 100% of that, even if that’s what they implicitly agree to do in the license.
If you have your package manager set up in a way that allows it to automatically upgrade/break your code, that's 100% on you.
Seriously, you expect anyone to audit all this? It's basically impossible for any solo dev / small org, and as I say, it's not even a big project. A vulnerability is like half a line, or sometimes a typo.
Clearly, very different proposal for a large org, but even then, no small task.
Do your job and make sure the code that's running is what you expect. There's no valid excuse not to.
Ultimately the 2017 Equifax data breach was the fault of the people who hacked into Equifax's website.
We need systems in place to defend against people doing malicious things, but yes ideally individual developers shouldn't be the ones tasked with reviewing all of their dependencies' code.
Operating System provided packages, for example, are generally reviewed by someone other than the author, which can lead to a more secure supply chain.
Rust's cargo-crev review system also seems like a possible solution the problem.
Which is why you schedule time each sprint/release to check your dependencies and upgrade them in a controlled fashion.
The same version number of a package should always link to the same version numbers of both its direct and nested dependencies. No?
NPM is highly optimized to make sharing code as easily as possible, but that comes at a heavy price.
Holding back updates is not a sane strategy either. You must review all the new patched versions of all dependencies daily or, failing that, do the work once to drop all the deps you can not afford to maintain reviews for.
Why do some JS devs import tiny packages to do simple things? I don’t feel like I’ve seen this behavior in Python. Is it because browsers are an awful environment?
So, people had incentive to write and use smaller packages.
Now, the situation has improved. If you use esmodules all the way, and only import what you need, then your bundler can remove unused modules from final build
That's kinda funny. These node tools I run into these days usually take forever to install, waste a lot of space due to creating their own package mirror and generally are prone to break because of dependencies.
There is usually nothing tiny about them, even thought they only have a few lines of code.
when we make a build for the browser we bundle all dependencies into fewer files that only contain the code that is used
I got tired of copying and pasting the same classes between projects. The worse part was I'd add new features to the newer projects and when I would have to go back to work on something from a year or two ago I'd have to spend time backporting all the new code. I also don't like how bloated a bunch of the "popular" packages are. Why do something in 40kB of JS when you can do it in 3kB. Smaller is faster which is important to me because one of my main selling points is that I build modern-looking marketing websites that load and render under 5 seconds on a slow 3G connection.
Why not create your own common library and publish it to a private repo? There's a lot of options between using a stranger's package and what you're describing.
Exactly what I thought. Fascinating how they can have missed this obvious solution.
Ironically, I'm working on a laravel package myself that i'm hoping to maintain (and turn into a viable side project) that's basically jetstream with SaaS components, and UI elements... (think ui component libraries + laravel jetstream + extra SaaS/ERP things like tenancy beyond just teams but..like Org which can have teams, projects, employees, and each user can belong to multiple orgs, teams, projects, and have attached profiles to each.
For a lot of the UI stuff, I've basically repacked MIT stuff for tailwindcss components, and laravel/livewire added some extra configurations and options, and made it so you don't need Jetstream, just this thing... so a lot of it is actually other's packaged code pulled into one package so, ideally there's one dependency that could even be easily forked and repurposed for a team's needs but cover a lot of boilerplate possibilities.
That hasn’t been true for 7 years now, it was changed after the left-pad incident and that article everyone keeps quoting is from 2016. Deleting a GitHub repo or a package does not remove it from npm as part of their policy.
Sure, but unless you carefully review the full diff of every package after every update, you wouldn't discover something slightly more subtle like
That kind of behavior would be practically impossible to code-review for lots of packages that rely on other dependencies.
Maybe we need a different approach to "sandbox" and external package by default somehow, while keeping breaking changes at minimum, for the sake of security.
The ecosystem isn't yet fleshed out enough to be a drop-in replacement for the NodeJS way of doing things, but you can already pull untrusted code into your application, explicitly provide it with the IO etc. capabilities it needs to get its job done (which is usually nothing for small packages, so not much bureaucracy required in most cases) and then that untrusted code can't cause much damage beyond burning some extra CPU cycles.
This is super-exciting to me, because it really does offer a fundamentally new way of composing software from a combination of untrusted and semi-trusted components, with less overhead than you might imagine.
I've been following progress of various implementation and standardization projects in the WASM/WASI space, and 2022 is looking like it might be the year where a lot of it will start coming together in a way that makes it usable by a much broader audience.
Graal is cool tech but it's not playing the same game Wasm is.
Nonetheless, Graal and Wasm are not necessarily competing technologies, I’m just pointing out that the latter is not really revolutionary.
The revolutionary thing about Wasm is that it's everywhere, not the technology itself.
Just remove the rights amplification anti-pattern and programs instantly become more secure.
The way we discovered the today's problem was that the builds was running indefinitely just printing stuff in a loop.
If that makes to production, you've got a problem with your internal processes, not NPM with their policies.
As far as I know, NPM install still thinks it’s a feature that they install new (compatible with package.json, but not with lockfile) versions.
- `npm install` should be renamed to `npm upgrade`
- `npm ci` should be renamed to `npm install`
"npm-crev" can't come soon enough...
https://web.crev.dev/rust-reviews/ https://github.com/crev-dev/cargo-crev
I'm personally a fan of using Debian/Ubuntu packages, because generally code goes through a human before it gets published. That human has already been trusted by the Debian or Ubuntu organization.
And while some packages have been distroized (eg. a lot of old perl packages, a lot of python packages, some java/node packages) I have no idea if any rust package is distro packaged separately. (Since rust is static linked there's no real reason to package source code. Maybe as source package. But crates.io is already immutable.)
I feel like the implicit trust makes even using popular packages such as react seem a bit sketchy. I’m betting react devs audit upstream packages, but I don’t know if any formal statements that they do. Multiply that by all the other common projects and you have a huge auditability issue.
I would think that the sheer popularity of the JavaScript (and therefore Node) ecosystems contributes partially to it - there's a massive industry out there about skilling new developers up in JavaScript, Node, and some front-end frameworks. But it definitely doesn't explain all of it.
- some sites favor security over UI/UX
- some organisations have the funding to review packages such as react
In the future I think security bureaucracy will prevent security conscious organisations from having nice new things. This happens in places like the military (who were known to use WinXP long after public EOL).
> are literally remote code exec vulns
with javascript/node, you can write 100 lines of code then pulling 100 modules, it's quite different and hard to assure its quality and safety over long period of time.
I heard rust also has a very small stdlib, that gave me concerns, but I don't code rust, I do hope though all languages can have a stdlib that is 20% the size covers 80% of normal needs.
It's the same with packages, it's FINE to have to redo a bit thousand separator logic, do you truly need a transitive dependency hell with ^1.1.1 in the package list that auto upgrade at random !!? I've had several cases where the whole company is all hands on deck because some dep somewhere moved up and all subsequent builds fail - what are people doing in node, we never had these issues in Java.
Something mentioned in this article caught my eye:
> While searching for Marak’s libraries, I found this npm-test-access library. This library seems to be used for what the name describes: to test access to NPM. Marak seems like a very capable software engineer, and it’s unclear to me why he’d need a package like this. So, this make me personally doubt a little bit if Marak is really behind all of this, or if maybe his account got compromised, or if something else it at play.
— [0] https://jworks.io/the-faker-js-saga-continues/
If you wanted to take over other people’s NPM packages by pushing a compromised update to one of your own widely used packages, the first thing it would do is check to see if the victim had access to publish to NPM.
Marek, who has just started publishing malicious updates to his own widely used packages, has just created a package to check for access to NPM.
I’d rather skip the character assassination based on hypothetical future actions please, and focus on what’s actually happened.
There are 20m+ weekly downloads of the colors package alone. He has what amounts to remote execution privileges to people using that package. When the subject of compromised packages comes up and he’s demonstrated that he’s willing to publish malicious updates, it’s completely fair to wonder what else he’s willing to do with that level of access to that many systems. It’s irresponsible not to consider what his packages can do to your systems.
No, it's speculation that Marek didn't do any of this, but that instead his account was hacked. You either responded to the wrong comment by mistake, or totally misunderstood the one you replied to.
The problem here is not packages but lack of stdlib and tendency of package writers to have shit ton of further dependencies.
But here we are, 10 years in, with nobody giving a damn about semantic versioning.
Life could be so much easier with an actual package manager that isn't just some git clone replacement.
Every time I see a client importing unsigned code with no evidence anyone they trust has reviewed it, I flag it as a supply chain attack vector in their audit and recommend mitigations.
Some roll their eyes, but I will continue to defend it is a serious issue almost every company has, particularly since I have exploited this multiple times to prove a point by buying a lapsed domain name that mirrors JS many companies import ;)
The problem you describe isn't Linux, it's Linux Distributions.
Where would you draw the line?
Source packages are available, and if the binaries don't match the code a distro would soon be outed a la "many eyes" thinking.
We have to trust some or none.
Get the top off that chip, see if the factory put an extra core in for the NSA (IME).
If you can't trust the team behind the distro, then sure, your supply chain is compromised, but it's significantly less likely for a single package developer to cause any damage, as all the big distros have rather extensive policy and procedures to prevent such things.
The crappy ones maybe. Proper distros build everything from source.
Linux code is always reviewed before deployment, goes through many eyeballs, people are careful about this. The same is not true of npm, or any of the other services (as this event clearly shows).
I'm talking about not just the kernel but all the various other things from libraries to servers to tools and everything in between.
Is there some tooling we can build?
Whatever the language or repository system reusing libraries like React, Requests, Apache commons, or lodash make sense after reviewing the pros and cons (functionality, security, size, performance etc). But blindly adding small repositories to your packages file without understanding the implications is only increasing the risk of trouble.
Node and npm for some reason seems to have encouraged this - remember leftpad.
Many people interpret it as "Free as in beer", not "Free as in speech", so expecting people to pay for it disqualifies it as FLOSS.
Only for people with the wrong expectations. Just because there's many of them doesn't make them right.
Great for corporates who can buy the support contract, but is also suspiciously similar to the "freemium" model were FLOSS devs are suddenly incentivised to make the FLOSS offering insecure in comparison to the paid licence.
In this case, Marak is his own bad-actor/saboteur, how would the support contract help? It would be far more likely to make free-users 2nd class users, and as such it might be better to simply keep the products managerially separate due to that conflict of interest.
And lets be honest here - when does something stop being a reasonable "maintenance fee", and start to become rent-seeking / extortion? I think if you want to get paid, you simply don't work with MIT/GPL, or fork to a different licence; Changing your mind halfway through isn't reasonable IMHO.
MIT basically means "anyone working on this codebase agrees to MIT terms for their code, and as such authorship isn't so important". If you change your mind, you broke your agreement. If suddenly your authorship matters, what about every other author who stuck to their MIT agreement?
Admittedly:
A) Both of these meta-repository tools are reactive rather than proactive: they flag bad versions rather than known good versions.
B) It doesn't take too many HN searches to find people don't trust `npm audit` or DependaBot either because both have provided a lot of false positives and false negatives over the years.
C) If someone does trust one or both, often the easiest course of action is to automate the acceptance of their recommendations and just blindly accept them leaving us about where we started and just blurring the lines between what is repository and what is "meta-repository". (Even the "Bot" in DependaBot's name implies this acceptance automation is its natural state, and the bot's primary "interface" is automated Pull Requests).
Also, no auto-update of packages.
Why is a program able to concoct a random string that conveys no authority, into a file handle that conveys monstrous authority potentially over an entire operating system, ie. file_open : string -> File.
That's just crazy if you think about it: a program that only has access to a string can amplify its own permissions into access to your passwords file. This anti-pattern is unfortunately quite pervasive, but it's a library design issue that can be tackled in most existing languages by using better object oriented-design: don't use primitive types, use more domain specific types, and don't expose stdlib functions whereby code can convert an object that conveys few permissions into one that conveys more permissions.
This means deeper parts of a program necessarily have fewer permissions, and the top-level/entry point typically has the most permissions. It makes maintenance and auditing easier to boot.
https://github.com/crev-dev/
But yes, I prefer to "vendor" my dependencies too, especially on large projects.
If you do not update you are vulnerable to piles of issues anyone can look up.
If you update blindly you may import new obvious supply chain attacks.
The solution is actually doing code review. If you can not afford to review 2000 dependencies then you can not afford 2000 dependencies. The extra effort to use a minimal framework and some cherry picked functions may be worth it for most orgs.
This offers no benefit in terms of security, over a package dependency locked at a specific version.
The end result is the same: the user ends up downloading the .php files, and testing them in deployment, but through composer instead of curl.
It doesn't contribute to security at all, it just makes it awkward for other people to use your code.
I would also assume that people are connecting your library to a package management system anyway, to overcome this unnecessary hurdle e.g. https://getcomposer.org/doc/05-repositories.md#loading-a-pac...
You can keep a dependency fixed at a particular version indefinitely. You can also point composer at a private vendored repository of the dependency if you don't trust the upstream server.
To the people who want to use my code, it is recommended prominently in multiple places that they not blindly trust the code and actually inspect it before using it. The friction in this process is intended.
The code I write is primarily for me. Other people can use it if they want to, and I hope it helps them, but I don't care much about how many chose to use it or not. If they do, they have to work with my preferred way of distributing code.
There have been times where third parties have included my code in their packages, but I'm explicitly not the package author in those cases, so it (the package) is not my responsibility.
There is nothing inherent in using packages that means you have to blindly trust the code, neither does providing a package mean you have to accept any more responsibility over providing a .php file (packages are just .php files with a few metadata files that allow them to be downloaded using composer rather than curl).
Fair enough if someone doesn't want to add metadata to allow their code to be downloaded by composer, but I disagree that that offers any security benefit.
Agreed, but packages are an additional layer of abstraction, and you and I both know that the vast, vast majority of devs will not "look under the hood".
Packages are often seen as a one-step plug-and-play solution. I don't want people to see my code that way. They should dive in and inspect it before using it (it is always written with this in mind - with extensive commenting and documentation).
> neither does providing a package mean you have to accept any more responsibility
Honestly, this is a personal thing for me. If people are using my code, I will feel responsible to some extent. IMO, the advantage of my method is that (at least a few) more people will test/audit my code as opposed to if it was available as a package. Which increases the likelihood of any possible bugs in the code getting caught.
> IMO, the advantage of my method is that (at least a few) more people will test/audit my code as opposed to if it was available as a package. Which increases the likelihood of any possible bugs in the code getting caught.
The person who unthinkingly installs a package will also unthinkingly include your script using 'require'.
The only thing that happens is anyone who is interested in auditing your code and uses composer is inconvenienced with busywork, that would otherwise be handled by composer, e.g. autoloading the library.
> Honestly, this is a personal thing for me. If people are using my code, I will feel responsible to some extent.
The point you made was that you would feel more responsibility for a package rather than a PHP file. There's no reason why this should be the case. Both methods result in your code being run by 3rd parties.
Yeah, everything I'm talking about is to make the latter a less likely occurance.
I'm going to leave this by saying that I think the idea that you can make developers more conscientious by increasing busywork, is false. All it achieves is creating more busywork. Unconscientious developers will do the busywork and not scrutinise the library anyway, conscientious developers will just have to do extra busywork.
A better solution would be to provide a composer metadata file and to publish each new release using a new major release number each time, which is arguably the proper way to signal to consumers of the library that each version needs careful scrutiny and testing, as major release numbers signal breaking changes.
The npm stories show that most people do this with npm though. This color thing shows many people will just install whatever without checking: manually or automatically.
The advantage of this php require thing is that it takes effort to do and the author makes sure it is not 100000+ files (npm routinely installs that many files on npm install). Package management is great; it works well with NuGet for instance. But those are a sane community; no one used leftpad and such, so the tree of source to audit is not so large, not counting MS, but then again, you are not auditing nodejs are you? Npm is worse than gems, nuget, whatever php has etc simply because the community is pretty broken in as much that everything has to be a package and, even though you can type the functionality faster than you can search for it (yeah yeah whine tests whine docs: for leftpad, nobody cares about those things; it's trivial functionality), people use those.
Now faker (don't know colors) is non trivial: question is, what makes this to happen here and not in, say, nuget popular packages? Is it still/again the community or something else...
I do not recommend being consistent with that position in other areas of your life otherwise you might quickly find yourself in a jungle, starving and naked. Given that relying on others for shelter, food, or clothing is clearly out of the question!
The actual code is downloaded from either a git or http server, not via attachments to the emails themselves.
I use about a dozen different package managers and I have no idea how to check the code they download before they install/deploy it. I often check the source on Github if I need to look something up, but I have no idea how I'd go about verifying that the code on Github is the same as whatever the package managers install.
More broadly, and I am sorry if I am wrong here, but what do you expect to glean from reading that code if you don’t bother reading the man page for your package manager?
Package managers just make it so convenient to use code without ever looking at it.
You can even experiment with the packages directly, by editing the files in vendor/.
Not really, if you pin your versions exactly and don't do auto-updates. This also means that you have to update your packages manually and inspect what the latest versions are doing -- which is good practive anyway. NPM packages can not be depublished any longer as far as I know.
You should still look over it and make sure it's not obviously malicious, but simply using forks or local repos of open source packages would probably save 98% of these kinds of headaches (with a 2% allowance for insecure/malicious open source code).
GitHub issues can work in theory but in practice, developers are often slow to respond i.e. GI is where problems go to die.
Reminder: this is why traditional Linux distributions exist.
As always it is a cost/benefit trade-off: what is the benefit to auditing every package vs. the cost of auditing every package?
atleast in this way we can secure ourselves from supply chain attacks
> get color and style in your node.js console
A picture is worth a thousand words → https://i.imgur.com/inxA7Pg.png
The library inserts ANSI escape sequences [1] between the text you want to colorize in order to, well, colorize it ¯\_(ツ)_/¯
Many people are obsessed with colors in the Terminal, and so, they reach out to libraries like this. They exist in every major programming language ecosystem, even though colorizing text is as simple as writing this \x1b[48;5;011<TEXT>\x1[0m . One of the disadvantages of this rudimentary colorization technique is that if you have short-term memory, you will quickly forget the meaning of these numbers, but you can solve that problem with constants, there is no reason to install a third-party library with potentially malicious code to add this type of functionality to a high-profile project like AWS-CDK [2].
I wish these libraries would support NO_COLOR [3] more consistently.
I have seen many “modern” CLI tools (the ones people like to build using Rust or Go) overuse colors with no option to disable them.
[1] https://stackoverflow.com/a/33206814
[2] https://github.com/aws/aws-cdk/pull/18324/files
[3] https://no-color.org
Laziness; and people’s infatuation for dependency trees, especially those in the Node.js & JavaScript ecosystems.
> even though colorizing text is as simple as writing this \x1b[48;5;011<TEXT>\x1[0m
JS devs will do anything to avoid this.
The problem is not in what the code does it’s a problem with the agreement for use.
What if this was addressed at the “platform” level, I’m thinking the package manager here, NPM.
If npm had paid plans that would essentially mop up larger corporations they could then auto-distribute funds Spotify style based on “number of listens”.
I’d personally want to see this work mainly as enterprise plans.
This seems like a pretty decent idea…
For programmers, you'd be correct. That only really be a replacement for patreons, tips, and donations, which would typically be a miniscule amount. It just redistribute it instead. (Your $x subscription just automatically gets allotted instead of manually allotted).
Expecting compensation for a gift is the error.
If you intend to keep it as the original gift, it will be called abandoned.
The software maintenance is also given out as a gift. That's a choice.
So he's perfectly entitled to choose not to do that any more, no?
This model where someone develops something for free and then those that benefit the most don't contribute back isn't sustainable. I don't know if the packages owner was conscious about it but this was a political act and hopefully the impact will be positive.
From where we are we have two options: (1) companies find a way to make open source financially rewarding; (2) companies use their own crap instead.
It's a choice to distribute software for "free" as in "free beer".
Is there a license that is like MIT but with special clauses for people making big bucket?
I don't think I can just put an extra clause on it that says something on the line of "if you are using this to earn more than X big macs by year then you have to pay me or be subject to a fine".
This radically changes things as it may void the liability clause and also make code less fungible. And there's the issue of fairness to contributors.
- Devs do open source and get rewarded
- Devs get hired to make crappy alternative software for companies
Yes, it's called paid software, but don't worry, it's going to be trendy again soon.
The days of free software contribution are almost over.
Devs want to be paid for their work. too many corporations made billions from open source projects while maintainers live in quasi poverty.
What is needed is a proper market place for paid open source software. Github isn't one.
CVE are the proper marketing. The perspective of having a package with a vulnerability and no-one to provide a fix is frightening to any software maintainer.
I think I could be extorted dozens of thousands for a single upgrade. Heck, when the electrician auditor says our office is not certified for 2022, I pay $700 for a professional to fix it. Software will be the same very soon.
Fittingly enough, all four solutions for this particular issue all goes back to Snyk, as seen in the bottom of the blog post. Seems like they are labeling this a DoS to justify being able to publish something in their database and blog.
Common sense.
If the maintainers wasn't acting maliciously, they could change this new version to count as a major release, and then it wouldn't be a DoS.
I think it most definitely falls into "malicious code" that certainly is not done in good faith, and if not handled properly by downstream users, can cause a lot of unexpected problematic failures.
Whatever labels are used to characterize this, whether to call it a vulnerability/DoS or not, are just a matter of arguing over semantic meaning.
If the author made the project GPL, someone else would create a similar library with a more permissive license, which would end up taking the market and making the original library irrelevant.
It's unclear if it's referring to current authoritarian turns in our western world, big corps using his software for free, or something else.
https://abc7ny.com/suspicious-package-queens-astoria-fire/64...
https://www.qgazette.com/articles/more-charges-possible-for-...
https://nypost.com/2020/09/16/resident-of-nyc-home-with-susp...
He might have been the unibomber in training.
Don't want to pile on, but dude clearly seems to be going through mental issues.
> When investigators entered Squires' apartment to look further, they found more bomb making items including potassium nitrate.
Magnesium powder, sulfur powder, copper powder, aluminum powder, hobby fuse and mixing cups were also discovered in the home.
"The chemicals separately are what they are, but taken together they can assemble an explosive device," Deputy Commissioner of Intelligence and Counterterrorism John Miller said. "There were books about military explosives, booby traps and other things...What we're looking at here is the totality of the circumstances that raised our concern to a level where we're going to need more investigation."
Does that sound to you like he wanted to make a candy rocket?
You can do chemistry all your want, but attempting to build a bomb, even of the attempt doesn't succeed, is illegal.
At the time of the article, the investigation was still ongoing. That's likely why they were no charges yet.
I will also say, that as a native New Yorker, doing this type of "kitchen chemistry" (if that was he was doing) is _extremely_ reckless in a dense residential neighborhood.
He was either was just a hobbyist who liked experimenting with explosives and he was fine with recklessly endangering an entire community.... or he was planning to commit a bombing.
A different article:
> On Thursday, law enforcement sources told News 4 the fire started because he had a box next to his stove that caught fire. He tossed it, trying to douse the flames, and it landed in his living room, which then also caught fire.
So he went to the hospital with severe burns on his hands. We don't know yet exactly what he was doing, but I don't think he deserves anybody's sympathy. That has its limits.
I suppose we should charge everyone who starts a fire while cooking with reckless endangerment too? I get that there are different standards of liberty in dense urban areas but I don't think this is beyond them. It's just cops and feds talking up their non-bust.
[0] https://news.ycombinator.com/item?id=29869547
He’s almost certainly going through major mental issues, along the lines of schizophrenia or something similar. He needs help.
as is usual with a lot of recent conspiracy theories, seems analogous to apophenia[0] to me, or something similar.
the non-conspiracy "fact" seems to be what most people here think about Swartz: that he killed himself after a overzealous prosecution. Nothing to do with Epstein or Swartz's role at Reddit.
[0] - https://en.wikipedia.org/wiki/Apophenia
The link seems to be “Swartz downloaded millions of scholarly articles using an MIT network, and Epstein/Maxwell donated money to MIT.” That seems to be about it? Not exactly a logical reason to conclude that Swartz was assassinated as part of an Epstein/Maxwell coverup.
Or both of those two.
Pinning all dependencies to this extent is extremely inconvenient. More inconvenient, arguably, than to deal with this shit once in a while...
And even if all of that works, you still run head first into the issue once you inevitably upgrade the dependencies.
Dependencies change. Dependencies of dependencies change. You can't not update any of your dependencies for too long or there will be other problems. Something is gotta give.
If anything, this is the reason you use pull-through proxies. Your proxy will hold the version you depend on, regardless of upstream drama. Keep your proxy backed up and you'll be able to use those dependencies until the end of time, or you finally decide to migrate to an alternative.
If your package system allows this switch to another one, like, right now.
NPM, Cargo, etc. don't allow this (they "unlist" versions, but they don't "remove" them, i.e. you can't search for them, but they are still there).
I'd say the likelihood is about 50% you have a NPM package in your dependencies right now that pulls some binary or whatever from a random S3 bucket during installation.
And that's among the reasons people have started to commit their node_modules folders.
It has the neat side-effect of making people take a closer look at all the crap their pulling in too.
NPM no longer allows this.
You do that. Your coworkers don't. And they'll complain to your boss if you try to make them.
If your boss doesn't take your side, at least you can say "I told you" when things go wrong.
And their PRs aren't merged until they do, because we'd have talked about it before hand and achieved consent around the idea that pinning dependencies is a practice our team will start doing, or some other specific practice that solves this problem.
Really? Have I led a sheltered life? I cannot rightly apprehend the state of mind that would see pinning deps as bad. It only helps you!
The biggest churn in package-lock.json files is from using different npm versions. It’s worth keeping them aligned within a dev team.
https://dpc.pw/cargo-crev-and-rust-2019-fearless-code-reuse
Most Linux distributions are using a single key for the signing of packages, and this might be an easier place to start changing over to a multiple signature model.
In this particular case is was a single jilted developer, but determined actor could easily attack someone with access to the keys to sign compromised packages, as per the obligatory xkcd on the matter https://xkcd.com/538/ .
Ideally you'd want a system which separates reputation from meatspace identity, so that well-trusted reviewers couldn't be easily targeted offline. Unfortunately that would require a lot of good opsec, and go against the financial incentive for someone to disclose their online identity as part of a salary negotiation, for example.
In essence, It seems to be the case of a developer getting screwed, being disillusioned, becoming political, making bombs?, attacking the ecosystem etc.
Many years ago, I recall another developer of popular NPM packages(Azer Koçulu) pulling a similar thing[0].
https://qz.com/646467/how-one-programmer-broke-the-internet-...
We followed each other on Twitter, I recall him being disillusioned with SV and angry to Wikipedia for some reason(I think he believed on some greater plan or agenda pushed by SV companies, including Wikipedia). I disagreed and got unfollowed and blocked. Later, if I recall correctly, he got married and was touring the world.
Now pin an older version and if you want fork it and develop it yourself. Whooptidoo.
Point is, maybe 10 people. And that’s if you like ramen.
Moderately skilled it engineers other of backgrounds and devs can make much more than $200k, just go check levels.fyi
To the parent comments point
> It's a bit wild that the sum total money spent on salaries for engineers handling potential problems stemming from this or defending against the possibility in the future could probably have covered paying the maintainer a living wage many times over.
The collective effort across numerous companies is much more than just $200k and you can bet your butt on that.
Even in America outside of the coasts and outside of FAANG, making $140-$150+ as a senior developer is very good (and compared to almost all other industries is absurd) - salary.com which doesn't just rely on self-reported info as levels does reports the median salary + bonus for senior software engineers as $120k
https://www.salary.com/tools/salary-calculator/senior-softwa...
Outside of the US, even in more expensive places in the EU, even the equivalent of $100k for a super senior lead architect would be Very Good - I don't know a single SWE in the midwest in the US - including senior embedded systems engineers working on medical devices, senior firmware devs working on networking equipment, or any web engineer that makes more than $175k and I know plenty of Very Good senior full-stack web devs that make $125-$150
$125k a year is still a top of the top salary in the US, so don't cry for them tho
That's why the number they used for a living wage is so low too.
Dumb comments saying software engineers make 100x a living wage (as if this would be a bad thing) are the flavor du jour. It’s hard not to respond in kind.
But thank you, for what it’s worth. I remember you from 2010. It was quite a time.
Individual contributors in large companies, especially, would want their companies to fund FOSS projects they use. But approval processes are generally extremely complicated and there's nothing to gain internally by doing it. And we're talking about money that these corporations spend each millisecond. They barely need approvals for many other activities costing 10x, 100x in other domains.
In theory, this was enforced by copyleft requiring derivative works to also be free software. In practice, companies use software with permissible licenses instead because then they can reap the benefits without any requirement to pay it forward.
> If you’re expecting to get paid for it, it’s not FOSS.
Being paid for your time has nothing to do with whether your source code is public or what freedoms users have when using your software. Conflating free software with volunteer labor is exactly what leads to situations like this one, where the author's business based on faker got copied wholesale by a competitor who simply ignored their attempts to reach out.
> Deciding you can’t maintain a project anymore is fine, but pulling it out from under the people that are using it is incredibly anti-FOSS.
The mechanisms that allow a rug-pull are entirely choices made by the users of the libraries for their own convenience; the author did everything needed for you to download a working copy and use it in perpetuity. It's your fault for choosing to rely on NPM, choosing to not cache your dependencies, and choosing not to pin your dependencies.
If you want to fix this, stop contributing to permissively-licensed software. If you have a change you want to make, make or find a GPL fork of it and contribute it to that instead.
- They're contributing while at work and work only allows permissive licenses - They're familiar with permissive libraries because of the previous point - Permissive licenses are perceived as simpler - They've been pushed away from the free software movement by the FSF/Stallman/Linus - They don't think copyleft is the right form of enforcement
The OP is also attempting to use a proven failure of a business model, and then throwing a tantrum when it fails. Sure he’s within his rights to do so, but he has no moral high ground here, and I don’t think he’s entitled to any sympathy for adopting a business model that everybody knows for sure doesn’t work.
> it stops being yours, but you benefit from having a huge number of people improve it for you
It stops being yours, but somehow everyone who works on it can say they're helping you. This isn't fair. You don't have to pay for it, but fixing and adding features to the software that you use to make a living can't be counted as charity work.
But what happened here is that free/open source software doesn't have a consistent stance on paying maintainers or contributors, and this author feels that it's unfair and (potentially in the midst of other personal issues, it seems?) took advantage of a problem with how the ecosystem pulls in dependencies to complain about it.
As a matter of practicality, commercial entities using a maintainers' work should donate to maintainers to incentive them to, well, at least not go rogue, or to be on their good side when they rogue. Companies pay their employees to incentive them to function in the interests of the company. While this isn't fool-proof (principal-agent problem), it lowers the odds of a pissed off employee having the will/self-righteous fury to pursue something more aggressive than resigning in a huff.
FOSS is licensing, not religion. Rug-pulling a project from people who are enjoying using it isn't "anti-FOSS" IMO. This may not apply to you, but for all the contrast that OSS people project between their pragmatism and Free Software people being insane religious zealots on a jihad against money, OSS advocates seem to imbue a lot of flaky new agey spiritism into what FOSS is or isn't.
Why should this one developer collect payment but not everyone else who contributed it?
Regardless, it’s ridiculous to give something away openly under a permissive license and then later get angry when people use it exactly as you license it.
Doing your best to live in a bad system does not invalidate the complaints you have about that system.
That's the goal. Or at least one goal. But you can't just press a button and do that.
Being in charge of and an expert on open source software can be a way get people to buy your labor, but it's much harder than it should be. Instead many companies will demand you work for free, because it's open source!
Also trying to do something good for the world shouldn't make it so hard to make money. The companies get value but don't want to pay even a pittance.
It's hard to get paid when you decide to give your work away. If only there was some way a person could enter into a contract in order to guarantee payment in exchange for their work. What a radical idea...
2. You shouldn't have to take the option that hurts everyone else just to get paid.
Exactly, no one said _only_ the lead maintainer should be compensated. _All_ of the labor, not just the labor that happens to have a day job that benefits from it, should be compensated. That includes non-coding labor like support or community management, too.
The maintainer wants to have their cake and eat it too - they likely believe in FOSS for moral reasons yet consider it immoral when companies take their software and use it freely under the terms offered.
If you want people to pay you for your work, don't give it away for free. If you give it away for free, don't have a temper tantrum if someone gets rich off of your work without compensating you, because those were the rules you chose to play under.
While it was seen as an unnecessary hurdle set up by us I hope it started some meaningful conversations in the teams and maybe even end up with them "reinventing" the wheel for the better.
I've seen it happen.*
EDIT: * While working in infosec, I'll add.
1. Robust build and deployment processes. Locked-down build servers, proxied/cached package registries, locked dependencies, automated dependency upgrades, tests, rollbacks, etc. Pretty much exactly what you need to mitigate unexpected breaking changes in dependencies, regardless of whether they're security risks or not.
2. Comprehensive dependency inventory. List of all your dependencies, where they're used, what vulnerabilities they're affected by, various other metadata, automated threat-hunting, manual review and annotation.
Trust but verify. No need for developers to fill out forms, wait on your (context-free) approval, resort to implementing worse versions of things themselves because they don't want to jump through hoops, etc.
stop trying to save a couple of bucks by reusing functionality that's not that hard to just develop in-house maybe?
Everyone decloud and only use the standard libraries compilers provide. What a wonderful world! Everyone is forced to do some system programming. Going to be pain in the beginning but then whoever really passionate about programming (not shipping products but programming) is going to be happy.
OK just joke :/ Although I do secretly wish to wake up one morning and find out we have to do things like in the early 90s.
As the article states, AWS CDK depends on colors, if I want to use AWS CDK, I have to use colors too. I don't get the choice to re-implement that myself unless I want to stop using the official CDK library
every time something like this happens, the reaction in the comments is the same: well you should test your dependencies. and yeah, i do that before release, but running an npm update on my dev branch and finding one of my dependencies has broken something is a bunch of work i could do without, and seems like work that is being duplicated by a ton of developers.
IT people for some reason tend not to be great at introspection which tells me this won't be fixed because this is more than just one asshole dude, this is a systemic issue with js and web development as a whole. This sort of thing would be really hard to accomplish in Linux for example because of the mentality they adopted early on when Linus had his first burn out moment, but the fact that even after leftpad nothing has changed really shows how systemically broken web dev is.
on a somewhat regular basis, as time allows, i run npm update to bring all my dependencies to the lastest version. then i test (including thorough manual testing / qa), commit, and release with updated dependencies. if some update is broken and i don't have time to fix it then yeah, i can roll back. but the point is to be on relatively up-to-date version of as much as possible, so if something is broken then it turns into a game of trying to figure out which library it was that broke things. i don't want to just not update any dependencies because something is broken.
A reasonable maintainer semvers their packages.
You shouldn't update to the latest version if you want to minimize the likelihood of issues. That's what stable releases are for
Worst case everyone is going to invent their own wheels, which is beneficiary to all lower-echelon programmers (but not so for managers/tech leads as they have responsibility to ship things) because I as one definitely want to invent as many wheels as possible.