I was trying to find Marak's motives for this, but I couldn't find much, does anyone know more?
On his Twitter Marak is claiming [1] that GitHub/NPM have suspended his account, which is an interesting move, but I guessing that that's the only tool available to prevent further "sabotage".
It seems to be partially that big corporations are using his library without payment – I can sympathise with that, although if you choose to release it under that license I'm not sure whether you can accuse the companies of doing anything particularly wrong.
But also seems to relate it to a pretty insane conspiracy theory that Ghislaine Maxwell was involved in the death of Aaron Swartz, almost entirely based on the debunked theory that an account on Reddit named "MaxwellHill" was hers.
He was arrested a while back for having bomb-making materials in his apartment. Based on that and some other public comments, he seems to be mentally ill.
> the debunked theory that an account on Reddit named "MaxwellHill" was hers.
I assume by "debunked" you are referring to the Vice article, which offered nothing beyond name-calling of the people investigating the theory, and ended with "u/MaxwellHill did not immediately respond to a request for comment sent through Reddit."[0]
An example of a relevant fact might be that vice.com was the most popular domain among all the links submitted to Reddit by the user MaxwellHill.[1]
It's also worth pointing out that another Reddit mod tried to debunk the theory by claiming that they knew MaxwellHill and that he was a Malaysian man, but this claim started to look suspicious when people found that MaxwellHill had mentioned he/she had "visited" Malaysia before (which is not something one typically says about one's home country).[2]
The point is there's been no strong evidence, so it's a huge leap to then decide this is a solid basis for "Aaron Swartz was killed because Ghislaine Maxwell was tight with the Reddit staff and he got wind of her child trafficking".
That's a pretty thorough debunking, thank you, and you're right that it's a huge leap to connect Aaron Swartz to this (which I've seen no evidence for). I do want to say, though, that it's a pity that the debunker didn't get better responses from believers on Twitter, as there are some reasonable counter-arguments.
Firstly, I'm not impressed by the (lack of) probability calculations. The article concedes that the Reddit user and Ghislaine are both born in December, which has, let's say, a 1 in 12 chance, and also says that there are "hundreds of thousands" of Maxwells in the world, so let's say 800,000 out of 8 billion, or 1 in 10,000. So the chance of these two details both coinciding is 1 in 120,000.
On the other hand, there are apparently 430 million active monthly users on Reddit, so we should expect there to be some December Maxwells in there, but considering the user's interests (US politics) and their use of British English spelling and phrases[0], I think the number of non-Ghislaine accounts with all these attributes would be very low.
So the fact that this Reddit user posts consistently for 14 years and then abruptly stops posting (publicly) right at the time of Ghislaine's arrest does seem statistically improbable. The debunker would have us believe that the account is still active sending private messages and posting in private subreddits (and no doubt has a girlfriend who lives in Canada), but if they're happy to have their activity disclosed like that, why wouldn't they also post publicly saying "I'm not Ghislaine you idiots!" or respond to the Vice journalist, for that matter?
Is it really that hard to imagine that influential people on Reddit would fake screenshots of post-arrest MaxwellHill activity to try to distance themselves from the actions of a notorious criminal? Do we just have to take these screenshots at face value, despite all the incentive to fake them, and the fact that a mod has already been caught out in a lie about whether MaxwellHill lived in or visited Malaysia?
The debunker also claims that someone trying to hide their identity (by falsely claiming to be male) wouldn't be so stupid as to use their actual name as part of their username, but obviously the motivation to hide what you are doing changes over time. When MaxwellHill was first registered, it probably seemed anonymous enough (there are "hundreds of thousands" of Maxwells in the world, after all), but over time, as the account became more influential (and Ghislaine got involved in more and more shady activities, and gave away more and more clues about her identity from her posts) she would develop a need to throw people off the trail by dropping some false biographic details.
I won't go into more details about the stylometry and contents of MaxwellHill's posts, as that is more subjective, but I will end by saying there is nothing contradictory about Ghislaine as a Redditor making negative comments about Trump (which is implied but not cited by the debunker) but Ghislaine in person bragging to someone about being a friend of Trump. It's possible that she hates Trump's current politics, but got on fine with him in a social setting 20 years ago, for example. In fact I suspect that one of her greatest talents is being able to seem friendly and trustworthy to powerful people whose actions she doesn't fully agree with.
If I voluntarily share my lunch with someone at work, I can't poison it even though they got free food. Even if someone at work is stealing my lunch, I can't poison it.
We have a right to basic safety even in situations we aren't paying for things. That extends to the digital world. You can't put malware in your open-source project without a giant disclaimer "THIS IS MALWARE. USE ONLY FOR RESEARCH PURPOSES" or something like that
>> . That extends to the digital world. You can't put malware in your open-source project without a giant disclaimer "THIS IS MALWARE. USE ONLY FOR RESEARCH PURPOSES" or something like that
My understanding is that he wasn't voluntarily sharing it. In November he pretty clearly told people to fork his code and move off his repository, and not to rely on his packages anymore. He used a graphic clearly warning of an impending "strike."
That doesn't make it not a dick move. But let's not pretend they still had permission to remote load directly from his repo. If I own a personal storage facility and I decide to demolish it, and I say months in advance for everyone to take their stuff out and store it somewhere else, and then I demolish it and people lose their property... the harmed individuals are not faultless.
Would the professional and responsible thing to do be to announce a cut-off date explicitly and make very clear that Bad Things were going to happen if you didn't stop relying on him? Yes, 100%, and I have a low opinion of him for not doing so.
However, while what he did was bad, it isn't quite as bad, in my opinion, as people are making it out to be.
You do not have a right to basic safety when you divest yourself of basic precautions any reasonable actor would have taken. A reasonable actor checks new code pulled down to ensure it actually works. You accept the risk integrating in a stranger's code without mirroring/audit.
The legal profession tends to take a more permissive stance when determining the limits of what's "reasonable", as can be seen from how the phrase "moron in a hurry" has become a term of art.
Nobody is forcing them to rewrite it or anything. Github canceled his account because knowingly posting malicious software is against the TOS, and it is not difficult to argue that putting infinite loops in the startup code of all your users is malicious. He still has the code on his computer to do whatever he pleases with. The rest of the world is just looking at it askew and calculating the odds something like this is going to happen again and what they should do about it. If you are going to be a dick to your users you should not be surprised that they will refuse to use your software afterwards.
In a way, it is kinda sad that this behavior drives him away from the financial security he so seemed to crave. "Yes I just caused your dev team to do hotfixes during the weekend and cost you XYZ dollars in downtime, can I have a senior dev position now plz" is not a very convincing line when interviewing.
I agree that it was a dick move from his side. Fork his project, or use an older version. I am not convinced that him breaking his code is something that ought to be punished.
GitHub gets the right to close their hand while Marak doesn't.
Deviant or not and corporation or not aren't at play. It'd be the same story if you had a video sharing site and didn't want to host my videos because I broke the "no comedy" rules by uploading a skit.
Exactly what? The government wasn't involved here so GitHub was allowed to decide what they wanted to host and the author was allowed to do what they want with the code?
> Well fuck them. It's his code, he should be able to do with it whatever he wants.
He can do whatever he wants with his codes. It didn't means that private companies have to accept that due to liabilities. He is using an private company platform to publish his code which could make the company liable for him. By removing or suspending his account, the liability will be minimized and shift it to the developer. If the private company keep it in their system and widely available for distribution, other companies that got hit by this malicious code could have a standing to sue the hosting private company for allowing the code to be published. Imagine thousand companies have a lawsuit against the hosting company. So the hosting private company don't want to be liable for this and shutting off the account is their best interest to keep the liability off on them.
Did Microsoft completely banned this developer from their other services? To my understanding, the developer account is suspended from GitHub. He is not banned from other MS products.
That is the difference. In Google case, Google completely ban on all of their services to those banned users. If they are banned in Gmail, then likely they are banned from Play Store, Workspace, Google Drive, etc (entire Google ecosystem). In this case, Microsoft didn't banned this developers from their other services, it only the developer's GitHub account is suspended without affecting other services that the developers are using.
I was about to point out the fact that all this guy's motivations seem to be mental illness YOLO and that it seems he recently lost his job. Kind of like the movie "falling down" but even more petty somehow
I never understood 9/11 truthers until someone explained it that it was more comforting to at least believe someone was pulling the strings in the background because at least there was a plan, even if it was sinister, compared to accepting a militarily inferior enemy half way across the world can upend everything about your life seemingly randomly.
Go pick up a copy of "Through our Enemies' Eyes." It's an account from a CIA agent who did extensive research on the events and circumstances that led up to 9/11 and the actors involved. His conclusion: It's not really about that at all; the answers are much more personal than that.
There was a plan, and it was by Osama Bin Laden. But he was naive. He wanted the American people to question themselves why something like this would happen to them, then find out about all the horrible things USA has done to other nations. It was never about hating freedom.
I don't understand how the former is more comforting. Military inferior enemy with one successful attack is preferable. It is safer the sinister plan inside.
I don't think Trump believes in conspiracies, generally speaking. I think he simply finds them useful for manipulating large groups of people. I could certainly be mistaken.
I hate this position. Imagine someone ranting about MKULTRA if the CIA never acknowledged it. Imagine it was just some papers on the internet with no providence. I like to think of it as a matter of diversity of though. We need the "crazy" people to explore the really unlikely bizarre scenarios that MIGHT be true that most mainstream people will never even humor. At the end of the day reality is stranger then fiction.
Everyone has some non-orthodox beliefs. I am sure I'm not exempt.
I'm not trying to describe someone who has a few odd beliefs, especially if they know they are a bit odd and can entertain good arguments.
I'm trying to describe where someone descends into the whole web of inter-related beliefs and accepts them with a religious determination, resisting even the suggestion they could be wrong.
Sure, 1% of conspiracy theories turn out to have some truth to them — but on the whole, if there’s no way of knowing in-advance which ones those are, I don’t know if “engage with all the conspiracy theories, spend time and energy attacking fictional problems and innocent people” is a better response than “reject all the conspiracy theories, spend time and energy on definitely-real problems, allow some dodgy organisations to get away with stuff”...
You also have the conspiracy theory that says that the most outlandish conspiracy theories are amplified much more than they naturally would be in an attempt to discredit all conspiracy theories. Which I do believe, myself.
The "crazy" people make it less likely that the bizarre scenarios that are real will actually be exposed, because the crazy people that do on rare occasions get one right usually are at the same time ranting about a dozen other things that actually are completely bonkers.
For instance if someone says they were abducted by aliens (from space), but they also say that COVID was designed by Gates and Fauci when they were roommates at Princeton to depopulate the world [1], and that Betty White was using Hollywood to promote the vast secret satanist pedophile network that traffics millions of US children each year, while her sister Barbara Bush did similar in the government, all at the behest of their father the satanist Aleister Crowley [2], and that in early January 2021 the military arrested Biden, Harris, Pelosi, Schumer, Democratic governors, Fauci, Gates, etc and took them to Gitmo where they were tried and executed, and restored Trump to the Presidency but are using robots and actors to make it look like all those dead people are still running things because they don't want to tip off the people running the underground adrenochrome harvesting operations until the children have been rescued [3], I'm not going to take their alien abduction seriously.
If on the other hand Neil deGrasse Tyson said he was abducted by aliens I'd take it seriously. I wouldn't necessarily believe he was actually abducted by aliens, but I'd believe there was a high probability that he had a good reason to believe he was, and that would be something worth investigating.
[1] Yes, there are people who believe that, and yes, they really say it was at Princeton even though neither of them went to Princeton.
[2] That started appearing on fringe sites shortly after Betty White's recent death.
Oops...I just realized alien abduction was a bad example, because it is not really in the same category as the other things I used for the crazy person's beliefs.
Those other things all involve belief in things they have been told but have not personally experienced. The alien abduction claim would be a claim that they have personally experienced alien abduction.
Observing a schizophrenic I knew made me realize people high in "intolerance of uncertainty" can probably discard reality checking to arrive at a firm conclusion. I googled that and found a psych paper claiming schizophrenics indeed tend toward high intolerance for uncertainty. It was a relatively new finding.
Astral Codex Ten posited that conspiracy theories are part of the Epistemic Minor Leagues [0]. In other words a way for people to flex the part of their brain that doing real research and discovery stimulates even if they don't feel they can belong to or contribute to "regular" intellectual activity like academic research.
He is mentally unstable. His motives are probably irrational. This is not the first time he reaches the news for strange criminal activities.
(Yes, it's his code. No, it isn't legal to maliciously add an infinite-loop to a library that you know is used by other companies. The license adds some liability protection, but it's not so simple.)
I'll leave it as an exercise for the reader to understand the difference between "I am not liable if I have a bug that ruins your production environment" and "I am not liable if I maliciously introduce a fatal bug knowingly into your production environment".
Most of those (malice, who introduced it to your environment, fatal bug) seem contestable, even if we grant for the purpose of argument that the as-is disclaimer does not cover all cases.
Any reasonable expert in the field will tell you you don't plug an auto-updating dependency into production. Marak wrote code. You, (the consumer), pulled, and deployed it without due diligence. That is entirely on you.
Not one person is obligated to keep your crap working except you. This has really outed all the people who really should know better.
The commit had a comment to the effect of being test / toy code not meant to be put into a release. I don't think a claim of randomly producing the snippet would be put forward in the hypothetical court case. Then there's the question of malice vs some other motive of expression in looping and printing some ASCII / zalgo art in your own terminal art lib.
Of course he did. Intent matters, and this was a reasonably foreseen consequence of the way the system is set up.
He knew how npm works and he knew the implication of adding that code is that hundreds of libraries and production systems would automatically upgrade and install it.
In fact, the whole point of what he did was to introduce the code into production environments.
If you put a bomb in a box and attach a button with a note that the button is provided as-is and author disclaims any liability, then leave it in public place and someone presses it, do you think you will not be found liable?
> No, it isn't legal to maliciously add an infinite-loop to a library that you know is used by other companies.
Could you cite an source for this? Because I got a impression that it "isn't legal" which mean it is not illegal based on your comment. I would assuming you are referring to USA Computer Fraud and Abuse Act?
"18 U.S.C. § 1030(a)(5)(A) knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;"
'without authorization' here is going to be tricky. author probably did have authorization to both github + npm? and didn't knowingly cause transmission to anywhere else? the rest of the steps were pull, not push.
If we're honest about the US justice system, this would be a subjective decision decided by non-technological lawyers, jurors, and judges. The purposeful malicious intent is working hard against his stance.
He did breach basic ethics and standards of professional conduct by his actions, for sure. I would lean against considering what he did illegal, but I think there is an argument to be made that it would be illegal under the CFAA.
It is on the person using code to do due diligence to ensure that any code pulled down in an update is good to utilize. You're seeing an implicit obligation where there has never been one.
In general, most just have the integrity, empathy and detachment to not do what he did, however, any programmer/developer who doesn't have a checklist item of "audit that code" before updates is committing an aggriegious breach of professional ethics;as this is the exact circumstance that everyone should be on the lookout for.
Everyone here assuming there is an obligation on Marak's part to continue to provide an interface in a non-molested form for their convenience are part of the problem. You should have mirrored, or paid the man.
>> The purposeful malicious intent is working hard against his stance.
OK, but should cloud providers similarly be held accountable for screwing their customers through negligent acts - to come full circle, like pulling these updates without doing any checks or QC?
Although it's a different area of law, product-defect liability attaches to all actors in the "stream of commerce" stretching end-to-end from the manufacturer to the retailer.
scotus in van buren (2020) let off a cop who was selling LPR searches for cash so in theory the days of aggressive interpretations are over? eff called it a 'victory for security researchers', though it's probably too soon to say whether it's that or just a victory for people selling LPR data
The precedent doesn't apply. The SCOTUS interpreted (and in effect, defined) that the "authorized access" in 18 U.S.C. § 1030(a)(2) can't be qualified and limited to less access. If I'm authorized to see usernames, and due to light hacking I can also see emails - I'm n̵o̵t̵ ̵a̵ ̵c̵r̵i̵m̵i̵n̵a̵l̵ maybe a criminal (EDITED). If I'm authorized to check license plates for some reasons, and despite employer policy I checked license plates for some other reasons - I'm not a criminal.
The issue we're discussing here is based on 18 U.S.C. § 1030(a)(5) (note the last digit) and "authorized access" is not mentioned there at all. This section deals with damage and not access.
hmm, not really my area. This coverage of van buren seems to show the court trying to make 'authorization' agree in meaning in different parts of (a)(2)?
This incident has nothing to do with (a)(2) as Marak didn't _access_ any system. The only sections violated are (a)(5) (_knowingly damaging_ a system) and, arguably, (a)(7) (extortion). (a)(7) is a lot harder to argue though as his extortion attempt doesn't have a named target or an explicit demand and is generally... lame.
Edit: Note that I'm seeing this the same as a virus, not the same as a data-extraction hack.
I agree with awinter-py, it will be tricky to use "without authorization" for this. The developer only published the code, that is up to other companies to verify the code and roll with it. If Company A installed the dependency, that could mean Company A "authorized" the code because they pulled the dependency to their system. So not sure how CFAA could protect this if the Company A proceed to download the code which in turn that the code are authorized. It is their responsibility to audit and verify the code before incorporating into their software.
Is it considered "authorized" if I knowingly visit a website but did not realize it would execute malicious JavaScript on my machine? Anyone who unknowingly installed this malicious package in their project is having that same problem.
Personally I think he did it to prove a point. IIRC he made a post last year where he said he would no longer be developing software for companies to use for free. This gripe is that he does all this work for free and companies then use his code to make money and do not contribute back to the ecosystem.
IMO this was not the most graceful way to make the point, but at the end of the day it's his project, his code. If you want the expectation of it always working, then pay him. Don't bitch and moan that the old man giving out free bread on the steps isn't here this morning.
I don't know in this case but in general a GPL fork must stay GPL and, AFAIUI, importing a GPL package in your code it's similar to linking to it, so if the code that uses your GPL package is published (on GH for example) that could be considered redistribution. Not sure about the legalities but it could create enough friction to keep companies not willing to contribute away.
I guess I'm thinking more in terms of faker, which I believe was a library for creating test data. Makes me wonder if licenses matter much for tools that are never intended to be included in the final distribution.
Internal changes aren't going to be detectable, so it feels like the best you can hope for is that they don't want the maintenance burden of a patch set on top of your project. At that point it's not much different than MIT.
I'm not judging the author here, but on one hand I understand the frustration, on the other hand the package probably owes its popularity in part to its very permissive license.
Anyway, a fork maintained by the companies that use the package would still be a better outcome than keep working for them for free (or remove the package entirely).
License is pretty good starting point for that. Or, simply making closed source only as overwhelming majority of programmers do.
The whole "contributing to open source" while effectively saying "I think open source should not exist and is rip of" makes no sense in world where literally nothing forces you to create open source.
Do you really need 40 kg of potassium nitrate for your hobbyist fireworks though? That amount can blow up a house.
Also if you absolutely can't live without the immature thrill of mixing explosives with your bare hands, at least don't do it in a residential neighborhood.
While it's not enough to generate a stable income, faker.js has received over $20k in donations through Open Collective. It's more than most other projects -- but I guess nowhere near what one could get if a big corporation wanted to sponsor the continued development of the project.
>> On his Twitter Marak is claiming [1] that GitHub/NPM have suspended his account
Interesting. Shouldn't they also block access to the repositories? Oh right, that would hurt their other users. This makes it clear who thinks they own code and who works for them for free.
> Essentially every open source software license points out that the code is made available with no warranty at all. Modern package managers need to be designed to expect and mitigate this risk.
It's almost like the package manager's job has become to protect users from their dependencies.
Until the opposite happens. Some big security hole is fixed in the dependency, npm gets the fixed version by default while Go is stuck in the tested unsecure version. Or Go mitigates the somehow?
Go doesn't mitigate that somehow. You get the code that you specify, not the code that someone else has decided is better.
In practice, for both npm and Go dependencies, you'll get a Dependabot PR that upgrades the dependency for you. Obviously that is Github-specific, so if you're on a different platform, you'll have to subscribe to security updates in some other way. I am guessing there are many services that you can subscribe to that do the same thing.
Some big security hole is introduced in one of thousands of dependencies, npm gets the insecure version on next npm install while Go is stuck in the tested secure version. How difficult is that to see? I'm not the one to believe in conspiracy theories but this is just nuts.
Protecting users from their dependencies should be the job of their language's standard library. With a good enough stdlib, the number of dependencies required is much lower.
NPM makes an interesting trade off - with the current scheme, security updates which improve the security posture of the application are accepted by default. Those are far more common than malicious updates. Would pinning dependencies lead to larger problems? Imagine if a log4j-like issue showed up tomorrow; most NPM-managed software would just require reinstallation to be fixed.
The way you do it is simple. Take ruby gems. In development you update as often as possible. When you see the lock file changed, you check you're happy with the upgrades and commit the lock file. When you don't want to keep the newer version, you change the gemfile (package.json) to point to a version you're happy with, ideally with a comment describing why you're pinning, and what you'd have to do to adopt the newer version. In CI/CD you exclusively use the lock file. This workflow is very difficult to emulate with npm/yarn.
Can you describe why this workflow is difficult? I'm probably missing or misunderstanding something, as we literally use this workflow at work, and we never had dependency issues with our npm projects...
Edit: npm has lock files and a way to install from the lock file (simples way being `npm ci`)
npm ci is relatively new, before that it was hard to get it to precisely respect the lock file. There is also an issue if your devs don't all decide to use either yarn or npm with the separate lockfiles.
I do agree that it's relatively new, but not _that_ new. I feel like it's been around for at least a couple of years (but my sense of time since the COVID pandemic started has been unreliable).
FWIW: Yarn has had lock files for a longer time. I know it's not technically npm, but they share the ecosystem.
Well I think it was introduced in npm 6, either way you slice it, 6 major versions is way to late for this basic functionality.
Yes but that is the issue with yarn: it has its separate lock file, which is why when I (used to) devops node apps, I required the devs all agree on either npm or yarn but not a mix.
The gap in this workflow is you have to go out of your way to get a diff. Never mind a diff of what's actually in the .gem. Glancing at changelogs on GitHub only reviews changes from good actors.
In practice most of us just update, often with live reload running, and move on.
We need mandatory peer review of updates before they're distributed in the first place.
Diffs are easily in the 10k if not 100k range. No company but proverbial FAANGs can cope with that. Exploits are hard to spot by design and may span commits and even major versions. Only a computer process can handle that scope.
I'm not saying you should at any other diff than that of the lockfile itself. From there figure out potential impact, run more complete tests, and other mitigations are available. In most famous instances (leftpad and this newsline), the problem would have been immediately obvious. Just because it's still possible to fall victim to sabotage doesn't mean you shouldn't be doing this.
So they would just have to be re-deployed? I find it insane you would put in production un-reviewed dependencies. Wouldn't it be better to update them explicitly and then redeploy? I don't see the benefit and I see huge downsides.
I'm not 100% on how npm handles the following but composer (PHP package manager) allows you to specifically lock a version for a particular del then when your ready to upgrade and test you can manually change that pinned version to get the next. Thus if Log4j happened, your project wouldn't automatically pull in the fix but you would know about it through media etc and then would go into your project and change the version to the one with the fix.
Npm has lock files, and the documentation tells you why they exist and when to use them (eg `npm ci` being the shortest/easiest way to avoid this incident).
> Those are far more common than malicious updates
Are they though? It seems to me that security updates that actually affect a given individual or company are few and far between.
Example: I use a library that both reads and writes a particular file format. In the past few years, it has literally had hundreds of potential vulnerabilities fixed in its parsing code. I only use it to write files, so most or all of its vulnerabilities are of no importance to me.
How often do log4j-level RCEs (or of equal severity) happen? And of those, how many occur in JS packages hosted on NPM?
In my opinion, when I'm using code from an untrusted source that neither I or anyone else has carefully reviewed, it is less risky to wait to patch until I am sure a change will improve my security posture versus YOLOing everything into production.
But there is no right answer to this problem. We are all unknowingly running vulnerable code, and will at some point likely will make an update that introduces more vulnerable code. With the current state of our industry, we will all be owned at some point. It becomes a blame game - "you didn't apply the security patches?!" versus "you deployed code you didn't review?!".
Does anyone have a cli command to check down the dependency tree for these packages? I'm seeing this issue on my company's app and trying to figure out which package(s) it stems from.
uhhh yes staying pinned to an old version forever solves some problems, but not other problems? article doesn't mention 'npm audit' and how there are cases where you want to encourage an upgrade
real long-term solve here is a code review community for widely-used public packages I suspect?
am not a huge blockchain fan but this is one thing that blockchain could conceivably do well, because reviews are public, need to be authenticated, exist as compact metadata that can fit on chain, and benefit from public reputation dynamics
A blockchain isn't needed for that. Authentication needs "crypto"graphy, but not "crypto"currency.
This wouldn't be a complete thread without someone mentioning Rust, so I'll do it. cargo-crev is a nice web-of-trust type code review system for Rust crates. https://github.com/crev-dev/cargo-crev
I swear I'm not normally this person but I think if 'web of trust' means a database that is replicated in parts across many different computers and uses cryptographic signing to authenticate messages, you're describing the thing known to the bros and the gen-Zs as a blockchain
Blockchain refers specifically to a linked-list-like data structure which utilizes cryptographic hashes at each node to store an authentication of the tail of the list on each head (node). If you have a similar structure using trees, it's a merkle tree. Replication + message signing does not imply either (necessarily).
The real long-term solution is for projects not to have hundreds of weird dependencies.
If dependencies are pinned until developers update them, you have massive security holes, because developers can't be trusted.
If you take the latest dependencies on every update, you have massive functional exposures (and security holes), because different developers can't be trusted.
If you have "community review", you create a new class of thankless work that nobody will want to do... except power-tripping control freaks who want to gatekeep over whatever their personal obsessions may be.
If users have to pick the versions, nothing will ever work, because users can't be trusted (and wouldn't want to do it anyway).
review is a compliance function and may be more likely to attract payment than writing OSS, paradoxically
also companies may devote in-house resources to do it
best case IMO is that normalizing paid review leads to normalizing paid bugfixes / feature requests. people need to start thinking about what monetized github looks like
This isn't like leftpad being deleted: he added an infinite-loop on purpose in a patch release to the package. This is a malicious attack. Only later did he delete packages.
No. It's a change he wanted to make to his code. Code is and has always been art. People have been consuming his code, and not keeping an eye on it to make sure it continues to mesh with their own work.
Intentionally adding code that has an infinite loop (the for loop literally uses "Infinity" as the target for the 'for' loop) sounds like an attack to me..
Indeed, it's common nowadays to label things (ideas, people, etc.) in order to frame them in a way that's convenient to the labeler and helps him advance his agenda. I think given the global situation, some people become more sensitive to this kind of tactic (which is often used), while others have shown just how susceptible they are to it.
The author of the software didn't attack anything. He just pushed some code into a place he had legitimate control of.
Some irresponsible (see what I did?) developers downloaded and executed this code without checking, and as a result their stuff broke.
If it’s his project, as far as I’m concerned he’s within his rights deciding to make it do something different to what it did before, even if that is malicious. There is precedent for this with Chrome addon devs selling their addons to malware companies on the quiet.
That said, it is an attack on his users and it’s a shitty thing to do. He’s likely ended his career as an open source developer, and likely a paid developer as well.
Yes I do strongly disagree with the wording (attack) here.
If publishing a package you control is considered an attack than the same could be said about the developer using the package or the admins deploying said package
Attack and rights are not exclusive concepts. I would venture to say that your comment is mostly nonsequitor.
It's an indirect attack against the lazy and complacent, at the very least.
How dare the developer do that to them?!
People hate it when you make more work for them and companies will actively fight back so the outrage is predictable. What's surprising is the lack of support for both user and developer agency. Some have gone so far as to say that users have some sort of ownership over someone else's licensed code they chose to blindly change (apply an update) by right of "community" because they used it when it did what they wanted.
just pin your code to specific versions, the package manager, build tools and linter can just give warnings and case study examples about why the warnings are relevant
And then in five years there's a log4j vuln and you've got to figure out how to upgrade a bunch of (potentially incompatible) versions to get a fix.
It's a different approach, with pros and cons. Personally I think the npm model has more downsides than other approaches, but it's not clearly always the wrong approach.
The author argues that version bounds should be treated as a maximum rather than a minimum, like Go does. e.g., if the latest version of colors is 1.0.3, and you have dependencies that request 1.0.1 and 1.0.2, you would end up with 1.0.2. The end result being, the exact resolved version will have been tested with at least one of your dependencies.
Let's say you have two dependencies each requesting colors:
colors@1.0.1-1.0.3
colors@1.0.1-1.0.4
With npm, you'll end up with 1.0.3 because it satisfies all constraints. OP wants to end up with 1.0.4 if at least one dependency tested with 1.0.4 (and reject 1.0.5). I don't know of a way to do this with npm today.
You might like the [1] "overrides" field in npm v8.3 although I would recommend using it with caution, changing your dependencies dependencies has unknown consequences... even a patch release can break everything...
You are reading this wrong. The OP is suggesting that if you have two dependencies that are requesting:
colors@^1.0.1
colors@^1.0.2
then npm should get you 1.0.2, instead of 1.0.4, because it's the "version as close as possible" to "the dependency version that the package was actually tested with".
OP is not suggesting that npm should ignore dependency constraints, just that the version that is picked is the closest to the tested version (among those that satisfy the constraints).
If you have a package that explicitly says it won't work with >1.0.3, installing 1.0.4 is silly.
> the standard way that this is done allows minor and point releases to be trusted.
I feel like this event (and previous ones) has taught me that one should NOT trust patch and minor version upgrades to work. Obviously we want them to, but I distinctly recall having "minor" patches that broke existing behavior in the past, and has bitten my team on multiple projects over the last several years. Pinned versions are a giant pain, but having builds suddenly stop working seems worse, because you can't plan ahead for the time to upgrade.
I've come to believe that pinned versions with an active dependency check is the way to go. A lot of the dependency checks/scans are build time rather than an "on going" approach.
If nothing else, that is a step in the direction of reproducible builds which are also in the Good Thing category.
This is likely going to be another maturing event for NPM and the community where they will need to decide how they want to move forward. The blind trust of a `^1.2.3` version specification is something that will likely be outgrown.
I still believe that one of the biggest problems that JavaScript libraries face is the transitive dependency explosion combined with the "always update" build policies and that in turn makes makes the issue of a suddenly untrustworthy developer more likely and more problematic.
Except that a shit-ton of developers will code and test with one version of a dependency, and never, ever, ever update it. If the dependency has a catastrophic security hole, that security hole will be pretty much permanent.
And what happens if project A pins projects B and C, which in turn pin DIFFERENT versions of project D? Is there any language or environment out there that can make that work?
Read it again. Any one dependency, or the root project, has the power to pull in the latest version. One laggard dependency does not stop that.
For your second question, yes, Rust handles that well. If you depend on ">=1.0" and ">=1.1", you end up with a single copy of 1.1. If you depend on "=1.0" and "=1.1", you end up with both copies of the library. Every crate uses the version it requested. You can argue whether that's good or bad, but at least it's principled. There's a lint if you dislike that behavior.
OK, so suppose I depend on anarchy, which wants shades >=1.0.1, and chaos, which wants shades >=1.0.2. The author of shades releases 1.0.3 because of a bad security hole in all prior versions. My project will still get 1.0.2, so it will still have the security hole. For that matter, it may ALSO break because anarchy is broken by a change made between shades 1.0.1 and 1.0.2... which is why the maintainer of anarchy hasn't updated their dependency.
On the whole, I think I'd prefer a solver that gave me 1.0.3 by default (but maybe would NOT give me 2.0.0 by default, depending on what the version numbers mean in this particular system). But the bottom line is that there is NO solver that can be SURE that what I eventually get will really work.
That's an interesting fact about Rust, and I didn't know it. On the whole, it sounds like it at least needs some serious tooling so you can make sure you're not dragging in a bunch of old versions that both bloat your code and open you to abuse. Can I ask for a warning if I'm getting two different versions linked into the same binary? If something depends on "=1.0", and the maintainer issues 1.1 with a flag that says "I really, really don't think think you should be using 1.0 any more", will that throw an error? And what happens if both versions get pulled in, but the package in question uses an external data file whose format changed between 1.0 and 1.1?
Oh, and another Rust question, if you don't mind. If I have both versions 1.0 and 1.1 of package X in my binary, and X defines a data type called foo, and one of my dependencies constructs a foo using X 1.0, is that value of a different type than a foo constructed by X 1.1? Or can version 1.0 foos wind their way through the code and up being processed by version 1.1 code?
> And what happens if both versions get pulled in, but the package in question uses an external data file whose format changed between 1.0 and 1.1?
If it uses something like `include!`, both copies will be compiled in (and maybe optimized later by the linker). If it's truly "external" like hosted on some website outside the package manager, then it just means the author broke their package. Maybe I misunderstood your question.
> one of my dependencies constructs a foo using X 1.0, is that value of a different type than a foo constructed by X 1.1?
I believe they are always different types. Cargo encourages but doesn't enforce semver, so anything can change between versions, including private fields or enum variants behind non_exhaustive, etc. So they're treated as different and you need to convert between them. Although this might only be true for major versions; I don't know off the top of my head.
To work around it you can convert the types at crate boundaries, or the package author can use the so-called "semver trick" [1]
By an "external data file", I mean that the package keeps a runtime database in a disk file or something, and will end up getting confused if two versions are reading and writing that file concurrently. The same would apply if the two versions had any way to end up sharing an in-memory data structure as well.
> is that value of a different type than a foo constructed by X 1.1
I encountered this case once, and they are considered to be different types (in my case the issue was that using v1.0::foo with v1.1::foo traits wouldn't compile), so either keep them separated or pick one. Error messages can be confusing though, if you don't suspect using two versions.
> For your second question, yes, Rust handles that well. If you depend on ">=1.0" and ">=1.1", you end up with a single copy of 1.1. If you depend on "=1.0" and "=1.1", you end up with both copies of the library. Every crate uses the version it requested.
IIRC rust only does that for MAJOR versions (1.1 and 2.0, or 0.2 and 0.3, since cargo consider releases before 1.0.0 to be major ones).
To clarify, if you depend on ">=1.0" and ">=1.1", you might also end up with a single copy of 1.2. Rust does not implement the "maximum version" scheme described above.
And what happens if project A pins projects B and C, which in turn pin DIFFERENT versions of project D? Is there any language or environment out there that can make that work?
Node and npm have always been able to do this. Once upon a time this was accomplished via very hierarchical node_modules directories. This caused some issues, so the directory was flattened, however the multi-version compatibility was always maintained.
I agree that this approach is much better for stability. The trade off is that a lot of systems will end up missing their needed security updates. Very few people are consciously balancing their type I and type II errors when considering upgrade strategies.
That means that if I depend on leftpad and leftpad depends on colors, and a new version of colors is released, the maintainer of leftpad has to be pestered about testing it with the new colors and doing a new release with absolutely no code change and only this (semver-compatible) dependency bump, otherwise no one using leftpad will be able to update their version of colors?
And the security of this new scheme depends entirely on the leftpad developer correctly assessing the security of the third-party colors package, possibly much bigger than his own?
The way Go's versioning works: no, the highest version wins, not the lowest. So anyone can force an upgrade by upgrading their minimum version. This includes your application's go.mod file, i.e. you can force updates of anything.
Which has other problems too, particularly where semver is not followed strictly, since it fairly often means that using an update of X might force an incompatible update of Y that you now have to go and fix. Go modules have no way to specify upper bounds to prevent or warn about this.
Just do it like Maven. You know why no-one talks much about dependency management on the JVM? Because everything works properly so it's all very boring.
I've used NPM shrinkwrap before back in versions 1-3 of NPM. It's a little confusing that the author of this post calls it "new", so I investigated.
Since NPM version ~2 (current is 8), you're allowed to publish a npm-shrinkwrap.json file[0] in your package. That's in contrast to a package-lock.json, which may not be included in a published package.
Back when I used shrinkwrap, it was pre-lockfiles and it was trying to accomplish something similar to what lockfiles accomplish today. (Lockfiles were added in v5 [1])
I dug up this old StackOverflow post to verify that the behavior of shrinkwrap hasn't changed[2] since many years ago.
That doesn't leave me very hopeful. If package maintainers aren't doing this already, and they've been able to for 6+ years, then it's unlikely they will in the near future.
(I'm currently investigating how to deal with bad deps with some Open Source tooling I'm building. Feel free to ping me if you have any thoughts to share)
It's because having an actual shrinkwrap file in a library introduces a huge number of conflicts. If library A has 20 transitive dependencies, and library B has the same 20 transitive dependencies, deduplication based on ranges can leave you with only 20 transitive dependencies. If both libraries have shrinkwrap files in them, you can end up with 40 transitive dependencies, even if those are nominally compatible with each other. When you consider that even with the current behavior projects end up with thousands of transitives, you're looking at adding literally thousands or 10s of thousands of duplicate dependencies to a project. That's not even considering that some duplicate dependencies will just break things if they show up multiple times in a project (older versions of react, for example).
Ah, that totally makes sense. I know that's what "peerDependencies" is used for by many projects. For example, `react-dom` has a peerDependency on `react`. This can either be pinned to an exact version (react-dom@1.1.1 must be used with react@1.1.1) or it can require a range (react-dom@1.1.1 may be used with react@^1.0.0 (any version 1.0.0 and up)).
That seems tricky with the shrinkwrap, as you describe above, because it might create additional versions everywhere instead of trying to collapse them. (If you have 3 versions of `react` in peerDependencies, but each package had a npm-shrinkwrap.json, then you'd end up with 3 different versions of `react` installed.)
Would the burden then fall on your package manager to resolve this, ie npm or yarn? If it sees 3 different shrinkwraps (1.1.1, 1.1.2, 1.1.3) but they all have compatible peerDependencies (>=1.1.1, >=1.1.2, >=1.1.3) it could then pick the version that is compatible with all of the peerDependencies but that also exists in a shrinkwrap (so 1.1.3).
That would protect you from this "colors" scenario of somebody publishing `1.1.4` with malicious changes because, unless somebody were to shrinkwrap it, then it wouldn't get picked up by default. (In contrast to the current semver, which likely would pick up 1.1.4 upon a fresh `npm install`).
The inverse side of this is highlighted by another comment on this post: That semver makes absorbing security updates an easier process in the event of a log4j-style vulnerability. There is always a tradeoff, for sure!
It is worth noting that this is not an impossible problem. I'm not a js expert, and it seems that js loves imports, but Nix solves exactly this problem on Linux. You have, for each end binary, a set of dependencies. If they are exactly the same, they can be shared among programs, otherwise you would have different versions of the dependency, all the way down, per program.
It can be bloaty, and you have to manually look at and test packages if you want to get everyone on the same set of dependencies, but you end up with reproducible environments.
This is probably the only sane way to proceed when our software has dependency chains more than a few levels deep. Establish mechanisms to try to prevent bloat, but otherwise make it possible upgrade independently and make it starkly apparent if the is duplication.
That is exactly how npm works. Except instead of one at least somewhat-aligned group of maintainers of a Linux distro trying to keep things under control, you have a much larger set of packages individually maintained (or abandoned) by totally independent people.
So practically nothing will ever exactly match, which leaves you right back at many slightly-different copies of hundreds or thousands of dependencies.
> Other package managers should take note too. Marak has done all of us a huge favor by highlighting the problems most package managers create with their policy of automatic adoption of new dependencies without the opportunity for gradual rollout or any kind of testing whatsoever.
(Emphasis added) - is this actually a widespread practice? That's certainly not how apt packages are handled...my impression was this is a problem unique to the js ecosystem.
I don't have an answer to your question, but I'm glad you've highlighted this part of the article. The idea that an attacker "has done all of us a huge favor" by attacking the free software community is so toxic that it needs to be called out, even if it was meant somewhat in jest.
If we don't reject this logic, then we'll get more attackers claiming "just a prank, bro!" and "social experiment!", like the University of Minnesota researchers carrying out human experimentation on kernel developers without their consent:
I'm not sure what this prank showed beyond what all the previous malicious NPM packages already showed, other than that developers of free software are unstable and can sometimes ruin your day for lolz.
Even if you accept the idea of vandalism being used for a positive purpose, a better form of protest would have been to make the package just print a message saying "This software has been abandoned by its author. Please pin your dependencies to known good versions." and then exit.
That would still have been annoying to the people having to do that unnecessary version pinning work, but would at least have preserved some shred of sympathy for the maintainer.
For this particular case obviously previous packages didn't show it clearly enough. And yes, if you give thousands of 3rd party devs (or anybody snatching their credentials) direct access to your build machines or production systems, you should absolutely expect some of them to be unstable in all kinds of ways.
Insider problem is hard enough to guard against when you know the people involved.
Checking in your dependencies with https://github.com/JamieMason/shrinkpack can help insulate you from these problems until you're ready to face them. I created this before left-pad and thankfully meant that we were unaffected.
A lot of developers, understandably, baulk at checking in dependencies, but there is a concrete benefit in being able to continue uninterrupted during outages.
They are used. A lot of the comments in this thread that are about NPM specifically are strawmen, since it has been standard to use lock files for years.
You could still have an issue if you need to update a dependency, for security or other reasons, since it could bring along a bunch of updated sub-dependencies of its own (and sub-sub-dependencies, and so on). But that problem is not unique to NPM and exists in any language or platform that includes package management.
Time we had a new FOSS licence that specifically forces commercial users to pay for it. Why should big greedy corps profit from using software that's free?
We already have them. If I'm wrong, then we wouldn't need them. Just change the license for future versions like SugarCRM did. They didn't break anything and now they have paying customers.
Maybe the JS language itself should vet and absorb some of these more common libraries. Printing pretty colors in a console could be a feature of the language itself.
There was a major push few years ago, but it died down. Basically performance benefits of built modules weren't worth it and with that standard library proposal also lost momentum.
> Maybe the JS language itself should vet and absorb some of these more common libraries. Printing pretty colors in a console could be a feature of the language itself.
Maybe Node.js should start shipping with a substancial standard library instead of making developers rely on NPM. Node.js can't even parse a multi-part request body without a third party library. So much for a HTTP server that doesn't even implement the HTTP spec...
rsc's post isn't (directly) about this colors package. It is marginally about npm and broadly about package managers.
rsc is stating: (paraphrasing) the current state of affairs in many package managers is not a good design. This is (yet another) reason why package managers should work differently then they often do by default.
It seems npm and GitHub do take action in cases like leftpad and colors.. and people don't like them (Microsoft) doing so unilaterally. Maybe part of the 0-day fix is go multiparty: allow weighted votes by dependents to take over the namespace to force a patch/reversion. You own your code etc, but the OSS distro service has its own community-owned rules, and you are free to run a competing package manager without them. By using community managed package managers, you signal your intent not to break your users & contributors, and give an explicit remediation mechanism for handling such trust chain violations. Instead of a hundred-message GH thread, we get a voted minor version bump same-day.
Though agreed with the sentiment of 'prefer stable' during install would be A+. 1am package scans was a cruddy way to start my vacation.
> Maybe part of the 0-day fix is go multiparty: allow weighted votes by dependents to take over the namespace to force a patch/reversion. You own your code etc, but the OSS distro service has its own community-owned rules, and you are free to run a competing package manager without them.
Not a good idea. You are pretty much asking to be review bomb. You would need very good moderation to be able to trust votes. That means npm spending a bunch of resources on hiring them, not sure if they are up for that (no, free mods aren't the solution).
Yep, so build in adversarial controls like thresholding. The status quo is already beyond what gh/npm are managing: they already need the mods they aren't hiring. Scanners are helping, but a big part of GH is community tools, so weird not to figure it out here too.
Those both have their own scaling issues, so not as clear of a step forward.
Ex: We pin & package lock our versions, but it's harder for libraries to (they should do ranges), including ones we release. Likewise, for our apps (non-libraries), updating pinned package locks is reasonable for our direct dependencies, nested dependencies are hard to really have confidence on. Colors and leftpad both exemplify this: both projects fail our standards for direct dependencies, so the concern is nested ones. Unlike apt, we don't want super stale versions of everything.
For stuff like security, defense in depth + minimal targeted mechanisms for targeted threat models are generally a win, and I think that applies here for mitigating rogue releases by rogue package owners. Shifts like bitcoin -- crypto mining, ransomware, and wallet theft -- have made stuff like package buyouts and sneaky patches a reality where we must 'assume breach' of bad releases, not just try to prevent.
Alternatively, instead of a futile attempt to reinvent the universe, services like NPM could stop pretending that dependencies are easy. They should be encouraging people to pin versions, keep track of updates, and avoid packages with poorly defined dependencies of their own.
> It seems npm and GitHub do take action in cases like leftpad and colors.. and people don't like them (Microsoft) doing so unilaterally.
A few people don't like this but I think that most people are quite happy not to have their projects compromised. I would not assume that the loudest voices are broadly representative, especially with Microsoft being a lighting rod for complaints.
> Maybe part of the 0-day fix is go multiparty: allow weighted votes by dependents to take over the namespace to force a patch/reversion.
I think there's a lot of promise in this approach, especially if it's all public and deliberative — e.g. freeze the attacker's account, remove the compromised package, etc. but then have a public voting period for a transfer. The other question I'd have is whether it'd make sense to have a restricted set of maintaining organizations — e.g. I'd be a lot more comfortable if, say, the new project got transferred to an organization like the Apache foundation than some random developer who could plausibly be preparing to quietly ship a crypto-miner or something.
I like what npm is doing in these cases and I think so do most developers.
Isn't this what people are always praising about linux package managers? That they modify the software in all kinds of ways and take things into their own hands, beyond the developers will?
For GitHub it's maybe not a great look. If there was content in the readme that they felt violated their ToS, fair enough, I guess. They aren't obliged to publish that.
Yes agreed.. except move from relying exclusively on just MS npm employees for shepherding OSS packages to the much larger JS community, and esp those tied to any specific project. MS is richer stewardship than when npm was a struggling VC funded company, but still a clearly struggling community moderation model, esp for OSS.
I'm skeptical of cryptocurrency but do like DAO style innovations (with no $ involved) because of this kind of issue. Many stepping stones to get there though and too many unknowns to go whole hog with an overly ambitious schemes. But seems inevitable to divest & decentralize from MS here. Something like community moderation has small components that can be iterated on for pushing to a community model vs current overwhelmed & overall inappropriate Microsoft moderation team model.
Sure, that is technically correct and if your only goal is to assign blame that's helpful to point out. But it does nothing to prevent this from happening again.
The point here is how we can make it easier for svgo and every other package to avoid the problem in the first place.
NOTE: this isn't an open source versus closed source thing. Linux has distributions which test included packages (to varying extents, I'm sure) and some of these are commercial operations. It's not impossible to have code whose verification you have paid for, to one extent or another, even with open source. (and hey, you can install malware with automatically updating closed-source see Solarwinds).
Is it? In retail, AFAIK, if a store sells you a defective product, they are liable (or at least partly liable). It doesn't matter that some other manufacture made the product. The point being, responsibility is shared.
You're responsible for every dependency you add to your project and that includes all sub-dependencies. Your users will sue you for not doing your due diligence. You may turn around and try to sue your suppliers but that doesn't absolve you of your responsibility.
> It's the responsibility of "svgo" to make sure it's direct dependencies are alright.
No, it's your responsibility to make sure your dependency tree is alright.
Your personal definition of 'alright' may be more easily satisfied if the packages you choose to depend on autonomously practice some level of responsibility towards their own dependencies. Choose wisely.
But there is no way to dictate your requirements to dependencies, or impose some kind of responsibility or demand some kind of warranty. You can accept what is offered, or not. In fact, if you are using svgo, even indirectly, you have agreed to this: https://github.com/svg/svgo/blob/main/LICENSE
>THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
So svgo doesn't have to care and you can't make them. Even if they do care, there's no guarantee they will meet your standards - and you still can't make them.
If you want someone to blame, find someone willing to sign a contract that says you can blame them.
Efforts within NPM and github to control this situation are simply the bare minimum of case-by-case disaster mitigation, in the interest of reputation alone. If you're using their infrastructure, you apparently find this acceptable.
Exactly. In this case, the package manager should use the version of colors that svgo asked for, not the one that appeared on the internet 5 minutes ago.
The problem is by default `npm install [package]` will put a "give me [package] that appeared on the internet 5 minutes ago" into your package.json file by default. Until `npm install` pins to a minimum version by default instead of a maximum version by default, this will keep happening. It's a mess. The benefit of "maximum by default" is that you pick up security fixes by default. So... pick your poison, I guess.
On the other side of the spectrum, if no one was installing 'latest', then no one would be testing latest, and we wouldn't catch these bugs until later. We would just all get surprised after 2 weeks (or whatever arbitrary delay) until the bad version became the blessed default version.
In other words.. If there isn't a trusted test pipeline then there's no benefit in delaying the latest version. Might as well just get latest.
In other contexts, the best ways to deal with trade-offs use randomness. Perhaps this sort of stunt would cause less damage if npm just randomly chose which versions to use. To avoid churn, the random choices could be stored in lockfiles. The point would be that not everyone is using the newest version of any particular package.
People starting from scratch would get latest, whereas people maintaining existing projects would be able to upgrade when they feel it’s safe to to so. Do you really want the people maintaining the nuclear reactor code being testers for latest?
It's damned if you do, damned if you don't for packages in the middle of your direct dependencies and a sub-sub dependency with an issue (be it security or tantrum based).
These middle deps could pin the exact version, but then when a security vuln is found and a patch issued, these libraries also need to update. This is like a traffic jam. If you're 6 hops from the vulnerable package you need to wait for not one, but six maintainers to push an update to npm before you can clear the security warning.
To get around this, middle packages list semver ranges. And then you have your occasional left-pad issue.
If I had to choose between those two ways to lose, I would use server ranges. The only way to win is to not play at all - have no dependencies.
as an example look at the log4j mess. "Everything" in the Java world needs log4j updates, but some dependecy's dependencies pull in some specific version.
There you need the way to say "get the new version from 5 minutes ago" without waiting for all levels of the dependency hierachy. Especially as there were a bunch of emergency releases in short sequence and younhabe tonmake sure youbget zhe latest one.
Dependencies are a mess. NIH can't be the solution either, though.
This isn't the problem being discussed: when something like log4j happens, everyone in the world needs to update. The difference is if one of your dependencies pin 2.14.0, you either have to wait for that project to also ship an update or do something more complicated whereas if they specify less than version 3 you can immediately ship that patch.
Something like log4j or many packages in the NPM world are so widely used that on any non-trivial project the odds approach certainty that multiple dependencies will specify versions. The more specific the pin, the more likely it is that you'll be unable to ship a security update without risk of a functional change or blocking incompatibility. You can mitigate that with good CI/CD infrastructure and lots of automated testing, but that means taking on more infrastructure expense.
What if the patch for the buggy `2.14.0` library was released as `3.0.0`? In semver MAJOR can be a superset of MINOR and PATCH, so this is a perfectly logical semver operation.
You still have the exact same problem you described with `^2.14.0`. Someone would have to manually update the package to get the security fix in 3.0.0.
Unless you're suggesting code should also automatically update major versions aswell?
Yes, if someone chooses not to follow semver correctly they can create problems but there are a somewhat unlimited number of ways in which an untrustworthy maintainer can do that. The difference is that following semver means it's easy to not do that since they can always ship 2.<latest + 1> with no changes other than the security fix.
> The difference is that following semver means it's easy to not do that since they can always ship 2.<latest + 1> with no changes other than the security fix.
If the line between bug and feature were clear. (Log4j worked 100% as specified btw. in regards to log4shell)
You don't have to wait till that lib updates. You update version yourself and overwrite it.
Second, log4j is bad examples. Libraries don't pin that at all and people report bug is they do. Libraries are supposed to depend on logging api in general and end project decides whether use log4j or slfj.
global for what? How much tracking of issues do I have to make? In an ideal world (which we can't have for multiple reasons) I get security fixes by default. (and no other breakage)
This is also saying you can't easily update from 1.2.3 to 1.2.4 without all of your dependencies which specify 1.2.3 also being updated without having an override mechanism. This isn't a simple problem where one option has no downsides.
Everybody else handles dependencies like npm, e.g. installs the latest versions that satisfy constraints. Some package managers have the ability to install from a lock file (e.g. Python's Poetry, Python's Pipenv, Rust's Cargo, npm) but will still grab the latest version when the lockfile doesn't exists (e.g. cloned a project with no lock file, or just added the dependency yourself), and have a command to update which updates every single package to the latest compatible versions.
You can argue that npm's commands are poorly named and guide users towards bad defaults, but saying that it works in a unique way is not true.
I guess my experience is limited. For example, maven dependencies usually specify the exact version required in the pom files. At least, on all the projects I've ever dealt with.
I’ve seen plenty of pom.xml files which are unversioned — not having a standard command to update means less disciplined developers say it’s too much work – but increasingly in other languages (and Gradle IIRC) there’s a distinction between the metadata file listing your direct dependencies and a lock file detailing exactly what was installed. The idea is that (using Node as an example) I’d say “I depend on the AWS SDK” in the main file, which changes infrequently when I add or remove direct dependencies, but my tool would use the lock file (npm-shrinkwrap) to record the exact versions of every package, preferably even by file hash, so you can have a highly repeatable build.
Separating the two is handy both due to frequent churn and to avoid transitive dependencies being kept unnecessarily — I’ve seen projects where libfoo stopped depending on libbar a while back but they were still installing it because someone had copy-pasted that block years before and was just incrementing the version.
> "Everything" in the Java world needs log4j updates, but some dependecy's dependencies pull in some specific version. There you need the way to say "get the new version from 5 minutes ago" without waiting for all levels of the dependency hierachy.
At least with Maven, that's extremely simple. Just add to your project a dependency on that "new version from 5 minutes ago" of log4j. Maven always prefers the version from a direct dependency. You don't have to wait for "all levels of the dependency hierarchy" at all.
The problem is that, AIUI, each NPM package installs its own, isolated set of dependencies. It has facilities to manage versions down in the dep tree, but it’s not as simple as other managers which ultimately install a single set of packages that satisfy all version constraints.
many java jars contain other jars withbthe dependency, so users don't have to run maven or something like thst, but jsut grab the jar. This is even worse. (there was no package manager early on, thus many java developers aren't taught that way, even decades after Maven was created ... and for end-user products it makes sense to bundle)
> should use the version of colors that svgo asked for
Another thing about setting exact dependencies is reducing duplication. Libraries are encouraged to use loose ranges, because if everyone pins exact versions for every dependency, then you could end up with 10 to 20 different copies of 'colors' in your tree, instead of having just 1 or 2 copies that work across the board.
Indeed and crying for the little developer is of no help, the little coffee owner is liable for everything that gets sold, the kitchen cleanness and the good state of the food being packaged.
The people posting that it's not their fault, that they can't possibly vet the code they ship, etc. are basically people who don't have any professional ethics.
Not only that, even if you excuse their lack of ethics, there's a basic competence issue: they have a basic failure to learn from history and the most basic part of this, which is that you shouldn't be pulling live from the internet.
Short term bean counting always sets you up for failure. The fact that this guy's packages were depended upon by so many projects and people were not actively supporting him with money shows how broken open source is. The projects with the most dependencies should be rooted out and collectively paid for or a new license should be made that excludes the use of OS software by corporations unless they pay a minimum amount based on their size. I'm sick of hearing about stories like this and it has discouraged me from releasing much of anything open source that any corporation could use without paying me. That's why small projects that I do release get GPL licenses and I don't put my best work out there for free.
I mean, the bare naked reality is that software, web software in particular, is still like... a hundred-billion dollar industry. Maybe more. As long as it's still profitable enough to deal with these supply chain attacks occasionally, and nobody legislates or regulates things, we'll keep lumbering along like this. I'm disappointed, and I don't like the current state of affairs, but we can't just expect anything to change in the absence of any kind of external pressure.
I'm very sympathetic to your arguments in this thread, but it seems like in practice doing it right in your eyes would mean either 1 - not using dependencies at all, or 2 - only using dependencies when you have read and understood every single line of code in both the immediate dependency and every sub-dependency in its tree. Neither is realistic, unfortunately, for building anything non-trivial without a huge team of programmers.
Perhaps you have another solution in mind? I think something along the lines of package-level content security policies with granular permissions has a lot of potential.
I did try (2) for awhile with some success. You basically vet everything you import.
This generally follows the same rules of supply chain validation expected _and common_ in other industries where people actually give a crap about what they are doing.
In the case of large projects, especially those with a solid track record of testing and security issue management, the vetting is organizational. You don't need to read every line of Kafka (though you should, as I did, actually look through it to see what you're getting into), as an example, at least today.
But for smaller projects, just like with smaller vendors, you need to go deeper. You do need to read the code. Then you look at their issue tracking and see if they have any concept at all of quality, sanity, security notices, what their maintainer attitude is, and so on. You definitely read the code. If you're sloppy, you at least read through a good sample of the code. Again, diligence - you have the obligation to your downstream to know what you are getting into.
This is no different than the vendor management you expect in pretty much every supply chain that you interact with. Food, cars, medicine, pharmaceuticals, building materials, heavy machinery, embedded software in medical devices, and so on. Software has gotten away with complete negligence for a long time and instead of addressing it, the common practice has gotten worse and worse. At some point, we are going to accidentally kill someone.
Then, on top of that, you lock it down. You mirror it, you pin it, you make an active plan to monitor the project to make sure you are aware if it is abandoned, hijacked, deleted, etc. You take responsibility for dealing with these possibilities. You mitigate immediate threat by not pulling random crap live when you build. You further make sure you partition your system such that code you have not classified as high quality has a reasonable blast radius and security boundaries.
But whatever, let's say you're a typical valley company that operates like a psychopath and you don't care about that. What companies should be worried about is liability. Yes, most of the security activities for most SaaS and other SW is laughable and mostly theater and checklists. But there is liability nonetheless. After all, you are about to represent it as high quality software and you are taking on the liability. No judge in the world is going to let you go with the "but it was this javascript we pulled in, not ours" excuse.
We built quite substantial systems using this approach, and it was important in the domain that we were working in to have done so. It is not without hassle and cost.
Really, what are we talking about here? It's _too expensive_ or _too difficult_ to have even the slightest quality verification of software? The industry is a complete joke if that is true.
This is part of why cars cost $30k apiece. If open source maintainers saw the same kind of revenue as car component suppliers, there'd be a lot more paid QA jobs.
Pretty sure they don't MRI every other batch of rust covered by paint received from yet another shitty frame manufacturer. Many PSU, GPU, etc manufacturers often ship first batch with hi-class components, and then "downgrade it a little" or "lose control of a dealer" when it sells nice.
Spot checks and manufacturing line quality monitoring absolutely do happen in the process you are describing, which is a hell of a lot more than the weak excuses for not even bothering to mirror the SW and look at diffs now and then accomplish.
I suppose they must. Because car is a hazard on road and may not end well if you don't put enough man-hours within design, documentation, testing etc.
Now, who's gonna pay you for writing that single functionality whole year? Sure, you tested, documented, did everything right, you may have almost no bugs, pretty code, decent test coverage, unit tests, integration tests, UI tests, performance tests, edge cases ironed out, top notch performance, UX no-one matches, one click deployment, static code analysis, linters, fuzzers, vulnerability scanner tools, monitoring, auto scaling... while you get there, your startup may be no more. Or maybe just a money sink: https://thedailywtf.com/articles/unseen-effort
Alright, I'm little off the track, but not every software project is comparable to automotive/air/rocket/med/... industry. Ofcourse you will put 10x+ more money/time/effort if human lives may be impacted with your commit.
To put that into context, let's appreciate SQLite - software that is thoroughly tested and your airplane shouldn't be afraid to run those. From the creator of SQLite:
> I’m going to write tests to bring SQLite up to the quality of 100% MCDC, and that took a year of 60 hour weeks. That was hard, hard work. I was putting in 12 hour days every single day. I was just getting so tired of this because with this sort of thing, it’s the old joke of, you get 95% of the functionality with the first 95% of your budget, and the last 5% on the second 95% of your budget. It’s kind of the same thing. It’s pretty easy to get up to 90 or 95% test coverage. Getting that last 5% is really, really hard and it took about a year for me to get there, but once we got to that point, we stopped getting bug reports from Android
The difference is that 1 person responsible for installing 1 valve can't disable every car sold in the last year if they decide they aren't being paid enough.
Unfortunately the way that this can be prevented would be to audit the packages before including them or having your own package management that you control.
There are tons of tiny libraries that have gotten themselves into the dependency chain of popular libraries, and any attempt to remove them is treated as a turf war.
I've tried to remove single-line dependencies only to have the PR rejected by the creator of the tiny library who happens to contribute to the parent library.
IMO the Javascript ecosystem really needs a decent standard library to remove the inordinate amount of power granted to these these tiny library squatters who wrote 5 lines of code and a package.json file 10 years ago.
What "inordinate amount of power" is this really though? If they're a contributor to the parent library then they can already put whatever they want into code that will get executed by many people; meanwhile having libraries properly broken out into smaller pieces is great for maintainability. Let's not throw the baby out with the bathwater.
The parent library in that situation was a popular library with a whole team of contributors who could reject malicious PRs. But a PR that just updates every dependency (including a malicious update to the tiny library) can easily go unnoticed.
And that was one situation. The mindset of the Javascript ecosystem is still to maximize code reuse, meaning even if the tiny library maintainer isn't a maintainer of the parent library, the parent library still frequently clings to their one-line dependencies when I've tried removing them. Thus granting the tiny library owner tons of power like the maintainer of "colors".
> I've tried to remove single-line dependencies only to have the PR rejected by the creator of the tiny library who happens to contribute to the parent library.
So fork it and throw out everything you don't like.
Reminds me of my time at a global bank. After many security incidents, they finally decided to step up their game, and implemented multiple security scans.
For any project using NPM, it was an absolute nightmare. Dependencies 3 or 4 levels deep would get flagged, and there wouldn't be a way to resolve them. The security teams didn't care, and multiple projects were left stranded. Since resolving those issues would mean having to contribute to open source, which we weren't allowed to do.
On the other hand, because we had specific security teams doing the scanning and assessing the results, you weren't allowed to question them, because they could just block your entire project. So developers were extremely unhappy with them, and what ended up happening is that developers just did what the security teams asked, without questioning. Which to me, let to more security concerns, because we were just pulling in libraries without knowing what had changed.
Sadly, the few devs who stopped relying on third party dependencies all left, because most of their work took longer (since they were implementing stuff that third parties usually do). Business teams took notice and questioned why everything was taking longer.
We used to use npm audit at my last job, and it’d regularly flag dependencies 20+ layers deep - usually with something like create-react-app or jest at the top, and some trivial obviously not actually a problem issue at the bottom.
The resolver tool we used couldn’t fix dependencies that deep even when there were fixed versions available. And since their internal processes didn’t use the same audit tooling, Facebook devs would often close issues requesting dependency upgrades to fix those not actually security issues. I think Dan Abramov wrote a post on overreacted about it at some point too. I’m not convinced any of the time we spent on those issues was well spent to be honest.
There's obviously very valid security concerns sometimes. We didn't use npm audit, but an external tool. And often it would highlight things like "use https", and when looking at which line, it was just a comment that referred to an external link.
Obviously a non-issue, but you still had to verify, and document everything. Best practice would be that we ran this scan on every commit, but the scan took about 15-20 minutes. It just ended up being something that was part of our release process, with the obvious danger of breaking our application right before release.
All in all, it stopped me from relying on third party modules, and tried to use as much vanilla code as possible.
I made a post yesterday where I advocated for this (with lockfiles) at https://maxleiter.com/blog/pin-dependencies but it was pointed out to me that pinning dependencies would cause duplicates in your rollup/webpack outputs. YMMV, as I haven't experienced that but haven't extensively looked into it
It's worth asking, I think, how much energy should be spent against guarding against an attack like this in the future. I think every organization's risk model is different.
In this case, the Net interpreted Marak's actions as damage and seems to have effectively routed around them. There was disruption, but we already have systems in place for flagging broken or malicious versions (`npm audit` being one example).
For people willing to have the system work 99% of the time with the occasional Marak-screw, we already have a working solution in what currently exists. For people who cannot tolerate that level of risk, you have to firewall your package sources and only allow use of vetted code dependencies in your software.
Open source relies, in the end, on trust. Marak broke trust and that's all. The question isn't "How do we prevent people breaking trust" (that's impossible; humans have free will) but instead "How do we mitigate damage / protect that which cannot be risked if someone chooses to break trust?"
714 comments
[ 2.4 ms ] story [ 342 ms ] threadOn his Twitter Marak is claiming [1] that GitHub/NPM have suspended his account, which is an interesting move, but I guessing that that's the only tool available to prevent further "sabotage".
[1] https://twitter.com/marak/status/1479200803948830724
But also seems to relate it to a pretty insane conspiracy theory that Ghislaine Maxwell was involved in the death of Aaron Swartz, almost entirely based on the debunked theory that an account on Reddit named "MaxwellHill" was hers.
Seems like he was just flailing wildly.
He doesn't charge for it.
I assume by "debunked" you are referring to the Vice article, which offered nothing beyond name-calling of the people investigating the theory, and ended with "u/MaxwellHill did not immediately respond to a request for comment sent through Reddit."[0]
An example of a relevant fact might be that vice.com was the most popular domain among all the links submitted to Reddit by the user MaxwellHill.[1]
It's also worth pointing out that another Reddit mod tried to debunk the theory by claiming that they knew MaxwellHill and that he was a Malaysian man, but this claim started to look suspicious when people found that MaxwellHill had mentioned he/she had "visited" Malaysia before (which is not something one typically says about one's home country).[2]
[0] https://www.vice.com/en/article/y3zbaj/incoherent-conspiracy...
[1] https://www.reddit.com/r/conspiracy/comments/hoqheb/vice_pub...
[2] https://rareddit.com/r/conspiracy/comments/hoopdf/why_is_vic...
The point is there's been no strong evidence, so it's a huge leap to then decide this is a solid basis for "Aaron Swartz was killed because Ghislaine Maxwell was tight with the Reddit staff and he got wind of her child trafficking".
Firstly, I'm not impressed by the (lack of) probability calculations. The article concedes that the Reddit user and Ghislaine are both born in December, which has, let's say, a 1 in 12 chance, and also says that there are "hundreds of thousands" of Maxwells in the world, so let's say 800,000 out of 8 billion, or 1 in 10,000. So the chance of these two details both coinciding is 1 in 120,000.
On the other hand, there are apparently 430 million active monthly users on Reddit, so we should expect there to be some December Maxwells in there, but considering the user's interests (US politics) and their use of British English spelling and phrases[0], I think the number of non-Ghislaine accounts with all these attributes would be very low.
So the fact that this Reddit user posts consistently for 14 years and then abruptly stops posting (publicly) right at the time of Ghislaine's arrest does seem statistically improbable. The debunker would have us believe that the account is still active sending private messages and posting in private subreddits (and no doubt has a girlfriend who lives in Canada), but if they're happy to have their activity disclosed like that, why wouldn't they also post publicly saying "I'm not Ghislaine you idiots!" or respond to the Vice journalist, for that matter?
Is it really that hard to imagine that influential people on Reddit would fake screenshots of post-arrest MaxwellHill activity to try to distance themselves from the actions of a notorious criminal? Do we just have to take these screenshots at face value, despite all the incentive to fake them, and the fact that a mod has already been caught out in a lie about whether MaxwellHill lived in or visited Malaysia?
The debunker also claims that someone trying to hide their identity (by falsely claiming to be male) wouldn't be so stupid as to use their actual name as part of their username, but obviously the motivation to hide what you are doing changes over time. When MaxwellHill was first registered, it probably seemed anonymous enough (there are "hundreds of thousands" of Maxwells in the world, after all), but over time, as the account became more influential (and Ghislaine got involved in more and more shady activities, and gave away more and more clues about her identity from her posts) she would develop a need to throw people off the trail by dropping some false biographic details.
I won't go into more details about the stylometry and contents of MaxwellHill's posts, as that is more subjective, but I will end by saying there is nothing contradictory about Ghislaine as a Redditor making negative comments about Trump (which is implied but not cited by the debunker) but Ghislaine in person bragging to someone about being a friend of Trump. It's possible that she hates Trump's current politics, but got on fine with him in a social setting 20 years ago, for example. In fact I suspect that one of her greatest talents is being able to seem friendly and trustworthy to powerful people whose actions she doesn't fully agree with.
[0] https://www.rareddit.com/r/conspiracy/comments/hnfx0r/not_co...
It certainly isn't "his code" anyway. Many people contributed to it with the good-faith belief that he would not use it as a Trojan horse.
We have a right to basic safety even in situations we aren't paying for things. That extends to the digital world. You can't put malware in your open-source project without a giant disclaimer "THIS IS MALWARE. USE ONLY FOR RESEARCH PURPOSES" or something like that
sourceforge?
GitHub Post: https://archive.fo/9fwGz
HN Discussion: https://news.ycombinator.com/item?id=25032105&p=2
That doesn't make it not a dick move. But let's not pretend they still had permission to remote load directly from his repo. If I own a personal storage facility and I decide to demolish it, and I say months in advance for everyone to take their stuff out and store it somewhere else, and then I demolish it and people lose their property... the harmed individuals are not faultless.
Would the professional and responsible thing to do be to announce a cut-off date explicitly and make very clear that Bad Things were going to happen if you didn't stop relying on him? Yes, 100%, and I have a low opinion of him for not doing so.
However, while what he did was bad, it isn't quite as bad, in my opinion, as people are making it out to be.
In a way, it is kinda sad that this behavior drives him away from the financial security he so seemed to crave. "Yes I just caused your dev team to do hotfixes during the weekend and cost you XYZ dollars in downtime, can I have a senior dev position now plz" is not a very convincing line when interviewing.
GitHub gets the right to close their hand while Marak doesn't.
https://www.eff.org/deeplinks/2012/10/privacy-ubuntu-1210-am...
He can do whatever he wants with his codes. It didn't means that private companies have to accept that due to liabilities. He is using an private company platform to publish his code which could make the company liable for him. By removing or suspending his account, the liability will be minimized and shift it to the developer. If the private company keep it in their system and widely available for distribution, other companies that got hit by this malicious code could have a standing to sue the hosting private company for allowing the code to be published. Imagine thousand companies have a lawsuit against the hosting company. So the hosting private company don't want to be liable for this and shutting off the account is their best interest to keep the liability off on them.
That is the difference. In Google case, Google completely ban on all of their services to those banned users. If they are banned in Gmail, then likely they are banned from Play Store, Workspace, Google Drive, etc (entire Google ecosystem). In this case, Microsoft didn't banned this developers from their other services, it only the developer's GitHub account is suspended without affecting other services that the developers are using.
I think it's a refuge for distressed people. Some belief that unconnected terrible events are somehow all controlled and "part of the plan".
I still don't get it and likely never will, but it at least aligns with anecdotal cases I've experienced of seemingly normal people going a bit off.
If you want a logical rebuttal, you have to make a logical argument, and not
> At least 70-80% of 'conspiracy theories' are true. Yes, even many of the 'crazy' ones.
plays along on the conspiracies as long it benefits him and shows him in good light
his psychopathy is a level beyond those of conspiracy believers
I'm not trying to describe someone who has a few odd beliefs, especially if they know they are a bit odd and can entertain good arguments.
I'm trying to describe where someone descends into the whole web of inter-related beliefs and accepts them with a religious determination, resisting even the suggestion they could be wrong.
For instance if someone says they were abducted by aliens (from space), but they also say that COVID was designed by Gates and Fauci when they were roommates at Princeton to depopulate the world [1], and that Betty White was using Hollywood to promote the vast secret satanist pedophile network that traffics millions of US children each year, while her sister Barbara Bush did similar in the government, all at the behest of their father the satanist Aleister Crowley [2], and that in early January 2021 the military arrested Biden, Harris, Pelosi, Schumer, Democratic governors, Fauci, Gates, etc and took them to Gitmo where they were tried and executed, and restored Trump to the Presidency but are using robots and actors to make it look like all those dead people are still running things because they don't want to tip off the people running the underground adrenochrome harvesting operations until the children have been rescued [3], I'm not going to take their alien abduction seriously.
If on the other hand Neil deGrasse Tyson said he was abducted by aliens I'd take it seriously. I wouldn't necessarily believe he was actually abducted by aliens, but I'd believe there was a high probability that he had a good reason to believe he was, and that would be something worth investigating.
[1] Yes, there are people who believe that, and yes, they really say it was at Princeton even though neither of them went to Princeton.
[2] That started appearing on fringe sites shortly after Betty White's recent death.
[3] Another real conspiracy theory.
Those other things all involve belief in things they have been told but have not personally experienced. The alien abduction claim would be a claim that they have personally experienced alien abduction.
[0] https://astralcodexten.substack.com/p/epistemic-minor-league...
(Yes, it's his code. No, it isn't legal to maliciously add an infinite-loop to a library that you know is used by other companies. The license adds some liability protection, but it's not so simple.)
Um, what?
The commit is here as far as i know, not deleted: https://github.com/Marak/colors.js/commit/074a0f8ed0c31c35d1...
Not one person is obligated to keep your crap working except you. This has really outed all the people who really should know better.
For fucks sake, people need to pin dependencies to a known good version at the very least.
He knew how npm works and he knew the implication of adding that code is that hundreds of libraries and production systems would automatically upgrade and install it.
In fact, the whole point of what he did was to introduce the code into production environments.
<no hands raised>
How can we claim he did anything to production if no one will admit they're dumb enough to push this latest version without testing it?
tbis ismt a bomb in a box its his project car you copied without any warranty or.gurantee of stability.
That's different to criminal liability.
Could you cite an source for this? Because I got a impression that it "isn't legal" which mean it is not illegal based on your comment. I would assuming you are referring to USA Computer Fraud and Abuse Act?
It is on the person using code to do due diligence to ensure that any code pulled down in an update is good to utilize. You're seeing an implicit obligation where there has never been one.
In general, most just have the integrity, empathy and detachment to not do what he did, however, any programmer/developer who doesn't have a checklist item of "audit that code" before updates is committing an aggriegious breach of professional ethics;as this is the exact circumstance that everyone should be on the lookout for.
Everyone here assuming there is an obligation on Marak's part to continue to provide an interface in a non-molested form for their convenience are part of the problem. You should have mirrored, or paid the man.
OK, but should cloud providers similarly be held accountable for screwing their customers through negligent acts - to come full circle, like pulling these updates without doing any checks or QC?
The issue we're discussing here is based on 18 U.S.C. § 1030(a)(5) (note the last digit) and "authorized access" is not mentioned there at all. This section deals with damage and not access.
https://www.natlawreview.com/article/supreme-court-ends-long...
Edit: Note that I'm seeing this the same as a virus, not the same as a data-extraction hack.
IMO this was not the most graceful way to make the point, but at the end of the day it's his project, his code. If you want the expectation of it always working, then pay him. Don't bitch and moan that the old man giving out free bread on the steps isn't here this morning.
I understand the sentiment, but why chose MIT license then?
GPL would be the right choice if one wants to stop companies profiting from free work without giving anything back.
Internal changes aren't going to be detectable, so it feels like the best you can hope for is that they don't want the maintenance burden of a patch set on top of your project. At that point it's not much different than MIT.
That said, AGPL would scare off most companies :p
I'm not judging the author here, but on one hand I understand the frustration, on the other hand the package probably owes its popularity in part to its very permissive license.
Anyway, a fork maintained by the companies that use the package would still be a better outcome than keep working for them for free (or remove the package entirely).
The whole "contributing to open source" while effectively saying "I think open source should not exist and is rip of" makes no sense in world where literally nothing forces you to create open source.
http://web.archive.org/web/20210704022108/https://github.com...
Also he seems to have somewhat of a Guerilla mindset, based on this:
https://nypost.com/2020/09/16/resident-of-nyc-home-with-susp...
I see "charged," not "convicted" in that article.
Real world: Innocent until proven guilty.
Internet: Always guilty, because justice doesn't scale.
Also if you absolutely can't live without the immature thrill of mixing explosives with your bare hands, at least don't do it in a residential neighborhood.
While it's not enough to generate a stable income, faker.js has received over $20k in donations through Open Collective. It's more than most other projects -- but I guess nowhere near what one could get if a big corporation wanted to sponsor the continued development of the project.
https://opencollective.com/fakerjs
Interesting. Shouldn't they also block access to the repositories? Oh right, that would hurt their other users. This makes it clear who thinks they own code and who works for them for free.
Seems like he had series of odd statements around the same time: liberty, Aaron Schwarts and some such.
Overall it sounded to me more like breakdown then anything else.
https://abc7ny.com/suspicious-package-queens-astoria-fire/64...
It's almost like the package manager's job has become to protect users from their dependencies.
Go modules did different then npm, and I argue Go modules did it correctly.
In practice, for both npm and Go dependencies, you'll get a Dependabot PR that upgrades the dependency for you. Obviously that is Github-specific, so if you're on a different platform, you'll have to subscribe to security updates in some other way. I am guessing there are many services that you can subscribe to that do the same thing.
I'm not sure why you think deploying untested updates is a good idea?
Edit: npm has lock files and a way to install from the lock file (simples way being `npm ci`)
FWIW: Yarn has had lock files for a longer time. I know it's not technically npm, but they share the ecosystem.
Yes but that is the issue with yarn: it has its separate lock file, which is why when I (used to) devops node apps, I required the devs all agree on either npm or yarn but not a mix.
In practice most of us just update, often with live reload running, and move on.
We need mandatory peer review of updates before they're distributed in the first place.
Are they though? It seems to me that security updates that actually affect a given individual or company are few and far between.
Example: I use a library that both reads and writes a particular file format. In the past few years, it has literally had hundreds of potential vulnerabilities fixed in its parsing code. I only use it to write files, so most or all of its vulnerabilities are of no importance to me.
How often do log4j-level RCEs (or of equal severity) happen? And of those, how many occur in JS packages hosted on NPM?
In my opinion, when I'm using code from an untrusted source that neither I or anyone else has carefully reviewed, it is less risky to wait to patch until I am sure a change will improve my security posture versus YOLOing everything into production.
But there is no right answer to this problem. We are all unknowingly running vulnerable code, and will at some point likely will make an update that introduces more vulnerable code. With the current state of our industry, we will all be owned at some point. It becomes a blame game - "you didn't apply the security patches?!" versus "you deployed code you didn't review?!".
real long-term solve here is a code review community for widely-used public packages I suspect?
am not a huge blockchain fan but this is one thing that blockchain could conceivably do well, because reviews are public, need to be authenticated, exist as compact metadata that can fit on chain, and benefit from public reputation dynamics
This wouldn't be a complete thread without someone mentioning Rust, so I'll do it. cargo-crev is a nice web-of-trust type code review system for Rust crates. https://github.com/crev-dev/cargo-crev
If dependencies are pinned until developers update them, you have massive security holes, because developers can't be trusted.
If you take the latest dependencies on every update, you have massive functional exposures (and security holes), because different developers can't be trusted.
If you have "community review", you create a new class of thankless work that nobody will want to do... except power-tripping control freaks who want to gatekeep over whatever their personal obsessions may be.
If users have to pick the versions, nothing will ever work, because users can't be trusted (and wouldn't want to do it anyway).
also companies may devote in-house resources to do it
best case IMO is that normalizing paid review leads to normalizing paid bugfixes / feature requests. people need to start thinking about what monetized github looks like
You think anyone will pay anything more than lip service to QA?
That's a good one.
The author of the software didn't attack anything. He just pushed some code into a place he had legitimate control of.
Some irresponsible (see what I did?) developers downloaded and executed this code without checking, and as a result their stuff broke.
That said, it is an attack on his users and it’s a shitty thing to do. He’s likely ended his career as an open source developer, and likely a paid developer as well.
If publishing a package you control is considered an attack than the same could be said about the developer using the package or the admins deploying said package
It's an indirect attack against the lazy and complacent, at the very least. How dare the developer do that to them?!
People hate it when you make more work for them and companies will actively fight back so the outrage is predictable. What's surprising is the lack of support for both user and developer agency. Some have gone so far as to say that users have some sort of ownership over someone else's licensed code they chose to blindly change (apply an update) by right of "community" because they used it when it did what they wanted.
It's a different approach, with pros and cons. Personally I think the npm model has more downsides than other approaches, but it's not clearly always the wrong approach.
The author argues that version bounds should be treated as a maximum rather than a minimum, like Go does. e.g., if the latest version of colors is 1.0.3, and you have dependencies that request 1.0.1 and 1.0.2, you would end up with 1.0.2. The end result being, the exact resolved version will have been tested with at least one of your dependencies.
I must admit I like the idea.
https://stackoverflow.com/a/22345808 - and especially the comment.
You will find a lot of `^1.2.3` in version specifications which means everything from `1.2.3` up to (but not including) `2.0.0` is allowed.
Specifying `1.0.1 - 1.0.3` is allowed too and would meet the desired functionality - but that isn't the culture of JavaScript developers.
The version range is allowed in other dependency management systems (e.g. Maven - https://www.mojohaus.org/versions-maven-plugin/examples/reso... ) but rarely do I see it used - most often its pinned to a specific known good version.
colors@1.0.1-1.0.3
colors@1.0.1-1.0.4
With npm, you'll end up with 1.0.3 because it satisfies all constraints. OP wants to end up with 1.0.4 if at least one dependency tested with 1.0.4 (and reject 1.0.5). I don't know of a way to do this with npm today.
[1] https://docs.npmjs.com/cli/v8/configuring-npm/package-json#o...
OP is not suggesting that npm should ignore dependency constraints, just that the version that is picked is the closest to the tested version (among those that satisfy the constraints).
If you have a package that explicitly says it won't work with >1.0.3, installing 1.0.4 is silly.
I feel like this event (and previous ones) has taught me that one should NOT trust patch and minor version upgrades to work. Obviously we want them to, but I distinctly recall having "minor" patches that broke existing behavior in the past, and has bitten my team on multiple projects over the last several years. Pinned versions are a giant pain, but having builds suddenly stop working seems worse, because you can't plan ahead for the time to upgrade.
If nothing else, that is a step in the direction of reproducible builds which are also in the Good Thing category.
This is likely going to be another maturing event for NPM and the community where they will need to decide how they want to move forward. The blind trust of a `^1.2.3` version specification is something that will likely be outgrown.
I still believe that one of the biggest problems that JavaScript libraries face is the transitive dependency explosion combined with the "always update" build policies and that in turn makes makes the issue of a suddenly untrustworthy developer more likely and more problematic.
And what happens if project A pins projects B and C, which in turn pin DIFFERENT versions of project D? Is there any language or environment out there that can make that work?
For your second question, yes, Rust handles that well. If you depend on ">=1.0" and ">=1.1", you end up with a single copy of 1.1. If you depend on "=1.0" and "=1.1", you end up with both copies of the library. Every crate uses the version it requested. You can argue whether that's good or bad, but at least it's principled. There's a lint if you dislike that behavior.
https://rust-lang.github.io/rust-clippy/master/#multiple_cra...
On the whole, I think I'd prefer a solver that gave me 1.0.3 by default (but maybe would NOT give me 2.0.0 by default, depending on what the version numbers mean in this particular system). But the bottom line is that there is NO solver that can be SURE that what I eventually get will really work.
That's an interesting fact about Rust, and I didn't know it. On the whole, it sounds like it at least needs some serious tooling so you can make sure you're not dragging in a bunch of old versions that both bloat your code and open you to abuse. Can I ask for a warning if I'm getting two different versions linked into the same binary? If something depends on "=1.0", and the maintainer issues 1.1 with a flag that says "I really, really don't think think you should be using 1.0 any more", will that throw an error? And what happens if both versions get pulled in, but the package in question uses an external data file whose format changed between 1.0 and 1.1?
Edited to change "<=" to ">="...
Right. To mitigate that you would regularly run `npm audit` or even just `npm upgrade` – and test afterwards of course.
I'm not completely sold, but I do think it's a very interesting idea.
> Can I ask for a warning if I'm getting two different versions linked into the same binary?
Yes. That was the lint I linked in my last post. Alternatively, you can run `cargo tree --duplicates`.
> "I really, really don't think think you should be using 1.0 any more"
That's called "yanking". Personally I think it has limited usefulness, but it exists.
https://doc.rust-lang.org/cargo/commands/cargo-yank.html
> And what happens if both versions get pulled in, but the package in question uses an external data file whose format changed between 1.0 and 1.1?
If it uses something like `include!`, both copies will be compiled in (and maybe optimized later by the linker). If it's truly "external" like hosted on some website outside the package manager, then it just means the author broke their package. Maybe I misunderstood your question.
> one of my dependencies constructs a foo using X 1.0, is that value of a different type than a foo constructed by X 1.1?
I believe they are always different types. Cargo encourages but doesn't enforce semver, so anything can change between versions, including private fields or enum variants behind non_exhaustive, etc. So they're treated as different and you need to convert between them. Although this might only be true for major versions; I don't know off the top of my head.
To work around it you can convert the types at crate boundaries, or the package author can use the so-called "semver trick" [1]
[1]: https://github.com/dtolnay/semver-trick
I encountered this case once, and they are considered to be different types (in my case the issue was that using v1.0::foo with v1.1::foo traits wouldn't compile), so either keep them separated or pick one. Error messages can be confusing though, if you don't suspect using two versions.
IIRC rust only does that for MAJOR versions (1.1 and 2.0, or 0.2 and 0.3, since cargo consider releases before 1.0.0 to be major ones).
Node and npm have always been able to do this. Once upon a time this was accomplished via very hierarchical node_modules directories. This caused some issues, so the directory was flattened, however the multi-version compatibility was always maintained.
And the security of this new scheme depends entirely on the leftpad developer correctly assessing the security of the third-party colors package, possibly much bigger than his own?
Which has other problems too, particularly where semver is not followed strictly, since it fairly often means that using an update of X might force an incompatible update of Y that you now have to go and fix. Go modules have no way to specify upper bounds to prevent or warn about this.
https://abc7ny.com/suspicious-package-queens-astoria-fire/64...
Since NPM version ~2 (current is 8), you're allowed to publish a npm-shrinkwrap.json file[0] in your package. That's in contrast to a package-lock.json, which may not be included in a published package.
Back when I used shrinkwrap, it was pre-lockfiles and it was trying to accomplish something similar to what lockfiles accomplish today. (Lockfiles were added in v5 [1])
I dug up this old StackOverflow post to verify that the behavior of shrinkwrap hasn't changed[2] since many years ago.
That doesn't leave me very hopeful. If package maintainers aren't doing this already, and they've been able to for 6+ years, then it's unlikely they will in the near future.
(I'm currently investigating how to deal with bad deps with some Open Source tooling I'm building. Feel free to ping me if you have any thoughts to share)
0: https://docs.npmjs.com/cli/v8/configuring-npm/npm-shrinkwrap...
1: https://nodejs.dev/learn/the-package-lock-json-file
2: https://stackoverflow.com/questions/44258235/what-is-the-dif...
That seems tricky with the shrinkwrap, as you describe above, because it might create additional versions everywhere instead of trying to collapse them. (If you have 3 versions of `react` in peerDependencies, but each package had a npm-shrinkwrap.json, then you'd end up with 3 different versions of `react` installed.)
Would the burden then fall on your package manager to resolve this, ie npm or yarn? If it sees 3 different shrinkwraps (1.1.1, 1.1.2, 1.1.3) but they all have compatible peerDependencies (>=1.1.1, >=1.1.2, >=1.1.3) it could then pick the version that is compatible with all of the peerDependencies but that also exists in a shrinkwrap (so 1.1.3).
That would protect you from this "colors" scenario of somebody publishing `1.1.4` with malicious changes because, unless somebody were to shrinkwrap it, then it wouldn't get picked up by default. (In contrast to the current semver, which likely would pick up 1.1.4 upon a fresh `npm install`).
The inverse side of this is highlighted by another comment on this post: That semver makes absorbing security updates an easier process in the event of a log4j-style vulnerability. There is always a tradeoff, for sure!
It can be bloaty, and you have to manually look at and test packages if you want to get everyone on the same set of dependencies, but you end up with reproducible environments.
So practically nothing will ever exactly match, which leaves you right back at many slightly-different copies of hundreds or thousands of dependencies.
(Emphasis added) - is this actually a widespread practice? That's certainly not how apt packages are handled...my impression was this is a problem unique to the js ecosystem.
If we don't reject this logic, then we'll get more attackers claiming "just a prank, bro!" and "social experiment!", like the University of Minnesota researchers carrying out human experimentation on kernel developers without their consent:
https://www.theverge.com/2021/4/30/22410164/linux-kernel-uni...
We should encourage old-style prank hacking.
Even if you accept the idea of vandalism being used for a positive purpose, a better form of protest would have been to make the package just print a message saying "This software has been abandoned by its author. Please pin your dependencies to known good versions." and then exit.
That would still have been annoying to the people having to do that unnecessary version pinning work, but would at least have preserved some shred of sympathy for the maintainer.
For this particular case obviously previous packages didn't show it clearly enough. And yes, if you give thousands of 3rd party devs (or anybody snatching their credentials) direct access to your build machines or production systems, you should absolutely expect some of them to be unstable in all kinds of ways.
Insider problem is hard enough to guard against when you know the people involved.
A lot of developers, understandably, baulk at checking in dependencies, but there is a concrete benefit in being able to continue uninterrupted during outages.
How is it that people aren't doing that today? For the sake of security and stability, lock files should be used.
You could still have an issue if you need to update a dependency, for security or other reasons, since it could bring along a bunch of updated sub-dependencies of its own (and sub-sub-dependencies, and so on). But that problem is not unique to NPM and exists in any language or platform that includes package management.
https://sugarclub.sugarcrm.com/engage/b/sugar-news/posts/sug...
You don't need to do something crazy like what Marak did.
https://github.com/tc39/proposal-built-in-modules
Even so, unlikely that a "Faker" would become part of any such standard library.
Maybe Node.js should start shipping with a substancial standard library instead of making developers rely on NPM. Node.js can't even parse a multi-part request body without a third party library. So much for a HTTP server that doesn't even implement the HTTP spec...
Javascript isn't just node and web browser scripting.
Things like UI should not be in the language.
https://news.ycombinator.com/item?id=29863672
rsc is stating: (paraphrasing) the current state of affairs in many package managers is not a good design. This is (yet another) reason why package managers should work differently then they often do by default.
Dev corrupts NPM libs 'colors' and 'faker', breaking thousands of apps - https://news.ycombinator.com/item?id=29863672 - Jan 2022 (955 comments)
Important NPM package colors from Marak causing console problems at the moment - https://news.ycombinator.com/item?id=29861560 - Jan 2022 (1 comment)
Creator of faker.js pushed an update of colors.js which has an infinite loop - https://news.ycombinator.com/item?id=29855397 - Jan 2022 (1 comment)
Marak adds infinite loop test to popular colors.js - https://news.ycombinator.com/item?id=29851065 - Jan 2022 (7 comments)
Marak's GitHub account suspended after he erased his faker project - https://news.ycombinator.com/item?id=29837473 - Jan 2022 (53 comments)
Faker.js Erased by Author - https://news.ycombinator.com/item?id=29822551 - Jan 2022 (2 comments)
Popular JavaScript package “Faker” replaced with message about Aaron Schwartz - https://news.ycombinator.com/item?id=29816532 - Jan 2022 (3 comments)
Faker.js Has Been Deleted - https://news.ycombinator.com/item?id=29806328 - Jan 2022 (9 comments)
Though agreed with the sentiment of 'prefer stable' during install would be A+. 1am package scans was a cruddy way to start my vacation.
Not a good idea. You are pretty much asking to be review bomb. You would need very good moderation to be able to trust votes. That means npm spending a bunch of resources on hiring them, not sure if they are up for that (no, free mods aren't the solution).
Ex: We pin & package lock our versions, but it's harder for libraries to (they should do ranges), including ones we release. Likewise, for our apps (non-libraries), updating pinned package locks is reasonable for our direct dependencies, nested dependencies are hard to really have confidence on. Colors and leftpad both exemplify this: both projects fail our standards for direct dependencies, so the concern is nested ones. Unlike apt, we don't want super stale versions of everything.
For stuff like security, defense in depth + minimal targeted mechanisms for targeted threat models are generally a win, and I think that applies here for mitigating rogue releases by rogue package owners. Shifts like bitcoin -- crypto mining, ransomware, and wallet theft -- have made stuff like package buyouts and sneaky patches a reality where we must 'assume breach' of bad releases, not just try to prevent.
A few people don't like this but I think that most people are quite happy not to have their projects compromised. I would not assume that the loudest voices are broadly representative, especially with Microsoft being a lighting rod for complaints.
> Maybe part of the 0-day fix is go multiparty: allow weighted votes by dependents to take over the namespace to force a patch/reversion.
I think there's a lot of promise in this approach, especially if it's all public and deliberative — e.g. freeze the attacker's account, remove the compromised package, etc. but then have a public voting period for a transfer. The other question I'd have is whether it'd make sense to have a restricted set of maintaining organizations — e.g. I'd be a lot more comfortable if, say, the new project got transferred to an organization like the Apache foundation than some random developer who could plausibly be preparing to quietly ship a crypto-miner or something.
Isn't this what people are always praising about linux package managers? That they modify the software in all kinds of ways and take things into their own hands, beyond the developers will?
For GitHub it's maybe not a great look. If there was content in the readme that they felt violated their ToS, fair enough, I guess. They aren't obliged to publish that.
I'm skeptical of cryptocurrency but do like DAO style innovations (with no $ involved) because of this kind of issue. Many stepping stones to get there though and too many unknowns to go whole hog with an overly ambitious schemes. But seems inevitable to divest & decentralize from MS here. Something like community moderation has small components that can be iterated on for pushing to a community model vs current overwhelmed & overall inappropriate Microsoft moderation team model.
Looking at one of our production projects, we use colors via: "css-loader" -> "cssnano" -> "svgo" -> "colors"
I wish I could say I spent the hours to go line by line through every dependency of our app. But that wouldnt leave much time for anything else.
The point here is how we can make it easier for svgo and every other package to avoid the problem in the first place.
NOTE: this isn't an open source versus closed source thing. Linux has distributions which test included packages (to varying extents, I'm sure) and some of these are commercial operations. It's not impossible to have code whose verification you have paid for, to one extent or another, even with open source. (and hey, you can install malware with automatically updating closed-source see Solarwinds).
You're responsible for every dependency you add to your project and that includes all sub-dependencies. Your users will sue you for not doing your due diligence. You may turn around and try to sue your suppliers but that doesn't absolve you of your responsibility.
No, it's your responsibility to make sure your dependency tree is alright.
Your personal definition of 'alright' may be more easily satisfied if the packages you choose to depend on autonomously practice some level of responsibility towards their own dependencies. Choose wisely.
But there is no way to dictate your requirements to dependencies, or impose some kind of responsibility or demand some kind of warranty. You can accept what is offered, or not. In fact, if you are using svgo, even indirectly, you have agreed to this: https://github.com/svg/svgo/blob/main/LICENSE
>THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
So svgo doesn't have to care and you can't make them. Even if they do care, there's no guarantee they will meet your standards - and you still can't make them.
If you want someone to blame, find someone willing to sign a contract that says you can blame them.
Efforts within NPM and github to control this situation are simply the bare minimum of case-by-case disaster mitigation, in the interest of reputation alone. If you're using their infrastructure, you apparently find this acceptable.
In other words.. If there isn't a trusted test pipeline then there's no benefit in delaying the latest version. Might as well just get latest.
These middle deps could pin the exact version, but then when a security vuln is found and a patch issued, these libraries also need to update. This is like a traffic jam. If you're 6 hops from the vulnerable package you need to wait for not one, but six maintainers to push an update to npm before you can clear the security warning.
To get around this, middle packages list semver ranges. And then you have your occasional left-pad issue.
If I had to choose between those two ways to lose, I would use server ranges. The only way to win is to not play at all - have no dependencies.
There you need the way to say "get the new version from 5 minutes ago" without waiting for all levels of the dependency hierachy. Especially as there were a bunch of emergency releases in short sequence and younhabe tonmake sure youbget zhe latest one.
Dependencies are a mess. NIH can't be the solution either, though.
Something like log4j or many packages in the NPM world are so widely used that on any non-trivial project the odds approach certainty that multiple dependencies will specify versions. The more specific the pin, the more likely it is that you'll be unable to ship a security update without risk of a functional change or blocking incompatibility. You can mitigate that with good CI/CD infrastructure and lots of automated testing, but that means taking on more infrastructure expense.
You still have the exact same problem you described with `^2.14.0`. Someone would have to manually update the package to get the security fix in 3.0.0.
Unless you're suggesting code should also automatically update major versions aswell?
If the line between bug and feature were clear. (Log4j worked 100% as specified btw. in regards to log4shell)
https://xkcd.com/1172/
Second, log4j is bad examples. Libraries don't pin that at all and people report bug is they do. Libraries are supposed to depend on logging api in general and end project decides whether use log4j or slfj.
We need to get the version that we tested with and know works, which is what it does. When we need to update, we say what version we want.
Java can be a pain, but I'm very glad it doesn't handle dependencies like npm.
I'm still glad that basically noone else handles dependencies like npm.
You can argue that npm's commands are poorly named and guide users towards bad defaults, but saying that it works in a unique way is not true.
Separating the two is handy both due to frequent churn and to avoid transitive dependencies being kept unnecessarily — I’ve seen projects where libfoo stopped depending on libbar a while back but they were still installing it because someone had copy-pasted that block years before and was just incrementing the version.
So that would be “get the one from 5 minutes ago, no questions asked”
You just specify the latest version. But you do it explicitly.
At least with Maven, that's extremely simple. Just add to your project a dependency on that "new version from 5 minutes ago" of log4j. Maven always prefers the version from a direct dependency. You don't have to wait for "all levels of the dependency hierarchy" at all.
Another thing about setting exact dependencies is reducing duplication. Libraries are encouraged to use loose ranges, because if everyone pins exact versions for every dependency, then you could end up with 10 to 20 different copies of 'colors' in your tree, instead of having just 1 or 2 copies that work across the board.
Our industry has gotten along for far too long with zero liability or accountability.
Not only that, even if you excuse their lack of ethics, there's a basic competence issue: they have a basic failure to learn from history and the most basic part of this, which is that you shouldn't be pulling live from the internet.
Completely agree, which is why I support such endeavors wholeheartedly.
The comments here have convinced me that the industry is basically beyond self help.
Perhaps you have another solution in mind? I think something along the lines of package-level content security policies with granular permissions has a lot of potential.
I did try (2) for awhile with some success. You basically vet everything you import.
This generally follows the same rules of supply chain validation expected _and common_ in other industries where people actually give a crap about what they are doing.
In the case of large projects, especially those with a solid track record of testing and security issue management, the vetting is organizational. You don't need to read every line of Kafka (though you should, as I did, actually look through it to see what you're getting into), as an example, at least today.
But for smaller projects, just like with smaller vendors, you need to go deeper. You do need to read the code. Then you look at their issue tracking and see if they have any concept at all of quality, sanity, security notices, what their maintainer attitude is, and so on. You definitely read the code. If you're sloppy, you at least read through a good sample of the code. Again, diligence - you have the obligation to your downstream to know what you are getting into.
This is no different than the vendor management you expect in pretty much every supply chain that you interact with. Food, cars, medicine, pharmaceuticals, building materials, heavy machinery, embedded software in medical devices, and so on. Software has gotten away with complete negligence for a long time and instead of addressing it, the common practice has gotten worse and worse. At some point, we are going to accidentally kill someone.
Then, on top of that, you lock it down. You mirror it, you pin it, you make an active plan to monitor the project to make sure you are aware if it is abandoned, hijacked, deleted, etc. You take responsibility for dealing with these possibilities. You mitigate immediate threat by not pulling random crap live when you build. You further make sure you partition your system such that code you have not classified as high quality has a reasonable blast radius and security boundaries.
But whatever, let's say you're a typical valley company that operates like a psychopath and you don't care about that. What companies should be worried about is liability. Yes, most of the security activities for most SaaS and other SW is laughable and mostly theater and checklists. But there is liability nonetheless. After all, you are about to represent it as high quality software and you are taking on the liability. No judge in the world is going to let you go with the "but it was this javascript we pulled in, not ours" excuse.
We built quite substantial systems using this approach, and it was important in the domain that we were working in to have done so. It is not without hassle and cost.
Really, what are we talking about here? It's _too expensive_ or _too difficult_ to have even the slightest quality verification of software? The industry is a complete joke if that is true.
;)
Now, who's gonna pay you for writing that single functionality whole year? Sure, you tested, documented, did everything right, you may have almost no bugs, pretty code, decent test coverage, unit tests, integration tests, UI tests, performance tests, edge cases ironed out, top notch performance, UX no-one matches, one click deployment, static code analysis, linters, fuzzers, vulnerability scanner tools, monitoring, auto scaling... while you get there, your startup may be no more. Or maybe just a money sink: https://thedailywtf.com/articles/unseen-effort
Alright, I'm little off the track, but not every software project is comparable to automotive/air/rocket/med/... industry. Ofcourse you will put 10x+ more money/time/effort if human lives may be impacted with your commit.
To put that into context, let's appreciate SQLite - software that is thoroughly tested and your airplane shouldn't be afraid to run those. From the creator of SQLite:
> I’m going to write tests to bring SQLite up to the quality of 100% MCDC, and that took a year of 60 hour weeks. That was hard, hard work. I was putting in 12 hour days every single day. I was just getting so tired of this because with this sort of thing, it’s the old joke of, you get 95% of the functionality with the first 95% of your budget, and the last 5% on the second 95% of your budget. It’s kind of the same thing. It’s pretty easy to get up to 90 or 95% test coverage. Getting that last 5% is really, really hard and it took about a year for me to get there, but once we got to that point, we stopped getting bug reports from Android
https://corecursive.com/066-sqlite-with-richard-hipp/#testin...
What we are missing on the software industry is proper liability.
By the time you got to the end, it would be time to go back to beginning and start over again...
I've tried to remove single-line dependencies only to have the PR rejected by the creator of the tiny library who happens to contribute to the parent library.
IMO the Javascript ecosystem really needs a decent standard library to remove the inordinate amount of power granted to these these tiny library squatters who wrote 5 lines of code and a package.json file 10 years ago.
And that was one situation. The mindset of the Javascript ecosystem is still to maximize code reuse, meaning even if the tiny library maintainer isn't a maintainer of the parent library, the parent library still frequently clings to their one-line dependencies when I've tried removing them. Thus granting the tiny library owner tons of power like the maintainer of "colors".
So fork it and throw out everything you don't like.
For any project using NPM, it was an absolute nightmare. Dependencies 3 or 4 levels deep would get flagged, and there wouldn't be a way to resolve them. The security teams didn't care, and multiple projects were left stranded. Since resolving those issues would mean having to contribute to open source, which we weren't allowed to do.
On the other hand, because we had specific security teams doing the scanning and assessing the results, you weren't allowed to question them, because they could just block your entire project. So developers were extremely unhappy with them, and what ended up happening is that developers just did what the security teams asked, without questioning. Which to me, let to more security concerns, because we were just pulling in libraries without knowing what had changed.
Sadly, the few devs who stopped relying on third party dependencies all left, because most of their work took longer (since they were implementing stuff that third parties usually do). Business teams took notice and questioned why everything was taking longer.
The resolver tool we used couldn’t fix dependencies that deep even when there were fixed versions available. And since their internal processes didn’t use the same audit tooling, Facebook devs would often close issues requesting dependency upgrades to fix those not actually security issues. I think Dan Abramov wrote a post on overreacted about it at some point too. I’m not convinced any of the time we spent on those issues was well spent to be honest.
Obviously a non-issue, but you still had to verify, and document everything. Best practice would be that we ran this scan on every commit, but the scan took about 15-20 minutes. It just ended up being something that was part of our release process, with the obvious danger of breaking our application right before release.
All in all, it stopped me from relying on third party modules, and tried to use as much vanilla code as possible.
https://www.reddit.com/r/ProgrammerHumor/comments/6s0wov/hea...
>Heaviest Objects in the Universe:
>Sun
>Neutron star
>Black hole
>node_modules
In this case, the Net interpreted Marak's actions as damage and seems to have effectively routed around them. There was disruption, but we already have systems in place for flagging broken or malicious versions (`npm audit` being one example).
For people willing to have the system work 99% of the time with the occasional Marak-screw, we already have a working solution in what currently exists. For people who cannot tolerate that level of risk, you have to firewall your package sources and only allow use of vetted code dependencies in your software.
Open source relies, in the end, on trust. Marak broke trust and that's all. The question isn't "How do we prevent people breaking trust" (that's impossible; humans have free will) but instead "How do we mitigate damage / protect that which cannot be risked if someone chooses to break trust?"