I feel like some of the disconnect comes from being a mandate rather than a persuasive recommendation. Imagine that they included some sort of case study to make it less abstract:
Company C uses database driver P. On 2020-01-02 they upgraded their app to the latest version. On 2020-01-04 they noticed that someone stole all their user data. Upon investigation, Package P had a version uploaded on 2019-12-24 that sends all user data to evil.example.com. The tooling to upgrade the package doesn't show diffs, so company C had no way to detect this malicious change. How did such a compromised version get uploaded? Looking at the audit logs, it appeared that there were 2398438 unsuccessful login attempts to package P author's account, before finally uploading the new patch release from the same IP. Company C lost 8 billion dollars as a result of this. If author A had used multi-factor authentication, then this wouldn't have happened, because it would have taken more login attempts than there are atoms in the Universe.
Then the author can see "hey I can save the shareholders of some random corporation 8 billion dollars if I make it harder for myself to release software". That's a better incentive than "because I told you to".
I know it sounds a bit disingenuous to say this is just for the shareholders of random corporations. I'm being a little snarky. It's also good for your reputation to not get hacked, and using 2FA to log in every time is probably less time overall than reacting to a single compromise. Imagine how many emails you're going to get. Big pain. I just think it's important to show authors the cost/benefit. 2FA is easy. Next time it might not be something that's as easy, though.
I have the feeling that’s because it’s a “best practice” and not a data-driven decision.
Wouldn’t they have led with, “we’ve witnessed X attempts and Y takeovers in the past year, so we’re introducing new security measures aimed at…”
To me, this seems like more frog-boiling authoritarianism “for our own good”, like the COVID lockdowns, censorship, etc.
I’m tired of technology forcing me to do things “for my own good” — and I recommend against companies who engage in those policies.
These are all the same arguments trotted out to justify Windows spyware and forced updates that routinely lag my computer for hours — because Microsoft knows better than me how to use my desktop and isn’t shy about pushing me out of the way to do so.
That leaves a strange taste in my mouth. I love writing open source for both me and other devs to use it and make their lives easier; I understand that big companies will also use it since that's one of the places where devs work, and since I make mostly tools or meta-tools ("dev solutions" and not "business solutions") I'm happy with this arrangement. When I do a more business-like solution, I tend not to publish it as open source and remain on my list of trying-to-monetize side projects.
But if you quantify it like this, "this company saved 8 billion dollars out of some extra work on your side, here is a pat in the back because they'd never even consider give back a single cent or helping in any way" it feels a bit strange. I know it's on me, and I just learned this contradictory feeling so need to unravel/understand it, but you def got the opposite feeling from me as expected.
Edit: you added the "I know it sounds a bit disingenuous to say this is just for the shareholders of random corporations." later, right? Which kinda reflects a bit of this feeling
With no data, I'm assuming this is all business-driven. Nobody is compromising software supply chains to make your 3D printer print 8% slower for the lulz. They're doing it to hack companies and compromise user data. If open source was just hobbyists making stuff for hobbyists, it might not even require a login to update a package. With companies involved, they to reduce this risk, and this comes at the cost of people not involved with the company's business. I can see why authors get annoyed; they are being required to change their workflow for a use case that doesn't interest them. Being asked to do boring work for free is not a great solution to anything.
Personally, I don't have a horse in this game. I always use 2FA when it's available. I have already made the necessary adjustments in my workflow. But I get why people are annoyed that they have to change something. Imagine that this was "we'll send you a check for $1000 if you enable 2FA on your account". Bet there would have been zero angry blog posts.
> Imagine that this was "we'll send you a check for $1000 if you enable 2FA on your account". Bet there would have been zero angry blog posts.
That is just entitlement on the part of some open source devs. PyPI is already providing a huge service for them: it is publishing their package free of charge. Furthermore, it is maintaining old versions, scanning others for vulnerabilities, replicating across the world, resolving dependencies etc. They have every right to ask anyone who is enjoying their completely free service to put in the minute amount of extra work required to set up 2FA.
Of course, anyone is also free but to want to put in that work, and so they can stop delivering their package on PyPI. Absolutely your right as an open source maintainer, and you have no one to answer to if that breaks some builds.
> Nobody is compromising software supply chains to make your 3D printer print 8% slower for the lulz. They're doing it to hack companies and compromise user data.
The high-profile NPM package compromises in recent years have mostly targeted hobbyists/been indifferent to who they were targeting. Crypto miner malware doesn't care what computer it's installed on.
> With no data, I'm assuming this is all business-driven. Nobody is compromising software supply chains to make your 3D printer print 8% slower for the lulz.
As far as I know common targets for malicious packages are crypto (mining, wallet swiping), and credentials (SSH, package repositories), none of which is limited to businesses. There's been some cases of politically motivated sabotage lately but mostly from the maintainer themselves so far - expect more of that though.
Often a compromised package will try to steal more credentials from downstream users. [1] I don't doubt that threat actors all the way from script kiddies to nation states are keeping a stash of credentials to package repositories, and it's going to suck for everyone when they decide to use them.
Right, so this is why I keep harking on the fact that we need a non-billionaires OS license. People will say "oh, no, that's not real open source" but it sure is as open as their chequebook. I'm hand waving a bit here, but if you have a stakeholder that's worth a billion dollars you can pay a $1m per year for the OSS library you use.
When I write open source software I'm not doing it for Bezos. I'm doing it for Heather down the street who is trying to get a cool little project off the ground and maybe pay some bills. I'm even writing it for Frank who runs a modest 20 person dev shop and maybe contributes back a bit here or there where they find the warts.
I'm not building it for the psychos that have lost all touch with humanity.
That makes a lot of sense to me. I don't know if I do open source even for Heather down the street, I mostly do it for myself. "I made a thing, come look." "Come look" is not really an invitation for "we looked and we took it and can you do all these things for me for free". It's really a tradeoff between ego/reputation and time. It's possible that people can ask you for more of your time than you get returned in reputation. (Ask any artist who has made something "for the exposure." People will do it! But they'd rather be getting paid.)
Art is a great analogy here because you don't get to have a say in the process when an artist is doing it for themselves. You can look and critique, but you don't get to have control. If you want that, you're welcome to commission a piece and pay for it.
I just consider future me as just another developer, and it's way too often that I've had to read my own docs to see how I was supposed to do something and thanked past me. I do have a preference of who uses my software, but in no way encode it since I believe it's just no realistic: future me > other OSS devs > indie devs > general devs > small companies > large companies.
Why are so many arguments acting like the maintainers of PyPI who imposed this change are not themselves open source developers whose time none of us, including those who publish their open source code on PyPI, pay for?
PyPI asking for 2FA is in no way different from an open source project asking you to run their unit tests before submitting a PR. Sure, it puts a burden on you, the open source contributor, but it is balanced by removing a burden from the project maintainers.
I don't see why people should be persuaded at all. PyPi is a software distribution mechanism to millions of PC globally. PyPi is putting their footdown and saying we are going to do everything in power to prevent PyPi from being used as a global malware distribution system. That involves not only securing our software, but ensuring the people who have the capability to do damage are following appropriate security protocol.
It's the responsible thing to do. If a package gets compromised its PyPi that has to do cleanup work. It's like complaining that the government forces you to wear your seatbelt.
What costs? You can stuff 2FA token generation in many password managers that run on the very computer on which you develop. It costs you exactly zero except a couple of minutes to set it up.
Yeah, except that the advent of password managers means that most 2FA users actually rely on something you own + something you own.
Everyone can agree that password managers are a good thing. However, it's debatable that, in many setups including the one that was just described, 2FA brings added security.
You’re only looking at one side of the cost equation. If a maintainer screws up and lets their account get compromised, and then it uploads malware, it’s PyPI who has to pay the immense cost of cleanup. How is that fair?
Well, it's PyPi who decides what is critical and what is not. If they're not happy with maintainers not wanting to take that responsibility, they can fork.
>Yeah, that's the very essence of being a distribution tool.
Also, conveniently, it's the thing whose cost is not paid by the people decided to get owned.
And again, the government doesn't cover the cost of your seatbelt, so I don't see your argument. PyPi also makes you use passwords, should I complain that they are forcing me to buy a keyboard?
> Then the author can see "hey I can save the shareholders of some random corporation 8 billion dollars if I make it harder for myself to release software". That's a better incentive than "because I told you to".
To be quite honest, I would rather the shareholders of some random corporation do lose 8 billion dollars. In fact, I would be willing to pay a small amount to see it.
There are many weights involved; PyPI is trying to strike a balance between them.
Among them:
* Project installs follow a power law distribution: the top 1% of packages account for 99% of installations, and the remaining 99% account for roughly 1% of installations. In other words: prioritizing the security of the 1% of packagers means prioritizing the 99% of users who depend on those critical projects.
* PyPI has tried really hard to make 2FA as effortless and non-invasive as possible: the current program includes giving away two free physical hardware tokens, as well as a long grace period to ensure that maintainers are not caught in a sudden lurch between their open source and professional obligations. Both of these don't work at scale: PyPI doesn't have the material resources to give every maintainer free hardware tokens, and the same packaging power law means that a large number of relatively inactive maintainers will be caught by any deadline that gets set.
Ultimately, the goal is to maximize the number of users protected, maximize the quality of protection, prevent package user disruption, and minimize maintainer disruption. The current scheme, in my view, does a good job at achieving all of these goals.
FD: I worked on PyPI's 2FA implementation, but I do not represent PyPI and am not a maintainer.
I forgot to mention another weight: maintainer time. A universal 2FA mandate would drastically increase the amount of support time maintainers spend helping users recover their accounts (a perverse problem to have!).
The weird bit is that quality control is so completely out of the question that it's not even mentioned, but authentication is seen as more important than anything else. I wonder how accurate that threat model really is.
I mean, node-ipc would have happened regardless of how many authentication factors were involved in uploading. And offers to buy ownership will keep coming. What happened to Sourceforge wasn't lack of security.
Turns out those distribution people did more than just wrap software in other formats. Turns out that multi stakeholder thing had merit. Who would have thought?
Quality control isn't out of the question, per se: it's part of a different threat model. The goal behind 2FA is to limit the latent threat of account takeover attempts, which increases as packages' criticality does.
Quality control belongs more closely to the "integrity" category of threats, which is much broader in scope than account takeover. It's something that we're thinking about as part of designing additional security schemes for PyPI; you can follow some of that work under the PEP 480 and PEP 458 umbrellas.
"Seems a bit janky to only require 2FA from “critical projects” and not all projects. Why cut corners?"
Agreed.
I don't know if these projects need 2FA "protection" or not - but I'm quite certain that the risk to any particular user is identical whether they are downloading a popular vs. unpopular package.
Total, global risk is, of course, higher for popular packages, but any individual user does not care if their intrusion vector was a popular vs. unpopular package ...
Well, there's an easy solution to square these viewpoints. People who prefer a caveat emptor can keep operating like that, and people who want to be responsible can drop the "AS IS" clause from the license and put their money where their mouth is. (And really, none of this "well yeah the license says that but really you should be held to account for other stuff"; either commit or don't.)
> If he doesn’t owe us anything, then we also don’t owe him anything. Perhaps he’d like to take back his code and the rest of the world can take back all the opportunities and other things he’s been given In exchange.
I feel like this is a rather silly take. Book-writing, speaking, etc opportunities are not ongoing burdens like open source maintainership. Even job opportunities you can choose to quit. I think ultimately if people are doing things for free you can't ask too much of them since they are well within their right to stop doing it for any capricious reason.
I feel like you’re taking the author out of context. His point is that without the open-source work fewer opportunities would exist for the person doing it. So it’s a bit disingenuous to act like you do all of this “purely for free” when it’s more like “self-marketing”. I get how imposing on people doing the former can be unreasonable, but asking things of people doing the later seems quite fair.
The original author seemed to think so and the regular speaking engagements and book deals seem consistent with that. It might not be the exclusive reason but it certainly seems a factor.
Yeah, the second sentence isn't a great argument because it's a voluntary relationship, you can quit at any time, and that doesn't mean you have to give back any benefits you received.
In particular, you can quit maintaining an open source package, or stop publishing it in an index and let someone fork it. (Maybe you feel a little guilty, but that seems like a feeling to get over if it's not working out for you.)
But it still works both ways. If you're not finding it fun anymore, that doesn't say anything in itself about who, if anyone, is to blame.
With all due respect to James, without whom I probably wouldn't have my current job (I've been programming in Django since the 0.96 days, that is 16 years now), I think Armin is right on this one:
> PyPI asks for 2FA today, what might they ask for tomorrow?
Yes, slippery slopes are real because they happen, especially in this very tense and very non-rational geo-political climate. Yes, today it might just be "use 2FA with you still want to actively maintain your own package", tomorrow it might be "boo-hoo-hoo, you've followed/liked Putin/Xi/whoever the powers that be don't like, you're a threat to liberal democracy, we can't leave critical infrastructure in your hands, bye-bye", and there will be nothing for the package maintainer left to do at that point.
The entirety of the last decade in open source software development can be summarized as this: it is morally reprehensible for your project to not support the causes du jour.
You must adopt a COC. You must rename your branch from "master" to "main". You must use "inclusive" language in your documentation.
Why? Because you must assume the responsibility for everything your project touches. A bad person uses your leftpad.js to perpetuate badness. You must stop them. Someone might get their feelings hurt. You must preemptively protect them.
This is an impossible task. 2FA is fine, sure. I use it myself, and advocate for others to do so as well.
But I'm sick and tired of the "harm reduction" mantra that means I have to entirely rewrite my documentation every 6 months.
Dude, how did you go from 2FA for a Python package to Putin in a single sentence? This makes no sense.
Look, publishing to PyPI is not a right. If typing 6 digits is too much burden you can drop a source tarball in some web page and call it a day, you don't have to go through PyPI. Done.
I, for one, I'm very very glad that a piece of _critical infrastructure_ that I depend on is a bit more secure.
> Dude, how did you go from 2FA for a Python package to Putin in a single sentence? This makes no sense.
The 2FA thingie is not implemented because of some random dude who might want to steal a bitcoin or two, is implemented against state actors. This being the US we're talking about (and the West more generally) the state actors taken into consideration are: China, Russia, Iran, probably North Korea (probably in this order). Hence my mention of China and Russia in my initial comment.
If they slip further down the slope you can, as a maintainer, protest there. 2FA is literally the least you can do to help prevent the kind of 'supply chain attacks' we've been seeing in the open source community for years now. If they continue, in spite of your protest, you can always distribute your library some other way or retire it.
The author of the article this thread is about is litterally pushing further down that particular slope:
> And if your package gets established and has people regularly using it, you have some responsibility not to mess with those users. If you want to make backwards-incompatible changes, for example, communicate them and bump the major version number. If you want to stop maintaining the package, post an announcement about it and be willing to hand over to someone else to continue work.
Not really, none of those things are things he is suggesting PyPI should do, but rather social expectations that come from being a maintainer of a large project.
> PyPI asks for 2FA today, what might they ask for tomorrow?
If and when they start policing political views of their maintainers, it would be fair to criticize them. But the theoretical possibility of such a thing happening has no relevance to whether 2FA is a reasonable request.
Are the 2FA dongles provided by Google available to maintainers that reside in countries the US (and the West generally) doesn't generally like? I'm thinking, explicitly, at China, Russia and Iran. Most probably no, even though I'd love to be proven wrong on this. As such, this is already a political thing.
TOTP is defined in a freely available RFC (https://www.rfc-editor.org/rfc/rfc6238) and is implemented by a wide variety of applications and libraries, both proprietary and open source.
Yes, I know, I've implemented it on the server side once, but I was curious why did Google (the company) feel the need to get involved with those thousands of physical dongles as long as almost any Android or iPhone can handle this, hence my question. Because I suppose there aren't thousands of contributors who don't have an iPhone nor an Android phone. Or maybe it's just a marketing ploy? I still don't get it.
Dongles are generally understood to be more secure than TOTP because they have to be physically stolen (unlike TOTP seeds, which are just information). Google is giving away thousands of physical keys maybe to drum up goodwill among the python community, maybe because it's a trivial expense for them and they have an institutional interest is seeing PyPI succeed. I dunno, but it doesn't really seem all that sinister.
Just FYI, you don't need a smartphone to use TOTP. You can download 'pyotp' from PyPI and use it with any python interpreter. Anybody publishing packages to PyPI would have access to that.
> And finally: if you’re determined to take something I’ve said above, find the least charitable interpretation of it you can come up with, and argue with that, please know that I don’t read Hacker News and haven’t for years, so I won’t see or respond to you.
from the linked article, i'll just leave that here
> More seriously: two-factor auth is such a reasonable bare-minimum and easy-to-do (from the account-holder end) thing for account security these days that the objections being raised make no sense to me.
As a security engineer, this is the thing I don't get about dissenters. Any account I actually care about has 2FA on it.
By today's security standards, NOT adding 2FA just means you don't really care about your project. Hand it over to someone else in that case.
Depends. Some websites insists on implementing 2FA in a poor way (e.g. installing dodgy custom apps on users' phones), so for accounts that users really care about, it might be the case that they would avoid these poor implementations to reduce attack surface.
(I am sure it doesn't apply to PyPI, but I am just saying both site operators and users need to adopt best practices in terms of 2FA)
> So, look, I get that there are some people who want to live in a world built on caveat emptor and the idea that it’s always and only your fault if something bad happens to you. I get that there are some people who think this is the only kind of world open source can be. Maybe Armin is one of those people, or maybe he just argues like one without realizing or intending it. [...] But no. Just… no. That would be a terrible world, and a terrible model for open source software.
Is "that would be a terrible world" supposed to be a counter-argument? The world is built on caveat emptor, especially when downloading random packages off the internet. That's why browsers are sandboxed. That's why people run virtual machines. The vetting process for adding new dependencies at big corporations can literally last months. The 2FA nonsense does basically nothing in practice (vetting will not and should not magically go away), and only adds an undue burden on developers.
The internet is a dark forest and I think it would do us a bit of good to treat it as such.
> The 2FA nonsense does basically nothing in practice (vetting will not and should not magically go away), and only adds an undue burden on developers.
This is true, but to play devil's advocate, how many HN articles have we seen where X critical NPM package had been compromised by a malicious user who gained control of the package maintainer's account? There are many examples of this. If a new package deployment system were to launch in 2022, I wouldn't be surprised if 2FA was required for all publicly listed packages, full stop, not just critical ones. Viewed through that lens, it really isn't that unreasonable.
That said, I think full power should be in the hands of the developers who maintain these packages when it comes to the actual content of the packages. Package systems should not be editorializing. If the author of [insert major package here] wants to upgrade a version that replaces the whole package with a console animation of a turtle, that's his/her choice as a developer, and simply should be viewed as a "breaking change". Intervening in THOSE scenarios I view as completely out of control on the part of package ecosystems. Let people leverage their creations however they like. Your trust is supposed to be in the maintainer. If they aren't trustworthy, you shouldn't use their software in the first place. If they decide to go in a new, breaking direction, that's their decision. Caveat emptor.
These scenarios though, are starkly different from scenarios where the package maintainer's own intent is subverted by a malicious hacker / user. That's a whole other can of worms, and I agree we need better controls to protect all packages from these types of scenarios.
Odd but serious question: could there be ways to distribute versioned software that doesn't require management of developer accounts (and the associated time-and-effort costs related to account takeovers)?
Not if the goal is to avoid vetting each new version as if it were a completely new dependency. The whole reason for the current system, and the very idea of new versions of "the same software", is that we want to be able to rely on the reputation of a project to make certain decisions. For example, we trust Linux 5.11.1 to be non-malicious and to be mostly stable etc largely on the reputation of the Linux project. We don't go around vetting the code of such s project except in efemer specific niches.
If we don't trust the provenance of the code we're getting though, that reputation becomes irrelevant. As such, this is a problem of identity assurances and access management, it can't be solved otherwise.
Ok: you've provided two requirements that I agree with:
- It should be possible to compare between two releases (I'd personally like to see a code diff, ideally with a complete path of the commits involved)
- Providing a reputation visibility mechanism (for publishers? author(s)?) across a series of releases is important
Those don't require user accounts necessarily, though. And responding to the end of your message: identity assurances, yep, those seem necessary; access management, I'm not so sure.
Access management is required to have identity assurances. If there's no way to ensure someone's identity is secure, how do we know it's them in the first place?
All major repositories editorialize to some extent, they would be awful if they did not.
For instance, PyPI will take down your software if we determine it to be malicious in some way, which is extending editorial control over what's on PyPI.
Generally speaking, though, PyPI does not concern itself with the contents of the packages shipped through it as long as they are reasonable packages. If a project wants to break compatibility, then that's on them. We strive to let projects manage themselves, in whatever way they think makes the most sense.
> For instance, PyPI will take down your software if we determine it to be malicious in some way, which is extending editorial control over what's on PyPI.
Where this breaks down and gets tricky I feel, though, is when you get into examples where the package maintainer makes the newest version essentially useless (but in an otherwise harmless way) in protest of some grievance. I think this is a perfectly legitimate action for a maintainer to take as long as they aren't literally installing malware or opening up security vulnerabilities. Package systems stepping in in these scenarios and "bailing out" all the companies that rely on the package by taking it over are effectively silencing the maintainer. My smell test for this is if the ISP or hosting provider wouldn't yank it, neither should the package system. Hosting providers will take down malware and illegal content, but they aren't going to do more than that, and that's how it should be imo in package systems. I think the benefit of developers' voices not being silenced vastly outweighs the humorous scrambling of a bunch of companies who failed to lock their version numbers for a few short hours. The same scramble happens all the time with real-world breaking changes anyway. If the maintainer wants to effectively yank their project, they should be allowed to do so. Otherwise you never really know if you are getting the maintainer's code, or whatever the package system decides should be the maintainer's code, which breaks the trust contract upon which package systems are built.
This is another reason it's great that a lot of languages, like Crystal for example, have adopted a decentralized model.
PyPI doesn't currently prevent deletions at all, so a maintainer can just flat out delete their stuff if they want. Though we are discussing if we want to tighten that up at all.
I would say that in the > 10 years of time I've been an admin on PyPI and even longer that I've been involved in Python's Packaging, I've never once seen anyone, but individuals placed front and center in who we're attempting to serve first and foremost.
Businesses and other large orgs do come up from time to time, and we've generally been pretty good at making decisions that allowed those businesses to solve their problems on their own, without requiring the rest of the ecosystem to do it for them. In other words, our features we add for businesses tend to be unblocking them from being able to implement something they want, rather than taking on the responsibility of whatever thing they want.
Critically, and something I forgot to mention, when the package system says it's from X developer, but they are actually serving some alternate content because editorially the package system has decided to override the developer's most recent version, saying it's from the developer is simply a lie, and is disgusting.
> The 2FA nonsense does basically nothing in practice (vetting will not and should not magically go away), and only adds an undue burden on developers.
I don't understand this claim -- are you saying that 2FA adds no protection of value in the context of package maintainership, or that does nothing in general? I'd like to understand the reasoning behind either, whichever it is.
My 0.02c is that 2FA has empirically reduced the incidence of ATOs on major services, and eliminates entire systemic weaknesses in account systems (like ATO via password reset). It stands to reason (to me) that PyPI is a service like all others, and benefits similarly.
> I don't understand this claim -- are you saying that 2FA adds no protection of value in the context of package maintainership, or that does nothing in general?
I'm trying to say two things: (1) that 2FA adds little, if any, protection of value in the context of package maintainership. In other words, if someone wants to be a malicious actor (say, they had a "legitimate" package from versions 1.0 to 2.1, but 2.2 contains a trojan), they can do it even with 2FA. And (2), it adds an undue burden developers, particularly brand new ones that just want to "publish their package," have to deal with. If you're 15 or 16 years old, excited to finally get your package on PyPI, but all of a sudden, you have to jump through all these hoops, including doxxing yourself and providing a real phone number, you would 100% be deterred.
> In other words, if someone wants to be a malicious actor (say, they had a "legitimate" package from versions 1.0 to 2.1, but 2.2 contains a trojan), they can do it even with 2FA.
That's a different threat model. We're talking about account takeover, which is a separate risk from maintainers going rogue. 2FA addresses the former; the latter is difficult to address in the general case. Nevertheless, it's something that people are tackling both with static analysis and "reputational" schemes.
> it adds an undue burden developers, particularly brand new ones that just want to "publish their package," have to deal with
You're right, which is why it's not mandatory for everyday users. The decision to enable 2FA for "critical" packages is a practical and calculated one: most critical packages are maintained not by brand new PyPI users, but by seasoned engineers who understand their centrality to the ecosystem and the importance of measures that secure their work. There's no current plan to make 2FA mandatory for every single user; the threat model doesn't justify it.
> including doxxing yourself and providing a real phone number, you would 100% be deterred.
I'm not sure what this has to do with PyPI. PyPI does not and has never supported SMS 2FA. This was an intentional design decision on my part.
The only supported 2FA methods are TOTP and WebAuthn, both of which can (and do, by default) preserve nominal anonymity. No phone numbers are involved.
2FA has never been described as protecting against (1) or (3). That’s not what it’s for; treating these as deficiencies is tantamount to criticizing a hammer for driving screws poorly.
2FA exists to make authentication more difficult for an impostor or attacker. The goal is to prevent account takeover, which has historically been a risk to critical packages in multiple ecosystems.
(2) is sort of ambiguous. I’m not aware of a major case of either TOTP or WebAuthn being compromised en masse because it was on the same device as the first factor. I’m not even aware of a case where WebAuthn has been meaningfully attacked, full stop, despite theoretical breaks of hardware stores. Even with a full break, WebAuthn provides detectability: cloning the factor means cloning the counter, which will alert the user on the next login. On whole, I think it's incorrect to frame 2FA as generally breakable without qualifying the threat and explaining how it applies to PyPI users.
> Some of this may just be due to incommensurable world-views. My view of the world is based on the idea that my actions may have consequences not only for me, but also for other people, and that I have some responsibility to at least consider those other people when deciding what actions I will take.
Very well then. Do whatever you think is necessary and appropriate.
There's a very simple way to make that opinion count, you can hire the maintainer.
Open source has been built on an endless stream of people making personal sacrifices and often burning out. Seeing friends slowly getting devoured by anxiety crisis or burying themselves to support commercially used software, barely being thanked for what they give to the community, has been one of the most depressing parts of this industry.
If you don't support that particular piece of infrastructure, your opinions on it are worth zero. Even if you're a well-known open source maintainer, and some times even more so. It is understandable that people getting a comfortable job thanks to their contribution on a high-profile project have a different view from people that don't, but that doesn't make it a better view.
In retrospective, open source was bound to have these dynamics, with some people benefiting from their open-source work not seeing why people not paid to participate don't enjoy being told what to do. I wonder whether we'll see a friendlier environment in the future, or whether open-source will end up being a cooperative of companies sharing code among themselves.
"There's a very simple way to make that opinion count, you can hire the maintainer."
I want to describe to you that this is very difficult - almost impossible.
A long time ago I decided that I did, in fact, want my opinions on open source and free software to have more weight than a typical non-contributor and I have made dozens of valid, reasonable - even generous - offers to individuals and projects to advance certain design goals or feature requests - or even simple bug fixes.
It's not that these offers (contract offers, bounties, straight up cash payments, sponsorship, etc.) were rejected - they are typically outright ignored. As in, radio silence. No response whatsoever. I know they got my message and I know they are ignoring it.
I have some pop-psych / cultural analysis / personal anecdata as to why this is (almost always) the case ... I think you can easily sketch a caricature of someone who does something out of love or passion or as a personal challenge but has no way to connect that to expectations or deadlines, or any other personal involvement whatsoever.
But whatever the explanation, be aware that your suggestion is not the magic bullet you think it is.
I've had the opposite experience, approaching a random FOSS maintainer asking them to add a feature my workplace wanted, they got it done pretty quickly. I wasn't involved in the contract/payment side of things though.
> In retrospective, open source was bound to have these dynamics, with some people benefiting from their open-source work not seeing why people not paid to participate don't enjoy being told what to do.
In particular, it was bound to have those dynamics once the radical possibilities of the Free Software movement were recuperated by capitalism in the form of Open Source.
The article touches on an important point: the set of open-source developers includes PyPI maintainers. A related distinction is that yes development effort required matters, but so does operational effort required. By enforcing 2FA, PyPI reduces their support burden a bit by not having to deal with account takeovers, worrying about account takeovers, and responding to account takeovers. Yes, by mandating 2FA that increases the developer's effort, but by refusing to use 2FA that increases the operational effort of PyPI. There's probably a discussion to be had about how much can PyPI lower its level of effort by large amounts by imposing small increases in effort on developers, and whether those effort values are large or small or whatever, but in this particular case I'm inclined to support the small amount of developer effort required to massively reduce the operational effort of both PyPI and everyone responsible for vetting packages for use
That's a very good point regarding operational cost of handling account takeovers.
I'm not sure I have much useful commentary to add, but it does occur to me that a sufficiently-sized pool of software users could inspect changes (either at individual-commit-time and/or at tagged-release-time) regardless of whether each changeset is by the same author or in fact a different person every time.
writing open-source doesn’t mean people can’t criticize you. It doesn’t mean the places you publish your software can ask you to do stuff or they’ll un-publish. It doesn’t mean others can’t republish them yourself.
It does mean you can ignore any and all criticism and do whatever you want. It does mean you can refuse to do what the package managers or community want. Even if your product has a glaring bug or security hole or is offensive! And you can put the burden of maintaining or patching your app to others.
And that’s fine, if you do so you’re not a bad person. Because others can maintain and patch your app. You have absolutely no obligation to maintain or address any complaints about your open source software.
But you do have to accept that people will make those demands and complaints. Not outright insults and hate speech, those are too far, but complaints about your software (even if harsh) are fine and expected. If they bother you, IMO the best thing to do is politely ask “please stop” and ignore them.
Choosing to use FOSS software to build products/services has always involved an element of caveat emptor, and even with the best of intentions, mistakes and errors are introduced sometimes, as they can be into any commercial software.
The technology industry (as the typical consumer of FOSS) generally understands that and introduces appropriate measures (dependency reviews, hiring developers with relevant experience, requesting professional security audits, keeping backups, ...).
Despite all those (sometimes expensive) measures, industry continues to develop (and indeed thrive) using FOSS, implying the trade-off is worthwhile. My guess is that it is in fact massively worthwhile, especially when comparing the technology economics of today with years and decades past.
Therefore I think it's reasonable to ask questions any time that barriers are raised -- however small -- on the production-side of FOSS. That's not where the bulk of the revenues are accruing.
(I also have a vague sense that 2FA could later be misused as an attempt to strongly-attribute blame, which again feels potentially unfair/unbalanced. if your business risk is high when upgrading packages, then you should review those updates more carefully and keep a record of the financial efforts and rewards)
Offering an opinion: the tech industry is invested in the success of PyPI -- perhaps not always in a literal monetary sense, you're right, but certainly in an ecosystem sense.
I keep seeing this claim that open source maintainers shouldn't be upset about a 3rd party mandating we increase our maintenance burden to follow their new rules, because it's all about responsibility.
I think we need to accept that it's not a simple matter of responsibility. It's a question of how much maintenance burden is OK to mandate. The claim that open source maintainers should be fine with this increased burden because "responsibility" is not logically sound.
Also, the idea that there's no problem, because OS maintainers can just remove their package from pypi, totally ignores the point that that would be a really bad outcome for everyone that uses that package. I this that's actually a pretty big problem.
> Also, the idea that there's no problem, because OS maintainers can just remove their package from pypi, totally ignores the point that that would be a really bad outcome for everyone that uses that package. I this that's actually a pretty big problem.
Is definitely not as big a problem as someone taking over their account and publishing malware under their name is.
Perhaps -- but not "definitely". It depends on what the frequency of each of those things is.
So far IIUC the frequency of critical projects being removed from pypi is 1 (in the space of a few days - although it came back), and account takeover of critical projects for malware publishing is 0 (in the space of many years).
The Atomicwrites maintainer/author has been using PyPIs servers and infrastructure to host and serve his project for free.
PyPI have no responsibility to do that, and that maintainer has always been free to publish his project himself. Instead the Atomicwrites maintainer doesn't want to do that, what they want is to be able to exploit someone else's infrastructure, and they want free rein on that infrastructure despite, again not paying a cent for that infrastructure. There is a weird belief that PyPI is the side of this equation that is exploiting the maintainer - by requiring literally the bare minimum of modern security - and not the Atomicwrites author exploiting the PyPI infrastructure to host their project.
Anyone arguing that PyPI is not allowed to make any demands of someone using their infrastructure, for free, because that is somehow unreasonable, is delusional. It's no different than having your own project, and someone push a PR, and you being required to take the change - after all they gave it to you for free.
> Anyone arguing that PyPI is not allowed to make any demands of someone using their infrastructure, for free, because that is somehow unreasonable, is delusional. It's no different than having your own project, and someone push a PR, and you being required to take the change - after all they gave it to you for free.
He's not arguing they're not allowed. He admits they are allowed, but he doesn't want them to do it. He thinks it's a bad idea.
I don't agree with him, but I think you are misrepresenting his views here.
The atomic writes maintainer was free to simply stop using their hosting service, but what he did was try to continue to use their hosting service without doing the bare minimum to protect that service.
So I don't believe I am misrepresenting them, nor the people who have defended their actions. They have made it _very_ clear that they believe PyPI should be forced to host every project whether they want to follow the rules or not.
> He was free to simply stop using their hosting service, but what he did was try to continue to use their hosting service without doing the bare minimum to protect that service.
the maintainer of atomicwrites, the person that started this whole saga, I guess that I had assumed context was sufficient here. I've updated the original text.
No I haven't. This maintainer got told that he needed to use 2FA for PyPI, instead of doing that most basic of actions he had a tantrum and removed his project:
I happen to work with that person so I had the enjoyment to watch this unfold as it happened. The maintainer thought the package would remain after deletion for existing users as it does for instance on crates.io. I also know that he uses 2FA.
With the belief that everyone is well intentioned and wants whats best for everyone, I think the thing that freaks me out about this whole thing is that through some process a developer has had their work declared 'critical'. If PyPI said this was a new requirement for everyone, then it would be make more sense, but the sudden declaration just seems problematic. I get there is a resource issue, but no one likes getting an e-mail that they're now subject to new rules not applied to others.
So you maintain a package 'X', that has been determined to be 'critical'. It just seems like some user of package 'X' needs to perform some action to insure the group safety. It just seems like the one party that benefits the most isn't doing anything to help the situation. Thinking about it, what if the three users of your package are the credit agencies? The whole criteria seems a bit arbitrary.
But there’s a reason why the slippery slope is a fallacy
> What determines if a project is a critical project?
> PyPI determines eligibility based on download counts derived from PyPI's public dataset of download statistics. Any project in the top 1% of downloads over the prior 6 months is designated as critical.
Its still not a great measure. A project with three downloads that are all from the credit rating agencies, or a project with 100 downloads from all banks would be considered 'critical' by most folks.
But not by PyPI! Whoever is in charge of supply chain security at those credit rating agencies and banks should think long and hard about what they've gotten themselves into, though.
I'm a bit unclear about your response. My contention is that the method of determination of critical is based on something that really doesn't tell you how critical it is. I do believe the customer has a responsibility but I am not seeing any of that in PyPI's actions.
Do I blame or think PyPI did anything wrong? No. I think everyone has the best intentions. I just think arbitrarily declaring someone work 'critical' without some involvement from the users of that developer's work is going to cause problems and not actually solve the issue. PyPI doesn't really have access to the information needed to declare something critical.
What I meant by my reply is that PyPI is using "critical" as a label for packages that meet a specific criterion. I agree that "critical" as an English word can be rather vague, but it is not in this case.
PyPI's interest here is in the integrity of accounts that can publish PyPI's most widely downloaded packages. If you believe based on the chosen label that in the future, PyPI will come in and expand the criteria for criticality, then I guess that's possible? But it's not what's happening now.
It's not arbitrarily because you've been told the criteria already. The problem you have is that you disagree with criteria which, well, not sure what to tell you except that there are no universal conditions everyone would agree to as clearly just demonstrated.
PyPI maintainers selected their own, as they have every right to and they are not obviously stupid as they seem to be a reasonably good proxy for which packages would afflict most developers.
Personally, as a developer who both publishes and uses packages from PyPI, I'd love to know who finds 2 minute 2FA set-up too burdensome (and if you save it in your password manager, that's all you'll every have to do) so I can avoid their packages. I have little faith in maintenance of packages for which a minimal one-time effort (per account, not even per package) is too big.
It's not arbitrarily because you've been told the criteria already
It's arbitrary because it doesn't seem to correspond to what is 'critical'. It might be an ok proxy, but I'm not really convinced of that either. I once again say I think everyone is being well intentioned, but I do not think they actually have enough information to build out a less than arbitrary criteria.
Personally, as a developer who both publishes and uses packages from PyPI, I'd love to know who finds 2 minute 2FA set-up too burdensome (and if you save it in your password manager, that's all you'll every have to do) so I can avoid their packages. I have little faith in maintenance of packages for which a minimal one-time effort (per account, not even per package) is too big.
If it is not burdensome, then they probably should of just required it from everyone going forward. It would have saved any debate as to singling out individuals for a higher maintenance demand.
As a non-native English speaker I will not argue about proper use of arbitrary. I will reiterate though that there is no universal criteria for what critical means and plausibly theirs (in sense that I am speculating) is not an unreasonable one: packages when compromised would affect the most and enough developers directly as they are the primary users of the packages and everyone else is downstream.
Downloads admittedly are not a perfect measure as it is not a fixed ratio between number of developers and downloads, but it is again a reasonable one as it is highly unlikely that with expected power law distribution of popularity the top 1% would not be also widely used.
The remaining quibble could be the cut off at 1% which I assume was derived from data and not an infatuation with 1.
I doubt mandating 2FA for all would save any debate at all as it seems mainly to be centered on "why are they doing this to me" and not "why am I being singled out", but personally I certainly wouldn't have a problem if they did. I certainly would prefer to know which packages are better protected than others.
There's also a reason why one wouldn't mandate it which is to make first steps in publishing easier for beginners with expected audience of only them.
I also share James' perspective that our obligations change with other people relying on us. However even if you don't, you are not forced to accept it. You only won't be able to publish new versions of the package, but you can always rename it and publish that if 2FA is really such a burden.
Number of downloads correlates strongly with "centrality," i.e. how highly connected (in the dependency graph) a package is to other packages. That, in turn, maps pretty well to "criticality," if your definition of "critical" is "a malicious takeover of this package would compromise large swaths of the Python ecosystem."
I am totally on board with critical packages offering enhanced security to protect the user.
But, I am deeply disturbed about the fact that we've reached this point where "enhanced_security === 2FA" by default (which I hate with a passion), with no alternatives considered.
118 comments
[ 2.6 ms ] story [ 188 ms ] threadCompany C uses database driver P. On 2020-01-02 they upgraded their app to the latest version. On 2020-01-04 they noticed that someone stole all their user data. Upon investigation, Package P had a version uploaded on 2019-12-24 that sends all user data to evil.example.com. The tooling to upgrade the package doesn't show diffs, so company C had no way to detect this malicious change. How did such a compromised version get uploaded? Looking at the audit logs, it appeared that there were 2398438 unsuccessful login attempts to package P author's account, before finally uploading the new patch release from the same IP. Company C lost 8 billion dollars as a result of this. If author A had used multi-factor authentication, then this wouldn't have happened, because it would have taken more login attempts than there are atoms in the Universe.
Then the author can see "hey I can save the shareholders of some random corporation 8 billion dollars if I make it harder for myself to release software". That's a better incentive than "because I told you to".
I know it sounds a bit disingenuous to say this is just for the shareholders of random corporations. I'm being a little snarky. It's also good for your reputation to not get hacked, and using 2FA to log in every time is probably less time overall than reacting to a single compromise. Imagine how many emails you're going to get. Big pain. I just think it's important to show authors the cost/benefit. 2FA is easy. Next time it might not be something that's as easy, though.
Wouldn’t they have led with, “we’ve witnessed X attempts and Y takeovers in the past year, so we’re introducing new security measures aimed at…”
To me, this seems like more frog-boiling authoritarianism “for our own good”, like the COVID lockdowns, censorship, etc.
I’m tired of technology forcing me to do things “for my own good” — and I recommend against companies who engage in those policies.
These are all the same arguments trotted out to justify Windows spyware and forced updates that routinely lag my computer for hours — because Microsoft knows better than me how to use my desktop and isn’t shy about pushing me out of the way to do so.
But if you quantify it like this, "this company saved 8 billion dollars out of some extra work on your side, here is a pat in the back because they'd never even consider give back a single cent or helping in any way" it feels a bit strange. I know it's on me, and I just learned this contradictory feeling so need to unravel/understand it, but you def got the opposite feeling from me as expected.
Edit: you added the "I know it sounds a bit disingenuous to say this is just for the shareholders of random corporations." later, right? Which kinda reflects a bit of this feeling
Personally, I don't have a horse in this game. I always use 2FA when it's available. I have already made the necessary adjustments in my workflow. But I get why people are annoyed that they have to change something. Imagine that this was "we'll send you a check for $1000 if you enable 2FA on your account". Bet there would have been zero angry blog posts.
That is just entitlement on the part of some open source devs. PyPI is already providing a huge service for them: it is publishing their package free of charge. Furthermore, it is maintaining old versions, scanning others for vulnerabilities, replicating across the world, resolving dependencies etc. They have every right to ask anyone who is enjoying their completely free service to put in the minute amount of extra work required to set up 2FA.
Of course, anyone is also free but to want to put in that work, and so they can stop delivering their package on PyPI. Absolutely your right as an open source maintainer, and you have no one to answer to if that breaks some builds.
The high-profile NPM package compromises in recent years have mostly targeted hobbyists/been indifferent to who they were targeting. Crypto miner malware doesn't care what computer it's installed on.
As far as I know common targets for malicious packages are crypto (mining, wallet swiping), and credentials (SSH, package repositories), none of which is limited to businesses. There's been some cases of politically motivated sabotage lately but mostly from the maintainer themselves so far - expect more of that though.
Often a compromised package will try to steal more credentials from downstream users. [1] I don't doubt that threat actors all the way from script kiddies to nation states are keeping a stash of credentials to package repositories, and it's going to suck for everyone when they decide to use them.
[1] https://www.bleepingcomputer.com/news/security/compromised-j...
When I write open source software I'm not doing it for Bezos. I'm doing it for Heather down the street who is trying to get a cool little project off the ground and maybe pay some bills. I'm even writing it for Frank who runs a modest 20 person dev shop and maybe contributes back a bit here or there where they find the warts.
I'm not building it for the psychos that have lost all touch with humanity.
PyPI asking for 2FA is in no way different from an open source project asking you to run their unit tests before submitting a PR. Sure, it puts a burden on you, the open source contributor, but it is balanced by removing a burden from the project maintainers.
It's the responsible thing to do. If a package gets compromised its PyPi that has to do cleanup work. It's like complaining that the government forces you to wear your seatbelt.
Also, conveniently, it's the thing whose cost is not paid by the people deciding to enforce it.
> If a package gets compromised its PyPi that has to do cleanup work.
Yeah, that's the very essence of being a distribution tool.
That's one of the reasons I believe 2FA is mostly security theater. That and people insisting they need their shitty app on my unsecured phone.
It's not two-factor if all of it is running on the same hardware.
It's something you know + something you own. That's 2FA.
Everyone can agree that password managers are a good thing. However, it's debatable that, in many setups including the one that was just described, 2FA brings added security.
>Yeah, that's the very essence of being a distribution tool.
Also, conveniently, it's the thing whose cost is not paid by the people decided to get owned.
And again, the government doesn't cover the cost of your seatbelt, so I don't see your argument. PyPi also makes you use passwords, should I complain that they are forcing me to buy a keyboard?
To be quite honest, I would rather the shareholders of some random corporation do lose 8 billion dollars. In fact, I would be willing to pay a small amount to see it.
https://news.ycombinator.com/item?id=32037562 "Congratulations: We now have opinions on your open source contributions"
https://news.ycombinator.com/item?id=32026624 "Atomicwrites' old versions have been purged from PyPI"
https://news.ycombinator.com/item?id=32058053 "PyPI is rolling out 2FA for critical projects, giving away 4k security keys"
Among them:
* Project installs follow a power law distribution: the top 1% of packages account for 99% of installations, and the remaining 99% account for roughly 1% of installations. In other words: prioritizing the security of the 1% of packagers means prioritizing the 99% of users who depend on those critical projects.
* PyPI has tried really hard to make 2FA as effortless and non-invasive as possible: the current program includes giving away two free physical hardware tokens, as well as a long grace period to ensure that maintainers are not caught in a sudden lurch between their open source and professional obligations. Both of these don't work at scale: PyPI doesn't have the material resources to give every maintainer free hardware tokens, and the same packaging power law means that a large number of relatively inactive maintainers will be caught by any deadline that gets set.
Ultimately, the goal is to maximize the number of users protected, maximize the quality of protection, prevent package user disruption, and minimize maintainer disruption. The current scheme, in my view, does a good job at achieving all of these goals.
FD: I worked on PyPI's 2FA implementation, but I do not represent PyPI and am not a maintainer.
I mean, node-ipc would have happened regardless of how many authentication factors were involved in uploading. And offers to buy ownership will keep coming. What happened to Sourceforge wasn't lack of security.
Turns out those distribution people did more than just wrap software in other formats. Turns out that multi stakeholder thing had merit. Who would have thought?
Quality control belongs more closely to the "integrity" category of threats, which is much broader in scope than account takeover. It's something that we're thinking about as part of designing additional security schemes for PyPI; you can follow some of that work under the PEP 480 and PEP 458 umbrellas.
Agreed.
I don't know if these projects need 2FA "protection" or not - but I'm quite certain that the risk to any particular user is identical whether they are downloading a popular vs. unpopular package.
Total, global risk is, of course, higher for popular packages, but any individual user does not care if their intrusion vector was a popular vs. unpopular package ...
I feel like this is a rather silly take. Book-writing, speaking, etc opportunities are not ongoing burdens like open source maintainership. Even job opportunities you can choose to quit. I think ultimately if people are doing things for free you can't ask too much of them since they are well within their right to stop doing it for any capricious reason.
In particular, you can quit maintaining an open source package, or stop publishing it in an index and let someone fork it. (Maybe you feel a little guilty, but that seems like a feeling to get over if it's not working out for you.)
But it still works both ways. If you're not finding it fun anymore, that doesn't say anything in itself about who, if anyone, is to blame.
> PyPI asks for 2FA today, what might they ask for tomorrow?
Yes, slippery slopes are real because they happen, especially in this very tense and very non-rational geo-political climate. Yes, today it might just be "use 2FA with you still want to actively maintain your own package", tomorrow it might be "boo-hoo-hoo, you've followed/liked Putin/Xi/whoever the powers that be don't like, you're a threat to liberal democracy, we can't leave critical infrastructure in your hands, bye-bye", and there will be nothing for the package maintainer left to do at that point.
You must adopt a COC. You must rename your branch from "master" to "main". You must use "inclusive" language in your documentation.
Why? Because you must assume the responsibility for everything your project touches. A bad person uses your leftpad.js to perpetuate badness. You must stop them. Someone might get their feelings hurt. You must preemptively protect them.
This is an impossible task. 2FA is fine, sure. I use it myself, and advocate for others to do so as well.
But I'm sick and tired of the "harm reduction" mantra that means I have to entirely rewrite my documentation every 6 months.
FWIW, I have multiple open source projects. I don't have a COC. I haven't renamed any of my branches. I haven't gotten any pushback.
Look, publishing to PyPI is not a right. If typing 6 digits is too much burden you can drop a source tarball in some web page and call it a day, you don't have to go through PyPI. Done.
I, for one, I'm very very glad that a piece of _critical infrastructure_ that I depend on is a bit more secure.
The 2FA thingie is not implemented because of some random dude who might want to steal a bitcoin or two, is implemented against state actors. This being the US we're talking about (and the West more generally) the state actors taken into consideration are: China, Russia, Iran, probably North Korea (probably in this order). Hence my mention of China and Russia in my initial comment.
> And if your package gets established and has people regularly using it, you have some responsibility not to mess with those users. If you want to make backwards-incompatible changes, for example, communicate them and bump the major version number. If you want to stop maintaining the package, post an announcement about it and be willing to hand over to someone else to continue work.
If and when they start policing political views of their maintainers, it would be fair to criticize them. But the theoretical possibility of such a thing happening has no relevance to whether 2FA is a reasonable request.
TOTP is defined in a freely available RFC (https://www.rfc-editor.org/rfc/rfc6238) and is implemented by a wide variety of applications and libraries, both proprietary and open source.
Just FYI, you don't need a smartphone to use TOTP. You can download 'pyotp' from PyPI and use it with any python interpreter. Anybody publishing packages to PyPI would have access to that.
from the linked article, i'll just leave that here
As a security engineer, this is the thing I don't get about dissenters. Any account I actually care about has 2FA on it.
By today's security standards, NOT adding 2FA just means you don't really care about your project. Hand it over to someone else in that case.
(I am sure it doesn't apply to PyPI, but I am just saying both site operators and users need to adopt best practices in terms of 2FA)
Is "that would be a terrible world" supposed to be a counter-argument? The world is built on caveat emptor, especially when downloading random packages off the internet. That's why browsers are sandboxed. That's why people run virtual machines. The vetting process for adding new dependencies at big corporations can literally last months. The 2FA nonsense does basically nothing in practice (vetting will not and should not magically go away), and only adds an undue burden on developers.
The internet is a dark forest and I think it would do us a bit of good to treat it as such.
This is true, but to play devil's advocate, how many HN articles have we seen where X critical NPM package had been compromised by a malicious user who gained control of the package maintainer's account? There are many examples of this. If a new package deployment system were to launch in 2022, I wouldn't be surprised if 2FA was required for all publicly listed packages, full stop, not just critical ones. Viewed through that lens, it really isn't that unreasonable.
That said, I think full power should be in the hands of the developers who maintain these packages when it comes to the actual content of the packages. Package systems should not be editorializing. If the author of [insert major package here] wants to upgrade a version that replaces the whole package with a console animation of a turtle, that's his/her choice as a developer, and simply should be viewed as a "breaking change". Intervening in THOSE scenarios I view as completely out of control on the part of package ecosystems. Let people leverage their creations however they like. Your trust is supposed to be in the maintainer. If they aren't trustworthy, you shouldn't use their software in the first place. If they decide to go in a new, breaking direction, that's their decision. Caveat emptor.
These scenarios though, are starkly different from scenarios where the package maintainer's own intent is subverted by a malicious hacker / user. That's a whole other can of worms, and I agree we need better controls to protect all packages from these types of scenarios.
If we don't trust the provenance of the code we're getting though, that reputation becomes irrelevant. As such, this is a problem of identity assurances and access management, it can't be solved otherwise.
- It should be possible to compare between two releases (I'd personally like to see a code diff, ideally with a complete path of the commits involved)
- Providing a reputation visibility mechanism (for publishers? author(s)?) across a series of releases is important
Those don't require user accounts necessarily, though. And responding to the end of your message: identity assurances, yep, those seem necessary; access management, I'm not so sure.
For instance, PyPI will take down your software if we determine it to be malicious in some way, which is extending editorial control over what's on PyPI.
Generally speaking, though, PyPI does not concern itself with the contents of the packages shipped through it as long as they are reasonable packages. If a project wants to break compatibility, then that's on them. We strive to let projects manage themselves, in whatever way they think makes the most sense.
Where this breaks down and gets tricky I feel, though, is when you get into examples where the package maintainer makes the newest version essentially useless (but in an otherwise harmless way) in protest of some grievance. I think this is a perfectly legitimate action for a maintainer to take as long as they aren't literally installing malware or opening up security vulnerabilities. Package systems stepping in in these scenarios and "bailing out" all the companies that rely on the package by taking it over are effectively silencing the maintainer. My smell test for this is if the ISP or hosting provider wouldn't yank it, neither should the package system. Hosting providers will take down malware and illegal content, but they aren't going to do more than that, and that's how it should be imo in package systems. I think the benefit of developers' voices not being silenced vastly outweighs the humorous scrambling of a bunch of companies who failed to lock their version numbers for a few short hours. The same scramble happens all the time with real-world breaking changes anyway. If the maintainer wants to effectively yank their project, they should be allowed to do so. Otherwise you never really know if you are getting the maintainer's code, or whatever the package system decides should be the maintainer's code, which breaks the trust contract upon which package systems are built.
This is another reason it's great that a lot of languages, like Crystal for example, have adopted a decentralized model.
I would say that in the > 10 years of time I've been an admin on PyPI and even longer that I've been involved in Python's Packaging, I've never once seen anyone, but individuals placed front and center in who we're attempting to serve first and foremost.
Businesses and other large orgs do come up from time to time, and we've generally been pretty good at making decisions that allowed those businesses to solve their problems on their own, without requiring the rest of the ecosystem to do it for them. In other words, our features we add for businesses tend to be unblocking them from being able to implement something they want, rather than taking on the responsibility of whatever thing they want.
I don't understand this claim -- are you saying that 2FA adds no protection of value in the context of package maintainership, or that does nothing in general? I'd like to understand the reasoning behind either, whichever it is.
My 0.02c is that 2FA has empirically reduced the incidence of ATOs on major services, and eliminates entire systemic weaknesses in account systems (like ATO via password reset). It stands to reason (to me) that PyPI is a service like all others, and benefits similarly.
I'm trying to say two things: (1) that 2FA adds little, if any, protection of value in the context of package maintainership. In other words, if someone wants to be a malicious actor (say, they had a "legitimate" package from versions 1.0 to 2.1, but 2.2 contains a trojan), they can do it even with 2FA. And (2), it adds an undue burden developers, particularly brand new ones that just want to "publish their package," have to deal with. If you're 15 or 16 years old, excited to finally get your package on PyPI, but all of a sudden, you have to jump through all these hoops, including doxxing yourself and providing a real phone number, you would 100% be deterred.
That's a different threat model. We're talking about account takeover, which is a separate risk from maintainers going rogue. 2FA addresses the former; the latter is difficult to address in the general case. Nevertheless, it's something that people are tackling both with static analysis and "reputational" schemes.
> it adds an undue burden developers, particularly brand new ones that just want to "publish their package," have to deal with
You're right, which is why it's not mandatory for everyday users. The decision to enable 2FA for "critical" packages is a practical and calculated one: most critical packages are maintained not by brand new PyPI users, but by seasoned engineers who understand their centrality to the ecosystem and the importance of measures that secure their work. There's no current plan to make 2FA mandatory for every single user; the threat model doesn't justify it.
> including doxxing yourself and providing a real phone number, you would 100% be deterred.
I'm not sure what this has to do with PyPI. PyPI does not and has never supported SMS 2FA. This was an intentional design decision on my part.
The only supported 2FA methods are TOTP and WebAuthn, both of which can (and do, by default) preserve nominal anonymity. No phone numbers are involved.
1. Doesn't protect against the developer actually being malicious
2. Is able to be broken, especially when both factors occur on the same mobile device.
3. Doesn't do anything to prevent the theft of API keys from CI/CD builds.
2FA exists to make authentication more difficult for an impostor or attacker. The goal is to prevent account takeover, which has historically been a risk to critical packages in multiple ecosystems.
(2) is sort of ambiguous. I’m not aware of a major case of either TOTP or WebAuthn being compromised en masse because it was on the same device as the first factor. I’m not even aware of a case where WebAuthn has been meaningfully attacked, full stop, despite theoretical breaks of hardware stores. Even with a full break, WebAuthn provides detectability: cloning the factor means cloning the counter, which will alert the user on the next login. On whole, I think it's incorrect to frame 2FA as generally breakable without qualifying the threat and explaining how it applies to PyPI users.
Very well then. Do whatever you think is necessary and appropriate.
For you. Don't heft your ideas on other people.
Open source has been built on an endless stream of people making personal sacrifices and often burning out. Seeing friends slowly getting devoured by anxiety crisis or burying themselves to support commercially used software, barely being thanked for what they give to the community, has been one of the most depressing parts of this industry.
If you don't support that particular piece of infrastructure, your opinions on it are worth zero. Even if you're a well-known open source maintainer, and some times even more so. It is understandable that people getting a comfortable job thanks to their contribution on a high-profile project have a different view from people that don't, but that doesn't make it a better view.
In retrospective, open source was bound to have these dynamics, with some people benefiting from their open-source work not seeing why people not paid to participate don't enjoy being told what to do. I wonder whether we'll see a friendlier environment in the future, or whether open-source will end up being a cooperative of companies sharing code among themselves.
I want to describe to you that this is very difficult - almost impossible.
A long time ago I decided that I did, in fact, want my opinions on open source and free software to have more weight than a typical non-contributor and I have made dozens of valid, reasonable - even generous - offers to individuals and projects to advance certain design goals or feature requests - or even simple bug fixes.
It's not that these offers (contract offers, bounties, straight up cash payments, sponsorship, etc.) were rejected - they are typically outright ignored. As in, radio silence. No response whatsoever. I know they got my message and I know they are ignoring it.
I have some pop-psych / cultural analysis / personal anecdata as to why this is (almost always) the case ... I think you can easily sketch a caricature of someone who does something out of love or passion or as a personal challenge but has no way to connect that to expectations or deadlines, or any other personal involvement whatsoever.
But whatever the explanation, be aware that your suggestion is not the magic bullet you think it is.
In particular, it was bound to have those dynamics once the radical possibilities of the Free Software movement were recuperated by capitalism in the form of Open Source.
I'm not sure I have much useful commentary to add, but it does occur to me that a sufficiently-sized pool of software users could inspect changes (either at individual-commit-time and/or at tagged-release-time) regardless of whether each changeset is by the same author or in fact a different person every time.
It does mean you can ignore any and all criticism and do whatever you want. It does mean you can refuse to do what the package managers or community want. Even if your product has a glaring bug or security hole or is offensive! And you can put the burden of maintaining or patching your app to others.
And that’s fine, if you do so you’re not a bad person. Because others can maintain and patch your app. You have absolutely no obligation to maintain or address any complaints about your open source software.
But you do have to accept that people will make those demands and complaints. Not outright insults and hate speech, those are too far, but complaints about your software (even if harsh) are fine and expected. If they bother you, IMO the best thing to do is politely ask “please stop” and ignore them.
The technology industry (as the typical consumer of FOSS) generally understands that and introduces appropriate measures (dependency reviews, hiring developers with relevant experience, requesting professional security audits, keeping backups, ...).
Despite all those (sometimes expensive) measures, industry continues to develop (and indeed thrive) using FOSS, implying the trade-off is worthwhile. My guess is that it is in fact massively worthwhile, especially when comparing the technology economics of today with years and decades past.
Therefore I think it's reasonable to ask questions any time that barriers are raised -- however small -- on the production-side of FOSS. That's not where the bulk of the revenues are accruing.
(I also have a vague sense that 2FA could later be misused as an attempt to strongly-attribute blame, which again feels potentially unfair/unbalanced. if your business risk is high when upgrading packages, then you should review those updates more carefully and keep a record of the financial efforts and rewards)
But I also see people claiming that it's not OK for the UK to require an increased maintenance burden for the same reason: https://news.ycombinator.com/item?id=32055756
I think we need to accept that it's not a simple matter of responsibility. It's a question of how much maintenance burden is OK to mandate. The claim that open source maintainers should be fine with this increased burden because "responsibility" is not logically sound.
Also, the idea that there's no problem, because OS maintainers can just remove their package from pypi, totally ignores the point that that would be a really bad outcome for everyone that uses that package. I this that's actually a pretty big problem.
Is definitely not as big a problem as someone taking over their account and publishing malware under their name is.
So far IIUC the frequency of critical projects being removed from pypi is 1 (in the space of a few days - although it came back), and account takeover of critical projects for malware publishing is 0 (in the space of many years).
The Atomicwrites maintainer/author has been using PyPIs servers and infrastructure to host and serve his project for free.
PyPI have no responsibility to do that, and that maintainer has always been free to publish his project himself. Instead the Atomicwrites maintainer doesn't want to do that, what they want is to be able to exploit someone else's infrastructure, and they want free rein on that infrastructure despite, again not paying a cent for that infrastructure. There is a weird belief that PyPI is the side of this equation that is exploiting the maintainer - by requiring literally the bare minimum of modern security - and not the Atomicwrites author exploiting the PyPI infrastructure to host their project.
Anyone arguing that PyPI is not allowed to make any demands of someone using their infrastructure, for free, because that is somehow unreasonable, is delusional. It's no different than having your own project, and someone push a PR, and you being required to take the change - after all they gave it to you for free.
He's not arguing they're not allowed. He admits they are allowed, but he doesn't want them to do it. He thinks it's a bad idea.
I don't agree with him, but I think you are misrepresenting his views here.
So I don't believe I am misrepresenting them, nor the people who have defended their actions. They have made it _very_ clear that they believe PyPI should be forced to host every project whether they want to follow the rules or not.
Who is "he"?
https://news.ycombinator.com/item?id=32026624
So that he could continue to use PyPI's infrastructure to publish his work for free.
So you maintain a package 'X', that has been determined to be 'critical'. It just seems like some user of package 'X' needs to perform some action to insure the group safety. It just seems like the one party that benefits the most isn't doing anything to help the situation. Thinking about it, what if the three users of your package are the credit agencies? The whole criteria seems a bit arbitrary.
But there’s a reason why the slippery slope is a fallacy
Yeah, if anything, its a certainty not a fallacy.
> What determines if a project is a critical project?
> PyPI determines eligibility based on download counts derived from PyPI's public dataset of download statistics. Any project in the top 1% of downloads over the prior 6 months is designated as critical.
Do I blame or think PyPI did anything wrong? No. I think everyone has the best intentions. I just think arbitrarily declaring someone work 'critical' without some involvement from the users of that developer's work is going to cause problems and not actually solve the issue. PyPI doesn't really have access to the information needed to declare something critical.
PyPI's interest here is in the integrity of accounts that can publish PyPI's most widely downloaded packages. If you believe based on the chosen label that in the future, PyPI will come in and expand the criteria for criticality, then I guess that's possible? But it's not what's happening now.
PyPI maintainers selected their own, as they have every right to and they are not obviously stupid as they seem to be a reasonably good proxy for which packages would afflict most developers.
Personally, as a developer who both publishes and uses packages from PyPI, I'd love to know who finds 2 minute 2FA set-up too burdensome (and if you save it in your password manager, that's all you'll every have to do) so I can avoid their packages. I have little faith in maintenance of packages for which a minimal one-time effort (per account, not even per package) is too big.
It's arbitrary because it doesn't seem to correspond to what is 'critical'. It might be an ok proxy, but I'm not really convinced of that either. I once again say I think everyone is being well intentioned, but I do not think they actually have enough information to build out a less than arbitrary criteria.
Personally, as a developer who both publishes and uses packages from PyPI, I'd love to know who finds 2 minute 2FA set-up too burdensome (and if you save it in your password manager, that's all you'll every have to do) so I can avoid their packages. I have little faith in maintenance of packages for which a minimal one-time effort (per account, not even per package) is too big.
If it is not burdensome, then they probably should of just required it from everyone going forward. It would have saved any debate as to singling out individuals for a higher maintenance demand.
Downloads admittedly are not a perfect measure as it is not a fixed ratio between number of developers and downloads, but it is again a reasonable one as it is highly unlikely that with expected power law distribution of popularity the top 1% would not be also widely used.
The remaining quibble could be the cut off at 1% which I assume was derived from data and not an infatuation with 1.
I doubt mandating 2FA for all would save any debate at all as it seems mainly to be centered on "why are they doing this to me" and not "why am I being singled out", but personally I certainly wouldn't have a problem if they did. I certainly would prefer to know which packages are better protected than others.
There's also a reason why one wouldn't mandate it which is to make first steps in publishing easier for beginners with expected audience of only them.
I also share James' perspective that our obligations change with other people relying on us. However even if you don't, you are not forced to accept it. You only won't be able to publish new versions of the package, but you can always rename it and publish that if 2FA is really such a burden.
But, I am deeply disturbed about the fact that we've reached this point where "enhanced_security === 2FA" by default (which I hate with a passion), with no alternatives considered.