35 comments

[ 3.4 ms ] story [ 61.8 ms ] thread
Very reasonable other side to this story, which doesn’t come as much of a surprise. Too bad it didn’t hit the front page.

People went WAY too far WAY too fast on this. There HAS to be urgency to this, the software supply chain is presently, undeniably, under attack.

Frankly, everyone blasting RubyCentral the last few days should feel shame and embarrassment. These aren’t evil suits at Microsoft, they’re normal people invested in maintaining a critical piece of infrastructure for the good of all who love and profit from Ruby.

So Ruby Central, by their own admission, agreed to take $$$$$ of funding on the premise that they would "secure RubyGems against supply chain attacks", and then sat on their hands not doing anything about it until a few days before the deadline, when it was too late to seek community consensus or figure out a good transition plan. So they ended up screwing over everybody who was actually doing work on the project in favor of their own funding. And also they apparently used this as an opportunity to consolidate their power in other ways (renaming the github org) for reasons that were unrelated to the self-imposed deadline. How does this make them look better?
To my untrained eye it looks like a board with a bunch of money and perhaps a fork on their hands.
This definitely has a Mullenweg-esque scent to it.
> [The Ruby Central board] is a small group of volunteers

is somewhat at odds with

> Some [...] companies specifically pay Ruby Central to ensure the security and stability of that part of the supply chain,

but not so much. Then the sentence goes on with

> but then discovered that people with no active affiliation or agreement in place had top level privileges to some of this critical infrastructure.

So something has been wrongly managed or wrongly sold.

Then the final part about the emotional conversations and the dilemma sounds honest or at least very plausible, but as they write, the critical mistake already happened.

> I can't speak for the board or the Ruby Central staff. But I know them and they are like me. They do this because they love Ruby and our community. I'm certain of that.

I don't know how to reconcile 'they love Ruby and our community' with moves that are actively hostile to the community.

I think that if they had been up front and transparent, and cut the PR bullshit corpospeak from their damage-control post, this would have been something that's much less embarrassing for all involved.

Something like:

"Hey all, RC here: with the very real threat of supply-chain attacks looming around us, one of the critical financial backers of our nonprofit org gave us a deadline around tightening access to the Github Account for rubygems/bundler. We tried and failed to arrive at a consensus with the open-source volunteers and maintainers for the best path forward and were forced to make a decision between losing the funding and taking decisive (if ham-fisted) action to keep Ruby Central financially healthy. We think RC's continued work is important enough that we stand by our decision, upsetting though it might be, but want to work out a better one ASAP. We are genuinely sorry for any fear/disruption this has caused."

Something simple that just owns the fact that they screwed up and tried to handle it as best they could. Doing this proactively as soon as they made the changes and broadcasting it would have been even better, but even posting this in reply to the controversy would have done more imo...

Locking out a guy like David Rodriguez (the main person I see doing bundler commits) in a dramatic fashion just seems like absolute craziness. I can't fathom doing it without a very good reason, which has yet to be revealed if it exists.
I'm truly hoping for a reasonable resolution on all sides for this situation. IMO Ruby is too small, and shrinking compared to Python and JS/TS especially in the AI era, to be able to afford any splintering of efforts.
It's such a weird thought process to have gone through, to write this. The sentiments expressed are basically:

"I WANT to apologize ... that I feel awful."

"How can you possibly talk to someone about changing access, when multiple people tell you no, you are wrong?! A coup is the only way!"

"Because funding deadline, we executed a coup, which will keep everyone safe from hostile actors... Taking over accounts and access"

This story is missing any context around what occurred. The only thing I was able to find was by searching, and I came to this PDF statement.

https://pup-e.com/goodbye-rubygems.pdf

> On September 9th, with no warning or communication, a RubyGems maintainer unilaterally:

> renamed the “RubyGems” GitHub enterprise to “Ruby Central”,

> added non-maintainer Marty Haught of Ruby Central, and

> removed every other maintainer of the RubyGems project.

> On September 18th, with no explanation, Marty Haught revoked GitHub organization membership for all admins on the RubyGems, Bundler, and RubyGems.org maintainer teams

Which is important context that was left out of this board member's statement.

I don't know more about the controversy than what's explained here, but, reading between the lines, it sounds like companies want Ruby Central to operate more like a for-profit company, where people carry out defined tasks in exchange for getting paid, than like a jury or the American Medical Association, where people do what seems best to them in exchange for a harder-to-define sense of collective social obligation. (When they work, of course; sometimes those institutions don't work very well.)

I am skeptical that the model where people carry out defined tasks in exchange for getting paid can properly discharge the obligations of trustworthiness and disinterest that are necessary for the proper functioning of software supply chains. I'm thinking that probably people whose motivation is primarily personal gain will seek out ways to exploit their users' trust for additional personal gain, for example by bundling adware and other malware into their software the way Microsoft does with Windows, or only releasing security updates to paying customers.

Open-source licensing provides some protection against this problem, because it guarantees you the legal right to switch to a non-malicious fork; but the whole reason we're talking about open-source supply chain security in the first place is that your vulnerability to your chosen upstream is still far from nonzero.

> Either Ruby Central puts controls in place to ensure the safety and stability of the infrastructure we are responsible for, or lose the funding that we use to keep those things online and going.

Seems pretty clear after reading this. If 1-2 companies pulling funding is enough for them to force you to to what they want, its hard to stay independent.

Agreeing with most of the other comments here that this discussion needs more context which we don't have...

If the request for additional access controls/access cleanup came from one of the Ruby Central funders, could we not know who that was and what exactly their ask consisted of? I am interested in knowing their side of the story, and what the motivation was. (But in general, cutting off long-time maintainers' access seems like a bad choice - as presumably they have long since proven their good will toward the ruby community as shepherds of these projects.)

The only reason why Ruby and other open source projects survive is because large companies can trust them to do the right thing. Given the critical nature of the supply chain attacks, what the board did was 100% right. Like he said, some people's egos got hurt but if no one can trust the maintainers, then Ruby has no future in the industry and it will die quickly.

This is basically like fixing technical debt. It's painful and it's political but sometimes you have to do the right thing for the community as opposed to trying to assuage individuals' egos.

[flagged]
I think a lot of people did read the OP and were left wanting - I know I did and was.

Breaking down the posted article, there’s a lot missing (which the author admits), and it’s not clear really what the goal of the post was other than to say “someone, not me, made an oops. But it’s fine, right, because the community needed this to happen.”

Parts that were particularly odd, that others have said with better words:

- Who imposed this ultimatum on RC? - How long was the timeline to “tighten things up”? It sounds like there was both a decent amount of time and an immediate urgency - it can’t be both. - “We’re nerds who can’t communicate well” (paraphrased) is such a poor argument - I get it, I’ve had to do a lot of work to figure out how to navigate social spaces and how to communicate effectively in professional settings. That said, the author is writing as if they’ve never had a single conversation with a technical person that they didn’t know well; that any conversation about removing or reducing access would be a catastrophe. That’s ridiculous.

It seems that either there was poor planning around this, or someone forgot about the deadline and YOLO’d it, or there was a malicious push to oust some of the biggest contributors under the guise of security.

One thing is clear, regardless of what the root cause of this all was: RC showed a deep lack of respect for the people that make their community what it is, and that stinks.

So basically they're a bunch of serfs
> A deadline (which as far as I understand, we agreed to) loomed. Either Ruby Central puts controls in place to ensure the safety and stability of the infrastructure we are responsible for, or lose the funding that we use to keep those things online and going.

This makes a lot of sense, and it puts the 'drastic' action in understandable light.

It also contrasts with the 'On September 9th, with no warning or communication, a RubyGems maintainer unilaterally...' from the Goodbye RubyGems letter. Perhaps that person did not have communications or insight?

Going forward I think we could judge the good faith, if it's uncertain, by if we do see people reinstated. Cutting off access (for urgency with a deadline) followed by reinstatement (because they contribute) would match this post. No doubt there will be hurt feelings on all sides, which is understandable, but I hope as humans everyone can get through it.

A lot of people are arguing about whether locking down access was justified to resolve the security issues. I guess it's debatable.

But I don't see any excuse for not putting out a statement when you do it. You have to know there will be a fight, and you will look like the bad guy. Perhaps I could see directly communicating to the maintainers that you expect that they'll be reinstated. But to say nothing? To let the post by duckinator float around for days without having a "we did this because of security concerns, we want to work together and find a resolution..." It's incomprehensible that they thought this would go well.

For any company that wants to secure and maintain critical source infrastructure for a language, community/maintainer relations is a fundamental responsibility. It is not to be waved away with quasi-candid admissions that you're just too small a team, too technical, etc. Even if this board member is being totally sincere about his feelings for Ruby and its community, it changes little.

> Some of those companies specifically pay Ruby Central to ensure the security and stability of that part of the supply chain, but then discovered that people with no active affiliation or agreement in place had top level privileges to some of this critical infrastructure.

This is the most candid bit of the article.

RubyCentral seems to have screwed up. The sense I get after reading this paragraph is that RC's non-apologies about poor communication are smoke. Why did they have to move this quickly/silently? Well...

If you are taking money from businesses in exchange for certain assurances about the security/soundness of RubyGems, you have a responsibility the minute pen leaves paper to KYC(ontributors). Not when there's suddenly a fire, or when your clients notice.

By all appearances, RC was negligent, if not necessarily in the legal sense. They were highly reactive in response to a problem they should have been across already, and they have paid for it with a chunk of the Ruby community's trust.

To now retcon this action as poorly-communicated but ultimately noble and security-minded does not sit very well.

This is a reasonable perspective but leaves a lot of unanswered questions and creates more questions. Who is the funder threatening to pull funding and why were they not more collaborative or flexible with Ruby Central? Did they know that this is how their request would be handled?

How much information and what information did Board members have when making their votes?

One thing that hasn’t been addressed is who was responsible for communications and implementation of this. It says here that the Director of Open Source did what the Board asked of him. Outside of the Board, which as stated here were heads down and trying to problem solve, Ruby Central’s website also shows a staff of several non-technical employees. Prominently, there is an Executive Director with a background in communications and non profit work per their LinkedIn. Where was this Executive Director and the other staff members during this? Were they involved with decision making and communication around this? How involved was the Board of Directors in implementation after the decision was made? It is a hollow statement to say they are just technical people trying to problem solve when there appears to be a whole team of non-technical staff members and an executive specializing in communications. Something clearly went wrong here and there are a lot of missing pieces around what happened after the vote took place. Most of this could have been mitigated with standard processes and simply communicating to maintainers and the community.

What I'm missing is what, if any, communication Ruby Central had with maintainers.

> How do you tell someone that has had commit and admin access to critical infrastructure long after that need has expired that you need to revoke that access without upsetting them?

Start by letting go of the goal of not upsetting them. Make sure you do communicate clearly. Just say what you said a paragraph earlier: open source ecosystems, including ours, are increasingly suffering supply chain attacks. To guard against this, we need to tighten access that has traditionally been fairly loose. Starting <date>, we're going to remove general access and ask that contributors sign <link to agreement> before re-enabling access.

I mean, maybe that is what happened -- as the OP says, he wasn't part of the conversations so can't say. From the earlier public posts, it doesn't _sound_ like that's what happened. But I'd say as a general rule, it's important to communicate disruptive changes ahead of time to those affected and give a clear path to how they can mitigate the disruption.

I hate the style of write up. It feels a bit gaslightly (it may or may not be but feels like it). And defensive.

Just drop all the facts. Acknowledge you fucked up. Or dont say anything at all?

A board position means responsibility not just "head down coding". And that means communicating with people.

For clarity I wasnt super keen on the original submission this is responding to, for similar reasons.

I think that most Rubyists want to forgive each other and move forward. The board, staff, and volunteers at Ruby Central are all people and people are fallible, that's fine. The way to receive forgiveness isn't to convince others (who weren't there and who don't have the full context) that what was done was reasonable or justified. It doesn't matter. It doesn't even matter who is at fault. What matters is who will take responsibility.

The actions taken by people in service of Ruby Central have had unintended consequences, including damaging the community's trust in Ruby Central's stewardship.

A new governance model will solve only the problem of there not being a governance model. There also has to be an acknowledgment that the lack of an existing appropriate governance model wasn't just a "fiduciary failure," but a failure which cased harm to the community and contributors. Contributors who—like the board—are volunteers, and would have probably liked to have their significant dedication shown more respect.

You show respect to someone by giving them important information from which they can use to make their own decisions. As opposed to withholding information because you are uncomfortable with the possibility that they may make a decision you don't want them to.

Best comment I’ve seen on this thread so far.
Honestly this description makes me even more concerned. There’s a lot of “I don’t know what happened” and “I wasn’t involved” and “apparently we agreed to”.

In particular, after a long winded introduction and setting of the scene, suddenly there’s a mention out of the blue of a 24 hour deadline to cut off access or face losing funding (forever)? But who was holding this deadline over the board’s head is not explained (in fact the author doesn’t seem to know???).

Overall this just reinforces the impression that the RC board handled this sloppily and in a rushed manner, and failed to communicate with long term community members, and thought of themselves as the only parties who mattered, while not taking responsibility for holding such an important position (see the opening paragraphs about how “we don’t have time to communicate to the public because we’re busy programmers without a PR team”).

The poster is the treasurer too. How can he not know where the money is coming from and what constraints its beholden to?

Wildly unprofessional or just willful lying.

Here's a little bit of nitpicking:

> I want to apologize, genuinely, to people who have felt (...) outrage (...) after reading some of what others have shared.

He's apologizing for what others have shared, not for what they (Ruby Central) did.

> I often go out of my way to avoid making people feel bad

"I'm the good guy."

> and so to be part of what's caused so much chaos lately has really been awful.

"_I_ feel awful."

"I'm sorry for what others have said about what we _did_. I feel awful for people being outraged" Amazing.

> this is a small group of volunteers spread out all over the globe. (...) It's just us.

You didn't, for a single moment, think about notifying the people involved that you are removing them? It's the very first thing to do - notify someone who's involved of the change in their status. If your communication skills didn't reach a level in which you thought that would be the thing to do, I don't know what to tell you.

> It is really boring stuff. So why do I do it?

So what? Should we feel sorry for you?

> I love the community. I love the people who use Ruby, (...) I love the people who give their time to Ruby and I love the people and companies who generously provide financial support for Ruby.

Cool.

> I can't speak for the board or the Ruby Central staff. But (...)

proceeds to speak for the board and the Ruby Central staff.

> Ruby Central has been responsible for RubyGems and Bundler for a long time.

This is a lie. RubyGems and Bundler have been maintained by a group of core maintainers. Some members of this group were also Ruby Central staff, but not all.

> It's not a new story that Ruby Central has been working on (or trying to at least) improve the governance model for Bundler and RubyGems.

It's a new story to me. If it's not a new story, do you mind sharing some links to past discussions?

> How do you tell someone that has had commit and admin access to critical infrastructure long after that need has expired that you need to revoke that access without upsetting them?

You learn some basic English. And then let them know. It's called communication.

> And what if other people who do still need that access claim things like "If you remove their access, I'll just add it back" or "If you remove their access, I'll quit".

It's called consensus. And communication. You talk. You speak with people. And then you agree on a decision.

> These are emotional conversations.

Yes, they are. Is that why we shouldn't have them? When you want to leave your wife, do you just leave? What a strong person with strong values.

> I wasn't a part of them and can't actually speak to the content of the conversations or how they were handled.

Bad. They were handled bad. Why did you write this post? You don't have information, you don't know what happened...you just love people and community and companies. Happy happy joy joy.

> we don't have a "communications team"

You don't need a communications team. You just need to have a communication channel public or private, where you can reach all of the core members. It could be an email with everyone in CC.

> A deadline (which as far as I understand, we agreed to) loomed.

If you're not sure whether it was agreed on, again, communication. Learn how to communicate. Which deadline? Who set this deadline?

> With less than 24 hours to go

Did someone give you 24 hours deadline? Why wasn't this discussed long before the deadline?

> Marty, Ruby Central's Director of Open Source

How the f is Marty? If he wasn't one of RubyGems maintainers, why is he suddenly being put as the main maintainer? Aside from communication issues, you also have decision making issues. All of the core members should come to an agreement, without Marty.

> I love this community and I love ...

Money. It's all about money. This is the only sentence from the post worth reading: "Either Ruby Central puts controls in place to ensure the safety and stability of the infrastructure we are responsible for, or lose the funding that we use to keep those things online and going"