I'd like to give them the benefit of the doubt, but this is written like an apology they know they must write. It does not come across as apologetic. It comes across as rationalization veiled as an apology, and it doesn't sit well with me.
Indeed. This letter is an attempt to justify and rationalise their actions. Essentially, it amounts to saying "we're sorry you were offended and felt hurt by our legitimate work but we had no choice but to lie to you and unethically experiment on you without your consent or we wouldn't have been able to do it". Their statement is not an actual apology, even if it is phrased in the language of apology, and it is an excellent example of what not to write if you are seeking forgiveness.
The core issue here is that the system under which they were working, and the researchers themselves, did not consider this work to amount to unethical human experimentation - even though the work directly involved infiltrating and exploiting humans and human social systems. There is no mention of this nor of any substantial desire to understand or address this systemic failing and, until there is an unconditional acceptance of responsibility for the harm caused and a real actionable change for the better with accountability, I don't see how we can consider this matter to have been apologised for.
> we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches.
This bit just reads like "sorry you're upset".
I would have expected something along the lines of "sorry, we should have asked the maintainers and we understand why it was wrong not to", instead of "sorry, we didn't ask the maintainers, but this is why we didn't".
Legitimately these folks have some kind of personality defect. They observed something that everyone already knew and then decided to act on it and pretend they were doing something interesting and new rather than just being twats. They should be treated exactly as should be based on their actions, regardless of their claims of being researchers. This would be a perfect cover for being on the take from a government agency. They should be investigated to ensure they aren't actual malicious actors.
It takes almost no imagination to think of hundreds of "security vulnerabilities" in the world, yet the vast majority of people realize that it's dumb to act on those observations.
They should have addressed why the kernel maintainers felt the way they do. As it is now they are trying to rationalize their actions instead of showing they have made an attempt to understand why their research was questionable.
> We are sorry for the harm we've caused by our unethical experimentation involving members of the Linux community. In accordance with community standards and a desire to avoid benefiting from our ethical failure, we have requested the immediate retraction of the papers and other published work resulting from this unethical experimentation. We are also working together with our institutional leadership to address the systemic and individual failings which permitted this unethical research to take place. As a result of this, we will publicly prepare a report regarding the actions taken and potential systemic changes which will ensure these actions do not reoccur and further harm is not caused. Out of an abundance of caution and because experience has shown our judgement, as is, to be questionable, no further research similar in nature to the research in question will be undertaken by this institution or these researchers until proper community consultation has taken place regarding the report and proposed changes to systemic review are implemented. Once again, we take full responsibility for any harm we have caused and we will work to ensure that such harm never occurs again. We hope that, at some point, we may regain your trust and confidence and, if you'll permit us, to work alongside you.
[signed personally, by every involved researcher, professor, IRB member, and all of the relevant university leadership including the board]
Something like that would do nicely, in my opinion. Then they'd have to absolutely follow through on it. Ideally, they'd also ask involved members of the community so harmed, among other experts, to independently review and watch the process to ensure no further harm is being done, progress is made, and public accountability is kept.
Good suggestion, especially getting the IRB involved. The board failed to recognize that the target of the research was not inanimate software but human beings.
I'd consider that a pretty good apology, if that were the one they had made. I'd like to see it expressed by the university ethics board, not just by the researchers.
I wouldn't trust those researchers again, not for anything. Free software depends heavily on trust. It's astonishing that it works so well; but if you break trust, it's very hard to repair the fractures.
The legal arrangements around free software are fairly fluid, because there hasn't been much case-law, because there's not much money sloshing around for Free Software developers to pay lawyers. Banning pull-requests from that college is much cheaper than sueing.
I hope the whole uni stays banned until the ethics board issues their own apology.
Then let it go, I say. People make mistakes. We shouldn't blame everyone for the mistakes of a few.
But we have to protect the Linux kernel; billions of people depend on it's correctness.
It's not so much a question of what they should have put in but what they should have left out. There was an awful lot in there justifying what they did, and relatively little that acknowledged that it was wrong.
I am not a stakeholder in this, and my comment is about apologies, even though as a lifelong linux user and technologist responsible for deploying it, I still think there should be a real investigation.
While I had a bunch of notes about what makes the sentiment behing an apology meaningful, it's really separate from the issue that this should have been done privately. A public apology is just another thing about them and their message. They're performing and trying to draw in sympathetic members of an imaginary audience. There is no humility in public displays. I would be surprised if anybody asked for it, and doing it without asking is just more of the same type of behavior. There is no such thing as a public apology.
That said, I have no doubt they have suffered personally over this, and have some compassion for that suffering itself, but not sympathy for this performance. There is no gesture that restores trust. When their contributions and acomplishments become more remarkable than this issue, they will have restored it, but like most things, there is no "back" to go to. Maybe that static analyzer will be the ultimate bug finder, as succeeding at that is probably their only out.
I think if it were a sincere apology, they would have apologized for what they did, not just that there were undesirable side effects of their research. The letter is a typical "I'm sorry you got mad" apology that people make when they're forced to, but don't think they did anything wrong.
People love to analyze apologies after the fact, but it seems totally unfair to me. Once someone has said an apology you can take the text of it and turn it into anything you want and say it proves they were lying.
Yep, it's called accountability. Would you prefer people acted without regard for others knowing that magic words can be spoken after the damage is done?
I would prefer you not go around saying someone is "apologizing wrong" because you've found the magic formula to turn all apology text into "I'm sorry you feel bad". There's no point in making a statement like that the original person can't respond to, anyway.
I get what you're saying and I don't want that that either. But I also don't think a simple apology that doesn't attempt to justify one's actions can be so easily flipped.
An good apology needs to demonstrate an understanding of why what was done was wrong and ideally includes concrete steps that will prevent anything similar from happening again.
You seem to be implying that one apology is as good as another and that is absolutely not true. To me is is absolutely unfair to expect that people will treat a zero effort "sorry your feelings were hurt" apology the same as a deeply heartfelt apology by someone who feels deep remorse.
By it's nature, an apology should be an act of making amends to a person your have wronged. If the words of your apology don't do that, you have failed, even if you had good intentions.
If the author of the apology isn't a native speaker of english, this seems like a document of sufficient significance to merit asking a friend or colleague for a proofread. (Which you should do in a case like this even if English is your first language.)
Agreed. This is like a psychologist apologizing for traumatizing children by testing a hypothesis that telling them scary stories before bedtime would give them bad dreams. "Oh, but it's important to learn what scares kids, for the greater good" just doesn't cut it.
Software is complex. Wasting maintainer's time is harmful to the maintainers and to those who use the software. If the goal is to explain what sort of exploits to be on the look-out for, just write the paper without doing the exploits.
As for the letter, their goal is to recover their reputations, not to apologize. I suppose a lawyer helped them in spots, but there is still a certain truth that shines through: they want to continue on doing this sort of thing, because they are oh-so-clever and their work is oh-so-valuable. Nice try, but the academic community may be better off without these folks doing this sort of research and influencing students to follow their methods.
I agree, but let's give them a little extra benefit of the doubt. I thought as I was reading it that it seemed stilted and forced, then I wondered if the author(s) don't speak / write English as a primary language.
I'll be interested to see how they react to feedback / responses.
> We just want you to know that we would never intentionally hurt the Linux kernel community and never introduce security vulnerabilities. Our work was conducted with the best of intentions and is all about finding and fixing security vulnerabilities.
Either this is a bald-faced lie or they didn't think through the implications of their research, neither of which bode well for their trustworthiness in future. However, if it's the latter, then they possibly deserve additional leeway.
On the other hand, I wonder whether this letter would have eventuated without the ban - if not, then they've likely been forced to write it, in which case any remorse is either selfish (i.e. they've been professionally reprimanded and feel bad about getting caught, or making such an egregious error in judgement), or hollow.
> I agree, but let's give them a little extra benefit of the doubt. I thought as I was reading it that it seemed stilted and forced, then I wondered if the author(s) don't speak / write English as a primary language.
The wording is definitely stilted and forced, but the content is also wrong for an apology. They spend a lot of time justifying their actions, and close with "this has been painful for us too". This may also be cultural, but it's overly self-centered for an apology.
I was just about to post: "This is a great apology."
Context is everything, of course. I think you're right that it's tainted by the fact that they absolutely did not have a choice, and coming from people who've deceived the same tribes they're now trying to apologize to.
And what of the merits? If the previous 190 patches were indeed legitimate it strikes me as overly vengeful to pull them in a "punish the son for the sins of the father and the father for the sins of the son" kind of way.
Agree that it could be seen as excessive, but this was not just in response to these commits, but also to the university's failure to act once they were informed of them.
They university, and indeed, the researchers, only acted once the ban was imposed - i.e. they were getting away with it without reputational damage, and only once it was stained did they actually do anything.
Or it's not in vengeance, but caution after the researchers have demonstrated that they put their needs above the community's. When the presumption of innocence is lost, it's appropriate to revert the patches until they are known to be good.
ETA: GKH on reverting the patches[1]
> This patchset has the "easy" reverts, there are 68 remaining ones that
need to be manually reviewed. Some of them are not able to be reverted
as they already have been reverted, or fixed up with follow-on patches
as they were determined to be invalid. Proof that these submissions
were almost universally wrong.
counterpoint:
The LF Technical Advisory Board is taking a look at the history of UMN's contributions and their associated research projects. At present, it seems the vast majority of patches have been in good faith, but we're continuing to review the work. Several public conversations have already started around our expectations of contributors.
I don't think it's particularly vengeful. I think it's a pretty natural result of the broken trust.
Yeah, the authors are claiming those past 190 patches are fine... but they also essentially did the same thing when they submitted the intentionally bad patches too.
So the question is, how much do you trust them right now? And given they just submitted intentionally bad patches, I don't think it's very surprising they don't have a whole lot of trust.
And if you don't trust them, then you've got 190 patches active from an untrusted source with a history of submitting bad patches. That's not a particularly good situation. Why wouldn't you revert them?
The specifics of the situation make the whole thing substantially murkier, in my (entirely unrelated) opinion. But take a broad enough view, and this is pretty much the default path. However the specifics make this a bit of an oddly exceptional situation, all things considered.
Why would you trust anyone who pulls this kind of stunt? IMO pulling all commits pending review and banning future commits is a reasonable precaution, particularly given this disingenuous apology and the failings of the University oversight board.
Problem is, hardly anyone is spending money on lawyers to defend Free Software. So the cheapest way to defend Free Software is to ban submissions from people who have a record of submitting crap.
Blocking an entire University seems extreme; but just think of it as a sanction on the University's ethics board. I think a (short, and very explicit) apology from the ethics board ought to suffice, to get the Uni off the hook.
And I think those researchers should not be allowed near Free Software again, unless their pushes are going to be rigourously scrutinised.
I think they could have really helped themselves by being humble and truly apologetic. Simple things like:
- "We are sorry for the harm we caused" instead of "We're sorry for any harm we caused"
- Not trying to explain their actions in the first paragraph of the apology! I think most folks are aware of their intent by now, and leading with yet another explanation just makes the whole thing feel disingenuous
- Avoid saying things like "this has been painful for us as well"
Unfortunately, it has so many hallmarks of a non-apology, and it's hard to look past those given the context.
As others have mentioned, they can clear this up with their actions going forward, but it will take time to rebuild trust.
Is that a thing? Like there's some non-apology Bingo card you can fill out? I don't see a connection between the criteria you listed and genuine-vs-false contrition.
You may perceive these things one way, but ultimately you can't know the minds of others well enough to tell if they are sincere or not about anything. You don't get to just declare yourself the arbiter of their feelings because they used "any" instead of "the".
I tend to have pretty dry affect at certain hours of the day, which has caused considerable frustration for myself and others when I know I'm sincere but they don't. Nor does it make much sense to me for that sort of mind-reading to take place democratically. What do we then gain from it?
At best, you get an apology for a misunderstanding over the previous apology, plus a second draft that the masses might like better. Then you still get that lingering contingent that says "you just did that to placate people! Now we really know you don't mean it!"
The cultural apparatus for "Saving Face" is completely broken on a build failure for missing dependencies.
Yeah, it's not a great apology. I would say as far as justifications go, I think it's a certain level of depth expected from academics, you shouldn't overthink it.
As far as giving benefit of the doubt, it's likely not done out of malice in the first place. Just a combination of poor reasoning and doesn't exactly clear up if they even considered alternatives to control their variables. Unfortunately, it seems like they were already given the benefit of the doubt from their first offense, but did not take any lessons away from that, so it would be understandable if the maintainers kept their ban.
I don't think they had malice, but the breach of elemental ethics is just appalling. They show no remorse for being trusted and abusing that trust and good faith. They show no remorse for using human beings as involuntary guinea pigs.
For me, it's not learning the lesson the first time around.
I completely agree their experiment is unethical. However, it's not actually clear cut to most researchers the ethical bounds of their work, especially for study papers that's never really been explored before. Ethics in of itself is largely a active subtopic for many areas in CS, not only security research. AI is one area where qualifying potential harm to human beings remains largely controversial. Ask any ML scientist, and they'll tell you that determining the ethics of a project is not their responsibility.
The ethics around research that involves deception have been pretty well established and are are several good comments here explaining them.
Every scientist is personally respsible for the ethics of the research they conducts. Full Stop, no caveats allowed.
If your research is in an area where the ethics are controversial or grey, that means your need to spend MORE time considering the ethics of your research, not that you get a free pass from being responsible.
If a any scientist espouses the opinion that determining the ethics of their projects is not their responsibility, they should be permanently barred from recieving grant money.
> The ethics around research that involves deception have been pretty well established and are are several good comments here explaining them.
Ethics in computing research remains an active research area. This incident will be used as a case study in the future, but it's not that well established. Many people have been using anecdotals, which honestly don't fit the scenario because so many variables and parameters distinguish other types of pentesting from this. And disappointingly, not a single post has actually produced the documents that establish this.
Arguably the first set of guidelines for ethics in computer security research [1] was published in 2012 and not yet widely taught in Ethics lectures (I only know about it because I learned Computer Security from one of the authors).
On identifying harms:
> "Challenges identifying harms in ICTR environments stem from
the scale and rapidity at which risk can manifest, the difficulty of attributing research risks to specific individuals and/or organizations, and our limited understanding of the causal dynamics between the physical and virtual worlds. As with all exploratory research, it can be challenging to articulate benefits such that subjects can make informed decisions. In ICTR our ability to qualitatively and quantitatively foresee the probable benefits is particularly immature."
On this type of research:
> "Research of criminal activity often involves deception or clandestine research activity, so requests for waivers of both informed consent and post hoc notification and debriefing may be relatively common as compared with research studies of non-criminal activity."
This isn't a huge change from 30 years ago since Moor [2] wrote his thesis on Computer Ethics, see:
> "A typical problem in computer ethics arises because there is a policy vacuum about how computer technology should be used. Computers provide us with new capabilities and these in turn give us new choices for action. Often, either no policies for conduct in these situations exist or existing policies seem inadequate. A central task of computer ethics is to determine what we should do in such cases, i.e., to formulate policies to guide our actions."
Researchers themselves are far from educated on this topic; you won't ever explore this in depth unless you're in this particular sub-field. IRB/REB boards are considered the most qualified but are possibly too outdated to navigate around this. It's a whole mess, there is currently a lot of questionable research in many areas of computing, but the clock moves forward.
Their opening statement has the most classic of all non-apology tactics: "We're sorry for any harm we caused". Maybe this wasn't their intent, but this is one of the oldest "say the words without having to mean anything" tactics in the book.
It's like telling your partner "I'm sorry if I hurt you". No, you hurt them. Apologize for hurting them.
An apology should be for actions taken, not for how the other party felt about it. Acknowledging those feelings is important, but taking credit for those feelings takes agency away from the other person.
I don’t think that really makes sense. Often in a personal relationship the “action taken” will be innocuous considered in itself. But you may have done it knowing full well that it would hurt the other person’s feelings. In a case where the person’s feelings are totally irrational and you had a strong independent justification for taking the action, it does get more complicated. In general, though, I think it’s a huge mistake to assume that you should never apologize for hurting someone’s feelings.
I don't think you can take the action outside of the context in which it is performed. If you take action knowing another person is likely to feel hurt, are you... not sorry you took that action?
Regardless, you can't genuinely apologize for things that aren't in your control. (You can and should empathize and sympathize with their feelings about it.) It may be a convenient fiction at times, but there are many downsides to this type of boundary blurring. I think you do the recipient a disservice by implying they have no agency—especially if they believe you. You also face the danger that your apology seems insincere or you misjudged their feelings, which could actually make matters worse.
Suppose my sibling and I call each other 'ugly' as a greeting and we normally enjoy this behavior. If one day I feel hurt, is my sibling responsible for that feeling? If yes, is my sibling responsible for the depth of that feeling? Even if, on this particular day, I feel more hurt by the comment because I had just broken up with a partner? Is my sibling responsible for the duration of the feeling? Even if my response to the experience is to repeat the comment over and over in my head even after my sibling apologizes? Is my sibling responsible for all the decisions in my life (including encouraging that behavior) that have lead me to be sensitive to the comment in that moment? Am I powerless to change (whether in the near term or long term) how I respond to that comment, whether through meditation, therapy, confidence training or whatever? Am I also powerless to remove my sibling from my life?
In my view, the extent to which we can influence the emotions of another is the extent to which the recipient themselves has given implicit permission to do so; which is to say, they have agency. I don't think that simplifies into being able to choose to feel great any old time we like. I also don't think it at all excuses bad behavior or justifies a lack of empathy. But I will say that I feel a lot more good moments and less frequent and intense bad moments since I adopted this view and took responsibility for my own emotions.
This highlights the importance of apologizing for the impact something has, vs. just apologizing for the act itself.
There is no universal rule that can be applied to determine when one vs. the other is appropriate, but seemingly innocuous actions can have negative impact, and apologizing for the innocuous thing may not always make sense.
To be clear, this is not universally true. There are situations where apologizing for the action makes sense, and situations where the action that led to a negative outcome are either inconsequential on their own, or there might be a myriad of factors leading to the need for an apology.
Apologizing for the action can also come across as disingenuous or passive aggressive in the wrong context. This is especially true when the action is well-intentioned, but has the wrong effect. Taken to an extreme, this sounds like "I'm sorry I tried helping".
You don't have to be a mind reader to see that this apology isn't an apology. They spend as much time justifying their actions as they do apologizing for the unintended outcomes, and they close by complaining that they've also been hurt.
You have to acknowledge (in full) what it is that you are apologising for. That's the most important thing. Promises are meaningless.
It's not possible to "be genuine"; especially in a public post, nothing is genuine, everything is PR and smoke. Both the researchers and their ethics board need to grovel. If they did that, this tornado-in-a-teacup could be over in a month or two.
The kernel community isn't a spurned lover or anything; the apology just has to admit fault, promise not to do that again and be truthful about it.
There is no need for the researchers to be in any specific state of mind or even apologetic as long as they don't try to submit bad faith patches again.
Really the only reason there is even a need for a public apology is to signal how much pressure they are under so other people think twice before doing the same silly thing. Otherwise they could just apologise to the maintainers directly.
To me, it's always a distinct sign that an apology isn't sincere when it hedges apologizing for "any harm" that they might have caused. In my mind, a sincere apology clearly acknowledges harm that has been caused and apologizes for that.
Social media insanity has greatly reduced our capacity to accept public apologies at face value. There is always a seething mob that is ready to question the sincerity of the apology once there are no more demands left to be made.
I agree. There are weasel-words throughout, starting with apologizing for "any harm". When you've been told what the harm was, don't apologize for "any harm" you might have caused. You should start an apology by demonstrating that you have listened and understood exactly what harm you caused.
Listing the harm you caused is also an important part of the public record, so that anyone else considering a similar scheme in the future can see what consequences were agreed to have happened the last time.
You're not. It does not come off as apologetic at all. It is clearly someone writing as if they wanted to go back in time and act like they legitimately were caring, thoughtful people who made a mistake. In fact, they thought this was worth trying and got caught.
The Linux kernel team had to revert changes in over 200 files. They will also have to go back and review hundreds of commits to make sure they do not contain any malicious code.
The technical effort in addition to all the time spent dealing with the backlash of this is a waste of time for a lot of people doing important work that benefits everyone.
So, what else can be said... Play stupid games, win stupid prizes. To Mr. Lu, Wu, and Pakki: enjoy your ban and worldwide press coverage.
-they claim asking permission would have defeated their research
-they claim the other 200 were legitimate patch attempts (a sample of which were, from my personal reading, from 'innocuous but useless' to 'slightly harmful').
> they claim asking permission would have defeated their research
But surely this proves malice aforethought. If you interact with someone under false pretences to deliberately mislead them, then you are effectively lying to them. In fact, if you are doing so in order to receive something of value from them (such as their time reviewing your code) then it could even be seen as fraud.
Did they tell their IRB that their research involved deception? Before declaring the project exempt from oversight, the IRB should have required that the researchers answer a question like "Does your research involve interacting with people who are not aware of your project and who would act differently if they were aware?"
Until the IRB process at the university contains at least this level of protection against such ethical failings (not just in Computer Science research but across all areas of study) I think it is right for the kernel team to refuse to interact with members of that institution.
I really appreciate the apology and as they stated, its unconditional nature. Good.
However, I find something very problematic. This quote shows it:
"We have learned some important lessons about research with
the open source community from this incident."
This is something I don't like. This is not something about "research with the open source community". If anything, they should have learned something about treating human beings as persons and not as involuntary guinea pigs. They should have learned something about not breaking anyone's (not only any open source community) good faith and trust, and respecting that.
They behaved like jerks and they still cannot see that.
It would make sense if they were talking about mice, in which case "on" is still more accurate but "with" is probably correctly assumed to be equivalent. Research with humans involves them being on the author list...
>They behaved like jerks and they still cannot see that.
They just come off as disconnected academics. Samething with the experiments with algorithms from Facebook. They are so wrapped up in what they are doing, they no longer see the "users" as human beings. There are some times this is almost required to survive like being an ER doctor. If you lose that many people, it could be debilitating unless you were able to disconnect. That's far and away much different than these robots.
I think there is something to be applauded about people who genuinely apologize even though they can't see things from the other person's point of view.
They didn't decide to conduct this research on a whim. They had full approval of their university ethics board as well. They published a paper and had it peer reviewed without (as far as I know) anyone immediately calling for their heads.
They can't immediately turn heel face and believe their actions are wrong, always were wrong, and the wrongness should have been obvious to them.
Yet, despite this they recognize the hurt they've caused and are genuinely apologetic and will never do it again. Asking them to dismantle their world view in a week is a bit much, even criminals are given a few years of quiet contemplation before being asked to tell a parole board they have changed their hearts and minds.
This paper had multiple issues that they where aware of before the ban, original ban email indicates that complaint was made before to there's university, on twitter feed one of the researchers stated that IEEE received also complaint regarding ethics of the paper, way before ban.
> we are very sorry that the method used in the “hypocrite commits” paper was inappropriate
This reads more as "sorry you were offended" than "It was inappropriate and we are sorry".
> As many observers have pointed out to us, we made a mistake by not finding a way to consult with the community and obtain permission before running this study; we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches.
Bringing up why you did something in an apology is very shaky. Like in this case, it can sound more like justifying / excusing.
Which I'm guessing is why they made it an open letter. This isn't about acknowledging their misdeeds and trying to fix their relationship with the OS community, this is them trying to sneak in spin and justifications to a public announcement to cover their collective asses, to put forth a false narrative that shows their actions in a better light than they acted in.
"If anything, they should have learned something about not treating human beings as persons and not as involuntary guinea pigs."
Perhaps the entire "tech" industry needs to learn that lesson. Non-technical end users should be entitled to that same level of trust as nerds. I can download free open source code, extract a tarball and build the software without worrying too much about scanning through all the files first for phone home/telemetry/OriginTrials nonsense.^1 However non-technical end users who use programs compiled for them by "tech" companies with ads and surveillance as their "business model" are not entitled to the same trust. I cannot think of any justification for the difference.
> Perhaps the entire "tech" industry needs to learn that lesson.
Audit studies are conducted all the time where researchers lie to people and waste their time.
> Factors Determining Callbacks to Job Applications by the Unemployed: An Audit Study
> We use an audit study approach to investigate how unemployment duration, age, and holding a low-level “interim” job affect the likelihood that experienced college- educated females applying for an administrative support job receive a callback from a potential employer. First, the results show no relationship between callback rates and the duration of unemployment. Second, workers age 50 and older are significantly less likely to receive a callback. Third, taking an interim job significantly reduces the likelihood of receiving a callback. Finally, employers who have higher callback rates respond less to observable differences across workers in determining whom to call back. We interpret these results in the context of a model of employer learning about applicant quality
This is a very good point. I did not find anything objectionable about that experiment when I first read about it. And if the companies experimented upon had complained like the kernel maintainers did, I would have dismissed it as them covering up their bad process. This is actually making me consider that the submitters of the bad patches shouldn't be treated differently than the job application researchers.
Imagine if instead they did this research with the closed source community. You know, get hired under false pretenses, sneak some vulns into some commercial product, write a paper about how easy it was. Pretty sure if they tried that they would be in jail right now.
So? The difference is consent (in your case of the owners of the company).
There are many crimes in the world that are only crimes if you do it without consent.
> How else do you prove your process can stop real infiltrators by state actors?
One of the common security controls against infiltration by foreign nation states is espionage being a capital offense. Well obviously not appropriate, i think that its pretty obvious that these researchers would have not succeded if they faced the electric chair like a spy would. So i don't think the comparison is apt.
So if we get consent from a C-level at a company, we don't need consent from every other lower level person we trick?
If you have approval from one person in an org of 100,000 is this still okay?
It has to be, because getting explicit consent from every person in the org at that size would be untenable, and make them artificially extra vigilant during a potential audit window.
Well now scale this to open source. The typical web project has 2k+ random nodejs libs in their dependency tree. These are all separate orgs/individuals.
Real world bad actors are backdooring these projects constantly. I specialize in this area of research.
If every white hat has to get permission from exponentially large dependency chains, they will never even come close to being able to compete with black hats here finding the weak links of the chain.
The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
> So if we get consent from a C-level at a company, we don't need consent from every other lower level person we trick?
You have to get consent from someone who has legal authority to give it to you.
> If every white hat has to get permission from exponentially large dependency chains
They don't. They just have to get permission from someone in authority. Different open source projects have different governance structures. Sometimes that is a single person.
> The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
If you are hacking someone elses system without their consent (not to mention for your own personal gain), you are a blackhat. By definition. Pretending to be a researcher doesn't change that.
And what about the hundreds of NPM dependencies in VS Code no one reviews?
Or the the thousands of brew packages that are blindly merged unsigned by 800 people with access?
Who pays to give a white hat as much freedom to find supply chain attack vectors here? Who gets consent from every random student whose code, if compromised, would compromise every major company?
We have created a massive mess, and I don't know that we will be able to hire enough researchers to fix it.
We are going to need unpaid volunteers, and a lot of them.
> We are going to need unpaid volunteers, and a lot of them.
that's absolutely orthogonal to the question at hand, it's not a matter of paid or unpaid, neither being volunteer or not: it's a matter of consent.
if you don't sought consent beforehand either via a contract relationship or a sponsored bug hunt program or the likes, you're a racketeer, not a white hat, and you should (and will) be treated as such.
This falls under behavioral research. I believe psychologists have solved the problem of how to research that is both blind and in line with ethical standards. All they had to do was ask one.
The apology reads sincere to me even if it could be better, but this is for me why it might be very difficult for the Linux community to recover trust in these guys.
This, and the previous mail that I found pretty insulting too and seemingly written in bad faith but I'd give the benefit of the doubt, one can react badly to a difficult situation.
> As many observers have pointed out to us, we made a mistake by not finding a way to consult with the community and obtain permission before running this study; we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches.
This shows one thing to me, clearly, they still don't understand how such studies are conducted. not authorized penetration testing is illegal, to a very large degree.
For example, as far I understand, and as many pointed out; you get permission from members for an study without specifying time or any details of a study, then conduct study in 6 month.
What is that supposed to mean? Conference publications are for research.
I pulled this from the website of Oakland 2021
Since 1980 in Oakland, the IEEE Symposium on Security and Privacy has been the premier forum for computer security research, presenting the latest developments and bringing together researchers and practitioners. We solicit previously unpublished papers offering novel research contributions in any aspect of security or privacy. Papers may present advances in the theory, design, implementation, analysis, verification, or empirical evaluation and measurement of secure systems.
You’re defining whether something is “research” not based upon what it is or what it contains, but just based upon whether or not it’s been published.
I was pointing out that this is like defining whether or not something is “art” not based upon the work itself, but instead based solely upon whether somebody bought it.
Incidentally, my recipe for apple pie was printed in a newspaper, and so therefore my recipe is “news”.
Does the response of the Linux community seem like the response of some corporations when security vulnerabilities are disclosed?
In each case, a vulnerability was disclosed. With Linux being used everywhere you can be sure intelligence services etc are likely sending in bad patches. If the Linux kernel requires only patchers with good intentions, and doesn’t have other means of catching this stuff, we are screwed.
> the response of some corporations when security vulnerabilities are disclosed
There's a big ethical difference between trying to exploit a piece of commercially produced software, and trying to exploit the time and actions of humans who are producing software which is given away for free.
Also, this wasn’t a vulnerability found in existing code that was disclosed. This was an attempt to introduce several of them in the form of innocent looking commits.
I don’t think the free vs commercial aspect is the main issue here; lots of kernel devs are paid for their work after all...
> Also, this wasn’t a vulnerability found in existing code that was disclosed.
You're right that the OP's analogy breaks down if the researchers were unsuccessful in getting their malicious patches accepted, because then there is arguably no vulnerability to report, and OP said "when security vulnerabilities are disclosed".
Steelmanning that analogy, though, what the researchers were doing was "probing for possible vulnerabilities", which some vendors also complain about, especially if the target is an online service rather than software running locally on the researcher's machine.
In that case, the main flaw in the analogy is still the difference between exploiting software and exploiting humans, so I probably should have focused on that. Nevertheless, there is a small ethical difference in some circumstances between software that is bought and software which is freely downloadable (regardless of the licences involved), since if you paid for something which is defective then you might deserve a refund.
However, when a vulnerability is found, you would expect that the path by which that vulnerability was introduced would be shut off, right? In this case, what they’ve done is to revert all patches from this group, pending further review, just in case something else slipped through.
They’re not sweeping issues under the carpet. They are actively addressing those issues.
No, not really. For starters, notice how (some) corporations threaten legal action when evidence of a bug is disclosed? And how, here, there was simply a public ban on account of no longer being a good faith participant?
Everyone knows that happens. The issue is that these are presumably university funded researchers who intentionally tried to introduce vulnerabilities into the Linux kernel without working with the maintainers. It is fine to perform experiments like this, with permission. I think people are upset because there is a moral, ethical and social obligation expected from academia to not intentionally do shit like this.
They then published a paper and thought that this was acceptable behavior from a research/ethical standpoint.
> I think people are upset because there is a moral, ethical and social obligation expected from academia to not intentionally do shit like this.
OSS security should not rely on ethics and trust. The default assumption should be that malicious actors are using every dirty tactic they can to get their changes accepted, and that someone@something.edu is no more trustworthy than internal.security.agency@china.gov.cn.
Feels like the OSS community got caught with their pants down and is now overreacting.
A BS non-apology is not enough. These assholes should at the very least be reprimanded by the University of Minnesota for unethical research practises (psychological experiments on humans without their consent) and bringing the whole institution in disrepute.
Whether it is appropriate or not, the Linux kernel is used in many mission critical and essential services. One could very convincingly argue that the open source Linux kernel is a vital part of mission critical infrastructure found all around the world. An apology, likely written under duress, for an inappropriate research method or for consuming the time of maintainers (either volunteer or paid) does not address the most egregious transgression:
One does not conduct security research or experimentation on mission critical infrastructure without the informed consent of the persons maintaining or responsible for said infrastructure.
The stark difference between white (or even gray) hat and black hack security operations is whether those operations are done with the informed consent of the owners of the system(s) being manipulated. To knowingly and deliberately attempt to manipulate critical infrastructure without the informed consent of the persons responsible for that infrastructure is, in my eyes, clearly unethical black hat hacking. It does not matter how many resources were expended nor does it matter if there was no personal injury or destruction of property. A quiet, relatively uneventful outcome of a black hat operation is nothing more than a dose of good luck.
There are many other circumstances in which this sort of clandestine operation would be obviously unacceptable. Consider if a researcher wished to surreptitiously tamper with vaccine supply chains, automotive or aircraft supply chains, fuel distribution supply chains, chemical manufacturing supply chains, etc. Even if the researcher claimed to have the best of intentions, and had a written plan to carefully back-out or otherwise reverse (or prevent from going to production) the intentionally flawed manipulations or modifications of critical supply chain components, I think very few people would consider the research ethical. There are an enormous number of issues, defects, and mishaps that occur even when a group of people tries their best to implement and maintain critical systems. When a clandestine operator decides to interfere, using whatever justification, then reliability of the system can be expected to suffer.
I do not believe the written apology addresses the underlying problem, which seems to be that the University of Minnesota does not exercise sufficient governance over the researchers in its employ.
To be fair, you wouldn’t expect these individual researchers to address the governance question. That should come from the University or the department in question.
Indeed, I agree -- the University of Minnesota should shoulder a degree of responsibility. That is even more so the case given that the researchers do not seem to understand (or clearly acknowledge that they understand) the importance of informed consent in both research and security operations, how much reputational damage (for them and the university) can result from unethical research, and the potential for actual harm to occur when research or security operations are conducted in an exclusively clandestine manner.
> the University of Minnesota should shoulder a degree of responsibility.
In fact the argument could be made that there is an ethical duty for every software project, and possibly every individual, to economically boycott the University of Minnesota until they put in place independently-verified changes to their IRB process such that research like this will not be approved again.
If the only "cost" to the university is that one of its researchers has to write a two-faced apology message, then there is nothing stopping future abuses of non-consenting humans. Moreover, with no real punishment meted out, it creates an incentive for other universities to similarly weaken their IRB process to the same level so that they can approve more research projects and remain competitive.
The university should shoulder a huge degree of responsibility. They hired these "researchers". The apology should be coming from those who sponsored and funded the effort. Who cares about any "apology" these "researchers" cobbled together?
This is the most reasonable response in this chain. The number of unintended consequences this "research" could have caused is not quantifiable. This entire episode is shocking, disgusting, and should not be forgiven easily. Actions speak louder than words; this is especially true of apologies. Their words are empty and their actions are nonexistent.
There is a major error made by the research group. It starts and ends here: "we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches."
I am a Red Teamer and work with companies to understand how their detective/preventative/recovery controls and processes are working. Here's how you resolve this:
You work with maintainers to get their coordination on the research. You work out a mechanism to prevent submitted patches from being merged (e.g. maintainers are notified before bad patches accepted by code review processes are merged).
You do not tell them when the patches are coming. You do not tell them which identities are going to be used for the patches (e.g. from which email addresses). You do not tell them which area of code will be targeted. You set rules and time bounds for the study.
You wait some amount of time before submitting such patches (weeks to months). Realistically this is all that's needed. If hypersensitive, set this up earlier and let it bake longer.
At this point, you submit patches from a variety of addresses (probably not associated with your university - it is easy to create many such identities). You also can coordinate with other researchers, universities, and companies to submit patches under identities as needed. You also study submitting from yandex, gmail, .cn and other email addresses (because isn't that interesting to know?).
The premise that there's some ultimatum between working with the community and performing the research is on its face incorrect. This is either ignorance or laziness on behalf of the researchers. Clearly, they hadn't taken the time to work with the community to work out an approach that could be mutually acceptable.
This is how it should've been done. Like a war game (at least,as depicted in movies like Periscope Down), the organizers of the study and the project maintainers need to agree on (and be aware of!) bounds of the study, without necessarily knowing all of the details.
OK, how do you test SocEng vectors? Do you obtain consent and coordinate with every employee that might be targeted to receive your e-mail?
>You also study submitting from yandex, gmail, .cn and other email addresses
No, the point was submitting from a known and respectable entity, which might affect the level of scrutiny. They weren't testing a whole patching process, but a specific human component of it.
You don’t. You work with the leadership & security team. Any employee that clicks your phishing email gets an extra dose of security training. Those that forward the email to abuse@corp.com get a nice compliment
Just clicking the link is enough to fail? No need to enter private info or execute downloaded files? Either your phising emails are badly crafted or you expect employees to see the future.
Agreed. This is something that made me very annoyed when they did it to us. I know exactly what I'm doing, I know that you can't infect a computer from opening a link unless the attacker possesses a new browser 0-day, and expecting your average employee to worry about new browser 0-days is ridiculous.
If it were an obvious phishing URL, like a variation on the company domain, then fine (maybe). But it wasn't.
1. (Windows specific) Opens up a Windows file share, which causes the person to authenticate to the file share, which through PtH/Responder results in their enterprise credentials being stolen.
2. Exploited a XSS or CSRF attack on an internal/management endpoint. Which in turns allows a pivot from external to internal access.
3. Steals a web session, cached password or authentication token, resulting in compromise of employee credentials to be used elsewhere (e.g. reused to access enterprise VPN).
These are just some not-a-browser-0day examples of a single click being game-over dangerous.
> 3. Steals a web session, cached password or authentication token, resulting in compromise of employee credentials to be used elsewhere (e.g. reused to access enterprise VPN).
How do you do this without a browser vulnerability (and assuming it's not also XSS/CSRF like the previous point)?
You can do this with chains of vulnerabilities, including but not limited to insecure redirects, CSP bypasses, insecure cookies. Another useful technique is session fixation - you give your victims sessions you've started and often their SSO experience will connect _their_ credentials to _your_ session.
Also to distinguish between #3 and, XSS in #2 was intended to mean "persistent stored XSS" as opposed to "reflected XSS". In the case of reflected XSS, this can be chained with CSP bypasses and insecure cookies to grab out e.g. bearer tokens.
My overall point is that heap fung shui 0day not required for 1-click ownage. In practice, I've not had to burn browser 0day to compromise organizations or their customers.
I assume you're right on those techniques, but 2 things:
1. It sounds like they'd have to be pretty well targeted against the precise systems of that particular company in order to work. Which would tend to suggest more targeted spear-phishing attacks and extensive recon being done against the company systems somehow before anybody launched a real black-hat attack.
2. At that point, it feels hard to blame the individual employee versus whoever misconfigured those corporate services in the first place. Though I would guess it's fairly common for those kind of things to happen due to many systems being set up without the help of true experts and the unlikeliness of a real attack against them without either a highly-skilled black hat targeting them or securing the services of a skilled and prices pen test team.
1. You are correct. I would measure the effort in terms of a small number (1-3) weeks of recon and targeting for a team of two.
2. I agree. Individual employees are not at all to blame. Companies who are blaming their employees for getting phished are doing it wrong. The correct action to take is to inform employees and build the other kinds of mitigations mentioned elsewhere in this topic tree.
I'm usually not testing only whether employees are easy to phish (the answer is pretty much 100% yes). I'm testing end-to-end: can you as a company prevent me from phishing through email protections? Can you detect when I'm phishing your employees? Will your employees report potential phishing emails? Can you figure out (without me telling you) which employees were targeted and which attacks were successful? Can you figure out which credentials/machines would need to be quarantined/rotated/examined?
That exploit could happen ANYWHERE on the web. Any freaking link. Anything on a newspaper website, or social network.
You're talking about vulnerabilities in components, or other software, here. The user, by himself, is doing nothing wrong.
Why don't you just restrict employees to the intranet, then? Why do you give them free roam on the whole internet, but then you tell them "don't click the wrong link!".
Phishing happens when the user does something which is actively wrong. When the user opens an Word/Excel with VBA from an untrusted source and bypasses security restrictions. If they execute/install something unsigned and untrusted from some random site.
Click = fail is just wrong. Links are how the internet works. You aren't teaching anything.
Megacorp I work at does this. I think I've had around 1 phishing mail per month for the last year or so, and yes, just clicking the link is enough to fail.
It's particularly annoying where I work, as the company itself sends out a completely unreasonable amount of internal spam every.single.day - often with bad spelling/grammar, and very often with the contents being a single image with rendered text (why?!?!).
A popular phishing test vendor populates message headers with a very specific word that you can build a mail rule from. In three years it has flawlessly identified every test sent my way with zero false positives.
So, we don't click on links anymore? Anywhere on the internet? Just about any site can deliver a malicious link.
How can I tell whether I can click on a link? Sometimes there's even something like linkprotector.outlook.com/[very_long_url] in corporate emails.
My usual approach if I'm unsure whether a link is malicious would be to open it in a private window (and probably in a different browser from the one I usually employ), or if I really think it's phishy, I would open it from within throwaway VM.
So, the blanket "click and fail" policy seems pointless to me. If I enter some login/PII, then I can agree I've failed the test. But a click on a link cannot be considered failure.
> You work out a mechanism to prevent submitted patches from being merged (e.g. maintainers are notified before bad patches accepted by code review processes are merged).
It is my understanding that this happened and that no bad patches were actually merged.
I also empathize with the plight of the researchers — Linux is a bit different than normal Red Team engagements, in that a normal organization has a hunch of administrative / management layers who typically do not participate in the operations of the system being tested. A VP of engineering at a medium to large company is unlikely to be committing code, much less maintaining the build pipeline etc.
This is not the case with Linux. The people “at the top” are also reviewers, and so it’s pretty likely that notifying them will result in a change of behavior.
I wonder if there is some way to build some sort of Red Team consent “blind trust” organization, such that willing open source projects could agree to responsible attacks, and the attackers could register their work (including disclosure / mitigation plans) with the blind trust ahead of time.
Why do you think there will be any change in behavior ?
It is not like community members are not for look out for bad commits on every new commit from most committers any way, since at least 2003, I think substantially earlier.
Wouldn't it be better if the kernel had a governance committee and such so that innocent mistakes like this could be smoothed over and the personalities over-reacting on the basis of nothing but years of expertise gained at efforts that benefit the public at large could be dismissed because of their personality problems?
Maybe we can find or make up something unrelated but unsavory about these characters objecting to those pissing in the font of Linux... That's the normal way these things are handled now, right?
The thing I’m still missing is a detailed explanation of what the heck was going on with the recent bogus commit that triggered the banning. Supposedly the “hypocrite commit” research was all done in 2020 and is now in the past. So what was going on with this latest bad commit? The student who submitted it claimed it was generated by a static analysis tool, which kernel maintainers have plausibly called bullshit on. Was that student lying? If so, what were they actually doing? If not, it seems absolutely necessary at this point to prove that they were telling the truth by publishing how that commit was produced, including the tool’s source code and how it was invoked. Even if the campaign to intentionally introduce security flaws was over in 2020, the more recent issue is that the same research group submitted an apparently intentionally incorrect patch and then seems to have lied about its provenance and why it was submitted. Until that’s cleared up any kind of apology feels premature and impossible to evaluate. How can anyone decide if an apology is sincere without understanding what was done?
The latest patch adds a null check around a call to gss_release_msg. The commit message says “ The patch adds a check to avoid a potential double free.”
According to other people in the conversation, this is already taken care of by reference counting (https://lore.kernel.org/linux-nfs/20210407153458.GA28924@fie... ) and the patch apparently does nothing. The commit doesn’t reference any specific tool they’re using, or any bug they faced.
Looks innocuous, but I guess past behaviour from this group left enough of a bad taste for Greg KH to be suspicious.
A later message claims (outraged at being accused of submitting intentionally broken code to the kernel, despite having previously done exactly that) that the patch was generated by a static analysis tool. Ok, what tool? How did you run it? The message where he claims this has since been deleted (by who? Edit: probably never sent to the list, see below) but here is a message from Greg KH which quotes it:
You can see that Greg KH is skeptical that the patch was generated by a tool. It’s also unclear to me if the patch is actually harmless or not. It would be good to get some definitive clarity on that but the lkml discussion of it seems inconclusive.
I would add that the student’s tone in this message feels really familiar as an open source maintainer: this is the tone of someone doing something they know is bad being called out on it and trying to deflect with outrage. The last time I got that tone, it was someone who had created multiple sock puppet accounts to try to discredit my project and force moderators to reverse or apologize for a (completely justified and mild) moderation action. So I can’t help but feel that something fishy is going on here and the UMN research group still isn’t being honest about it.
My understanding is that the "not found" messages were never archived, rather than deleted. Likely they were not CC'd to the list, and the replies added the list back to CC.
My experience of GKH's work isn't generally very favourable; but my estination of him has been increased by this repair. He's created work for himself, but it needed doing. Credit where it's due.
> Looks innocuous, but I guess past behaviour from this group left enough of a bad taste for Greg KH to be suspicious.
The idea of the kind of research they previously did is to submit patch requests with some kind of trick or hidden agenda first, then do some analysis of the results, and then later on publish a paper explaining what they did.
Now here they are again submitting a weird looking and seemingly poorly conceived patch. Who knows what they're really doing? Perhaps they're working on some kind of new paper, with who know what purpose. Maybe they'd find out in 6 months. Maybe it's a failed line of research similar to the previous ones which they won't actually publish. Or maybe they just have no idea what they're doing. Either way, it seems like a waste of time at best. The mass ban and revert sounds like an appropriate move.
Supply chain attacks are the security buzzthreat of day, at least since Solarwinds. Mucking about with the Linux kernel would be a really juicy target for nation-state actors. If you wanted to study this risk, how would you go about doing it?
I haven't done kernel work since BSD4.3 so my opinion isn't particularly interesting. With that said, I would agree that the researchers were naive and their approach has caused collateral damage. However, I'd also argue that the (apparent) topic of research is quite valid. There are a lot of extremely well funded adversaries who are quite talented at obfuscation. "Be careful out there."
Agreed, and as such the Linux community should counter signal to gain deterrence. If you do this kind of thing and you get caught, a simple apology is not enough: you get banned. It's a kind of a game theoretic approach.
> If you wanted to study this risk, how would you go about doing it?
The main factor in doing this ethically is obtaining consent from your research subjects. I believe that some Red Teams test precisely these sorts of security mechanisms in commercial software projects and there are solid comments else discussing procedures they use.
The question always is, could you have learned the same thing with less "invasive" methods? And I think there's a good argument here that that's the case: I.e. the researchers first looked at bugs in the kernel and how their history, finding that such scenarios happened by accident. They could likely have gotten bugs past reviewers they had told that they were trying something like this. lots of options, so going for the most extreme option to "find" something that's not exactly a new or widely disagreed idea is quite unnecessary.
Linux Kernel Community, forever above board, pure, pristine and virtuous, is perhaps eager to cast the first stone against these researcher / activists in order to clean their own clothes by dirtying others, or deflect attention from their own failings?
Quick search through the Lore for "problematic" phrases:
In future, all communities, and all participants, should only be headed by the most ethical, upstanding, righteous, virtuous, pure, pristine, moral, kind, fair, balanced, unbiased, and wise humans, and these traits must be established through rigorous vetting and investigations.
Any violations should require extensive and genuine apologies, the genuineness of which must be assessed through committees convened for that purpose, and apologies must be examined from every possible angle to determine if they do really feel genuine contrition or are just faking it. In all circumstances the omniscience of the LKML in knowing the true feelings of apologizers is final, perfect and can never be questioned. /S
But you have a choice to interact with community or not to interact with an community, difference is that researcher's of “hypocrite commit” did not give such choice to community members, as of today there is overwhelming indication that this researcher's violated basic and fundamental principles of scientific ethical research.
But if I choose to interact with the community, does my choice to present myself, justify any and all abusive actions they may volley at me? Oof coarse not.
The point would ordinarily be a good one, but I think you've misused it.
The Linux kernel community, by your logic, chose to interact with and be open to patches which were not properly vetted.
This framing seems secondary to a seemingly more important point, which is, nobody is fucking perfect.
In LKLM or in UMinn. The light now shines on UMinn, but to me, the feigned horror of LKML at UMinn's transgressions, and the lack of commensurate outrage at the many abusive, toxic and hostile transgressions of the LKML community, presupposes a terrifying "normalization" of abusive behavior within the LKLM community.
By sooch notion, it's safer to interact with the UMinn transgressors, than it is with LKML, who here fail to even once question their own guilt, introspect upon their own created violations of trust and ethics, but heartily condemn the egregious breaches of trust of others.
I'd rather hang with the contrite, or at least hangdog, scallywag, than the quicktongued accusers blind to their own abuses. I'd feel safer there anyhoow.
Context who does what actions is important, you seem to equate university "professionals" research actions to volunteer community actions, it is false to large degree, like comparing apples to orangeries, additionally, consent to be deceived is critical factor. No one in Linux community promises you to be nice[0], and Linux has stepped aside due to his past behavior[1]. To me main point who initiates actions eg. did you willfully performed some actions or you where manipulated to perform some actions.
So it's bad for the researchers to deliberately deceive the community because a rule says patches must be good faith but it's not bad for the community to be abusive because there's no rule that says it shouldn't?
so the community can be abusive but no one may abuse the community?
If so, I think that sounds like the basis for bullying culture which perhaps unsurprisingly is what is observed.
I think it's naive for them to assume that all submissions are done in good faith, just as it's naive for participants to assume that all interactions will be nice. It's not the community's fault they were deceived nor is it the participants fault they were abused.
My point is there is an equality. That personal responsibility, ethics and integrity should apply to everyone not just to some labeled group.
What it sounds to me like you're saying is, it's the researchers fault that they deceived us and we have a right to be outraged but it's not our fault that we're abusive to participants and they have no right to be surprised at that. Am i hearing you wrong?
my point i think is to draw attention to the apparently unequal application of these principles of ethics responsibility and integrity. Additionally I'm a little worried that your reluctance to compare your different groups is an attempt to conceal, and justify, a continued unequal application of responsibility ethics and integrity.
That sounds pretty harsh towards you and so I doubt that's actually what you're saying because I don't think you're someone who lacks a moral character like that. I think probably you and I are just misunderstanding what each other is saying. Perhaps.
For instance I don't think either of us believe that bad behavior on either side whether the deception of the researchers or the abusiveness of the community is okay.
I suppose what I'm doing is addressing, or trying to, to contrast the reactions of either side to those things and expose what I see as an unfairness or a double standard or an unequal application. hope that helps you understand what I'm saying. Anyway have a good doi
It is not just bad researchers behavior, it not just unprofessional, it is unacceptable in scientific community, this is an fact, this is where most of outrage is coming from, but at same time similar unethical behavior is legal to a degree[0] in business organization.
No one is arguing that any online community perfect, no organization is perfect, but organization imperfections is not a justification for unethical behavior toward given imperfect organization, to a large degree, there rare case where it is justified, but this not the case here, at all what so ever.
> I think it's naive for them to assume that all submissions are done in good faith.
This is false, there where numerous attempts to submit backdoor to linux kernel, and most in the community are aware that such attempts will never stop, this why also some people criticize that there's study is just another scientific garbage paper.
What is unacceptable in name of science submitting commits in bad faith without informed consent.
What is unacceptable in name of science or otherwise ATTEMPTING an unauthorized penetration testing, this is illegal.
I guess you think that it's okay for the community to be bad in some way but it's not okay for researchers because science ethics should be higher than community ethics.
It's an interesting perspective I guess it comes from the idea that communities are sort of unstructured and it's not as important that they have ethical standards as high as science because science should be more objective and people need to be able to rely on it and trust it. I guess someone could adopt this perspective that seems to excuse community bad behavior while condemning researchers behavior because they just want to get out of any accountability in this particular case. but I'm assuming that you genuinely feel like this and have had this view from before this particular incident.
I simply disagree with this view, I think everyone should behave well or at least held the same standards.
It also seems that you're (or you were when you commented) very angry and hurt by the actions of the researchers and so I guess that you're probably involved in the Linux kind of in a way and felt betrayed by this action or I had some similar experience with scientific people in the past. I wonder how much the opposite of the community is a result that they were successfully deceived and if they had caught these patches and prevented them from being merged and their reaction may be entirely different even if the research is behavior was the same. But I suspect that would still be outrage because probably a lot of people share your view to some extent but the researchers behaviors should be held to a high standard than their own.
Just seems to me that you're not really open to hearing my point of view on this and that's got nothing to do with the merits of my view it's just where you're at. But because it seems like you're not able to hear my view on this or engage with that I just didn't really feel there was a point to continue the discussion. Have a good one
Yeah. Please find another project, that doesn't involve deliberately breaking other people's work, and screwing up the kernel that a billion people rely on.
If that means changing jobs, most of us have been there. Looking for a job sucks, but if you screw up that badly, then you do have to consider stacking shelves in a supermarket; you're not fit to work on a public project of such importance.
While there are other requirements, a sincere apology cannot in any way entertain doubt about the fact that there WAS harm.
Truly acknowledging the harm done is foundational to a real apology, and most of us (myself included) end up sneaking in weasel words or phrases like this.
Psychologically, its nice for the apologizer, since it allows one to think "i'm being good by apologizing, but maybe I didn't do anything bad after all?".
But from the apologizee standpoint, these phrases are often devastating and can make it clear that the apologizer has no real recognition or care of what happened.
Personally I've worked pretty hard to try to remove these sorts of phrases from my apologies. It's not easy. It makes you feel much more vulnerable and you really have to let whatever you did sit with you in a very uncomfortable way. But it's worth it.
> a sincere apology cannot in any way entertain doubt about the fact that there WAS harm
Someone will always be apologizing wrong for some. Some interpretation of what harm there was, is not necessarily the same as my interpretation. There are too many ways to construe what harm there was or may have been according to others to satisfy everyone addressed. This is an efficient wording that doesn't explicitly satisfy your (and many people's) specific issues out of "the community", which illustrates the point.
"If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him."
I understand doubting the sincerity of the authors, but this argument hinges here on them saying "any" instead of "all" and they are often used interchangeably in casual conversation.
The issue I have with the letter (not being involved at all and just following) is that it feels like an apology of the bully at school that took your lunch money, and the parents found out.
Honestly, for all we know this letter could be meaningless. A real good actor would also reveal the other commits in order to have a full disclosure.
Trust isn't based on promises, trust is based on past actions. Therefore they should be treated as such, as their actions are evidence for being a potentially malicious actor.
I'm still convinced Greg did the right thing here, as it's better to be safe than sorry in this case due to the sheer scale of an attack vector that the actors might have introduced.
Even if other commits of good actors at the same University are now treated with more attention to their code, I think they are a casualty that was predictable in the moment the ethics commitee signed off the paper's research procedure.
In researcher's shoes, I would retract paper from IEEE, only then send open latter. Because they did not do this, does not looks like there's open letter is sincere, I am sure researcher's aware that there where multiple complaints regarding there's paper before ban. The least they could have done is acknowledge such failures in open letter.
> end up sneaking in weasel words or phrases like this.
Honestly, based on the way they handled their "research" in the first place, sneaking weasel words into an open apology letter is entirely par for the course...
This interpretation looks flawed. You discredit the entire message based on a single word--a very general word whose meaning you pinned to a definition which results in the most negative interpretation--and you also ignore the fact that they explicitly state the damage it caused in the same paragraph.
To clarify: I'm not arguing this is a good or bad apology; just that the justification provided here looks flawed. Human speech isn't a programming language; it's not well defined and it's more than the sum of its parts. One can't derive conclusions about a text by analyzing a tiny subset out of context.
If this was a transcript of a spoken apology I might be of your train of thought, but this was a composed letter looked over by (ostensibly) multiple folks. The wording was a choice.
Strongly agree with your sentiment; any premeditated/composed message is held to a higher standard (and rightfully so). I'm not arguing that one shouldn't analyze the wording, but that the analysis here is flawed (for the reasons I stated).
I would argue at this point in collective awareness of public communications, using a form of "sorry if I did any harm" regardless of specific words used is an explicit decision by the writer to not accept full culpability. I agree with original comment, when you see a weasel apology introduction, reading the rest is of little value.
> I would argue at this point in collective awareness of public communications, using a form of "sorry if I did any harm" regardless of specific words used is an explicit decision by the writer to not accept full culpability.
Based on my experience with the general public, with academics, and with industry folks, the criteria for what constitutes a sincere apology is not known by the majority. Furthermore, many of those who have heard the criteria do not accept it as a gold standard, and disagree with it (even those who are not being looked to for an apology).
The recipient can always choose to accept or not an apology. However, I find it quite distasteful to attribute intentions to someone simply because they did not follow a recipe (even if the choice not to was intentional).
I find it nicely clear in German. If you wrong someone, you're loading guilt unto yourself ("mit Schuld beladen"). You'll then ask to be forgiven ("um Entschuldigung bitten" is "asking to be relieved of the guilt"), and the other party can lift the guilt ("entschuldigen"). You can also say "ich entschuldige mich", which skips that step and is essentially "I absolve myself".
While it's common in colloquial German to use the one-step-absolution and skip over the possibility of the other party not absolving you, it's also often considered rude when it matters and has a taste of "but not really". It's fine when you accidentally stepped on someone's foot, but not so much when you've stolen their car.
“I’m sorry if I hurt you,” is a very different statement from, “I am sorry I hurt you.” One makes responsibility conditional, and the other takes responsibility.
Strong disagree. You should always apologize for _what you did_ not the _effects_ of what you did. Lead with and more strongly emphasize your actions and how they were inappropriate.
I think it's both unfair and non-constructive to pick apart a apology letter based on one word like that. Let's assume good faith, especially when the writer's English might not be their first language (based on their name).
We are talking about deliberate security vulnerabilities in the Linux kernel, a piece of critical infrastructure. Frankly, this is not the place to assume good faith.
The "hypocrite commits", according to the researchers' own paper, originated from "random Gmail addresses", and the researchers continue to claim none of those got merged (though since they haven't told us what they were, ...)
The additional claim is that some of the commits not covered by their "hypocrite commits" (and thus, submitted from their UMN addresses) contained security bugs, deliberate or not, and the loss of trust in the researchers is sufficient to justify reverting all of their commits until they can be reviewed.
I think this response misses the point. The purpose of an apology is to make the recipient feel that you're sorry. That means thinking about how word choice is received is critical in crafting a good one. Your parent isn't saying the author's choice of words is in "bad faith", they're saying the author's word choice falls short of an effective apology due to a mistake that's common and easy to make. I agree, and I have done this myself in the past.
It seems like a nitpick to me, since the rest of the apology uses the proper choices of words, though. "The method used was inappropriate", "we made a mistake", etc.
The apology is also specific about what they did wrong despite their intentions. It really is a good apology after reading past the first six words.
There isn’t much difference and both are really correct. The problem with ‘any’ as raised here is the ambiguity. It could be taken to mean ‘any harm, if it occurred’ or ‘any single bit of harm that did occur’. So in this sense ‘all harm’ would be safer and less ambiguous.
Not correct, at least in this context. "Any harm" means there could be no harm, or some harm, but "all harm" admits there was harm. I.e. "any" is not an admission of doing harm, or an acceptance that it happened.
If you want to apologise, "all harm" admits you caused harm, which seems necessary for an apology. "Any" is a bit like those "I'm sorry if you were offended" non-apologies, though in this case the rest of the message does better than that.
That depends on the sentence context.[1] As used here it is fully correct; "we sincerely apologize for all harm our research group did" would be highly anomalous, whereas the phrasing they actually used is conventional.
"We sincerely apologize for all the harm our research group did" would be a little less unnatural than "...for all harm...", but it's still an unusual choice of wording that ends up sounding like you want to emphasize that you did an unusually large amount of harm.
A more standard phrasing that includes the word all would be "We apologize for any and all harm...".
The standard form I know is actually "[I am deeply sorry] for any harm I may have caused..."; the letter here does not use a modalized verb. (Compare their "We sincerely apologize for any harm our research group did...".) In that sense it's more definite than usual. The criticism above is strange.
(The reason for the modality in the standard form isn't really to leave open the possibility that you didn't do any harm. It's to leave open the possibility that you did harm you don't even know about -- and therefore can't apologize for specifically.)
[1] In general, any occurs only in negative sentences, though in the details there are several types of sentences that are sort of "honorarily negative" for the purposes of allowing any and other words that obey the same restrictions.
You want my opinion? It's possible, it's less standard than the "for any harm" form, but it means roughly the same thing.
It wouldn't have occurred to me to discuss it above, because I thought I was contrasting any with all, and neither is present in that example. But I'd rate it above most of the alternatives discussed (while still below "for any harm").
1) "We apologize for any harm we might have caused"
2) "We apologize for any harm that we caused"
3) "We apologize for all the harm that we caused"
(1) is the least apologetic. This could be interpreted as saying "we might or might not have caused harm, and we think we didn't but you think we did, so we're going to apologize for your sake, but we're not really sorry because we didn't do anything wrong from our perspective".
(3) is the most apologetic. You acknowledge that you did something wrong, and that you're apologizing for it. This might be followed up with a specific list of the things that you did wrong, and that you are apologizing for. That would make for the most sincere apology. This could be interpreted as "we caused harm and we're sorry, whatever harm you think that we did, we agree that you are right, and we are sorry for all of it".
(2) is partway between (1) and (3). You acknowledge you caused harm, but you won't enumerate the things that you did wrong. So, you leave yourself a little bit of wiggle room. This could be interpreted as "we caused harm and we're sorry, but we didn't cause that much harm, we think it was actually quite little, you think it was a lot, but we're saying sorry, so let us go with this apology".
A sincere written apology should be 3. You needn’t misrepresent your own intentions to do this, but it does require thinking about the specific harms you have caused (perhaps unintentionally) and enumerating them in a way that doesn’t minimize their import. That is the anatomy of a true apology. It requires taking the other perspective as fact.
I would go with "We apologize for the harm [that] we caused" which seems the most natural choice while explicitly acknowledging that some harm was caused. "All the harm" does indeed come across as a bit inflated, as other commenters have mentioned.
As others have said, 'any harm' implies that there might be none, while 'all harm' implies there is definitely some harm. So, 'all harm' is definitely the better choice of words for an apology (in this context.)
I am surprised to see people hung up on this, because the rest of the letter acknowledges specific harms done. To me, it really seems like a minor thing, and something that I might have written (as a native English speaker.)
I don't think it's that weak of an assumption. First, considering all the people with foreign names in relevant groups (multinational corporations, or in this case, a researcher/ student at a university), I would say the chances that their first language is not English is high enough to be a consideration. And even if it is, there are also regional variations regarding what are common and accepted ways of phrasing things. Second, similar to the rule used here at HN, giving the text a more charitable read should be the starting point.
I read it as a comment that was criticizing the language, not the intention of the apology. As such they are not inferring that the apology was insincere but pointing out language that in the apology that the posted has often added themselves that gives the appearance of insincerity of the recipient. To me that is the strongest possible interpretation of the comment you are replying to.
those guidelines only apply to members of this community. Not an outside article. We should assume all users on hn honour the guidelines. We don't need to assume outside articles do.
This is probably because they don't mean the apology, they were forced to write it by their administrators.
And to be honest, I kind of agree with them. I don't really see what they did here as particularly bad. They demonstrated a very serious vulnerability in the linux kernel development process. I guess the harm they caused was wasting maintainers time, a bit? But what we all got out of it is the knowledge that real bad actors could easily have done this too. It's odd to me that people are focusing on like, the etiquette here when such an important vulnerability was demonstrated.
Sure, but so does every website that AB tests a landing page. Consent is only relevant when there is some risk to the subject (or something they value, like privacy, etc).
What about the risk to the reputation of anyone who approves these patches? Or of intentinally buggy patches making it into the wild and being exploited?
There is also the risk that a third party sees the patch on a mailing list, and merges it to his internal branch because it looks like it is fixing one of his problem.
But I'd argue it is his own fault to use unsupported patches.
Me and common sense. I don't believe consent is relevant when there is no risk of harm to the subject. IRBs and the GDPR are overly aggressive on this point, probably as a reaction to real and important violations of privacy. But the idea that A/B testing the color of your CTA button on a landing page requires informed consent is absurd.
There's lots of systemically horrible things that can happen if you're not careful about what you allow. If you're interested why these ethical principles exist in the United States, I suggest you at least skim the Belmont Report
> There's lots of systemically horrible things that can happen if you're not careful about what you allow.
Of course there are. But what specifically are the harms that are going to be caused by either this research or a landing page A/B test without a click through pop up? The existence of theoretical harms for broad categories of potential research does not have a whole lot of bearing on these specific lines of research.
> If you're interested why these ethical principles exist in the United States, I suggest you at least skim the Belmont Report
I read it. It was interesting, but I don't think it's particularly relevant here. The only prong of its test that would be relevant to this experiment is "respect for persons". The idea that somehow not revealing the bug or the experiment was intrinsically harmful as a violation of a person's moral autonomy.
I don't buy that line of reasoning, and I can't really think of any valid consequentialist justification for it, and the report itself does not attempt to justify it either, as far as I can tell.
If I sit across the street from your house in a van and observe your life, noting down the times you come and go, and logging what I can see through your window, and then using that data to market things to you, would you agree that you'd rather be able to provide and withdraw consent for this activity? No harm done to you.
> But the idea that A/B testing the color of your CTA button on a landing page requires informed consent is absurd.
A/B testing on major platforms is much more sophisticated than that[1].
It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
So, yes, it should definitely require informed consent and opting in.
> A/B testing on major platforms is much more sophisticated than that[1].
On some it is, and some it isn't.
> It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
This is just a long and emotionally laden way of saying "changing images and text to see which works best".
> So, yes, it should definitely require informed consent and opting in.
Guess we're just going to have to agree to disagree then. I don't see any reason whatsoever to think that this practice is harmful to the subjects being experimented on.
At worst, it's harmful to society in general because it incentivizes consumerism and potentially self destructive behavior. But that is completely orthogonal to the issue of informed consent. If you got perfectly informed consent from 10,000 people to tease out the perfect pitch text, and then deployed it against the rest of the population, the effect would be exactly the same, whether or not you got consent.
Ya, they wasted a bit of their time. That wasn't very nice. But it's hardly the great crime it's being made out to be, and what they did was a huge public service. It's impossible to over-state the importance of linux kernel security.
You don't get to decide what you think is important and then burden others pushing it.
If you think testing security in random attack way is important tell the people who run the project. They are the ones qualified to make the calls.
It's just a scammy move at best. To me it's borderline criminal to sabotage a project like that.
This is the part I'm personally most interested in. Research generally has pretty strict ethical regulations about consent. I'm wondering if this would qualify as a violation of any of their university's or the conference's rules.
The purpose of the research was to publicly announce that that the research subjects (who did not consent to being studied) messed up. The research consisted of submitting patches specially crafted to make sure the (non-consenting) subjects did indeed mess up.
Ya, I get that. But the point was to demonstrate that the project was vulnerable to this kind of attack - which it was. That's an extremely important finding.
Their biggest crime seems to be they just aren't great developers. If you look at the patches they presented which to their knowledge were correct you'd see they couldn't have done anything useful. Because of their earlier work this was taken as intentional malice, but if they had been submitting great work it wouldn't be an issue.
For me, it reads like this "With the best intentions, we were helping you. Unfortunately, you are too stupid to not realise this. We are sorry that we hurt your feelings, but please let us continue in helping you."
When you get such an apology, best thing is to avoid such people. Because it's clear they do not understand where they went wrong.
They should have either apologized with "we made a very big mistake that had a negative impact, it did more harm than good". Or they should have argued with real evidence on how they improve the Linux kernel, like refer to real exploits that they fixed.
This is just a "we're sorry that you don't realize we are helping you"
Your critique seems more about waffle words in apologies and less about this specific apology. While they do waffle in that sentence, on the whole, their apology seems pretty sincere. They acknowledge their error and the harms they perceive they caused. As a linux user, I am inclined to accept.
Basically, warn in that you’re going to do it at some point in the future. Then do it in a time-frame of maybe 6 months or so from another ID. Maybe get some of the higher-ups to sign off (say Linus or Greg in this case) beforehand. At the very least, engage with the community...
383 comments
[ 3.9 ms ] story [ 352 ms ] threadI hope I'm just being overly sensitive here.
The core issue here is that the system under which they were working, and the researchers themselves, did not consider this work to amount to unethical human experimentation - even though the work directly involved infiltrating and exploiting humans and human social systems. There is no mention of this nor of any substantial desire to understand or address this systemic failing and, until there is an unconditional acceptance of responsibility for the harm caused and a real actionable change for the better with accountability, I don't see how we can consider this matter to have been apologised for.
This bit just reads like "sorry you're upset".
I would have expected something along the lines of "sorry, we should have asked the maintainers and we understand why it was wrong not to", instead of "sorry, we didn't ask the maintainers, but this is why we didn't".
Legitimately these folks have some kind of personality defect. They observed something that everyone already knew and then decided to act on it and pretend they were doing something interesting and new rather than just being twats. They should be treated exactly as should be based on their actions, regardless of their claims of being researchers. This would be a perfect cover for being on the take from a government agency. They should be investigated to ensure they aren't actual malicious actors.
It takes almost no imagination to think of hundreds of "security vulnerabilities" in the world, yet the vast majority of people realize that it's dumb to act on those observations.
[signed personally, by every involved researcher, professor, IRB member, and all of the relevant university leadership including the board]
Something like that would do nicely, in my opinion. Then they'd have to absolutely follow through on it. Ideally, they'd also ask involved members of the community so harmed, among other experts, to independently review and watch the process to ensure no further harm is being done, progress is made, and public accountability is kept.
I wouldn't trust those researchers again, not for anything. Free software depends heavily on trust. It's astonishing that it works so well; but if you break trust, it's very hard to repair the fractures.
The legal arrangements around free software are fairly fluid, because there hasn't been much case-law, because there's not much money sloshing around for Free Software developers to pay lawyers. Banning pull-requests from that college is much cheaper than sueing.
I hope the whole uni stays banned until the ethics board issues their own apology.
Then let it go, I say. People make mistakes. We shouldn't blame everyone for the mistakes of a few.
But we have to protect the Linux kernel; billions of people depend on it's correctness.
While I had a bunch of notes about what makes the sentiment behing an apology meaningful, it's really separate from the issue that this should have been done privately. A public apology is just another thing about them and their message. They're performing and trying to draw in sympathetic members of an imaginary audience. There is no humility in public displays. I would be surprised if anybody asked for it, and doing it without asking is just more of the same type of behavior. There is no such thing as a public apology.
That said, I have no doubt they have suffered personally over this, and have some compassion for that suffering itself, but not sympathy for this performance. There is no gesture that restores trust. When their contributions and acomplishments become more remarkable than this issue, they will have restored it, but like most things, there is no "back" to go to. Maybe that static analyzer will be the ultimate bug finder, as succeeding at that is probably their only out.
You seem to be implying that one apology is as good as another and that is absolutely not true. To me is is absolutely unfair to expect that people will treat a zero effort "sorry your feelings were hurt" apology the same as a deeply heartfelt apology by someone who feels deep remorse.
By it's nature, an apology should be an act of making amends to a person your have wronged. If the words of your apology don't do that, you have failed, even if you had good intentions.
If the author of the apology isn't a native speaker of english, this seems like a document of sufficient significance to merit asking a friend or colleague for a proofread. (Which you should do in a case like this even if English is your first language.)
Software is complex. Wasting maintainer's time is harmful to the maintainers and to those who use the software. If the goal is to explain what sort of exploits to be on the look-out for, just write the paper without doing the exploits.
As for the letter, their goal is to recover their reputations, not to apologize. I suppose a lawyer helped them in spots, but there is still a certain truth that shines through: they want to continue on doing this sort of thing, because they are oh-so-clever and their work is oh-so-valuable. Nice try, but the academic community may be better off without these folks doing this sort of research and influencing students to follow their methods.
I'll be interested to see how they react to feedback / responses.
Either this is a bald-faced lie or they didn't think through the implications of their research, neither of which bode well for their trustworthiness in future. However, if it's the latter, then they possibly deserve additional leeway.
On the other hand, I wonder whether this letter would have eventuated without the ban - if not, then they've likely been forced to write it, in which case any remorse is either selfish (i.e. they've been professionally reprimanded and feel bad about getting caught, or making such an egregious error in judgement), or hollow.
> I agree, but let's give them a little extra benefit of the doubt. I thought as I was reading it that it seemed stilted and forced, then I wondered if the author(s) don't speak / write English as a primary language.
https://www-users.cs.umn.edu/~kjlu/
It looks like you might be right about this, but I think that reading it as stilted and forced is still accurate.
Context is everything, of course. I think you're right that it's tainted by the fact that they absolutely did not have a choice, and coming from people who've deceived the same tribes they're now trying to apologize to.
They university, and indeed, the researchers, only acted once the ban was imposed - i.e. they were getting away with it without reputational damage, and only once it was stained did they actually do anything.
ETA: GKH on reverting the patches[1]
> This patchset has the "easy" reverts, there are 68 remaining ones that need to be manually reviewed. Some of them are not able to be reverted as they already have been reverted, or fixed up with follow-on patches as they were determined to be invalid. Proof that these submissions were almost universally wrong.
[1] https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...
https://lwn.net/Articles/854064/
What the authors did is wrong, so did GKH actions. Let's be honest in calling BS out.
Yeah, the authors are claiming those past 190 patches are fine... but they also essentially did the same thing when they submitted the intentionally bad patches too.
So the question is, how much do you trust them right now? And given they just submitted intentionally bad patches, I don't think it's very surprising they don't have a whole lot of trust.
And if you don't trust them, then you've got 190 patches active from an untrusted source with a history of submitting bad patches. That's not a particularly good situation. Why wouldn't you revert them?
The specifics of the situation make the whole thing substantially murkier, in my (entirely unrelated) opinion. But take a broad enough view, and this is pretty much the default path. However the specifics make this a bit of an oddly exceptional situation, all things considered.
Blocking an entire University seems extreme; but just think of it as a sanction on the University's ethics board. I think a (short, and very explicit) apology from the ethics board ought to suffice, to get the Uni off the hook.
And I think those researchers should not be allowed near Free Software again, unless their pushes are going to be rigourously scrutinised.
- "We are sorry for the harm we caused" instead of "We're sorry for any harm we caused"
- Not trying to explain their actions in the first paragraph of the apology! I think most folks are aware of their intent by now, and leading with yet another explanation just makes the whole thing feel disingenuous
- Avoid saying things like "this has been painful for us as well"
Unfortunately, it has so many hallmarks of a non-apology, and it's hard to look past those given the context.
As others have mentioned, they can clear this up with their actions going forward, but it will take time to rebuild trust.
Is that a thing? Like there's some non-apology Bingo card you can fill out? I don't see a connection between the criteria you listed and genuine-vs-false contrition.
You may perceive these things one way, but ultimately you can't know the minds of others well enough to tell if they are sincere or not about anything. You don't get to just declare yourself the arbiter of their feelings because they used "any" instead of "the".
https://www.google.com/search?q=non+apology+bingo
I would expect it to be there and would minimize attempts to marginalize my apology with direct, precise, and inclusive language.
For every word created or omitted to those ends, the number of negative comments will be reduced.
Fact is people take it how they take it and feel what they feel.
There really is no "can't" in any of that.
...which is why the consistent feedback to those ends is here in the discussion.
How else is this to be done and be meaningful, not easily gamed?
Serious question.
I tend to have pretty dry affect at certain hours of the day, which has caused considerable frustration for myself and others when I know I'm sincere but they don't. Nor does it make much sense to me for that sort of mind-reading to take place democratically. What do we then gain from it?
At best, you get an apology for a misunderstanding over the previous apology, plus a second draft that the masses might like better. Then you still get that lingering contingent that says "you just did that to placate people! Now we really know you don't mean it!"
The cultural apparatus for "Saving Face" is completely broken on a build failure for missing dependencies.
The first two sentences seem sincere. This reads as a real apology.
As far as giving benefit of the doubt, it's likely not done out of malice in the first place. Just a combination of poor reasoning and doesn't exactly clear up if they even considered alternatives to control their variables. Unfortunately, it seems like they were already given the benefit of the doubt from their first offense, but did not take any lessons away from that, so it would be understandable if the maintainers kept their ban.
In sum, they show not remorse for doing wrong.
I completely agree their experiment is unethical. However, it's not actually clear cut to most researchers the ethical bounds of their work, especially for study papers that's never really been explored before. Ethics in of itself is largely a active subtopic for many areas in CS, not only security research. AI is one area where qualifying potential harm to human beings remains largely controversial. Ask any ML scientist, and they'll tell you that determining the ethics of a project is not their responsibility.
The ethics around research that involves deception have been pretty well established and are are several good comments here explaining them.
Every scientist is personally respsible for the ethics of the research they conducts. Full Stop, no caveats allowed.
If your research is in an area where the ethics are controversial or grey, that means your need to spend MORE time considering the ethics of your research, not that you get a free pass from being responsible.
If a any scientist espouses the opinion that determining the ethics of their projects is not their responsibility, they should be permanently barred from recieving grant money.
Ethics in computing research remains an active research area. This incident will be used as a case study in the future, but it's not that well established. Many people have been using anecdotals, which honestly don't fit the scenario because so many variables and parameters distinguish other types of pentesting from this. And disappointingly, not a single post has actually produced the documents that establish this.
Arguably the first set of guidelines for ethics in computer security research [1] was published in 2012 and not yet widely taught in Ethics lectures (I only know about it because I learned Computer Security from one of the authors).
On identifying harms:
> "Challenges identifying harms in ICTR environments stem from the scale and rapidity at which risk can manifest, the difficulty of attributing research risks to specific individuals and/or organizations, and our limited understanding of the causal dynamics between the physical and virtual worlds. As with all exploratory research, it can be challenging to articulate benefits such that subjects can make informed decisions. In ICTR our ability to qualitatively and quantitatively foresee the probable benefits is particularly immature."
On this type of research:
> "Research of criminal activity often involves deception or clandestine research activity, so requests for waivers of both informed consent and post hoc notification and debriefing may be relatively common as compared with research studies of non-criminal activity."
This isn't a huge change from 30 years ago since Moor [2] wrote his thesis on Computer Ethics, see:
> "A typical problem in computer ethics arises because there is a policy vacuum about how computer technology should be used. Computers provide us with new capabilities and these in turn give us new choices for action. Often, either no policies for conduct in these situations exist or existing policies seem inadequate. A central task of computer ethics is to determine what we should do in such cases, i.e., to formulate policies to guide our actions."
Researchers themselves are far from educated on this topic; you won't ever explore this in depth unless you're in this particular sub-field. IRB/REB boards are considered the most qualified but are possibly too outdated to navigate around this. It's a whole mess, there is currently a lot of questionable research in many areas of computing, but the clock moves forward.
[1] https://www.dhs.gov/sites/default/files/publications/CSD-Men...
[2] https://web.cs.ucdavis.edu/~rogaway/classes/188/spring06/pap...
These guys are malicious clowns, who came up with an idea that would hurt Linux, but advance their careers, and they went all in on it.
The words are there. It's not up to us to decide if they're "genuine". No one's a mindreader.
The tendency to view apologies as fake seems more often to reflect how harshly we view the one making it.
It's like telling your partner "I'm sorry if I hurt you". No, you hurt them. Apologize for hurting them.
Agency returns as trust does.
Regardless, you can't genuinely apologize for things that aren't in your control. (You can and should empathize and sympathize with their feelings about it.) It may be a convenient fiction at times, but there are many downsides to this type of boundary blurring. I think you do the recipient a disservice by implying they have no agency—especially if they believe you. You also face the danger that your apology seems insincere or you misjudged their feelings, which could actually make matters worse.
If we really had complete agency over our feelings then I guess we’d all just choose to feel great all the time. Doesn’t seem to work that way.
In my view, the extent to which we can influence the emotions of another is the extent to which the recipient themselves has given implicit permission to do so; which is to say, they have agency. I don't think that simplifies into being able to choose to feel great any old time we like. I also don't think it at all excuses bad behavior or justifies a lack of empathy. But I will say that I feel a lot more good moments and less frequent and intense bad moments since I adopted this view and took responsibility for my own emotions.
There is no universal rule that can be applied to determine when one vs. the other is appropriate, but seemingly innocuous actions can have negative impact, and apologizing for the innocuous thing may not always make sense.
To be clear, this is not universally true. There are situations where apologizing for the action makes sense, and situations where the action that led to a negative outcome are either inconsequential on their own, or there might be a myriad of factors leading to the need for an apology.
Apologizing for the action can also come across as disingenuous or passive aggressive in the wrong context. This is especially true when the action is well-intentioned, but has the wrong effect. Taken to an extreme, this sounds like "I'm sorry I tried helping".
The phrasing is such that it acknowledges harm was done. It's an apology, full-stop.
You have to acknowledge (in full) what it is that you are apologising for. That's the most important thing. Promises are meaningless.
It's not possible to "be genuine"; especially in a public post, nothing is genuine, everything is PR and smoke. Both the researchers and their ethics board need to grovel. If they did that, this tornado-in-a-teacup could be over in a month or two.
But keep those researchers banned.
There is no need for the researchers to be in any specific state of mind or even apologetic as long as they don't try to submit bad faith patches again.
Really the only reason there is even a need for a public apology is to signal how much pressure they are under so other people think twice before doing the same silly thing. Otherwise they could just apologise to the maintainers directly.
To me, it's always a distinct sign that an apology isn't sincere when it hedges apologizing for "any harm" that they might have caused. In my mind, a sincere apology clearly acknowledges harm that has been caused and apologizes for that.
Well constructed apologies are hard to seethe over.
This one could be better.
Many, myself included, believe it should have been.
Listing the harm you caused is also an important part of the public record, so that anyone else considering a similar scheme in the future can see what consequences were agreed to have happened the last time.
The Linux kernel team had to revert changes in over 200 files. They will also have to go back and review hundreds of commits to make sure they do not contain any malicious code.
The technical effort in addition to all the time spent dealing with the backlash of this is a waste of time for a lot of people doing important work that benefits everyone.
So, what else can be said... Play stupid games, win stupid prizes. To Mr. Lu, Wu, and Pakki: enjoy your ban and worldwide press coverage.
No, they didn't have to. That seems like quite a bit of an over-reaction by mr Kroah-Hartman.
There were months of opportunity for contrition.
They seemed a little too proud of their paper for Oakland'21, and this note comes much too late, for the message to be sincere.
-they apologize for the three patches in 2020
-they claim asking permission would have defeated their research
-they claim the other 200 were legitimate patch attempts (a sample of which were, from my personal reading, from 'innocuous but useless' to 'slightly harmful').
Hard to believe, but plausible.
But surely this proves malice aforethought. If you interact with someone under false pretences to deliberately mislead them, then you are effectively lying to them. In fact, if you are doing so in order to receive something of value from them (such as their time reviewing your code) then it could even be seen as fraud.
Did they tell their IRB that their research involved deception? Before declaring the project exempt from oversight, the IRB should have required that the researchers answer a question like "Does your research involve interacting with people who are not aware of your project and who would act differently if they were aware?"
Until the IRB process at the university contains at least this level of protection against such ethical failings (not just in Computer Science research but across all areas of study) I think it is right for the kernel team to refuse to interact with members of that institution.
However, I find something very problematic. This quote shows it:
"We have learned some important lessons about research with the open source community from this incident."
This is something I don't like. This is not something about "research with the open source community". If anything, they should have learned something about treating human beings as persons and not as involuntary guinea pigs. They should have learned something about not breaking anyone's (not only any open source community) good faith and trust, and respecting that.
They behaved like jerks and they still cannot see that.
They just come off as disconnected academics. Samething with the experiments with algorithms from Facebook. They are so wrapped up in what they are doing, they no longer see the "users" as human beings. There are some times this is almost required to survive like being an ER doctor. If you lose that many people, it could be debilitating unless you were able to disconnect. That's far and away much different than these robots.
They didn't decide to conduct this research on a whim. They had full approval of their university ethics board as well. They published a paper and had it peer reviewed without (as far as I know) anyone immediately calling for their heads.
They can't immediately turn heel face and believe their actions are wrong, always were wrong, and the wrongness should have been obvious to them.
Yet, despite this they recognize the hurt they've caused and are genuinely apologetic and will never do it again. Asking them to dismantle their world view in a week is a bit much, even criminals are given a few years of quiet contemplation before being asked to tell a parole board they have changed their hearts and minds.
If they lack knowledge of why it was bad, what will prevent them from doing another garbage study next time.
This study could have been done with consent, with limited bias, but they chose to go the idiot route.
Are we not allowed to discuss the possibility of someone hacked an university email address and then submit a patch?
> we are very sorry that the method used in the “hypocrite commits” paper was inappropriate
This reads more as "sorry you were offended" than "It was inappropriate and we are sorry".
> As many observers have pointed out to us, we made a mistake by not finding a way to consult with the community and obtain permission before running this study; we did that because we knew we could not ask the maintainers of Linux for permission, or they would be on the lookout for the hypocrite patches.
Bringing up why you did something in an apology is very shaky. Like in this case, it can sound more like justifying / excusing.
Why is that a problem?
Which I'm guessing is why they made it an open letter. This isn't about acknowledging their misdeeds and trying to fix their relationship with the OS community, this is them trying to sneak in spin and justifications to a public announcement to cover their collective asses, to put forth a false narrative that shows their actions in a better light than they acted in.
After all before a ban, there where complaint(s) made regarding this paper, by other researchers and community members.
Perhaps the entire "tech" industry needs to learn that lesson. Non-technical end users should be entitled to that same level of trust as nerds. I can download free open source code, extract a tarball and build the software without worrying too much about scanning through all the files first for phone home/telemetry/OriginTrials nonsense.^1 However non-technical end users who use programs compiled for them by "tech" companies with ads and surveillance as their "business model" are not entitled to the same trust. I cannot think of any justification for the difference.
Audit studies are conducted all the time where researchers lie to people and waste their time.
> Factors Determining Callbacks to Job Applications by the Unemployed: An Audit Study
> We use an audit study approach to investigate how unemployment duration, age, and holding a low-level “interim” job affect the likelihood that experienced college- educated females applying for an administrative support job receive a callback from a potential employer. First, the results show no relationship between callback rates and the duration of unemployment. Second, workers age 50 and older are significantly less likely to receive a callback. Third, taking an interim job significantly reduces the likelihood of receiving a callback. Finally, employers who have higher callback rates respond less to observable differences across workers in determining whom to call back. We interpret these results in the context of a model of employer learning about applicant quality
http://www.econ.ucla.edu/tvwachter/papers/audit_study_Farber...
How else do you prove your process can stop real infiltrators by state actors?
There are many crimes in the world that are only crimes if you do it without consent.
> How else do you prove your process can stop real infiltrators by state actors?
One of the common security controls against infiltration by foreign nation states is espionage being a capital offense. Well obviously not appropriate, i think that its pretty obvious that these researchers would have not succeded if they faced the electric chair like a spy would. So i don't think the comparison is apt.
If you have approval from one person in an org of 100,000 is this still okay?
It has to be, because getting explicit consent from every person in the org at that size would be untenable, and make them artificially extra vigilant during a potential audit window.
Well now scale this to open source. The typical web project has 2k+ random nodejs libs in their dependency tree. These are all separate orgs/individuals.
Real world bad actors are backdooring these projects constantly. I specialize in this area of research.
If every white hat has to get permission from exponentially large dependency chains, they will never even come close to being able to compete with black hats here finding the weak links of the chain.
The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
You have to get consent from someone who has legal authority to give it to you.
> If every white hat has to get permission from exponentially large dependency chains
They don't. They just have to get permission from someone in authority. Different open source projects have different governance structures. Sometimes that is a single person.
> The blackhats set the rules of engagement, for better or worse. White hats should be free to go for it just like with any other vulnerability they evaluate.
If you are hacking someone elses system without their consent (not to mention for your own personal gain), you are a blackhat. By definition. Pretending to be a researcher doesn't change that.
there lies the difference, duh.
Or the the thousands of brew packages that are blindly merged unsigned by 800 people with access?
Who pays to give a white hat as much freedom to find supply chain attack vectors here? Who gets consent from every random student whose code, if compromised, would compromise every major company?
We have created a massive mess, and I don't know that we will be able to hire enough researchers to fix it.
We are going to need unpaid volunteers, and a lot of them.
> We are going to need unpaid volunteers, and a lot of them.
that's absolutely orthogonal to the question at hand, it's not a matter of paid or unpaid, neither being volunteer or not: it's a matter of consent.
if you don't sought consent beforehand either via a contract relationship or a sponsored bug hunt program or the likes, you're a racketeer, not a white hat, and you should (and will) be treated as such.
This, and the previous mail that I found pretty insulting too and seemingly written in bad faith but I'd give the benefit of the doubt, one can react badly to a difficult situation.
This shows one thing to me, clearly, they still don't understand how such studies are conducted. not authorized penetration testing is illegal, to a very large degree.
For example, as far I understand, and as many pointed out; you get permission from members for an study without specifying time or any details of a study, then conduct study in 6 month.
What makes you think this isn’t research?
Not justifying what they did but this apparently is still research.
I pulled this from the website of Oakland 2021
Since 1980 in Oakland, the IEEE Symposium on Security and Privacy has been the premier forum for computer security research, presenting the latest developments and bringing together researchers and practitioners. We solicit previously unpublished papers offering novel research contributions in any aspect of security or privacy. Papers may present advances in the theory, design, implementation, analysis, verification, or empirical evaluation and measurement of secure systems.
https://www.ieee-security.org/TC/SP2021/cfpapers.html
I was pointing out that this is like defining whether or not something is “art” not based upon the work itself, but instead based solely upon whether somebody bought it.
Incidentally, my recipe for apple pie was printed in a newspaper, and so therefore my recipe is “news”.
In each case, a vulnerability was disclosed. With Linux being used everywhere you can be sure intelligence services etc are likely sending in bad patches. If the Linux kernel requires only patchers with good intentions, and doesn’t have other means of catching this stuff, we are screwed.
There's a big ethical difference between trying to exploit a piece of commercially produced software, and trying to exploit the time and actions of humans who are producing software which is given away for free.
I don’t think the free vs commercial aspect is the main issue here; lots of kernel devs are paid for their work after all...
You're right that the OP's analogy breaks down if the researchers were unsuccessful in getting their malicious patches accepted, because then there is arguably no vulnerability to report, and OP said "when security vulnerabilities are disclosed".
Steelmanning that analogy, though, what the researchers were doing was "probing for possible vulnerabilities", which some vendors also complain about, especially if the target is an online service rather than software running locally on the researcher's machine.
In that case, the main flaw in the analogy is still the difference between exploiting software and exploiting humans, so I probably should have focused on that. Nevertheless, there is a small ethical difference in some circumstances between software that is bought and software which is freely downloadable (regardless of the licences involved), since if you paid for something which is defective then you might deserve a refund.
They’re not sweeping issues under the carpet. They are actively addressing those issues.
They then published a paper and thought that this was acceptable behavior from a research/ethical standpoint.
OSS security should not rely on ethics and trust. The default assumption should be that malicious actors are using every dirty tactic they can to get their changes accepted, and that someone@something.edu is no more trustworthy than internal.security.agency@china.gov.cn.
Feels like the OSS community got caught with their pants down and is now overreacting.
One does not conduct security research or experimentation on mission critical infrastructure without the informed consent of the persons maintaining or responsible for said infrastructure.
The stark difference between white (or even gray) hat and black hack security operations is whether those operations are done with the informed consent of the owners of the system(s) being manipulated. To knowingly and deliberately attempt to manipulate critical infrastructure without the informed consent of the persons responsible for that infrastructure is, in my eyes, clearly unethical black hat hacking. It does not matter how many resources were expended nor does it matter if there was no personal injury or destruction of property. A quiet, relatively uneventful outcome of a black hat operation is nothing more than a dose of good luck.
There are many other circumstances in which this sort of clandestine operation would be obviously unacceptable. Consider if a researcher wished to surreptitiously tamper with vaccine supply chains, automotive or aircraft supply chains, fuel distribution supply chains, chemical manufacturing supply chains, etc. Even if the researcher claimed to have the best of intentions, and had a written plan to carefully back-out or otherwise reverse (or prevent from going to production) the intentionally flawed manipulations or modifications of critical supply chain components, I think very few people would consider the research ethical. There are an enormous number of issues, defects, and mishaps that occur even when a group of people tries their best to implement and maintain critical systems. When a clandestine operator decides to interfere, using whatever justification, then reliability of the system can be expected to suffer.
I do not believe the written apology addresses the underlying problem, which seems to be that the University of Minnesota does not exercise sufficient governance over the researchers in its employ.
In fact the argument could be made that there is an ethical duty for every software project, and possibly every individual, to economically boycott the University of Minnesota until they put in place independently-verified changes to their IRB process such that research like this will not be approved again.
If the only "cost" to the university is that one of its researchers has to write a two-faced apology message, then there is nothing stopping future abuses of non-consenting humans. Moreover, with no real punishment meted out, it creates an incentive for other universities to similarly weaken their IRB process to the same level so that they can approve more research projects and remain competitive.
Since when it is a ok to acquire data through unethical means and then publish it in a journal ?
I am a Red Teamer and work with companies to understand how their detective/preventative/recovery controls and processes are working. Here's how you resolve this:
You work with maintainers to get their coordination on the research. You work out a mechanism to prevent submitted patches from being merged (e.g. maintainers are notified before bad patches accepted by code review processes are merged).
You do not tell them when the patches are coming. You do not tell them which identities are going to be used for the patches (e.g. from which email addresses). You do not tell them which area of code will be targeted. You set rules and time bounds for the study.
You wait some amount of time before submitting such patches (weeks to months). Realistically this is all that's needed. If hypersensitive, set this up earlier and let it bake longer.
At this point, you submit patches from a variety of addresses (probably not associated with your university - it is easy to create many such identities). You also can coordinate with other researchers, universities, and companies to submit patches under identities as needed. You also study submitting from yandex, gmail, .cn and other email addresses (because isn't that interesting to know?).
The premise that there's some ultimatum between working with the community and performing the research is on its face incorrect. This is either ignorance or laziness on behalf of the researchers. Clearly, they hadn't taken the time to work with the community to work out an approach that could be mutually acceptable.
OK, how do you test SocEng vectors? Do you obtain consent and coordinate with every employee that might be targeted to receive your e-mail?
>You also study submitting from yandex, gmail, .cn and other email addresses
No, the point was submitting from a known and respectable entity, which might affect the level of scrutiny. They weren't testing a whole patching process, but a specific human component of it.
If it were an obvious phishing URL, like a variation on the company domain, then fine (maybe). But it wasn't.
1. (Windows specific) Opens up a Windows file share, which causes the person to authenticate to the file share, which through PtH/Responder results in their enterprise credentials being stolen.
2. Exploited a XSS or CSRF attack on an internal/management endpoint. Which in turns allows a pivot from external to internal access.
3. Steals a web session, cached password or authentication token, resulting in compromise of employee credentials to be used elsewhere (e.g. reused to access enterprise VPN).
These are just some not-a-browser-0day examples of a single click being game-over dangerous.
How do you do this without a browser vulnerability (and assuming it's not also XSS/CSRF like the previous point)?
Also to distinguish between #3 and, XSS in #2 was intended to mean "persistent stored XSS" as opposed to "reflected XSS". In the case of reflected XSS, this can be chained with CSP bypasses and insecure cookies to grab out e.g. bearer tokens.
My overall point is that heap fung shui 0day not required for 1-click ownage. In practice, I've not had to burn browser 0day to compromise organizations or their customers.
1. It sounds like they'd have to be pretty well targeted against the precise systems of that particular company in order to work. Which would tend to suggest more targeted spear-phishing attacks and extensive recon being done against the company systems somehow before anybody launched a real black-hat attack.
2. At that point, it feels hard to blame the individual employee versus whoever misconfigured those corporate services in the first place. Though I would guess it's fairly common for those kind of things to happen due to many systems being set up without the help of true experts and the unlikeliness of a real attack against them without either a highly-skilled black hat targeting them or securing the services of a skilled and prices pen test team.
2. I agree. Individual employees are not at all to blame. Companies who are blaming their employees for getting phished are doing it wrong. The correct action to take is to inform employees and build the other kinds of mitigations mentioned elsewhere in this topic tree.
To test if employees are easy to pish, or was it for real (black hat)?
I'm usually not testing only whether employees are easy to phish (the answer is pretty much 100% yes). I'm testing end-to-end: can you as a company prevent me from phishing through email protections? Can you detect when I'm phishing your employees? Will your employees report potential phishing emails? Can you figure out (without me telling you) which employees were targeted and which attacks were successful? Can you figure out which credentials/machines would need to be quarantined/rotated/examined?
Even more fun if you were allowed to social engineer your way into the office and steal someone's powered on not-screenlocked laptop :-)
You're talking about vulnerabilities in components, or other software, here. The user, by himself, is doing nothing wrong.
Why don't you just restrict employees to the intranet, then? Why do you give them free roam on the whole internet, but then you tell them "don't click the wrong link!".
Phishing happens when the user does something which is actively wrong. When the user opens an Word/Excel with VBA from an untrusted source and bypasses security restrictions. If they execute/install something unsigned and untrusted from some random site.
Click = fail is just wrong. Links are how the internet works. You aren't teaching anything.
It's particularly annoying where I work, as the company itself sends out a completely unreasonable amount of internal spam every.single.day - often with bad spelling/grammar, and very often with the contents being a single image with rendered text (why?!?!).
Yes, clicking on links is dangerous[0].
0. https://www.bleepingcomputer.com/news/security/google-fixes-...
How can I tell whether I can click on a link? Sometimes there's even something like linkprotector.outlook.com/[very_long_url] in corporate emails.
My usual approach if I'm unsure whether a link is malicious would be to open it in a private window (and probably in a different browser from the one I usually employ), or if I really think it's phishy, I would open it from within throwaway VM.
So, the blanket "click and fail" policy seems pointless to me. If I enter some login/PII, then I can agree I've failed the test. But a click on a link cannot be considered failure.
They weren't, though - their own paper explicitly says they used newly created Gmail addresses for the patches...
"We submit the three patches using a random Gmail account to the Linux community and seek their feedback—whether the patches look good to them."
It is my understanding that this happened and that no bad patches were actually merged.
I also empathize with the plight of the researchers — Linux is a bit different than normal Red Team engagements, in that a normal organization has a hunch of administrative / management layers who typically do not participate in the operations of the system being tested. A VP of engineering at a medium to large company is unlikely to be committing code, much less maintaining the build pipeline etc.
This is not the case with Linux. The people “at the top” are also reviewers, and so it’s pretty likely that notifying them will result in a change of behavior.
I wonder if there is some way to build some sort of Red Team consent “blind trust” organization, such that willing open source projects could agree to responsible attacks, and the attackers could register their work (including disclosure / mitigation plans) with the blind trust ahead of time.
It is not like community members are not for look out for bad commits on every new commit from most committers any way, since at least 2003, I think substantially earlier.
In your experience, do you create a "fail safe"?
So for this study, some way for the researchers to prevent any of their patches from ever being released.
Maybe we can find or make up something unrelated but unsavory about these characters objecting to those pissing in the font of Linux... That's the normal way these things are handled now, right?
(scorching sarcasm intended)
According to other people in the conversation, this is already taken care of by reference counting (https://lore.kernel.org/linux-nfs/20210407153458.GA28924@fie... ) and the patch apparently does nothing. The commit doesn’t reference any specific tool they’re using, or any bug they faced.
Looks innocuous, but I guess past behaviour from this group left enough of a bad taste for Greg KH to be suspicious.
https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah...
You can see that Greg KH is skeptical that the patch was generated by a tool. It’s also unclear to me if the patch is actually harmless or not. It would be good to get some definitive clarity on that but the lkml discussion of it seems inconclusive.
I would add that the student’s tone in this message feels really familiar as an open source maintainer: this is the tone of someone doing something they know is bad being called out on it and trying to deflect with outrage. The last time I got that tone, it was someone who had created multiple sock puppet accounts to try to discredit my project and force moderators to reverse or apologize for a (completely justified and mild) moderation action. So I can’t help but feel that something fishy is going on here and the UMN research group still isn’t being honest about it.
Apparently, they sent html mails, which the list refuses to deliver.
The idea of the kind of research they previously did is to submit patch requests with some kind of trick or hidden agenda first, then do some analysis of the results, and then later on publish a paper explaining what they did.
Now here they are again submitting a weird looking and seemingly poorly conceived patch. Who knows what they're really doing? Perhaps they're working on some kind of new paper, with who know what purpose. Maybe they'd find out in 6 months. Maybe it's a failed line of research similar to the previous ones which they won't actually publish. Or maybe they just have no idea what they're doing. Either way, it seems like a waste of time at best. The mass ban and revert sounds like an appropriate move.
I haven't done kernel work since BSD4.3 so my opinion isn't particularly interesting. With that said, I would agree that the researchers were naive and their approach has caused collateral damage. However, I'd also argue that the (apparent) topic of research is quite valid. There are a lot of extremely well funded adversaries who are quite talented at obfuscation. "Be careful out there."
The main factor in doing this ethically is obtaining consent from your research subjects. I believe that some Red Teams test precisely these sorts of security mechanisms in commercial software projects and there are solid comments else discussing procedures they use.
Linux Kernel Community, forever above board, pure, pristine and virtuous, is perhaps eager to cast the first stone against these researcher / activists in order to clean their own clothes by dirtying others, or deflect attention from their own failings?
Quick search through the Lore for "problematic" phrases:
- site:lkml.org intext:moronic ~ 46 results
- site:lkml.org intext:idiotic ~ 278 results
- site:lkml.org intext:fucking (or f*cking) ~ 350 results
In future, all communities, and all participants, should only be headed by the most ethical, upstanding, righteous, virtuous, pure, pristine, moral, kind, fair, balanced, unbiased, and wise humans, and these traits must be established through rigorous vetting and investigations.
Any violations should require extensive and genuine apologies, the genuineness of which must be assessed through committees convened for that purpose, and apologies must be examined from every possible angle to determine if they do really feel genuine contrition or are just faking it. In all circumstances the omniscience of the LKML in knowing the true feelings of apologizers is final, perfect and can never be questioned. /S
The point would ordinarily be a good one, but I think you've misused it.
The Linux kernel community, by your logic, chose to interact with and be open to patches which were not properly vetted.
This framing seems secondary to a seemingly more important point, which is, nobody is fucking perfect.
In LKLM or in UMinn. The light now shines on UMinn, but to me, the feigned horror of LKML at UMinn's transgressions, and the lack of commensurate outrage at the many abusive, toxic and hostile transgressions of the LKML community, presupposes a terrifying "normalization" of abusive behavior within the LKLM community.
By sooch notion, it's safer to interact with the UMinn transgressors, than it is with LKML, who here fail to even once question their own guilt, introspect upon their own created violations of trust and ethics, but heartily condemn the egregious breaches of trust of others.
I'd rather hang with the contrite, or at least hangdog, scallywag, than the quicktongued accusers blind to their own abuses. I'd feel safer there anyhoow.
[0] https://www.kernel.org/doc/html/v4.10/process/code-of-confli...
[1] https://www.newyorker.com/science/elements/after-years-of-ab...
so the community can be abusive but no one may abuse the community?
If so, I think that sounds like the basis for bullying culture which perhaps unsurprisingly is what is observed.
I think it's naive for them to assume that all submissions are done in good faith, just as it's naive for participants to assume that all interactions will be nice. It's not the community's fault they were deceived nor is it the participants fault they were abused.
My point is there is an equality. That personal responsibility, ethics and integrity should apply to everyone not just to some labeled group.
What it sounds to me like you're saying is, it's the researchers fault that they deceived us and we have a right to be outraged but it's not our fault that we're abusive to participants and they have no right to be surprised at that. Am i hearing you wrong?
my point i think is to draw attention to the apparently unequal application of these principles of ethics responsibility and integrity. Additionally I'm a little worried that your reluctance to compare your different groups is an attempt to conceal, and justify, a continued unequal application of responsibility ethics and integrity.
That sounds pretty harsh towards you and so I doubt that's actually what you're saying because I don't think you're someone who lacks a moral character like that. I think probably you and I are just misunderstanding what each other is saying. Perhaps.
For instance I don't think either of us believe that bad behavior on either side whether the deception of the researchers or the abusiveness of the community is okay.
I suppose what I'm doing is addressing, or trying to, to contrast the reactions of either side to those things and expose what I see as an unfairness or a double standard or an unequal application. hope that helps you understand what I'm saying. Anyway have a good doi
No one is arguing that any online community perfect, no organization is perfect, but organization imperfections is not a justification for unethical behavior toward given imperfect organization, to a large degree, there rare case where it is justified, but this not the case here, at all what so ever.
> I think it's naive for them to assume that all submissions are done in good faith.
This is false, there where numerous attempts to submit backdoor to linux kernel, and most in the community are aware that such attempts will never stop, this why also some people criticize that there's study is just another scientific garbage paper.
What is unacceptable in name of science submitting commits in bad faith without informed consent.
What is unacceptable in name of science or otherwise ATTEMPTING an unauthorized penetration testing, this is illegal.
[0] https://www.theatlantic.com/technology/archive/2014/06/every...
It's an interesting perspective I guess it comes from the idea that communities are sort of unstructured and it's not as important that they have ethical standards as high as science because science should be more objective and people need to be able to rely on it and trust it. I guess someone could adopt this perspective that seems to excuse community bad behavior while condemning researchers behavior because they just want to get out of any accountability in this particular case. but I'm assuming that you genuinely feel like this and have had this view from before this particular incident.
I simply disagree with this view, I think everyone should behave well or at least held the same standards.
It also seems that you're (or you were when you commented) very angry and hurt by the actions of the researchers and so I guess that you're probably involved in the Linux kind of in a way and felt betrayed by this action or I had some similar experience with scientific people in the past. I wonder how much the opposite of the community is a result that they were successfully deceived and if they had caught these patches and prevented them from being merged and their reaction may be entirely different even if the research is behavior was the same. But I suspect that would still be outrage because probably a lot of people share your view to some extent but the researchers behaviors should be held to a high standard than their own.
Just seems to me that you're not really open to hearing my point of view on this and that's got nothing to do with the merits of my view it's just where you're at. But because it seems like you're not able to hear my view on this or engage with that I just didn't really feel there was a point to continue the discussion. Have a good one
Please, you've contributed enough already, really.
If that means changing jobs, most of us have been there. Looking for a job sucks, but if you screw up that badly, then you do have to consider stacking shelves in a supermarket; you're not fit to work on a public project of such importance.
This doesn't admit to harm done, and sounds like an apology that doesn't admit guilt.
"We sincerely apologize for any harm..."
While there are other requirements, a sincere apology cannot in any way entertain doubt about the fact that there WAS harm.
Truly acknowledging the harm done is foundational to a real apology, and most of us (myself included) end up sneaking in weasel words or phrases like this.
Psychologically, its nice for the apologizer, since it allows one to think "i'm being good by apologizing, but maybe I didn't do anything bad after all?".
But from the apologizee standpoint, these phrases are often devastating and can make it clear that the apologizer has no real recognition or care of what happened.
Personally I've worked pretty hard to try to remove these sorts of phrases from my apologies. It's not easy. It makes you feel much more vulnerable and you really have to let whatever you did sit with you in a very uncomfortable way. But it's worth it.
Someone will always be apologizing wrong for some. Some interpretation of what harm there was, is not necessarily the same as my interpretation. There are too many ways to construe what harm there was or may have been according to others to satisfy everyone addressed. This is an efficient wording that doesn't explicitly satisfy your (and many people's) specific issues out of "the community", which illustrates the point.
I understand doubting the sincerity of the authors, but this argument hinges here on them saying "any" instead of "all" and they are often used interchangeably in casual conversation.
Honestly, for all we know this letter could be meaningless. A real good actor would also reveal the other commits in order to have a full disclosure.
Trust isn't based on promises, trust is based on past actions. Therefore they should be treated as such, as their actions are evidence for being a potentially malicious actor.
I'm still convinced Greg did the right thing here, as it's better to be safe than sorry in this case due to the sheer scale of an attack vector that the actors might have introduced.
Even if other commits of good actors at the same University are now treated with more attention to their code, I think they are a casualty that was predictable in the moment the ethics commitee signed off the paper's research procedure.
Honestly, based on the way they handled their "research" in the first place, sneaking weasel words into an open apology letter is entirely par for the course...
To clarify: I'm not arguing this is a good or bad apology; just that the justification provided here looks flawed. Human speech isn't a programming language; it's not well defined and it's more than the sum of its parts. One can't derive conclusions about a text by analyzing a tiny subset out of context.
Based on my experience with the general public, with academics, and with industry folks, the criteria for what constitutes a sincere apology is not known by the majority. Furthermore, many of those who have heard the criteria do not accept it as a gold standard, and disagree with it (even those who are not being looked to for an apology).
The recipient can always choose to accept or not an apology. However, I find it quite distasteful to attribute intentions to someone simply because they did not follow a recipe (even if the choice not to was intentional).
While it's common in colloquial German to use the one-step-absolution and skip over the possibility of the other party not absolving you, it's also often considered rude when it matters and has a taste of "but not really". It's fine when you accidentally stepped on someone's foot, but not so much when you've stolen their car.
https://news.ycombinator.com/newsguidelines.html
I think it's both unfair and non-constructive to pick apart a apology letter based on one word like that. Let's assume good faith, especially when the writer's English might not be their first language (based on their name).
The "hypocrite commits", according to the researchers' own paper, originated from "random Gmail addresses", and the researchers continue to claim none of those got merged (though since they haven't told us what they were, ...)
The additional claim is that some of the commits not covered by their "hypocrite commits" (and thus, submitted from their UMN addresses) contained security bugs, deliberate or not, and the loss of trust in the researchers is sufficient to justify reverting all of their commits until they can be reviewed.
The apology is also specific about what they did wrong despite their intentions. It really is a good apology after reading past the first six words.
If you want to apologise, "all harm" admits you caused harm, which seems necessary for an apology. "Any" is a bit like those "I'm sorry if you were offended" non-apologies, though in this case the rest of the message does better than that.
"We sincerely apologize for all the harm our research group did" would be a little less unnatural than "...for all harm...", but it's still an unusual choice of wording that ends up sounding like you want to emphasize that you did an unusually large amount of harm.
A more standard phrasing that includes the word all would be "We apologize for any and all harm...".
The standard form I know is actually "[I am deeply sorry] for any harm I may have caused..."; the letter here does not use a modalized verb. (Compare their "We sincerely apologize for any harm our research group did...".) In that sense it's more definite than usual. The criticism above is strange.
(The reason for the modality in the standard form isn't really to leave open the possibility that you didn't do any harm. It's to leave open the possibility that you did harm you don't even know about -- and therefore can't apologize for specifically.)
[1] In general, any occurs only in negative sentences, though in the details there are several types of sentences that are sort of "honorarily negative" for the purposes of allowing any and other words that obey the same restrictions.
It wouldn't have occurred to me to discuss it above, because I thought I was contrasting any with all, and neither is present in that example. But I'd rate it above most of the alternatives discussed (while still below "for any harm").
2) "We apologize for any harm that we caused"
3) "We apologize for all the harm that we caused"
(1) is the least apologetic. This could be interpreted as saying "we might or might not have caused harm, and we think we didn't but you think we did, so we're going to apologize for your sake, but we're not really sorry because we didn't do anything wrong from our perspective".
(3) is the most apologetic. You acknowledge that you did something wrong, and that you're apologizing for it. This might be followed up with a specific list of the things that you did wrong, and that you are apologizing for. That would make for the most sincere apology. This could be interpreted as "we caused harm and we're sorry, whatever harm you think that we did, we agree that you are right, and we are sorry for all of it".
(2) is partway between (1) and (3). You acknowledge you caused harm, but you won't enumerate the things that you did wrong. So, you leave yourself a little bit of wiggle room. This could be interpreted as "we caused harm and we're sorry, but we didn't cause that much harm, we think it was actually quite little, you think it was a lot, but we're saying sorry, so let us go with this apology".
I am surprised to see people hung up on this, because the rest of the letter acknowledges specific harms done. To me, it really seems like a minor thing, and something that I might have written (as a native English speaker.)
This is a pretty weak assumption. Someone's name (and physical appearance) don't give you an accurate information about anything.
And to be honest, I kind of agree with them. I don't really see what they did here as particularly bad. They demonstrated a very serious vulnerability in the linux kernel development process. I guess the harm they caused was wasting maintainers time, a bit? But what we all got out of it is the knowledge that real bad actors could easily have done this too. It's odd to me that people are focusing on like, the etiquette here when such an important vulnerability was demonstrated.
This is a risk, but it's exposing vulnerabilities is always preferable to leaving them in place.
> Or of intentinally buggy patches making it into the wild and being exploited?
There was no real risk of this. They specifically did not allow it to make its way into real releases.
If they had allowed it into actual releases, I would have a serious problem with that.
But I'd argue it is his own fault to use unsupported patches.
Says who? I don't think that's true according to IRB standards in the US. Nor is it true for websites who A/B test according to the GDPR
Me and common sense. I don't believe consent is relevant when there is no risk of harm to the subject. IRBs and the GDPR are overly aggressive on this point, probably as a reaction to real and important violations of privacy. But the idea that A/B testing the color of your CTA button on a landing page requires informed consent is absurd.
https://en.wikipedia.org/wiki/Belmont_Report
Of course there are. But what specifically are the harms that are going to be caused by either this research or a landing page A/B test without a click through pop up? The existence of theoretical harms for broad categories of potential research does not have a whole lot of bearing on these specific lines of research.
> If you're interested why these ethical principles exist in the United States, I suggest you at least skim the Belmont Report
I read it. It was interesting, but I don't think it's particularly relevant here. The only prong of its test that would be relevant to this experiment is "respect for persons". The idea that somehow not revealing the bug or the experiment was intrinsically harmful as a violation of a person's moral autonomy.
I don't buy that line of reasoning, and I can't really think of any valid consequentialist justification for it, and the report itself does not attempt to justify it either, as far as I can tell.
A/B testing on major platforms is much more sophisticated than that[1].
It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
So, yes, it should definitely require informed consent and opting in.
[1]: https://www.bbc.com/news/technology-28051930
On some it is, and some it isn't.
> It's a crucial practice for advertisers and marketing teams that dives deep into researching the psychological response towards images, text and dozens of other variables. Human subjects are acting as lab rats in order to extract some data points to drive the next test and campaign.
This is just a long and emotionally laden way of saying "changing images and text to see which works best".
> So, yes, it should definitely require informed consent and opting in.
Guess we're just going to have to agree to disagree then. I don't see any reason whatsoever to think that this practice is harmful to the subjects being experimented on.
At worst, it's harmful to society in general because it incentivizes consumerism and potentially self destructive behavior. But that is completely orthogonal to the issue of informed consent. If you got perfectly informed consent from 10,000 people to tease out the perfect pitch text, and then deployed it against the rest of the population, the effect would be exactly the same, whether or not you got consent.
It's just a scammy move at best. To me it's borderline criminal to sabotage a project like that.
https://abcnews.go.com/US/tsa-fails-tests-latest-undercover-...
https://news.ycombinator.com/item?id=26929797
For me, it reads like this "With the best intentions, we were helping you. Unfortunately, you are too stupid to not realise this. We are sorry that we hurt your feelings, but please let us continue in helping you."
When you get such an apology, best thing is to avoid such people. Because it's clear they do not understand where they went wrong.
They should have either apologized with "we made a very big mistake that had a negative impact, it did more harm than good". Or they should have argued with real evidence on how they improve the Linux kernel, like refer to real exploits that they fixed.
This is just a "we're sorry that you don't realize we are helping you"
Indeed: "The Court held that "the word 'any' is to be considered all-inclusive"
https://www.michbar.org/file/generalinfo/plainenglish/pdfs/9...
A more substantial criticism is that blending we’re sorry with excuses and mitigating explanations is what makes it lose impact.
But really, they acknowledge the harm they did and they said sorry and I hope you give them some credit for doing so.
I would have listed the specific harms in a concise manner instead.
Maybe save that word “any” for the end, maybe and only as a future tense of profusely avoidance of future harm.
For you security people out there, how do red teams handle this issue?
Basically, warn in that you’re going to do it at some point in the future. Then do it in a time-frame of maybe 6 months or so from another ID. Maybe get some of the higher-ups to sign off (say Linus or Greg in this case) beforehand. At the very least, engage with the community...