The premise of this article is better than its execution. The author even points in the direction of open and honest sharing, saying "others could surely have learned from the mistakes I made". But the only mistake pointed to is "I stopped reviewing his original data". Is that the root cause and only way to prevent these problems? If so, drive the point home more firmly.
The analysis in the article is very shallow and raises more questions than answers. What motivated the postdoc to fabricate the data? What kind of project were they working on? A postdoctoral position can be a perilous journey, especially if the project involved is a high-risk one (which would be my guess, given the author’s urge to get recognized in her area). If they don’t generate enough data and publications within those few years, a postdoc can leave the lab effectively empty-handed. There is a tremendous pressure for productivity, at all costs.
While I'd love to see bullet points on takeaways, I wouldn't put the 'load' for the lack of them on the author. Even if he expressed the intention to do such thing and then couldn't deliver... I mean, seems like revictimization in a way. He was a victim (even compares the situation to a theft). In these cases, maybe there's not much you can really do to be safe. It's really up to the other person, who should not be dishonest.
That being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..
In my neck of science/AI/ML we've been talking about negative results being important as a community for a long time. But it never really happens.
It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
> Anyone can produce an unlimited number of negative results by being incompetent.
This is well said, I'm surprised I haven't heard it expressed before.
Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
I can see the reason for not rewarding individual negative results, but not for not publishing them at all. Any single positive result is almost certainly the cumulation of lots of work, most of them being negative. You can’t pretend that you arrived at this positive result with zero failed attempts. Why can’t we ask for past negative results along with every positive result, just for the sake of completeness? The authors could have just released a bunch of Jupyter notebooks or whatever original format the negative experiment results are in - why everything in academia has to be written in a certain format where every word is carefully scrutinized to be considered useful?
This doesn't make any sense. You get grants for having ideas (and enough preliminary results to convince the reviewers that you will be able to publish the ideas). Many junior PIs struggle precisely because they are better at doing research than at having ideas. You get a PI position for being good at reasearch, but the job you get is very different from what you are used to.
There is nothing special ahout the Arc Institute. It's a research institute that can hire professional researchers to do research, because it's not a university department that's supposed to train people to do research. The funders just are generous enough that the researchers don't have to apply for funding. Any organization could do the same with funders like that.
The Crick in London is maybe a better example. As I understand it there you apply to work on treatment or understanding a specific human disease and you receive a one time 7 year grant which provides a stipend for you, two Ph.D. studentships and one lab assistant + a sum to equip your lab and access to a bunch of services (mice, bugs, fish, compute, chemicals...)
You can get a single renewal - but that's all. No matter what you are done after 14 years.
I have witnessed first hand how many old PIs use PhD students and postdocs to come up with ideas and draft grants. Obviously as ghost authors. After discussion with other colleagues, this seems much more common than what I thought.
Arc is indeed special because all funding is internal, just like it was at LMB or CSHL during their golden years. This encourages small groups where everyone does research, instead of creating a class of middle managers.
There are many research institutes in the world with all kinds of funding arrangements. Also, Arc is apparently attempting to recreate the usual "middle manager" structure:
> Phase 1 of the institute involves hiring 10–15 Core Investigators, each of whom may employ 10–20 trainees, researchers, or engineers.
Those are large groups by academic standards.
The middle manager structure itself arises from many causes. Funders like competitive grants, because that gives them more control over the research they fund. Universities are in the business of training the next generation of researchers. And universities save money by hiring dedicated teachers instead of professors, leaving a smaller number of research-active people in charge of the trainees.
It’s already very nice of the author to reflect and share his past mistakes aloud, as this would be a good start for others to raise the same question towards the science community in general. Pressing the author for detail into this specific mistake that happened years ago is nothing compared to lots of existing labs whose integrity has not been questioned at all.
“I had depended on the idea that scientific research is a sacred trust”
And that is part of the problem. Having gone through a phd, this unspoken religiosity that underlies scientific research where you are expected to be monks for science contributes to stuff like this.
But selfless dedication to the truth is actually the standard you want. ISTM a bigger issue is the practice of grading on a curve, which incentivizes cheating; and once a student gets away with that as an undergrad, it's a pattern that's likely to persist.
Some architect grads I knew once mentioned hiding their building models in the classroom ceiling in the leadup tp their final exams. When I asked why, they said that there was a non-trivial risk that if you left your model out in the classroom other students would damage it to let a leg up on the competition, and the more ambitious the final project the greater the risk. In my innocence I was quite shocked, but as so often, once you become aware of such behavior you start noticing it in many other contexts.
I think you have a poor interpretation of monks. Monks in the catholic orders screw up all the time, make penance, meditate on their mistakes. They recognize that they are human, though sometimes divinely inspired, and seek to correct themselves.
At least, some do, and the better orders are built that way. There are plenty of bad ones, too, as I'm sure we've all heard, who don't do any of that - Who lord their pledges of service to the lord over those they consider lesser. That style seems to be more what modern academia is following.
> John’s fraud left me feeling personally violated, as if a thief had come to my home and rifled through my drawers and closets. Scientific fraud is, indeed, a kind of theft. Apart from the wasted time and ill-spent money, it is a blow to one’s reputation, and it damages a lab’s credibility.
I love how the author perpetrated scientific fraud and is playing the victim.
Part of the responsibility of a PI is supervision. The author put their name to the paper and the presentation. It is his responsibility to verify this.
Also what kind of lab culture did they create. Was there a super focus on discovery at all costs? Was the author just exploiting the post docs instead of actually properly supervising and mentoring them?
It kind of reminds me of the Wells-Fargo CEO who created a culture of high demands and no supervision then acted all surprised and innocent when fraud occurred. I think the people at the top of the hierarchy need to receive more blame, rather than push blame to the overworked, underpaid lowest person on the hierarchy.
The point is that trust is a major component of scientific work and how it functions collectively. A effect being that when that is violated a lot breaks down with a god amount of collateral damage.
With increasing complexity in research and pressure to produce and publish more, it’s a growing weakness.
Bottom line is that team work and trust are now indelible parts of research in a culture predicated on individual success and contributions.
I see it as a symptom of our culture where benefits flow uphill and blame flows downhill.
When writing papers and presenting at conferences, the PI was all about how he was an integral part of the research and if you asked why he was presenting instead of the post-doc he would have talked about his responsibilities to supervise and guide.
Once the fraud came out, he is just a poor victim who can’t be expected to know the post-doc wasn’t even using the equipment and was making up data completely.
The CEO gives the talk about the awesome product because they set the roadmap.
Benefits flow uphill, blame flows downhill is literally how any hierarchical organization works.
Now, you might want to say that science shouldn't be hierarchical, but we don't have any flat organizations like this that involve the kinds of mentoring and training required to build new scientists.
For example, a ship’s captain takes ultimate responsibility for any collision or sinking, whether or not they were actually steering the ship. There was a tradition of the captain being the last one to leave a sinking ship.
Many cultures had a tradition of executing commanders that suffered defeat.
I think our culture is an anomaly where blame does not flow uphill as well.
Science is based on trust. All incentices are against research that verifies results: you don’t get funding, you don’t get published, you are not doing original research etc. If we cannot trust the results, everything is a house of cards, because not only are the papers wrong, also all papers depending on that paper may be affected. The scientific system should not be based on blind trust but have processes in place that makes it more resilient.
It's possible that there was perhaps insufficient supervision by the PI, or not enough senior people in the lab hierarchy, but I suspect most labs are not equpped to handle someone who is outright deceitful and sufficiently good at lying.
The problem in this case is not that it’s not detected at all. I think the more interesting part is how the academic culture made everyone reluctant to conduct the investigation and admit that a misconduct occured.
So glad these types of conversations are happening. There are often difficult conversations but the fraud in research seems rampant and can cause so much damage to humanity it’s important to guard against.
> I had already presented John’s data at several national conferences, and we were writing a paper I thought would have a huge impact on our field.
Sounds like this person was part of the problem if their idea of "supervision" was flying around taking credit for their postdoc's fraudulent research while being so out of the loop that they didn't notice the postdoc fabricating data whole cloth.
I bet that if the government goes after PIs who take federal money and then run a paper mill with inadequate supervision of postdocs, the situation vastly improves.
If you know you can go to jail if your postdocs fabricate data, I bet you will be a lot more thorough in checking all the original data.
The problem of scientific research is misaligned incentives. There is tremendous upside for publishing novel ideas and very little downside for fraud, especially for the PI who can pawn it off on the overworked, underpaid post-doc.
Rather than an post-hoc investigation model, how about promoting reproducibility in publication, with an expectation that others can run the same study with identical equipment, models, codes, playbooks, etc. For PIs getting government funding, true reproducibility should be a requirement,
and other researchers should be funded to report pro or con on the reproducibility of studies, and then ethical lapses could be much more visible. Science based solely on faith in the ethics of other researchers is just religion with more complicated rituals.
Reproduction is not valued by the community in any way that matters. There’s no prize for reproducing, or failing to reproduce, others work. Getting a grant to do reproduction is much harder than getting the initial grant. Nobody is citing reproduction papers, so their authors can’t get that sweet sweet citation count.
We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.
> We as a society don’t actually value the discipline of science or the pursuit of knowledge.
The discipline of science is exactly how you get knowledge in a society that doesn't value its pursuit. It's an imperfect process of imperfect individuals that usually nevertheless arrives at knowledge eventually.
Fraud is just another part of this process. Additional checks for fraud or errors cost time and money. Effective checks would basically mean science costs 2x as much. How much fraud can we tolerate? Is it worth spending this much to reduce the amount of fraud compared to investing in more research? Can we slightly change incentives (as opposed to inventing some utopia that cannot be implemented) and thereby reduce fraud more cheaply? The academic discipline has been changing ever since and will continue to change and progress.
Publishing reproducible research would disincentivize fraud by providing a
clear way to expose fraud, and could tend to benefit other researchers in
the field, by effectively sharing specific methods as well as results.
Incentivizing reproducibility would be hard as a grassroots effort,
but would be enforcible if it were tied to funding. Even if only a
small percentage of published studies were reproduced, and only a small
percentage of them were found to be fraudulent, the disincentive for
fraud would be present. We do not need more Andrew Wakefields.
> Effective checks would basically mean science costs 2x as much.
By that logic, we would need to enforce speed limits by having one
police cruiser following every driver.
Absolutely his behavior allowed the situation to happen. This is probably the real original sin in all of this. The OP ego was so huge that he considered himself above the actual work —- even critical work —- that defines his trade.
In this case “John” probably understood this about him and was likely just giving him the validation he so desperately craved.
I like how he claims the art of science to be a sacrament yet can’t be bothered with the actual work. A complete bastardization of the actual core of the work.
This doublespeak of loving science in the abstract and totally ignoring it as applied to yourself is everywhere in academia. It’s one of the most profound disappointments of my adult life that the world is filled to overflowing with poser scientists.
This sounds like your standard “hands-off” PI who doesn’t dig too deeply in the day-to-day research and analysis. That can work, but at least make sure your postdoc has collaborators you trust will keep them honest. If your lab doesn’t have a culture where people give productive honest feedback (warts and all) on work before it’s publicized, then that’s a failure solely of leadership.
There are different tiers of hands-off PIs though. Most can probably be deceived by manipulated data. But you’re on another level if you can’t even catch someone producing groundbreaking data without being physically present in the lab using equipments.
Professors are basically CEOs for deep-tech start-ups (at least in science and engineering). Yes, they know the theory, technology, tools, etc, (from their own time as PhDs and Postdocs), but they need to delegate much of the work to their team. Like a CEO, the PI's main jobs are to strategize (choose research direction), form partnerships (collaborations), and fund raise (find grant money). This money is used to pay said team (students) to work on the products (research projects). In this analogy, each student is basically a project manager working on their own product, sometimes with a MASc or UG student to help with the labor intensive parts (experiments). When the product is ready for release (publication), the PI helps market it (at conferences) and puts their name on it (as coauthor). This is not stealing credit, this is amplifying the team's success so everyone wins. It builds visibility and credibility for the team members involved, and keeps the ball rolling by allowing the PI to get more funding based on the win. Note that the PI is instrumental in choosing and guiding the work, leveraging their experience to brainstorm solutions when stuck, etc, but nobody thinks they are actually in the lab doing it all.
So, I guess my point is, a good CEO would eventually figure out if work is bogus, but they'll probably be the last to know. BTW, noticing the data problems when working on the draft publication is literally the first opportunity a PI would have to really see all the final data in one place. Conferences are 'low stakes' and it's not unusual to present some preliminary and/or partial data.
I know this opens a HUGE can of worms if you really think about all the repercussions (e.g. who really deserves Nobel prizes and awards?), but that's a story for another day.
Nobody wants to assume fraud in science, unless you have unassailable evidence that someone has committed fraud you really cannot risk your own reputation by dishing out allegations. The solution to this is trust but verify, but verifying data would require twice the amount of work, which not every lab has the funding to do, even if it was breakthrough research.
The sad truth about a lot of science is that science is built on trust. The psychology replication crisis is proof of that, there are so many ways to manipulate data to make it look like you have a result and also so much pressure to get a result that people resort to the former to get the latter.
There are plenty of historical scandals about how scientific fraud has taken way too long to investigate, people like Victor Ninov and Hwang Woo-Suk are two examples of just how far fraud can go before it's found.
The article here is comparatively tame compared to how far Huang and Ninov got.
Academia is funded to the tune of billions of dollars a year. If they don't have the funds to check their own work then it is a damning indictment of the funding process, one that suggests governments are utterly unqualified to be funding research at all.
But in this case funding doesn't seem to be the problem. The author says why she didn't check: she wanted to be famous.
No because "we" don't pay academics, universities do. It's their decision to hire more professors instead of paying the existing ones more, it's their decision to create siloed departments etc. The system has plenty of money, but universities and grant agencies allocate it very badly despite being mostly staffed by academics and ex academics.
Yeah this really stands out to me too. I'm still just a PhD student, and my PI is fairly hands-off (in that he tells me roughly what problems need addressing and their priorities and trusts me to work autonomously), but as a result he's also hands-off with taking credit. He doesn't present my work in detail on his own, typically he tries to have me present to do that myself.
It'd be pretty fair for this person to be held partly responsible if he was presenting it at conferences, which outside of exceptional circumstances (eg the postdoc was supposed to present but had to cancel with very short notice), would justify the assumption that he was familiar enough with the work to speak on it.
I must admit I was surprised to see the specific study and falsified research wasn’t discussed.
I can appreciate this may have been more of a reflection than reprisal (with remediations no doubt made at the time) but if that was the case, would have been good to read it ‘unredacted’.
Had the falsified data not made it out of the lab I would understand this article’s approach - the author clearly uninterested in a fruitless smear campaign - but given “lies” are out there, would have been interesting to see exactly what it was and to what extent the findings proliferated.
Maybe the author didn’t want to give any more weight to incorrect information. I can only make assumptions.
A challenging and nuanced situation all round no doubt.
The explanation is pretty simple; she’s a coward and seeking closure by unburdening herself without doing the hard thing and being fully transparent about what happened.
If you helped build someone’s reputation by not bothering to verify their work and just taking credit for your involvement with them, you have an uncomfortable but undeniable responsibility to be just as strident in getting out the truth.
Easy to say that behind a computer screen, but we're in this mess because from a game theory perspective there is no gain to be had and a huge risk to your career and/or finances by publicly naming someone.
This also happened several years ago, when there was even less honesty and discussion around this topic.
sounds to me like the author was naive and idealistic, nothing wrong with that, it hurts everyone when people do the wrong thing by others. maybe what they are doing by publishing the story is trying to warn others not to just take people at face value and be aware of your own biases. its a good lesson. no further naming or shaming required.
> The author was running a paper mill with underpaid postdocs students and presenting their work as her own with very minimal supervision.
The postdocs aren't students.
How do you know that they were underpaid?
How do you know that she was presenting there work as hers - science is collaborative in the extreme now. It is very rare for a paper to be produced by one author. At one time I wrote a few such papers and I was told that it was actually a black mark against me for a post because I was someone who couldn't collaborate!
You’d think this crowd would have gotten the lesson from Wargames: sometimes, the winning move is not to play. In many real world scenarios, this is by far the optimal solution for the average individual.
> she’s a coward and seeking closure by unburdening herself without doing the hard thing and being fully transparent about what happened.
I don’t see how one can draw that conclusion from reading one text she wrote. She can easily have written other articles about the case.
This article isn’t about the fraud or about its correction, but about her soul-searching years later.
I also think the article indicates she has worked on being transparent. She reported the case to the university integrity officer and hired a scientist to check the data.
She also writes “I had barely escaped the shame of having to retract a published article”, which I interpret (that “escaped” makes it a bit ambiguous to me; if that “barely” hadn’t been there I would interpret it as there not being a retraction) as that an article got retracted.
> cowardice is behavior, her behavior is that of a coward.
Again: I don’t see how one can draw that conclusion from this one article.
Also, IMO, there’s a huge difference between “behaving like a Foo” and “being a Foo”.
The former leaves open that they change, the latter doesn’t. Compare “(s)he killed somebody” with “(s)he’s a killer”. Who do you think can deserve parole?
The article hadn’t been published (it says so in the article), she escaped retraction because it came up before it was published. In fact it reads as if it hadn’t even been submitted yet and was still being written.
The biggest problem: there's no signs anywhere in the article that she tried to stop John moving on and doing it all over again. She didn't even tell her colleagues about the fraud let alone his new employers.
This sorry tale is why legions of us longer trust "experts". She cares so little about science that she did her best to ignore the possibility of fraud and then did nothing to stop it happening again.
Academic culture is truly challenged when it comes to being scientific.
The paper is redacted and John would've been first author. Pretty obvious, especially within his field. Her department would have known the full story, it's just she didn't deliver a post-mortem presentation with learnings
Also, academia works on references, sure you can get older referees but the referees would be suspicious as well as the person receiving. The reference culture can be career killing if you just worked with someone toxic as you're blind to the references, they assume the worst if your most recent supervisor isn't providing one etc. then even if you're good you have to apply to a lot of jobs and waste a lot of people's time submitting to portals and writing references.
The problem is when dishonest and no-skill researchers aren't fired or don't leave like Iddo Drori
The PI was asleep at the wheel. When my father worked in a lab he knew who was fabricating data amongst his peers in the other adjacent labs not just his colleagues in the same labs. There was no was no way stop them but he knew and it made for frequent dinner conversation.
That's all very well, but whistleblowers can find themselves out of a job without getting any result if they aren't in the right place at the right time. What if the OP's dad was of a minority ethnicity in a tenuous position in a racist environment? What if the family were poor with a sick child? What if he was an only parent?
What if not everyone has the courage to do the right thing all the time because they are just ordinary people and not the star of a movie? What if they are scared and we don't judge them for that?
There's a big difference between a corporate executive with (say) $2m in the bank that keeps their head down and lets corruption ride, and a college professor with a family that keeps his beak out of things.
You seem to be trying to justify academic fraud as ethically acceptable for poor people to do. Is that right? Or are you just explaining how people could be motivated to participate in it despite it being unethical?
There is almost no protection for whistle blowers in university. If you report fraud you get kicked out of the lab and blackballed and have to restart your phd if someone even wants to take you in.
I would never recommend whistle blowing to anyone under those circumstances. It’s career suicide and may leave you personally devastated as well. The system is designed to protect bad actors.
Additionally, if you report outside the university, the univerisity is tainted because word of the investigation will spread farther than word of the results. So the choices are report internally and likely get retaliated against, report externally and suffer similar reputational damage, or keep quiet and hope someone with power notices and takes action.
For sure you will be labelled as a trouble maker, the worst thing is that there is a lot to hide out there and people are very ruthless about hiding it!
PI's have a lot to do, she admits that she wasn't focusing enough on the work that was being developed. I think you are making a good point though (she does as well), science as we currently understand it works on the basis of trust. This is because it's highly competitive and highly compelling. Compelling in the sense that people believe that their contributions may make significant contributions to wider society and other people, competitive in that the winners get the spoils.
I think we need to moderate both of these pressures. We need to stop the valourization of potentially high impact work - until it is high impact, and we need to emphasize the long value chains and problems with exploitation as well. A good example is the recent talk from John Burke about Quantum Computing and Quantum Clocks [1].
In terms of to the winners the spoils we have to develop a funding system that recognizes that people who make a single significant impact on a field early in their career are no more likely than anyone else to make further impacts. Indeed the more I hang around the edges of the research community the more I am convinced that there is a very weak correlation between any prior success and any future impact if you account for the funding that early winners receive. I feel that there are very many truly outstanding but unlucky researchers who are culled after 3 post-docs. I think that population is basically the same in capability as the folks who get tenure. We need to stop selecting so randomly - it's not working and it has bad effects (see this article).
In the past the way things worked was a "good fellows" club - where connections and patronage were what counted. In some parts of the world (Europe and Asia) this still has a massive effect. I think it's dissipated in North America and the UK but what's replaced it (savage competition) isn't really better - it just suits the political biases of the people funding the system.
> PI's have a lot to do, she admits that she wasn't focusing enough on the work that was being developed.
Her name was on the papers and she presented the work at conferences.
If she was not really involved in the work, then it was scientific fraud to have presented it as if it were hers.
If you were so out of the loop as to not realize that no experiments were being performed and the data was being fabricated wholesale, then it is fraudulent to put her name on the scientific paper as an author and to present as an author. She is not conducting research but is a renting out a paper mill.
That kind of behavior needs to be strictly condemned
That's not false authorship. You can get authorship just for getting funding to allow the paper to exist (researchers, reagants and equipment aren't free). You should still read and verify.
False authorship definitely happens a lot though but mostly with papers when 1-2 guys did the real work but 10 people are listed. Very common in many big departments.
This is not false authorship at all. Many modern scientific works are too big for a single individual to implement themselves: it’s much like modern software development in that sense. Hence the project is a team effort: there are people who are responsible for the design of the experiments, and then specific subtasks are handled by different team members, often with some review. When the work is presented, it’s presented with a complete author list. The presenter is not claiming full credit for the work.
The challenge with malicious team-members in science is the same one you face in software. As another person in the thread said, how do you safely develop programs when one contributor is adding “Underhanded C contest”-level backdoors to the code? You can architect your development processes to defend against this threat, but in practice very few software development teams do this because it devours limited resources. Resources in science are even more limited than those in for-profit software development firms, and often involve taxpayer funds.
The instinct on HN is to approach this as an angry blame game. But successful industries don’t do this: for example, the aviation industry performs root-cause analysis after each air disaster and then makes intelligent cost/benefit decisions about how to update processes to avoid future occurrences. In science it’s not clear what the right approach is on a cost/benefit level. Should we reduce scientific output by 50% to catch fraud in the first instance, rather than later on when important studies fail to replicate? Is that a good use of society’s limited resources? I don’t know the answer to this, I just want people to behave effectively rather than grinding an axe. We’re not getting out of the messes we’ve made on this planet without a whole lot of science.
This is orthogonal to the reproducibility crisis, but not really part of it. When you claim a breakthrough even just within a sub-field, if you simply invented an elaborate lie, it's always gonna be caught it's just a waste of everyone's time in the end. This person is lucky she and her lab caught it themselves before other people started to work further on it. And I guess the other lab members were already trying to reproduce it, as you would.
That guy who faked all this can't really be thought of as a sophisticated bad actor, neither was he really gaming the system because it never would have worked. He was just leaping into his own pit.
Research needs to first move to being fully open access and end of tens of millions of taxpayer dollars going to journals. This is something tangible you can legislate. Then, you need to incentivize high-quality research; that's not easy to do at all, but you can do things if you have hiring power within your field I guess...
Really minor nit, but the comments in this thread refer to the PI as a 'he' or 'him'.
The PI is Rosalind A. Coleman. I did not thoroughly investigate the preferred pronouns of this individual but canonically Rosalind would be assumed a 'she' and the top Google images returned for the name present as female.
I strongly disagree with the other comments. This PI seemed fully involved in, did the appropriate amount of oversight over, their lab's work. They studied the data their postdoc gave them, discussed its implications, and understood it. That's adequate. It's unreasonable to expect any team to be robust against a malicious insider attack—to be able to audit things with sufficient aggressiveness, that they could catch sophisticated liars who manipulate raw instrument data.
I don't have a *clue* what the solution is; but, asking ordinary people to turn into FBI detectives is not a viable one. It won't work in the real world, with real humans. It's a fantasy.
I sympathize with the outrage in this thread. But, go past your initial hot-takes: would you apply the same standards to yourself? Suppose you're in software, and one of your coworkers is sneaking subtle, obfuscated security bugs into your production code. Is it your fault if you fail to catch that? Would you consider yourself culpable if you did a code review and failed to notice some "Underhanded C Contest"-tier sophisticated backdoor? Because IMHO that is utterly impossible; and I think it's similarly impossible to catch data manipulation, reliably, if it's sophisticated enough.
Solutions need to be grounded in what's achievable.
> that they could catch sophisticated liars who manipulate raw instrument data
There was no raw data manipulation, but fabrication of derived data. These kind of too-good-to-be-true results are easily auditable, if anything by replicating the experiment. The PI’s blind enthusiasm prevented her to do her due diligence.
> would you apply the same standards to yourself?
While I was in academia I checked basically every kind of raw biological data I was getting. Routine QC allowed me, for instance, to find contaminants in some of the reagents the lab was using.
For many labs, there are no resources, time, or ability to rerun every bit of data collection that happens. When I was in academia, our lab had a hard enough time recruiting participants (we were a cog neuro lab) and our experiments were several hours long per participant with enough individual variation to be useless without several months of data collection and analysis to find any sort of signal in the noisy data. And that’s a single lab, with PIs overseeing several and collaborating with even more, an element of trust is going to be needed if anything is to ever get done.
In the case of this article, the experiment entailed putting a sample in an instrument, do the measurement and check the values. And the checks I propose are nothing out of the ordinary. Many academic fraud postmortems tell that the fabricated data had repeated or perfectly rounded values, strange timestamps, numeric data in a different format than expected, etc. Take a look at the (raw or processed) data, there’s unfortunately no way around it.
“I had depended on the idea that scientific research is a sacred trust, not only to the other individuals in one’s group, but also to the research enterprise as a whole.”
The concern of the author is ultimately one of status, preeminence. Of course people motivated by prestige will not always be trustworthy. Her sin isn’t being too trusting, it’s a lack of self awareness.
98 comments
[ 4.4 ms ] story [ 161 ms ] threadThat being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..
It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
This is well said, I'm surprised I haven't heard it expressed before.
Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
Nowadays, professors tend to play a middlemen role. Apply for grants, advertise results, and claim credit. Nothing else.
Most of the time, they do not come up with ideas, nor care about them or do any of the hard work.
Places like the Arc Institute have been born to cut PI middlemen out and get research out of this tar pit.
There is nothing special ahout the Arc Institute. It's a research institute that can hire professional researchers to do research, because it's not a university department that's supposed to train people to do research. The funders just are generous enough that the researchers don't have to apply for funding. Any organization could do the same with funders like that.
You can get a single renewal - but that's all. No matter what you are done after 14 years.
Arc is indeed special because all funding is internal, just like it was at LMB or CSHL during their golden years. This encourages small groups where everyone does research, instead of creating a class of middle managers.
> Phase 1 of the institute involves hiring 10–15 Core Investigators, each of whom may employ 10–20 trainees, researchers, or engineers.
Those are large groups by academic standards.
The middle manager structure itself arises from many causes. Funders like competitive grants, because that gives them more control over the research they fund. Universities are in the business of training the next generation of researchers. And universities save money by hiring dedicated teachers instead of professors, leaving a smaller number of research-active people in charge of the trainees.
And that is part of the problem. Having gone through a phd, this unspoken religiosity that underlies scientific research where you are expected to be monks for science contributes to stuff like this.
Some architect grads I knew once mentioned hiding their building models in the classroom ceiling in the leadup tp their final exams. When I asked why, they said that there was a non-trivial risk that if you left your model out in the classroom other students would damage it to let a leg up on the competition, and the more ambitious the final project the greater the risk. In my innocence I was quite shocked, but as so often, once you become aware of such behavior you start noticing it in many other contexts.
At least, some do, and the better orders are built that way. There are plenty of bad ones, too, as I'm sure we've all heard, who don't do any of that - Who lord their pledges of service to the lord over those they consider lesser. That style seems to be more what modern academia is following.
I love how the author perpetrated scientific fraud and is playing the victim.
Part of the responsibility of a PI is supervision. The author put their name to the paper and the presentation. It is his responsibility to verify this.
Also what kind of lab culture did they create. Was there a super focus on discovery at all costs? Was the author just exploiting the post docs instead of actually properly supervising and mentoring them?
It kind of reminds me of the Wells-Fargo CEO who created a culture of high demands and no supervision then acted all surprised and innocent when fraud occurred. I think the people at the top of the hierarchy need to receive more blame, rather than push blame to the overworked, underpaid lowest person on the hierarchy.
The point is that trust is a major component of scientific work and how it functions collectively. A effect being that when that is violated a lot breaks down with a god amount of collateral damage.
With increasing complexity in research and pressure to produce and publish more, it’s a growing weakness.
Bottom line is that team work and trust are now indelible parts of research in a culture predicated on individual success and contributions.
Honestly not sure how science adjusts.
When writing papers and presenting at conferences, the PI was all about how he was an integral part of the research and if you asked why he was presenting instead of the post-doc he would have talked about his responsibilities to supervise and guide.
Once the fraud came out, he is just a poor victim who can’t be expected to know the post-doc wasn’t even using the equipment and was making up data completely.
The CEO gives the talk about the awesome product because they set the roadmap.
Benefits flow uphill, blame flows downhill is literally how any hierarchical organization works.
Now, you might want to say that science shouldn't be hierarchical, but we don't have any flat organizations like this that involve the kinds of mentoring and training required to build new scientists.
For example, a ship’s captain takes ultimate responsibility for any collision or sinking, whether or not they were actually steering the ship. There was a tradition of the captain being the last one to leave a sinking ship.
Many cultures had a tradition of executing commanders that suffered defeat.
I think our culture is an anomaly where blame does not flow uphill as well.
This isn’t subtle fudging of numbers or something that can be harder to detect.
Sounds like this person was part of the problem if their idea of "supervision" was flying around taking credit for their postdoc's fraudulent research while being so out of the loop that they didn't notice the postdoc fabricating data whole cloth.
If you know you can go to jail if your postdocs fabricate data, I bet you will be a lot more thorough in checking all the original data.
The problem of scientific research is misaligned incentives. There is tremendous upside for publishing novel ideas and very little downside for fraud, especially for the PI who can pawn it off on the overworked, underpaid post-doc.
We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.
The discipline of science is exactly how you get knowledge in a society that doesn't value its pursuit. It's an imperfect process of imperfect individuals that usually nevertheless arrives at knowledge eventually.
Fraud is just another part of this process. Additional checks for fraud or errors cost time and money. Effective checks would basically mean science costs 2x as much. How much fraud can we tolerate? Is it worth spending this much to reduce the amount of fraud compared to investing in more research? Can we slightly change incentives (as opposed to inventing some utopia that cannot be implemented) and thereby reduce fraud more cheaply? The academic discipline has been changing ever since and will continue to change and progress.
Incentivizing reproducibility would be hard as a grassroots effort, but would be enforcible if it were tied to funding. Even if only a small percentage of published studies were reproduced, and only a small percentage of them were found to be fraudulent, the disincentive for fraud would be present. We do not need more Andrew Wakefields.
> Effective checks would basically mean science costs 2x as much.
By that logic, we would need to enforce speed limits by having one police cruiser following every driver.
In this case “John” probably understood this about him and was likely just giving him the validation he so desperately craved.
I like how he claims the art of science to be a sacrament yet can’t be bothered with the actual work. A complete bastardization of the actual core of the work.
So, I guess my point is, a good CEO would eventually figure out if work is bogus, but they'll probably be the last to know. BTW, noticing the data problems when working on the draft publication is literally the first opportunity a PI would have to really see all the final data in one place. Conferences are 'low stakes' and it's not unusual to present some preliminary and/or partial data.
I know this opens a HUGE can of worms if you really think about all the repercussions (e.g. who really deserves Nobel prizes and awards?), but that's a story for another day.
Whoever's at the top has culpability for this type of fraud. Either they knew or they should have known.
Professor would be responsible of fraud if they learn about it and ignore it.
The sad truth about a lot of science is that science is built on trust. The psychology replication crisis is proof of that, there are so many ways to manipulate data to make it look like you have a result and also so much pressure to get a result that people resort to the former to get the latter.
There are plenty of historical scandals about how scientific fraud has taken way too long to investigate, people like Victor Ninov and Hwang Woo-Suk are two examples of just how far fraud can go before it's found.
The article here is comparatively tame compared to how far Huang and Ninov got.
But in this case funding doesn't seem to be the problem. The author says why she didn't check: she wanted to be famous.
It'd be pretty fair for this person to be held partly responsible if he was presenting it at conferences, which outside of exceptional circumstances (eg the postdoc was supposed to present but had to cancel with very short notice), would justify the assumption that he was familiar enough with the work to speak on it.
I can appreciate this may have been more of a reflection than reprisal (with remediations no doubt made at the time) but if that was the case, would have been good to read it ‘unredacted’.
Had the falsified data not made it out of the lab I would understand this article’s approach - the author clearly uninterested in a fruitless smear campaign - but given “lies” are out there, would have been interesting to see exactly what it was and to what extent the findings proliferated.
Maybe the author didn’t want to give any more weight to incorrect information. I can only make assumptions.
A challenging and nuanced situation all round no doubt.
If you helped build someone’s reputation by not bothering to verify their work and just taking credit for your involvement with them, you have an uncomfortable but undeniable responsibility to be just as strident in getting out the truth.
This also happened several years ago, when there was even less honesty and discussion around this topic.
Naive and idealistic like the Wells Fago CEO who had no idea that his employees were actually committing fraud trying to meet his impossible demands.
The author was running a paper mill with underpaid postdocs students and presenting their work as her own with very minimal supervision.
The postdocs aren't students.
How do you know that they were underpaid?
How do you know that she was presenting there work as hers - science is collaborative in the extreme now. It is very rare for a paper to be produced by one author. At one time I wrote a few such papers and I was told that it was actually a black mark against me for a post because I was someone who couldn't collaborate!
there are always rationalizations for your behavior, but it may still be cowardice.
When the easy thing is cowardice (aka "strategy") why wouldn't you call out not doing the easy thing as exemplary?
I don’t see how one can draw that conclusion from reading one text she wrote. She can easily have written other articles about the case.
This article isn’t about the fraud or about its correction, but about her soul-searching years later.
I also think the article indicates she has worked on being transparent. She reported the case to the university integrity officer and hired a scientist to check the data.
She also writes “I had barely escaped the shame of having to retract a published article”, which I interpret (that “escaped” makes it a bit ambiguous to me; if that “barely” hadn’t been there I would interpret it as there not being a retraction) as that an article got retracted.
Just as greed got her there, cowardice keeps her there.
Again: I don’t see how one can draw that conclusion from this one article.
Also, IMO, there’s a huge difference between “behaving like a Foo” and “being a Foo”.
The former leaves open that they change, the latter doesn’t. Compare “(s)he killed somebody” with “(s)he’s a killer”. Who do you think can deserve parole?
It doesn't fucking matter, people will make the judgement _for_ them.
This sorry tale is why legions of us longer trust "experts". She cares so little about science that she did her best to ignore the possibility of fraud and then did nothing to stop it happening again. Academic culture is truly challenged when it comes to being scientific.
Also, academia works on references, sure you can get older referees but the referees would be suspicious as well as the person receiving. The reference culture can be career killing if you just worked with someone toxic as you're blind to the references, they assume the worst if your most recent supervisor isn't providing one etc. then even if you're good you have to apply to a lot of jobs and waste a lot of people's time submitting to portals and writing references.
The problem is when dishonest and no-skill researchers aren't fired or don't leave like Iddo Drori
What if not everyone has the courage to do the right thing all the time because they are just ordinary people and not the star of a movie? What if they are scared and we don't judge them for that?
A really big difference.
I would never recommend whistle blowing to anyone under those circumstances. It’s career suicide and may leave you personally devastated as well. The system is designed to protect bad actors.
I think we need to moderate both of these pressures. We need to stop the valourization of potentially high impact work - until it is high impact, and we need to emphasize the long value chains and problems with exploitation as well. A good example is the recent talk from John Burke about Quantum Computing and Quantum Clocks [1].
In terms of to the winners the spoils we have to develop a funding system that recognizes that people who make a single significant impact on a field early in their career are no more likely than anyone else to make further impacts. Indeed the more I hang around the edges of the research community the more I am convinced that there is a very weak correlation between any prior success and any future impact if you account for the funding that early winners receive. I feel that there are very many truly outstanding but unlucky researchers who are culled after 3 post-docs. I think that population is basically the same in capability as the folks who get tenure. We need to stop selecting so randomly - it's not working and it has bad effects (see this article).
In the past the way things worked was a "good fellows" club - where connections and patronage were what counted. In some parts of the world (Europe and Asia) this still has a massive effect. I think it's dissipated in North America and the UK but what's replaced it (savage competition) isn't really better - it just suits the political biases of the people funding the system.
[1] https://www.hpcwire.com/2023/12/19/dod-takes-a-long-view-of-...
Her name was on the papers and she presented the work at conferences.
If she was not really involved in the work, then it was scientific fraud to have presented it as if it were hers.
If you were so out of the loop as to not realize that no experiments were being performed and the data was being fabricated wholesale, then it is fraudulent to put her name on the scientific paper as an author and to present as an author. She is not conducting research but is a renting out a paper mill.
That kind of behavior needs to be strictly condemned
False authorship definitely happens a lot though but mostly with papers when 1-2 guys did the real work but 10 people are listed. Very common in many big departments.
The instinct on HN is to approach this as an angry blame game. But successful industries don’t do this: for example, the aviation industry performs root-cause analysis after each air disaster and then makes intelligent cost/benefit decisions about how to update processes to avoid future occurrences. In science it’s not clear what the right approach is on a cost/benefit level. Should we reduce scientific output by 50% to catch fraud in the first instance, rather than later on when important studies fail to replicate? Is that a good use of society’s limited resources? I don’t know the answer to this, I just want people to behave effectively rather than grinding an axe. We’re not getting out of the messes we’ve made on this planet without a whole lot of science.
That guy who faked all this can't really be thought of as a sophisticated bad actor, neither was he really gaming the system because it never would have worked. He was just leaping into his own pit.
Research needs to first move to being fully open access and end of tens of millions of taxpayer dollars going to journals. This is something tangible you can legislate. Then, you need to incentivize high-quality research; that's not easy to do at all, but you can do things if you have hiring power within your field I guess...
There's a lot being claimed without peer-review, and a lot of it is bullshit and wastes my time...
The PI is Rosalind A. Coleman. I did not thoroughly investigate the preferred pronouns of this individual but canonically Rosalind would be assumed a 'she' and the top Google images returned for the name present as female.
Require the postdoc to repeat the experiments in her presence could be one of the several solutions available
I don't have a *clue* what the solution is; but, asking ordinary people to turn into FBI detectives is not a viable one. It won't work in the real world, with real humans. It's a fantasy.
I sympathize with the outrage in this thread. But, go past your initial hot-takes: would you apply the same standards to yourself? Suppose you're in software, and one of your coworkers is sneaking subtle, obfuscated security bugs into your production code. Is it your fault if you fail to catch that? Would you consider yourself culpable if you did a code review and failed to notice some "Underhanded C Contest"-tier sophisticated backdoor? Because IMHO that is utterly impossible; and I think it's similarly impossible to catch data manipulation, reliably, if it's sophisticated enough.
Solutions need to be grounded in what's achievable.
There was no raw data manipulation, but fabrication of derived data. These kind of too-good-to-be-true results are easily auditable, if anything by replicating the experiment. The PI’s blind enthusiasm prevented her to do her due diligence.
> would you apply the same standards to yourself?
While I was in academia I checked basically every kind of raw biological data I was getting. Routine QC allowed me, for instance, to find contaminants in some of the reagents the lab was using.
The concern of the author is ultimately one of status, preeminence. Of course people motivated by prestige will not always be trustworthy. Her sin isn’t being too trusting, it’s a lack of self awareness.