I see it from another angle: it was resources spent to save the public from a lie. It's good investment. Desiring a world where no scientist lies or falsifies data is wishful thinking, in my opinion.
I thought this was very well explained, but the tests that K Sheldrick did were not actually particularly complicated. I remember learning about the Fisher Exact Test in an introductory college stats class. I have to wonder why any of the nine other groups that tried to replicate the results didn't ask for a statistician to look at the original results first.
This is flat out fraud, and if the research used any public funds the guilty party/parties deserves criminal prosecution.
> if the research used any public funds the guilty party/parties deserves criminal prosecution.
That sort of prompts the question though: Should it be a crime for any scientist to deliberately publish falsified data?
Obviously there would need to be a very high bar for proving that it was deliberate and not an honest mistake (or even complete incompetence), and we wouldn't want juries to have the power to stop new scientific theories emerging, but other professions are subject to criminal penalties if they fail at their job.
Scientific fraud is deeply immoral because it has the potential to subvert our sapience, harming our ability to act rationally and morally. Such behaviour is criminal from basic principles of justice and human laws should reflect that.
I guess academic fraud is fraud in the legal sense. The problem is that the 'fraud' statute requires 'damages' and 'intent' to have standing, and they're difficult to prove in academic fraud.
Nearly always, lies are protected speech. The Supreme Court has decided, rightly I think, that the government is not capable of determining what is opinion and what is fact.
Well, you would need to prove it was intentional and done by the person you're charging. I.e. if you could prove the assistant did it, you should be able to charge the assistant.
That's not really different from most crimes... sometimes being hard to prove doesn't mean we just make it legal.
I disagree that you need to prove who was responsible. The authors of the paper signed off on its content. They are responsible, in the same way a Professional Engineer signing off on drawings is responsible: it doesn't matter who did the work, they're guaranteeing it.
(As my username suggests, I have published scientific papers where I'm the corresponding author. I've also been added to truly shitty but honest papers against my will or knowledge, in cases where I did some of the work but left before the paper was written. I would never have approved the papers that were published had I been involved. This is an interesting wrinkle if criminal liability is on the table!)
Except it is nothing like a professional engineer, as an engineer is explicitly placing his professional license in jeopardy and assuming legal liability for those 'drawings'. Engineers have a far higher standard than academics.
You'll notice engineers tend to be scarce in arenas such as politics, it's safe to say people accustomed to and willing to face repercussions are a poor fit.
> Engineers have a far higher standard than academics.
That's exactly why I'm arguing for raising standards in academia. When you publish a paper with a traditional publisher, you have to sign a bunch of stuff. You are aware that you are literally signing off on something and claiming responsibility for it.
In my opinion that responsibility should extend to legal liability for fraud with criminal penalties if mens rea can be established.
And, yes, I'd freely apply that standard to every paper I've ever published. Because they might or might not be wrong, but they're not fraudulent!
> That sort of prompts the question though: Should it be a crime for any scientist to deliberately publish falsified data?
I'd go with maybe based on the context. If you take money, such as grants, etc, for studies, then lie about your results that might be fraud (IANAL). If you use those incorrect results to apply for more grants for further study that is definitely fraud. Padding your CV with publications where you lied about the results may not be fraud but would seem the same as any other lies on a CV and could justifiably result in you getting fired.
As you stated, other professions disbar, disqualify, ban, in addition there are civil, criminal penalties for (gross) negligence.
I'd not be worried about "honest mistakes" so long as a juries are involved, with respect to specific studies with direct, actionable consequences.
So there would be a difference between some nut jobs proposing a flat earth theory (unqualified), which they simply can't reproduce, to running a medical trial where potentially the medical technicians, MDs, surgeons, etc. might cite and apply a study.
Its fine to have some crazy theory so long as you are not going to present it as some actionable truth - if you publish a study and show negligence and cannot reproduce, then it is known as such. You are welcome to incomplete, possibly outright incorrect theories, so long as they are academic and not cited as mainstream "technology". Its when your bullshit becomes measurably impactful, is when you decide to impose consequences.
Not sure how to write that up in legal terms, but.
Big effect sizes tend to be fabricated. It's really disturbing. I do lots of biostatistics in academia and professors openly acknowledge many trials cherrypick data.
The outcome is that if you are honest it's hard to get funded because your results look bad.
Then, if we talk about honest results only, crappy statistics also leads to crazy optimistic effect estimates. Regularizing priors and pooling should be the norm. Andrew Gelman posts a lot about this: https://statmodeling.stat.columbia.edu/
I hope (but doubt) that the repeated fraudulent studies, the lack of replicable results followed by very strong negative results from more ethical scientists, and the self-serving grift that the HCQ/IVM/Zinc promoters have engaged in is enough to get people to really step back and consider why they believed these charlatans in the first place.
Plenty of ~intelligent people on HN and elsewhere, who would never sign up for a Gwyneth Paltrow detox regime, talked themselves into believing that there was some forbidden knowledge being exposed by these quacks. It was always so transparently silly.
> kills too many normal cells to be safe to do in living people.
Not an oncologist, but that sounds and awful lot like how cancer is currently treated with chemotherapy and radiation, kill everything and hopefully get all the cancer before the patient dies.
I think I heard about it on 60 Minutes, so... grain of salt there. Information here [1] details the news thread that must have led to the 60 Minutes piece. Interesting how the FDA jumps in to do rare enforcements, cautioning against its use for the risks, when the risks don't sound worse than conventional cancer treatment, and the ACA is also piling on. I think it is suspicious that not only would no one be interested in more research because the drug is unpatentable, but that also FDA and ACA are saying there's nothing to see here, when there is obviously something to be seen. Cancer is big business. I always figured it would never be cured because it is too many things, but maybe it will never be cured because it is too profitable.
> Not an oncologist, but that sounds and awful lot like how cancer is currently treated with chemotherapy and radiation, kill everything and hopefully get all the cancer before the patient dies.
Sure? That makes it a tool, and far from something you could simply call a "cure" for cancer. If it works as a cheaper chemo drug, great, but even if you make chemo drugs free that doesn't make cancer cured or cheap.
Seems like DCA is significantly more toxic than other chemo agents, and wasn't justified at the doses used. It might be good adjunct therapy with neuroprotectants or something...but that's an area for serious chemistry research!
> but that's an area for serious chemistry research!
Biology was never my thing, but it stands to reason, even if it is cynical, that if a cancer cure could be developed, except that there was no possible way to profit from the cure, then a cure would not be developed. Studies and research and development of treatments is very expensive, so the results need to be profitable or they will not be pursued.
That wasn't my claim, and if there is a good amout of research done by BigPharma without the goal being massive profits, they should really tell us about it.
My claim is not that BigPhatma spends too much money researching every drug. My claim is that BigPharma doesn't research new drugs or treatments unless they're going to be profitable, which means that BigPharma does not invest in development or research of unpatentable drugs that already exist.
> My claim is that BigPharma doesn't research new drugs or treatments unless they're going to be profitable
If that's your claim then fine, that's pretty true.
But what you originally said was that a cure would not be developed. Which means by anyone, not just big pharma (why are you making it one word with intercaps?).
And while studies tend to be expensive, the better something works the smaller of a study you can use.
Yes, but it wasn't really the drug I was drawing attention to but instead that if a cure can't be profitable, it will not be developed. It wasn't always this way. The polio vaccine was developed and never patented. Edward Murrow asked Jonas Salk who owned the patent. Salk’s famous reply was “Well, the people, I would say. There is no patent. Could you patent the sun?"
> It cures cancer, but it is never prescribed because it is unpatentable.
The article I found says:
> The occurrence of side effects such as neurotoxicity as well as the suspicion of DCA carcinogenicity still restricts the clinical use of DCA.
I am thoroughly anti-establishment and particularly anti patent as well. I am a libertarian communist aka an anarchist. But you're not going to win any arguments misrepresenting things like that, saying "the cure for cancer exists but they won't use it because they can't profit".
It sounds like what they are trying to do is figure out what mechanism DCA uses to work, so they can find away to do the same thing without the toxicity issues.
Of course everything in our world is touched and marred by the profit motive. So perhaps they have not studied DCA enough because of this profit issue in general. But perhaps it does a good profit opportunity since DCA is toxic - they have to invent something new that does the same thing as DCA without the toxicity.
Anyway I sympathize with your feelings about the world, but your obvious black and white views and grand claims aren't going to make you very popular in online discussions. The profit motive does suck, but the world is still a very complex place.
You seem to have very strong anti-establishment opinions. There are a lot of off-patent drugs (i.e. generics) that are dirt cheap and widely prescribed. Metformin is one example. Unfortunately, your world-view prevents you from seeing through the logical holes of your own statements. If dichloroacetate cured cancer, it would require a global conspiracy to conceal that fact. Doctors, hospitals, and insurance companies have quite the opposite incentives from big pharma. Like you, they dislike the excessive profiteering from pharmaceutical companies. Why would they be party to this conspiracy?
There's a balance to be found: we want enough "quacky" people to try crazy things to find what might work, especially when dealing with newly emergent diseases. On the other hand, we do not want those people to falsify data, and we want to minimize the incentive to do so. But the fame that comes with finding an off-label treatment that works is its own incentive to fraud/being sloppy.
Yep - especially during a novel pandemic of the scale of Covid19 - I'm really glad they looked at all of the available treatment options. Unfortunately it was extremely damaging for them to hold onto them as long as they did while crying "censorship". There were tons of people who refused to get vaccinated since they had access to IVM - the /r/hermancainaward subreddit was filled with people saying as much before they ended up in the ICU.
"Strong convictions loosely held" is an admirable way forward on testing everything we have to save lives. Unfortunately when it became so politicized and the unethical doctors and scientists realized they could print money by claiming they had a secret cure being suppressed, the "loosely held" bit was left behind.
> Plenty of ~intelligent people on HN and elsewhere, who would never sign up for a Gwyneth Paltrow detox regime, talked themselves into believing that there was some forbidden knowledge being exposed by these quacks. It was always so transparently silly.
I'm afraid that this is thanks to political tribalism. Especially in the system where you have only two sides.
Part of the problem is that the more reputable hydroxychloroquine studies were abandoned based on an even more blatently fradulent paper in the Lancet claiming it actually increased the risk of patients dying - like, it wasn't even some subtle statistical issue, there was no way the authors could've had access to the data they claimed to use and the number of patients supposedly in the study from some countries was implausibly high. So there wasn't really any good-quality evidence for a long time. Also, there's reason to suspect that the reason the Lancet was so willing to publish that paper is partisan; their editor-in-chief's Twitter had a blatant political axe to grind the whole time.
There's a ridiculous number of youtubers who are basically Gwyneth Paltrow For Men, and selling bogus supplements is a lucrative business that enables a self-sustaining disinformation ecosystem.
Also people really want to "own the libs" at any cost.
Single data point, but my grandfather was treated for a systemic infection at Sentara Norfolk General by Dr. Marik using his Vitamin C protocol and it brought him back from the brink of death when nothing else worked.
Is the study fraudulent? Could be, and his fight to use Ivermectin on Covid patients always struck me as quackery as well. That said, it seems to me that there is something about the Vitamin C protocol that should not be dismissed.
It's incredibly unlikely that the Marik Protocol is what made the difference. Following this fraudulent paper, the protocol was tested in 9 randomized controlled trials and they were all negative.
And as a statistician I can tell you with absolute certainty this paper is fraudulent.
As a non-statistician I can tell you this blog is absolutely unconvincing, because it doesn't explain anything about what the test is.
Apparently this test is how likely it is for the same number of people to have the primary diagnosis in each group because all the 1.0 have equal number of patients with that diagnosis or off by 1. Like if they said, we have 8 people with COPD in the first group and then waited for COPD patients to show up at the ER for the second group until they had 8 then you would get a perfect 1.0 match since they made that variable the same? And then the other variables would be randomish, but maybe correlated so more than 0.5?
Maybe you can explain how this layman take is wrong, but if all that's going on is they selected patients with the same diagnoses and didn't make that clear in the paper then I don't understand what the big deal is.
I'll give it a try. The study is two small populations that are assumed to be similar. If this were the case, then knowing the study ID (1 or 2) would sometimes wrongly look predictive for different conditions. However none of the measurements have this flaw, and that it self is very very unlikely. The claim is the data looks like it was designed to not have any imbalance from between sets 1 and 2, to an extend that is itself unlikely due to sampling.
The entire second group was picked after the first one, and there were imbalances on almost half of the diagnoses. So if I understand this right, the problem is not the 1.0s since if they were selecting the same number of patients with the same primary diagnoses for the second group then those would be expected, but that there should be some 0.2s in there as well.
"if they said, we have 8 people with COPD in the first group and then waited for COPD patients to show up at the ER for the second group until they had 8"
Did you not read this? I asked whether if they did this they would come up with the perfect 1 scores on those items. Whether they actually did or not is a separate matter.
What troubles me more than whether this study was faked or not is the certainty HN readers have that it was without being able to answer simple questions about why they believe that. Is this test just measuring the likelihood of each group having the same primary diagnosis? If they added people to the second group so they had the same number of a primary diagnosis, would that result in a 1.0 p value on this test?
These should not be difficult questions to answer for somebody certain that fraud occurred.
1. The procedure you describe would make patient recruitment non-consecutive, contrary to the reported procedure. Consecutive means you don't skip anyone who has the condition you're trying to treat. You would have to skip people if you're selecting matched treatment and control groups. In this case, people with sepsis who don't fit your matching design have to be skipped and excluded from the study.
2. Patient subgroup counts matched perfectly not only on COPD, but on about a dozen other conditions. Difficulty of matching subgroup counts grows rapidly with number of dimensions. To match subgroup counts for Group A and Group B near-perfectly on a dozen different dimensions, there's no substantially easier method than just one-to-one perfect matching, which requires a very, very large number of patients. You have to take Patient A1, who has subset X1 of 12 different conditions, and find Patient B1 with that exact same subset. Then repeat, 47 times in this case.
It is already quite hard to find the person B1 who has the exact subset X1 of conditions that person A1 had. For example, if there are 12 conditions, each condition is present in half the people that come into your clinic, and the conditions are independent, you'll need to go through 2^12 = 4096 people, on average, to find another exact match. The conditions may be a bit correlated but this can only help you so much when you're talking about 12 different conditions.
To repeat that feat 47 times is very hard. You'd have to churn through 10s or 100s of thousands of sepsis patients to get your matching subset. This would require access to an enormous pool of sepsis patients and constant reporting of all the conditions they have, that you want to match on. For this to be done without any mention in the paper is utterly beyond belief.
And the fact that the counts match near-perfectly in an off-by-1 fashion does not help the situation at all.
See this is fascinating to me. You are the statistician I originally replied to, and yet you also won't clearly answer the questions! This is like pulling teeth, but the implication seems to be my take on this statistical test was basically correct.
> Difficulty of matching subgroup counts grows rapidly with number of dimensions. ... and the conditions are independent, you'll need to go through 2^12 = 4096 people [for 12 conditions]
Except many of these are "primary" diagnoses, which presumably you have one of hence the name, so it's not possible for a person to have both a primary "Pneumonia" diagnosis and a primary "Other" diagnosis. So for each person maybe you're matching 2^3 or 2^4, not 2^12 - in any case, far less than your conclusions are based on.
> You'd have to churn through 10s or 100s of thousands of sepsis patients to get your matching subset.
So if the matching was 1/1000th as hard as you thought it was, that would be 10s or 100s of patients. A million cases a year, several doctors in major metro areas, there's probably 10s to 100s of patients at any given time in their hospital systems.
> Consecutive means you don't skip anyone who has the condition you're trying to treat.
Sepsis is very serious and common, so they'd have to treat them all basically simultaneously with the experimental treatment. I'm no expert, but I'd expect they'd want to be able to abort the trial if people started dying from it. It also doesn't even make sense as a study, because you want to test like for like as much as possible; maybe the treatment works fantastically on patients with cirrhosis and nobody else.
I think a plausible scenario is each day before normal rounds they did a med search for a patient matching the next one from first group, generally found a good match, worked with their doctor to change the treatment and added them to the study. Maybe two in a day, or skipping a day, and a month later they have really good matching data. Lots of cases to choose from, many exclusive variables. What do the statistics say for a charitable interpretation? Pretty good odds, right?
So maybe they didn't report their methods accurately, maybe it was assumed from domain knowledge or was a mistake when they cut and pasted from a template. Could be fraud too, but that seems like a huge leap to be certain of.
Am too a non-statistician. If I understand correctly: The two groups have very strong statistical relations across multiple variables (not just COPD), yet the study clearly states they were chosen almost randomly. This is extremly unlikely to happen in a perfectly randomized test.
What you describe is selecting the group in advance so they match (though you use only one variable). There are cases where doing this selection is reasonable and is useful. However, the study describes a different selection process. If the selection process described was not the one actually used, what else in the study does not match reality?
Vitamin C, I think studies finally proved about 10 years ago, puts the immune system into overdrive. So perhaps Vitamin C kicked up his immune system, and his own immune system is what beat the infection and put him on the mend.
Vitamin C is simply an electron donor for other chemical reactions. That can be very very important if not critical. However more is not better. Endurance athletes know that taking more than say 200mg (twice the RDA) of C per day will defeat training adaptations because it calms the stress response too much.
However people get more than a bit overboard and woo-woo about Vitamin C.
Linus Pauling was a chemist and was obsessed with it but at least he was scientific about his quest.
ps. meta analysis of 60 years of Vitamin C studies with placebo controls showed the opposite of what you are remembering, C did not prevent or reduce the common cold symptoms/severity - and the electron donor principle can be googled or it's right on that LPI link above.
But ha I have the same size complaint. I have to sit there with a safety blade and cut 500mg tablets into four and then of course it's so imprecise.
They market what sells. People don't want to buy 100mg if 500mg is next to it for the same price. Everyone think more is better, super-size it.
BTW you can buy high quality Vitamin C gummies (ie. Vitafusion Power C) which because of their limited capacity per gummy are around only 100mg and the bottle says to take 4-6 but of course you can just take one. Studies show gummies have the same absorption quality of pills, downside is only the added sugar, which is tiny.
> ps. meta analysis of 60 years of Vitamin C studies with placebo controls showed the opposite of what you are remembering, C did not prevent or reduce the common cold symptoms/severity
Looks like there were two conflicting studies [1][2]. One concludes Vitamin C reduces frequency of common cold, but has no effect on severity, and the other concludes Vitamin C reduces severity of common cold, but has no effect on incidence. This is disconcerting, because I recall quite clearly that the study was around 2010, was a massive study and concluded as I described above in my reply to OP. But these studies are from 2005 and 2013. Another from 2005, concerning a boost to immune system, states "These trials document that adequate intakes of vitamin C and zinc ameliorate symptoms and shorten the duration of respiratory tract infections including the common cold. Furthermore, vitamin C and zinc reduce the incidence and improve the outcome of pneumonia, malaria, and diarrhea infections, especially in children in developing countries." [3]
Still nobody could explain with logic how American "BigPharm" and their profits and patents influence other countries from Europe ,China ,russia. Say CheepX cures Covid why would non american use it? same for the cancer miracle cure? Do all this countries ahte their people?
> Single data point, but my grandfather was treated for a systemic infection at Sentara Norfolk General by Dr. Marik using his Vitamin C protocol and it brought him back from the brink of death when nothing else worked.
And this is exactly why scientists rightfully ignore "single data points" such as this. People get sick and recover all the time. And sometimes people get very, very, very sick, and they still recover on their own.
So there would be a non-trivial percentage of people who would have gotten very sick, and just based on timing it wouldn't be unusual that they would be given some treatment and then credit that treatment when they recovered.
Not only is this study blatant fraud, nine other studies tried to replicate the results and showed no benefit to sepsis outcomes.
Not to mention, a mild vitamin C deficiency will screw with the immune system. In such cases, I can imagine vitamin C treatments having a clear and measurable effect.
The same applies for quite a wide range of other nutrients as well.
Yeah, wasn’t there a good meta-analysis of the “Ivermectin treats Covid” papers that basics said “Ivermectin seems to treats Covid in countries where certain parasites are endemic, and if doesn’t where they don’t”.
That's from an Alexander post (slate star codex fame), but there is good argument for effect beyond parasitic reduction... The politics and money involved have insured we will not come to the truth of the matter
For farm animals, acids are used to reduce the pH value of drinking water and to increase health. If you consume vitamin C via citrus fruits like oranges, then you are making a more hostile environment for bacterial infections.
> For farm animals, acids are used to reduce the pH value of drinking water and to increase health.
I have never heard of this practice, and can find no evidence of it online. Can you provide a citation?
> If you consume vitamin C via citrus fruits like oranges, then you are making a more hostile environment for bacterial infections.
This is utter nonsense. The pH of a living organism is tightly regulated, and a failure of this regulation can be rapidly fatal. Consuming small (sub-gram) quantities of vitamin C does not have any meaningful effect on blood or extracellular pH.
> This is utter nonsense. The pH of a living organism is tightly regulated, and a failure of this regulation can be rapidly fatal. Consuming small (sub-gram) quantities of vitamin C does not have any meaningful effect on blood or extracellular pH.
Translated for laypeople:
This is BS. The pH of your body is constant and your body has mechanisms to make sure it stays constant. If your pH changed as claimed, you’d die.
The mechanisms that maintain a stable pH are driving outcomes.
Your statement is accurate, just doesn't acknowledge the changes required for homeostasis. There is considerable effect merely in osmotic pressure relative to cellular function. To dismiss that carries as much error
Sample size of 1 here too. My dad started drinking MMS (Miracle Mineral Solution) , which is a solution you dilute in water to create a compound that cleans the body or something. He started feeling better, no more headaches, better humour, started loosing weight, was sleeping great, etc. Turns out my brother replaced it with water the next day he bought it. Placebo was the real miracle here.
Your brother definitely did the right thing, given that "MMS" is actually chlorine dioxide, which is an industrial bleach. Drinking it can seriously damage the digestive system.
So this is a recent study but surely there’s prior ones as well? How many people actually go and check studies like these for validation? I know there’s reproducibility issues with certain social sciences but how wide spread is this? Is anything being done to address such a glaring dependability issue?
The protocol is vitamin c + thiamine + hydrocortisone.
I'm 100% convinced hydrocortisone is bioactive is sepsis patients, like it is in anyone else . Also 100% convinced vitamin C boosts antioxidant levels in sepsis patients who're vitamin-C-depleted.
This is the same Marik behind efforts to treat COVID with Ivermectin. The paper he co-authored about treating COVID was found to have major issues and was retracted.
As for this particular paper about the Vitamin C trial (which hasn't been retracted yet): numerous other groups tried replicating it and none could. Millions and millions and millions of dollars, countless hours wasted on conducting controlled trials all because of this guy's fraud.
My two takeaways:
1) Kudos to Sheldrick, we absolutely need more people like him. We need statisticians to pick apart data in papers and see if it smells funny. This should have been caught earlier.
2) There should be a higher price to pay for this kind of fraud. For the confusion and mistrust that ensued, for the money and human capital that was wasted, he needs to be behind bars for a long, long time.
It would be better that independent studies tried to detect errors, not fraud.
Sometimes it is difficult to distinguish between both but of there is a track record of fraud no reputable journal should accept more papers from that source.
Edit: As I re-read my comment I worry because research should not really depend on "prestige", it should be based on cold facts. On the other hand I don't see any other way to deal with malicious actors.
There will always be fraud and we will always need to invest in trying to reproduce everything in order to sort them out.
It's absurd to think there will be a way to know which studies need reproduction or not. The whole purpose of reproducing is to find out which ones are true and which are bogus
We need to reproduce every single study, including the fraudulent ones. Gosh, ESPECIALLY the fraudulent ones.
But we can't possibly know in advance which are which. That's the whole purpose of trying to reproduce.
Without reproduction efforts, scientific papers are of no value, as we can't trust them.
In this sense, a paper without reproduction effort is not a "window" in the analogy. Only after reproduction efforts it becomes a window.
So saying it's a waste to try reproducing an article is just absurd. It's like saying it's a waste to produce the "window" in the first place because a kid might break it in the future (in case of the paper, that we might figure out the paper was fraudulent).
It's still valuable to find out a paper was fraudulent. It builds trust in the whole scientific endeavor.
There wouldn't be a point to reading papers/textbooks, buying equipment from others, etc., if you thought that it was all part of some elaborate deception.
The general presumption would tend to be that other scientists are trying to do good, honest work. Scientific progress does better when that presumption turns out to be more true.
To be clear, I was responding to their apparent disregard for the harm done by the original alleged-fraud.
Checks are good, definitely. And they help to limit the damage caused by stuff like fraud. You were right about that stuff.
The point I think folks were contesting was the tone some may've inferred from [this comment](https://news.ycombinator.com/item?id=30787447 ), which may sound a bit like downplaying the damage caused by fraud under the premise that reproduction ought to catch such things anyway.
For a computing analogy, TCP uses mechanisms that enable it to tolerate packet-loss. But packet-loss still slows things down, limiting bandwidth and increasing latency, until a connection would be impractical. For a connection with many hops, even a small packet-loss-rate along each hop would be a huge problem, even though TCP actively checks for packet-loss and tries to fix it.
Likewise, science-with-replication presumably ought to be able to fix a lot of fraud and errors -- but those problems take a heavy, debilitating toll on scientific progress; it's not a small problem that replication-studies alone could satisfactorily rectify.
What the people inferring from my comment are missing is that the reproduction efforts actually saved us from damage.
The real damage would have been everyone believing the paper conclusions were solid and start to act according to this false belief.
Reproduction efforts saved us from that damage.
These efforts were not the damage.
Whoever is seeing differently doesn't understand one of the basic foundations of how the scientific endeavor works.
And again: anyone expecting a world where there are no dishonest scientists or where the dishonest ones call themselves out in advance to save from the "damage" of people spending time reproducing their papers, is just playing wishful thinking, which is not a recipe for success...
The difference is that extinguishing a fire has no net benefit other than fixing the problem caused. The data collected in attempting to reproduce something is valuable even without original claims (especially is shared appropriately).
So I agree with OP the time wasn't wasted, but it could have been spent on better opportunities.
I feel like there should be criminal charges, as the treatments and their side effects have affected people and robbed them of proper care. The amount of people affected by this is humongous.
I respectfully disagree: public shaming should be enough. After all, we have a scientific community for this exact reason: we can tell frauds, or blatant mistakes if we want to feel charitable, by examining them collectively. And that serves as an immunity system against these cases. No other vengeanceful punishment is really necessary.
Also, if we really are fond of consequences, frauds carry over the price of having to confront, for future publishing, with a scientific community that has grown a bias against the frauder; and the latter will have to regain trust if he/she wants to publish again.
The criminal charges should be for those who use unproven scientific results to endorse their own foul agenda.
I don’t know, it seems like the consequences of fraud change significantly once someone has stepped into the realm of medical research. Publishing fraudulent research seems like it is, at the very least, medical malpractice and should affect the author’s ability to practice medicine.
The thing is, the impact of this fraud is not simply monetary damage or wasted time of researchers, the impact is lives lost or permanent damages done to affected patients health and well being. The consequences are quite different.
If a system allowed me to publish fraudulent research or publish bad research recklessly, with the only consequence being "public shaming" (which is sometimes not really consequential if one has the right connections to the right people), what's to prevent me (and everyone else) from doing it over and over again?
TBH from a legal perspective knowingly publishing false information for personal gain is already fraud. You're arguing that authorities should stay away because academia should somehow have immunity? That's a very disturbing line of thought.
> 2) There should be a higher price to pay for this kind of fraud.
At the moment, University incentives are to protect the prestige and not for honesty.
There should also be a University list of shame, based on author affil of papers accused of fraud.
The trouble and defense will be that any rando can accuse a paper of fraud, and instead one should use finalized decisions (such as retractions). That would cause delays and not lead to any resolution in many cases, and I doubt that the number of papers false accused of fraud based on any substantial statistical analysis is large.
Karma would help. One could then keep track of people who falsely accused papers of fraud in the past, and correctly pointed out papers which were later retracted, and then weigh the University shame list based on that.
Call me old-fashioned, but I suspect Sheldrick doesn't really take the measure of the risks. It's like walking on a wire. Very impressive, but just one misstep and you're joining the crocodiles (MD profs) below. Some of them really have unsuspected reach and power. Actually, the sheer dishonesty and nastiness of the MD hierarchy is probably what surprised me the most at the onset of my career.
> ...I have not requested access to the raw data or contacted the authors for explanation as the case is audacious...
Happily from the sidelines - it probably is fraud and this isn't really an article targeted at the general audience. But the statistical evidence proves beyond a doubt that the data wasn't collected by the methods described in the paper. That doesn't prove it is fraudulent because formally speaking he can't rule out that they just did a very bad job of explaining their methods.
Just from life experience, any time anyone goes in guns blazing based on statistical evidence - without actually talking to the people involved - it is often a mistake. Just because someone can't think of an alternative reason says more about their imagination than the reality of the situation. Particularly coming in hot off a twitter argument.
That being said, these are unusual and unexpected patterns that show irregularities in the data, and I would certainly share his concerns.
You need to consider the historical context too. The referenced authors has been found performing statistical fraud before. P(Mistake) is probably lower.
No there is no reason to be skeptical of the author here because he based his evidence on statistics.
As the author describes, the p-values listed are effectively impossible to get from real data. Because of the magic of mathematics, we know exactly how p-values are meant to behave when you calculate them repeatedly - uniform random between 0 and 1. There are a battery of tests you can use to interrogate this very well understood distribution. Those tests, as the author states, prove without any doubt that the p-values in table 1 are effectively impossible. You can't make it to those p-values by mistake.
P values are, effectively, the probability of a result occurring because of a process. These p values are overwhelming evidence that the result did not occur because of the process described in the study.
That does not mean fraud (although, I stress, in this case I can see why it is probably fraud). For example, maybe the study is done in some sort of hospital with some sort of patient scheduling system and the front desk is playing a funny joke on the doctors in this study. It is quite common for experimenters to not be as much in control of their environment as a paper suggests - that would be an honest reason for why the data is weird.
In a more plausible example - I don't know how "matching" works, but maybe the patients were matched, the report writer didn't know that and the sentence slipped through proofreading. Unlikely, but certainly possible. That mistake would happen in the wild from time to time.
You don't need much imagination to find examples other than fraud. Although it does take a little imagination, which is concerning. But, frankly, there should almost always be an attempt at dialog with someone before accusing them publicly of being a fraud. Unless they're going to shoot you or something extreme.
The author of this work addressed the notion that patients were matched and the methods were described incorrectly, even in that situation the distribution of p values is so highly unlikely that it’s most likely fraud.
Sloppy research methods leading to bogus results are can be construed as fraud, particularly if you don’t know the motivation for the lack of care.
A diligent researcher might take these results which were “incredible” and attempt to replicate them before publishing.
Sheldrick has just caused massive reputational damage, got thousands of strangers reading & discussing how Marik's work looks like fraud (despite few of us actually being qualified to assess its merits) & is posing a serious professional attack to Marik's wellbeing. It is unlikely but in the realm of possibility that someone would commit suicide or get some sort of trauma from this sort of attention.
He's doing that based on an interpretation of some study, which I suspect is approximately a 20 page document.
Now on the one hand, he is probably right. On the other hand, this is a very brave approach - the damage and the care taken aren't in good proportion here. This sort of thing is easy to get wrong. He should have talked to the paper's authors first.
People publishing a study that is so blatantly against everything that is statistically likely should be called out.
The author published an email he also - before publication - sent to the people involved.
Science happens in public for a reason. The reason is that others should be able to check the results being published. Authors should therefore be very aware of the fact, that their results might come under scrutiny.
If one were to publish claims that look so unlikely based on valid scientific methods they should at least address the unlikeliness of the results and provide a rationale why this is still valid, even if it looks unlikely.
That didn't happen - and so they get called out. The extreme unlikeliness is what validates the claim of this being fraudulent data. Now it is on the original study`s authors to react to that public answer to their publication.
I am guided by the "I have not requested access to the raw data or contacted the authors for explanation as the case is audacious no other explanation is possible." comment in the letter.
That's a dangerous statement. I'm pretty sure I got p-values of over 1 or under 0 by mistake. People will make every mistake you think is impossible. (Not that I think it's a mistake here)
No. Absolutely not. There is no non-fraudulent explanation here.
In the best case of "poorly described methods", we're talking about authors who blatantly lied about how they sampled their patients and deliberately omitted what would have been an extremely laborious process, involving many thousands of extremely non-consecutive patients, to produce an extremely well-matched treatment and control group.
You've got your choice as to the exact nature of the enormous lie or fabrication, but there's no possibility of innocent omission.
I think it's difficult to eliminate the possibility of a tremendous degree of incompetence, without outright fraud, based just on the p-values near 1.
As noted in other comments here, one possibility is that the authors failed to correctly describe the procedure they used. Sheldrick does consider this, pointing out that even if they actually did try to match subjects, unlike what they said in the paper, it would be very unlikely that they could match them well enough to obtain the reported p-values. But another possibility is that they were totally incompetent at using their statistical software - that the reported p-values are not actually the real p-values for the tests that they said they did. Maybe they accidently used the wrong dataset, for example. Or perhaps they copied numbers from the wrong column of the table the software produced.
In a wider sense, though, this would still essentially be fraud - it's dishonest to publish a paper that purports to be a competent study when really you are totally clueless about how to do research, or care so little about correctness that you put no effort into checking that you did things correctly. (And surely they would have to suspect that they don't really know what they're doing...)
This isn't about reported p-values. Sheldrick computed p-values from the reported treatment and control group samples. The latter are basic count data, not the output of statistical software. So here we are talking about either the raw data being wrong, or this tiny amount of raw data being incorrectly summarized.
Something like a column duplication would have resulted in perfectly identical data between treatment and control groups, which this data does not show.
The best case scenario is an errant copy-paste or Excel formula that somehow presented both groups of data in a way that actually only looked at one of the two groups, but also somehow introduced small differences along the way.
How exactly this could happen is beyond my imagination, but perhaps it introduces a sliver of doubt as to whether the paper is fraud by intention or merely by negligence.
Remind me the OTC company I have invested year ago, one of the cofounder had a really decent record, I am not expert in the bio research area but i had an impression that he was one of the respected authority in stem cell research. Then I try to find answers for "why stocks are so cheap", turns out his group confidently started fast trials at the beginning of COVID and resulted some deaths. It's hard to judge due to nature of pandemic executing a fast trial but his way of handling the aftermath was totally disaster for his company
In light of recent trends toward many more authors, I think it is worth revisiting scientific analysis of Ouija boards to look at how many participant processes arrive at a conclusion they find satisfactory without being aware of their own bias.
Once I had an argument with a PhD biologist at my university because she was doing a Fisher's exact test in the most insane way I had ever seen: taking floating point ratios instead of integers, and because the stats program she used didn't take floating points as valid inputs for the test---obviously---, her workaround was to multiply these numbers by 100 to get integers. She refused to back down on this madness because "her supervisor did it like this, and his supervisor before him". In the end, her paper was peer reviewed and published and it was the first in a series of let downs that made me decide not to pursue a career in academia. Sometimes it's not like researchers want to tamper with their results. It's just that statistical ignorance is widespread and affect even the reviewers that were supposed to be the first line of defense. Which is a huge problem for an environment that relies too much on p-values and not so much on explaining them.
"A 2016 poll of 1,500 scientists reported that 70% of them had failed to reproduce at least one other scientist's experiment (50% had failed to reproduce one of their own experiments)."
or at least - this is what it used to say! This has been memory holed.
"According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist's experiments."
It seems poignant to acknowledge that while this type of potential deception or fraud hiding in statistical analysis may be to the detriment of society or patients at large, a greater fraud is being perpetrated on the whole of humanity in the 'one size fits all', 'gold standard' of clinical trials.
The human body is unique between individuals, drug interaction is unique to body chemistry.
This isn't up for debate nor is it absurd to call for genotype specific studies of efficacy. We, collectively, have allowed real miraculous discoveries to flounder as a result of this failure to appreciate personalized medicine.
We will look back on this Era in awe with the ineptitude of our methods.
Randomised trials are for removing bias. You can certainly run precision medicine genomics randomised clinical trials. They are rarely done because so far our ‘omics tools haven’t been good enough to use, but that is changing now.
Quite a bit. Table 1 is standard fare in human clinical trials and you could certainly check the summary stats in it for sanity. But you would find a lot of edge cases where your automated system would struggle.
This is sorta done for meta-analyses that aggregate the results of several studies systematically. I think there are people trying to automate more of it.
Shame on Chest Magazine's editors for not catching this either way.
Most likely, one report-writer didn't understand that these were MATCHED consecutive patients. The data is so clearly matched, it's hard to believe it's fraudulent.
This is addressed by the author. Even if matched, these are impressively matched datasets. To the extent that it is questionable the hospital had the number of potential study participants to match them this well.
156 comments
[ 3.4 ms ] story [ 213 ms ] thread“Remember that Vitamin C cures sepsis paper that could never be replicated in 9 RCTs?
Turns out there is a good reason why: it’s very likely fraudulent.
More brilliant statistical sleuthing by @K_Sheldrick.”
https://twitter.com/nickmmark/status/1506402015936581635?s=2...
This is flat out fraud, and if the research used any public funds the guilty party/parties deserves criminal prosecution.
That sort of prompts the question though: Should it be a crime for any scientist to deliberately publish falsified data?
Obviously there would need to be a very high bar for proving that it was deliberate and not an honest mistake (or even complete incompetence), and we wouldn't want juries to have the power to stop new scientific theories emerging, but other professions are subject to criminal penalties if they fail at their job.
I guess academic fraud is fraud in the legal sense. The problem is that the 'fraud' statute requires 'damages' and 'intent' to have standing, and they're difficult to prove in academic fraud.
That role is reserved for the tribunals. Judges are supposed to determine what is protected speech, and what is not.
That's not really different from most crimes... sometimes being hard to prove doesn't mean we just make it legal.
(As my username suggests, I have published scientific papers where I'm the corresponding author. I've also been added to truly shitty but honest papers against my will or knowledge, in cases where I did some of the work but left before the paper was written. I would never have approved the papers that were published had I been involved. This is an interesting wrinkle if criminal liability is on the table!)
You'll notice engineers tend to be scarce in arenas such as politics, it's safe to say people accustomed to and willing to face repercussions are a poor fit.
That's exactly why I'm arguing for raising standards in academia. When you publish a paper with a traditional publisher, you have to sign a bunch of stuff. You are aware that you are literally signing off on something and claiming responsibility for it.
In my opinion that responsibility should extend to legal liability for fraud with criminal penalties if mens rea can be established.
And, yes, I'd freely apply that standard to every paper I've ever published. Because they might or might not be wrong, but they're not fraudulent!
I'd go with maybe based on the context. If you take money, such as grants, etc, for studies, then lie about your results that might be fraud (IANAL). If you use those incorrect results to apply for more grants for further study that is definitely fraud. Padding your CV with publications where you lied about the results may not be fraud but would seem the same as any other lies on a CV and could justifiably result in you getting fired.
I'd not be worried about "honest mistakes" so long as a juries are involved, with respect to specific studies with direct, actionable consequences.
So there would be a difference between some nut jobs proposing a flat earth theory (unqualified), which they simply can't reproduce, to running a medical trial where potentially the medical technicians, MDs, surgeons, etc. might cite and apply a study.
Its fine to have some crazy theory so long as you are not going to present it as some actionable truth - if you publish a study and show negligence and cannot reproduce, then it is known as such. You are welcome to incomplete, possibly outright incorrect theories, so long as they are academic and not cited as mainstream "technology". Its when your bullshit becomes measurably impactful, is when you decide to impose consequences.
Not sure how to write that up in legal terms, but.
The outcome is that if you are honest it's hard to get funded because your results look bad.
Then, if we talk about honest results only, crappy statistics also leads to crazy optimistic effect estimates. Regularizing priors and pooling should be the norm. Andrew Gelman posts a lot about this: https://statmodeling.stat.columbia.edu/
Plenty of ~intelligent people on HN and elsewhere, who would never sign up for a Gwyneth Paltrow detox regime, talked themselves into believing that there was some forbidden knowledge being exposed by these quacks. It was always so transparently silly.
https://xkcd.com/1217/
Not an oncologist, but that sounds and awful lot like how cancer is currently treated with chemotherapy and radiation, kill everything and hopefully get all the cancer before the patient dies.
I think I heard about it on 60 Minutes, so... grain of salt there. Information here [1] details the news thread that must have led to the 60 Minutes piece. Interesting how the FDA jumps in to do rare enforcements, cautioning against its use for the risks, when the risks don't sound worse than conventional cancer treatment, and the ACA is also piling on. I think it is suspicious that not only would no one be interested in more research because the drug is unpatentable, but that also FDA and ACA are saying there's nothing to see here, when there is obviously something to be seen. Cancer is big business. I always figured it would never be cured because it is too many things, but maybe it will never be cured because it is too profitable.
[1] https://en.wikipedia.org/wiki/Dichloroacetic_acid#Cancer
Sure? That makes it a tool, and far from something you could simply call a "cure" for cancer. If it works as a cheaper chemo drug, great, but even if you make chemo drugs free that doesn't make cancer cured or cheap.
Biology was never my thing, but it stands to reason, even if it is cynical, that if a cancer cure could be developed, except that there was no possible way to profit from the cure, then a cure would not be developed. Studies and research and development of treatments is very expensive, so the results need to be profitable or they will not be pursued.
If that's your claim then fine, that's pretty true.
But what you originally said was that a cure would not be developed. Which means by anyone, not just big pharma (why are you making it one word with intercaps?).
And while studies tend to be expensive, the better something works the smaller of a study you can use.
But something that causes significant nerve damage in 15 out 15 test patients during a study is not a top candidate for chemotherapy.
Edit: It seems potentially promising but also potentially toxic. Needs more study.
Yes, but it wasn't really the drug I was drawing attention to but instead that if a cure can't be profitable, it will not be developed. It wasn't always this way. The polio vaccine was developed and never patented. Edward Murrow asked Jonas Salk who owned the patent. Salk’s famous reply was “Well, the people, I would say. There is no patent. Could you patent the sun?"
> It cures cancer, but it is never prescribed because it is unpatentable.
The article I found says:
> The occurrence of side effects such as neurotoxicity as well as the suspicion of DCA carcinogenicity still restricts the clinical use of DCA.
I am thoroughly anti-establishment and particularly anti patent as well. I am a libertarian communist aka an anarchist. But you're not going to win any arguments misrepresenting things like that, saying "the cure for cancer exists but they won't use it because they can't profit".
It sounds like what they are trying to do is figure out what mechanism DCA uses to work, so they can find away to do the same thing without the toxicity issues.
Of course everything in our world is touched and marred by the profit motive. So perhaps they have not studied DCA enough because of this profit issue in general. But perhaps it does a good profit opportunity since DCA is toxic - they have to invent something new that does the same thing as DCA without the toxicity.
Anyway I sympathize with your feelings about the world, but your obvious black and white views and grand claims aren't going to make you very popular in online discussions. The profit motive does suck, but the world is still a very complex place.
"Strong convictions loosely held" is an admirable way forward on testing everything we have to save lives. Unfortunately when it became so politicized and the unethical doctors and scientists realized they could print money by claiming they had a secret cure being suppressed, the "loosely held" bit was left behind.
I'm afraid that this is thanks to political tribalism. Especially in the system where you have only two sides.
And imnsho a bigger problem: that they were in turn trying to dupe others. This amplification effect is at least as harmful as the initial fraud.
Also people really want to "own the libs" at any cost.
Is the study fraudulent? Could be, and his fight to use Ivermectin on Covid patients always struck me as quackery as well. That said, it seems to me that there is something about the Vitamin C protocol that should not be dismissed.
And as a statistician I can tell you with absolute certainty this paper is fraudulent.
Apparently this test is how likely it is for the same number of people to have the primary diagnosis in each group because all the 1.0 have equal number of patients with that diagnosis or off by 1. Like if they said, we have 8 people with COPD in the first group and then waited for COPD patients to show up at the ER for the second group until they had 8 then you would get a perfect 1.0 match since they made that variable the same? And then the other variables would be randomish, but maybe correlated so more than 0.5?
Maybe you can explain how this layman take is wrong, but if all that's going on is they selected patients with the same diagnoses and didn't make that clear in the paper then I don't understand what the big deal is.
Did you not read this? I asked whether if they did this they would come up with the perfect 1 scores on those items. Whether they actually did or not is a separate matter.
What troubles me more than whether this study was faked or not is the certainty HN readers have that it was without being able to answer simple questions about why they believe that. Is this test just measuring the likelihood of each group having the same primary diagnosis? If they added people to the second group so they had the same number of a primary diagnosis, would that result in a 1.0 p value on this test?
These should not be difficult questions to answer for somebody certain that fraud occurred.
2. Patient subgroup counts matched perfectly not only on COPD, but on about a dozen other conditions. Difficulty of matching subgroup counts grows rapidly with number of dimensions. To match subgroup counts for Group A and Group B near-perfectly on a dozen different dimensions, there's no substantially easier method than just one-to-one perfect matching, which requires a very, very large number of patients. You have to take Patient A1, who has subset X1 of 12 different conditions, and find Patient B1 with that exact same subset. Then repeat, 47 times in this case.
It is already quite hard to find the person B1 who has the exact subset X1 of conditions that person A1 had. For example, if there are 12 conditions, each condition is present in half the people that come into your clinic, and the conditions are independent, you'll need to go through 2^12 = 4096 people, on average, to find another exact match. The conditions may be a bit correlated but this can only help you so much when you're talking about 12 different conditions.
To repeat that feat 47 times is very hard. You'd have to churn through 10s or 100s of thousands of sepsis patients to get your matching subset. This would require access to an enormous pool of sepsis patients and constant reporting of all the conditions they have, that you want to match on. For this to be done without any mention in the paper is utterly beyond belief.
And the fact that the counts match near-perfectly in an off-by-1 fashion does not help the situation at all.
> Difficulty of matching subgroup counts grows rapidly with number of dimensions. ... and the conditions are independent, you'll need to go through 2^12 = 4096 people [for 12 conditions]
Except many of these are "primary" diagnoses, which presumably you have one of hence the name, so it's not possible for a person to have both a primary "Pneumonia" diagnosis and a primary "Other" diagnosis. So for each person maybe you're matching 2^3 or 2^4, not 2^12 - in any case, far less than your conclusions are based on.
> You'd have to churn through 10s or 100s of thousands of sepsis patients to get your matching subset.
So if the matching was 1/1000th as hard as you thought it was, that would be 10s or 100s of patients. A million cases a year, several doctors in major metro areas, there's probably 10s to 100s of patients at any given time in their hospital systems.
> Consecutive means you don't skip anyone who has the condition you're trying to treat.
Sepsis is very serious and common, so they'd have to treat them all basically simultaneously with the experimental treatment. I'm no expert, but I'd expect they'd want to be able to abort the trial if people started dying from it. It also doesn't even make sense as a study, because you want to test like for like as much as possible; maybe the treatment works fantastically on patients with cirrhosis and nobody else.
I think a plausible scenario is each day before normal rounds they did a med search for a patient matching the next one from first group, generally found a good match, worked with their doctor to change the treatment and added them to the study. Maybe two in a day, or skipping a day, and a month later they have really good matching data. Lots of cases to choose from, many exclusive variables. What do the statistics say for a charitable interpretation? Pretty good odds, right?
So maybe they didn't report their methods accurately, maybe it was assumed from domain knowledge or was a mistake when they cut and pasted from a template. Could be fraud too, but that seems like a huge leap to be certain of.
What you describe is selecting the group in advance so they match (though you use only one variable). There are cases where doing this selection is reasonable and is useful. However, the study describes a different selection process. If the selection process described was not the one actually used, what else in the study does not match reality?
However people get more than a bit overboard and woo-woo about Vitamin C.
Linus Pauling was a chemist and was obsessed with it but at least he was scientific about his quest.
https://en.wikipedia.org/wiki/Linus_Pauling#Medical_research...
and it gave us the Linus Pauling Institute which is a great thing for factual nutrition, I use it often as a reference:
https://lpi.oregonstate.edu/mic/vitamins/vitamin-C
But ha I have the same size complaint. I have to sit there with a safety blade and cut 500mg tablets into four and then of course it's so imprecise.
They market what sells. People don't want to buy 100mg if 500mg is next to it for the same price. Everyone think more is better, super-size it.
BTW you can buy high quality Vitamin C gummies (ie. Vitafusion Power C) which because of their limited capacity per gummy are around only 100mg and the bottle says to take 4-6 but of course you can just take one. Studies show gummies have the same absorption quality of pills, downside is only the added sugar, which is tiny.
Looks like there were two conflicting studies [1][2]. One concludes Vitamin C reduces frequency of common cold, but has no effect on severity, and the other concludes Vitamin C reduces severity of common cold, but has no effect on incidence. This is disconcerting, because I recall quite clearly that the study was around 2010, was a massive study and concluded as I described above in my reply to OP. But these studies are from 2005 and 2013. Another from 2005, concerning a boost to immune system, states "These trials document that adequate intakes of vitamin C and zinc ameliorate symptoms and shorten the duration of respiratory tract infections including the common cold. Furthermore, vitamin C and zinc reduce the incidence and improve the outcome of pneumonia, malaria, and diarrhea infections, especially in children in developing countries." [3]
[1] https://www.nature.com/articles/1602261
[2] https://pubmed.ncbi.nlm.nih.gov/23440782/
[3] https://pubmed.ncbi.nlm.nih.gov/16373990/
Still nobody could explain with logic how American "BigPharm" and their profits and patents influence other countries from Europe ,China ,russia. Say CheepX cures Covid why would non american use it? same for the cancer miracle cure? Do all this countries ahte their people?
And this is exactly why scientists rightfully ignore "single data points" such as this. People get sick and recover all the time. And sometimes people get very, very, very sick, and they still recover on their own.
So there would be a non-trivial percentage of people who would have gotten very sick, and just based on timing it wouldn't be unusual that they would be given some treatment and then credit that treatment when they recovered.
Not only is this study blatant fraud, nine other studies tried to replicate the results and showed no benefit to sepsis outcomes.
The same applies for quite a wide range of other nutrients as well.
I have never heard of this practice, and can find no evidence of it online. Can you provide a citation?
> If you consume vitamin C via citrus fruits like oranges, then you are making a more hostile environment for bacterial infections.
This is utter nonsense. The pH of a living organism is tightly regulated, and a failure of this regulation can be rapidly fatal. Consuming small (sub-gram) quantities of vitamin C does not have any meaningful effect on blood or extracellular pH.
Translated for laypeople:
This is BS. The pH of your body is constant and your body has mechanisms to make sure it stays constant. If your pH changed as claimed, you’d die.
If you trust anecdotes, your conclusions will be wrong most often than right. You might be lucky here and there, but will lose in the longrun.
Science is about being right most of the time and winning in the long-run.
I'm 100% convinced hydrocortisone is bioactive is sepsis patients, like it is in anyone else . Also 100% convinced vitamin C boosts antioxidant levels in sepsis patients who're vitamin-C-depleted.
As for this particular paper about the Vitamin C trial (which hasn't been retracted yet): numerous other groups tried replicating it and none could. Millions and millions and millions of dollars, countless hours wasted on conducting controlled trials all because of this guy's fraud.
My two takeaways:
1) Kudos to Sheldrick, we absolutely need more people like him. We need statisticians to pick apart data in papers and see if it smells funny. This should have been caught earlier.
2) There should be a higher price to pay for this kind of fraud. For the confusion and mistrust that ensued, for the money and human capital that was wasted, he needs to be behind bars for a long, long time.
That's how science work. No matter what we do to discourage fraud, we'll always invest tons of capital and time to find and filter it out.
Sometimes it is difficult to distinguish between both but of there is a track record of fraud no reputable journal should accept more papers from that source.
Edit: As I re-read my comment I worry because research should not really depend on "prestige", it should be based on cold facts. On the other hand I don't see any other way to deal with malicious actors.
It's absurd to think there will be a way to know which studies need reproduction or not. The whole purpose of reproducing is to find out which ones are true and which are bogus
But it was known not to be true, and so people were sent under false pretenses to go and reproduce and verify it. That is far from well invested.
We will always need to invest in finding error and fraud, no matter how we perceive scientists honesty.
The moment we stop doing these investment, we're doomed.
Unpossible!
We need to reproduce every single study, including the fraudulent ones. Gosh, ESPECIALLY the fraudulent ones.
But we can't possibly know in advance which are which. That's the whole purpose of trying to reproduce.
Without reproduction efforts, scientific papers are of no value, as we can't trust them.
In this sense, a paper without reproduction effort is not a "window" in the analogy. Only after reproduction efforts it becomes a window.
So saying it's a waste to try reproducing an article is just absurd. It's like saying it's a waste to produce the "window" in the first place because a kid might break it in the future (in case of the paper, that we might figure out the paper was fraudulent).
It's still valuable to find out a paper was fraudulent. It builds trust in the whole scientific endeavor.
The overall process assumes trustworthiness. The better that assumption holds, the better science performs.
The general presumption would tend to be that other scientists are trying to do good, honest work. Scientific progress does better when that presumption turns out to be more true.
To be clear, I was responding to their apparent disregard for the harm done by the original alleged-fraud.
But it's reasonable to think that some reproduction is better than no reproduction.
The point I think folks were contesting was the tone some may've inferred from [this comment](https://news.ycombinator.com/item?id=30787447 ), which may sound a bit like downplaying the damage caused by fraud under the premise that reproduction ought to catch such things anyway.
For a computing analogy, TCP uses mechanisms that enable it to tolerate packet-loss. But packet-loss still slows things down, limiting bandwidth and increasing latency, until a connection would be impractical. For a connection with many hops, even a small packet-loss-rate along each hop would be a huge problem, even though TCP actively checks for packet-loss and tries to fix it.
Likewise, science-with-replication presumably ought to be able to fix a lot of fraud and errors -- but those problems take a heavy, debilitating toll on scientific progress; it's not a small problem that replication-studies alone could satisfactorily rectify.
The real damage would have been everyone believing the paper conclusions were solid and start to act according to this false belief.
Reproduction efforts saved us from that damage.
These efforts were not the damage.
Whoever is seeing differently doesn't understand one of the basic foundations of how the scientific endeavor works.
And again: anyone expecting a world where there are no dishonest scientists or where the dishonest ones call themselves out in advance to save from the "damage" of people spending time reproducing their papers, is just playing wishful thinking, which is not a recipe for success...
I'm pretty sure everyone firmly agrees with this.
So I agree with OP the time wasn't wasted, but it could have been spent on better opportunities.
You can't flag "this scientist is honest, never reproduce anything from him anymore". He can turn dishonest the next day you do this.
If there are more dishonest scientists, we'll just find more useless papers. But the time to reproduce will be the same.
Also, if we really are fond of consequences, frauds carry over the price of having to confront, for future publishing, with a scientific community that has grown a bias against the frauder; and the latter will have to regain trust if he/she wants to publish again.
The criminal charges should be for those who use unproven scientific results to endorse their own foul agenda.
If a system allowed me to publish fraudulent research or publish bad research recklessly, with the only consequence being "public shaming" (which is sometimes not really consequential if one has the right connections to the right people), what's to prevent me (and everyone else) from doing it over and over again?
TBH from a legal perspective knowingly publishing false information for personal gain is already fraud. You're arguing that authorities should stay away because academia should somehow have immunity? That's a very disturbing line of thought.
At the moment, University incentives are to protect the prestige and not for honesty.
There should also be a University list of shame, based on author affil of papers accused of fraud.
The trouble and defense will be that any rando can accuse a paper of fraud, and instead one should use finalized decisions (such as retractions). That would cause delays and not lead to any resolution in many cases, and I doubt that the number of papers false accused of fraud based on any substantial statistical analysis is large.
Karma would help. One could then keep track of people who falsely accused papers of fraud in the past, and correctly pointed out papers which were later retracted, and then weigh the University shame list based on that.
That said, this particular author is egregious and I'm sure his infamy will precede his name in future papers.
https://en.wikipedia.org/wiki/Goodhart%27s_law
Funnily enough, if you should just go online and check the typical duration of a visual migraine, the results may shock you!
(where the placebo effect begins)
I’ve had this issue, arguing with bad faithed opportunists, whose approach was to just bury competing ideas in FUD.
Seems someone summarized it in this succinct statement:
[“The amount of energy needed to refute bullshit is an order of magnitude larger than is needed to produce it.”][1]
[1]: https://en.m.wikipedia.org/wiki/Brandolini%27s_law
Good luck to this man!
Happily from the sidelines - it probably is fraud and this isn't really an article targeted at the general audience. But the statistical evidence proves beyond a doubt that the data wasn't collected by the methods described in the paper. That doesn't prove it is fraudulent because formally speaking he can't rule out that they just did a very bad job of explaining their methods.
Just from life experience, any time anyone goes in guns blazing based on statistical evidence - without actually talking to the people involved - it is often a mistake. Just because someone can't think of an alternative reason says more about their imagination than the reality of the situation. Particularly coming in hot off a twitter argument.
That being said, these are unusual and unexpected patterns that show irregularities in the data, and I would certainly share his concerns.
As the author describes, the p-values listed are effectively impossible to get from real data. Because of the magic of mathematics, we know exactly how p-values are meant to behave when you calculate them repeatedly - uniform random between 0 and 1. There are a battery of tests you can use to interrogate this very well understood distribution. Those tests, as the author states, prove without any doubt that the p-values in table 1 are effectively impossible. You can't make it to those p-values by mistake.
That does not mean fraud (although, I stress, in this case I can see why it is probably fraud). For example, maybe the study is done in some sort of hospital with some sort of patient scheduling system and the front desk is playing a funny joke on the doctors in this study. It is quite common for experimenters to not be as much in control of their environment as a paper suggests - that would be an honest reason for why the data is weird.
In a more plausible example - I don't know how "matching" works, but maybe the patients were matched, the report writer didn't know that and the sentence slipped through proofreading. Unlikely, but certainly possible. That mistake would happen in the wild from time to time.
You don't need much imagination to find examples other than fraud. Although it does take a little imagination, which is concerning. But, frankly, there should almost always be an attempt at dialog with someone before accusing them publicly of being a fraud. Unless they're going to shoot you or something extreme.
Sloppy research methods leading to bogus results are can be construed as fraud, particularly if you don’t know the motivation for the lack of care.
A diligent researcher might take these results which were “incredible” and attempt to replicate them before publishing.
He's doing that based on an interpretation of some study, which I suspect is approximately a 20 page document.
Now on the one hand, he is probably right. On the other hand, this is a very brave approach - the damage and the care taken aren't in good proportion here. This sort of thing is easy to get wrong. He should have talked to the paper's authors first.
Sheldrick did exactly that. It is the first line of the article.
"Below is an email I have sent to Sentara Norfolk General Hospital, the editor of CHEST Journal, and prof Paul E Marik"
The author published an email he also - before publication - sent to the people involved.
Science happens in public for a reason. The reason is that others should be able to check the results being published. Authors should therefore be very aware of the fact, that their results might come under scrutiny.
If one were to publish claims that look so unlikely based on valid scientific methods they should at least address the unlikeliness of the results and provide a rationale why this is still valid, even if it looks unlikely.
That didn't happen - and so they get called out. The extreme unlikeliness is what validates the claim of this being fraudulent data. Now it is on the original study`s authors to react to that public answer to their publication.
That's a dangerous statement. I'm pretty sure I got p-values of over 1 or under 0 by mistake. People will make every mistake you think is impossible. (Not that I think it's a mistake here)
In the best case of "poorly described methods", we're talking about authors who blatantly lied about how they sampled their patients and deliberately omitted what would have been an extremely laborious process, involving many thousands of extremely non-consecutive patients, to produce an extremely well-matched treatment and control group.
You've got your choice as to the exact nature of the enormous lie or fabrication, but there's no possibility of innocent omission.
As noted in other comments here, one possibility is that the authors failed to correctly describe the procedure they used. Sheldrick does consider this, pointing out that even if they actually did try to match subjects, unlike what they said in the paper, it would be very unlikely that they could match them well enough to obtain the reported p-values. But another possibility is that they were totally incompetent at using their statistical software - that the reported p-values are not actually the real p-values for the tests that they said they did. Maybe they accidently used the wrong dataset, for example. Or perhaps they copied numbers from the wrong column of the table the software produced.
In a wider sense, though, this would still essentially be fraud - it's dishonest to publish a paper that purports to be a competent study when really you are totally clueless about how to do research, or care so little about correctness that you put no effort into checking that you did things correctly. (And surely they would have to suspect that they don't really know what they're doing...)
Something like a column duplication would have resulted in perfectly identical data between treatment and control groups, which this data does not show.
The best case scenario is an errant copy-paste or Excel formula that somehow presented both groups of data in a way that actually only looked at one of the two groups, but also somehow introduced small differences along the way.
How exactly this could happen is beyond my imagination, but perhaps it introduces a sliver of doubt as to whether the paper is fraud by intention or merely by negligence.
https://en.wikipedia.org/wiki/Replication_crisis
"A 2016 poll of 1,500 scientists reported that 70% of them had failed to reproduce at least one other scientist's experiment (50% had failed to reproduce one of their own experiments)."
or at least - this is what it used to say! This has been memory holed.
and
https://www.bbc.com/news/science-environment-39054778
"According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist's experiments."
If you look at this page, you feel as if you have stumbled into some psychological/sociological discussion.
This, to me, is an example of how wiki editors are trying to dampen down the implications of the replication crisis.
Take a look at the current version against previous versions, eg: https://en.wikipedia.org/w/index.php?title=Replication_crisi...
The strong "Overall" statement early in the article is gone, and instead we have lots of dull, makeweight context and background.
This isn't up for debate nor is it absurd to call for genotype specific studies of efficacy. We, collectively, have allowed real miraculous discoveries to flounder as a result of this failure to appreciate personalized medicine.
We will look back on this Era in awe with the ineptitude of our methods.
This is sorta done for meta-analyses that aggregate the results of several studies systematically. I think there are people trying to automate more of it.
Most likely, one report-writer didn't understand that these were MATCHED consecutive patients. The data is so clearly matched, it's hard to believe it's fraudulent.