You, like me, are a proud second-degree vegan: we only eat animals that eat plants.
If you find yourself around vegans, you might also at some point want to casually throw the word 'vegavore' around, and then explain it means 'someone who conciously only eats plants, as opposed to people that do so accidentally or unwittingly'. With some luck they won't realize the deception and start using it themselves ;-)
Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial
Conclusion: Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention. However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps. When beliefs regarding the effectiveness of an intervention exist in the community, randomized trials might selectively enroll individuals with a lower perceived likelihood of benefit, thus diminishing the applicability of the results to clinical practice.
My favorite part: “We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.”
This paper is intended as a sarcastic critique of the push towards conducting randomized trials in academia/medicine, but it’s an obvious straw man. You could write this sarcastic article for any research method - “see, why do research using method X when we already know the answer from other methods?”
The problem is that there are plenty of research questions where RCTs show us that previous non-RCTs were wrong.
Is that a problem? It can be true in isolation of the fact that sometimes, what's going to happen without a RCT is just obvious, because an experiment is not needed to answer every conceivable question.
"obvious straw man" researchers should be worrying about the non-obvious strawmen. Because these extreme conditions help up check the boundaries of what is possible to verify or not.
> The problem is that there are plenty of research questions where RCTs show us that previous non-RCTs were wrong.
And that is fine. When you are able to use an RCT.
For a serious take on their point: With parachutes, we have detailed physical models of how they work.[0] With anecdotal medicinal practices, however, AFAIK there's little other than "this works sometimes."[1] We don't understand the body half as well as we understand physics, so I really don't think it's fair to compare the two.
If we had no observational evidence for something, but we instead had an underlying model that made many correct predictions, then it would be fair to draw their conclusion that clinical trials aren't the end-all-be-all. But when you're operating solely on raw data with little to no understanding, you need data.[2]
Of course, I am most decidedly an amateur in all the fields I've mentioned, so take all this with a grain of salt :-)
[0] I don't actually know how parachutes were discovered, so maybe they were discovered purely experimentally. I'm inclined to doubt it, given a presumed lack of willing test subjects—but if I'm wrong, hopefully my comment will have value nonetheless.
[1] To be fair, my understanding is that this applies to mainstream medicine too (isn't "how do general anesthetics work" still an open question?).
[2] I feel this is a massive problem with current AI/ML trends as well. My undergrad "intro to ML" class briefly covered the statistics of it (VC dimensions and all that). The prof. basically said, "here's the amount of data you need to get statistical guarantees of correctness... and in practice, you'll have orders of magnitude less data than this." And yet we put this half-baked tech into cars, because it works on the happy cases, and totally ignore the lack of statistical confidence regarding the bad cases.
You can put the mask “controversy” under this perspective: do we know how surgical masks and airborne diseases work? Do we need more data to prove their effectiveness?
It's a great debating trick. Its very funny. It doesn't invalidate RCT as a statistical model.
The underlying ethics behind RCT are complex. There are instances of RCT being abandoned early because of abundant evidence the tested drug worked, and there are instances of RCT post-treatment for participants who were denied the drug under test by virtue of being the control, and there are examples of widespread concern at the ethics of RCTs.
"The Emperor of All Maladies" by Siddhartha Mukherjee discusses the history of RCT in the context of Cancer treatment, where the drugs were so cycotoxic the hospitals were extremely concerned at the ethics of even trying to use them.
What it invalidates is the notion that research that does not include RCT is automatically wrong.
We currently see this in action in the way corona is being treated. When we were told that masks don't help stop the spread of corona, what was really meant was "there has been no RCT that demonstrates that the spread of corona is less in a mask-wearing group" - in other words, the issue was simply untested, but this was formulated in such a way that a lay person would believe it was already known that masks have no effect, which is something completely different. The use of Ivermectin is arguably in the same situation right now.
25 comments
[ 3.0 ms ] story [ 25.4 ms ] threadI am one of the world’s best skiers. I have just chosen to never engage in that activity.
If you notice, not a single skiing olympian has ever beaten me. I am truly undefeated in skiing competitions!
If you find yourself around vegans, you might also at some point want to casually throw the word 'vegavore' around, and then explain it means 'someone who conciously only eats plants, as opposed to people that do so accidentally or unwittingly'. With some luck they won't realize the deception and start using it themselves ;-)
Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial
Conclusion: Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention. However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps. When beliefs regarding the effectiveness of an intervention exist in the community, randomized trials might selectively enroll individuals with a lower perceived likelihood of benefit, thus diminishing the applicability of the results to clinical practice.
The problem is that there are plenty of research questions where RCTs show us that previous non-RCTs were wrong.
> The problem is that there are plenty of research questions where RCTs show us that previous non-RCTs were wrong.
And that is fine. When you are able to use an RCT.
If we had no observational evidence for something, but we instead had an underlying model that made many correct predictions, then it would be fair to draw their conclusion that clinical trials aren't the end-all-be-all. But when you're operating solely on raw data with little to no understanding, you need data.[2]
Of course, I am most decidedly an amateur in all the fields I've mentioned, so take all this with a grain of salt :-)
[0] I don't actually know how parachutes were discovered, so maybe they were discovered purely experimentally. I'm inclined to doubt it, given a presumed lack of willing test subjects—but if I'm wrong, hopefully my comment will have value nonetheless.
[1] To be fair, my understanding is that this applies to mainstream medicine too (isn't "how do general anesthetics work" still an open question?).
[2] I feel this is a massive problem with current AI/ML trends as well. My undergrad "intro to ML" class briefly covered the statistics of it (VC dimensions and all that). The prof. basically said, "here's the amount of data you need to get statistical guarantees of correctness... and in practice, you'll have orders of magnitude less data than this." And yet we put this half-baked tech into cars, because it works on the happy cases, and totally ignore the lack of statistical confidence regarding the bad cases.
The underlying ethics behind RCT are complex. There are instances of RCT being abandoned early because of abundant evidence the tested drug worked, and there are instances of RCT post-treatment for participants who were denied the drug under test by virtue of being the control, and there are examples of widespread concern at the ethics of RCTs.
"The Emperor of All Maladies" by Siddhartha Mukherjee discusses the history of RCT in the context of Cancer treatment, where the drugs were so cycotoxic the hospitals were extremely concerned at the ethics of even trying to use them.
We currently see this in action in the way corona is being treated. When we were told that masks don't help stop the spread of corona, what was really meant was "there has been no RCT that demonstrates that the spread of corona is less in a mask-wearing group" - in other words, the issue was simply untested, but this was formulated in such a way that a lay person would believe it was already known that masks have no effect, which is something completely different. The use of Ivermectin is arguably in the same situation right now.