> Cochrane says that the lost NIHR funding — around £4.2 million (US$5.3 million) — does not affect its core income, which was £8.9 million in 2022. However, a more existential threat looms. Around £6.8 million of that core income came from subscriptions to the Cochrane Library, but Cochrane aims to make all its reviews open access by 2025
What the actual fuck, those are peanuts for any single country or health care system. What a tragedy and a blow to evidence based medicine.
There are a lot of systematic reviews. They're a very popular "Grad student's first paper" in public health fields, because if you're writing Chapter 1 of your dissertation, you're already halfway there.
Evidence based medicine (EBM) = population, intervention, comparison and outcome (PICO). In other words, PICO = RCT Randomized Control Trials. To me, EBM is medical instrumentalism (instrumentalism is one of philosophies of sciences): pick a treatment that works, without understanding the underlying mechanism. Even today, we use many drugs with unknown mechanisms of action[0].
Often times, you can find cures just by discovering mechanism of action, without PICO/RCT: for example, Helicobacter pylori causing peptic ulcers.
Placebo effect is NOT magic, but we don't understand the mechanism. Good relationship between doctors and patients help recover patients. Patients prefer to see particular doctors, even if the latter prescribe same medications as others, yet the differential effects on patients (when treated by preferred doctors vs others) is non-trivial.
It's a bit more important than that number suggests. For one, 7% of global systematic reviews is not a trivial proportion.
But more importantly Cochrane allows for the pre-registration of such reviews, i.e. where the methodology and goals (hypotheses) are stated before the review is conducted. This is probably the most scientifically rigorous method of assessing evidence, and is not performed for most systematic reviews.
Except this sort of funding has multiplicative effects. Funding for a single doctor helps only that doctor and their patients. Funding for this sort of work helps all doctors and all patients. When austerity cannibalizes your force multiplying efforts, that’s truly the end.
Edit:
For context the thing you linked says UK medical residents (in US language) make $36k/year, and the median household income is also $36k/y[0] .
In the US the average non specialist income is $60k[1], and the median household income is $69k[2]
This tells me things aren’t out of whack.
They also say 40% of residents in the UK want a different job. But residencies are temporary. In a few years they will have a new job - a full doctor.
I’m not saying there’s not a crisis, but that sheet doesn’t explain to me what the crisis actually is. They seem to be doing on par with US residents.
$230k, which when indexed by median household income, is comparable to the UK at 100k (I assume you used dollars? If not that’s $126k).
Edit: I’m not making any tax statements because taxes are unavoidable and applied to everyone in the nation, so are part of the indexing. That said US taxes sound comparable or more in some areas (taxes in NYC for instance can exceed 50%)
>Earn £100,000 in 2023/24 and you'll take home £67,049. This means £5,587 in your pocket a month. Over the year you'll pay £27,432 income tax and £5,519 in national insurance.
My brother has lived in all three of the US, UK and Australia, doing non-professional jobs (construction, agriculture, gardening, cellar hand in the wine industry).
He didn't last long in London – he was working as an unskilled labourer in the construction industry, the pay was terrible, and the basic necessities of life were so expensive, he could hardly afford to eat. He'd worked similar jobs in Australia and got paid a lot more, and found life more bearable.
Some countries pay high because everything is expensive. Some countries pay low because everything is cheap. The UK seems to be a place where people get paid low because everything is expensive.
Absolutely true. But saying UK doctors are underpaid because they’re paid half of a US doctor while ignoring everyone is paid half of a US person for everything is disingenuous. Everyone in the UK is inexplicably underpaid, and everything is expensive.
He lived in a small town in Oregon with his first wife. While they weren’t paid a lot, life wasn’t expensive either. I think he would have stayed except the marriage didn’t work out. Obviously that isn’t an entirely fair comparison to London, but I still think the pay-to-expenses ratio would be more favourable in small-town US than small-town UK
While it depends on the specialty, in Australia it is not uncommon for consultants to be on AUD 300-400K (= 150-200K GBP, 200-250K USD). I don't understand why salaries in the UK are so low, even for highly educated professionals.
My grandfather was a GP. He hated the NHS so much, he left the UK and never went back. Pay was likely a factor, but he also viewed the NHS as a denial of his professional freedom. While Australia eventually adopted something akin to the NHS (Medicare), I don't think he objected so much to that, since it involved less government control over the how of his job.
It varies quite widely by location because the income often reflects the number of patients and what source their income is from (Medicare, private insurance, uninsured). The med school debt for family medicine doctors can be quite daunting to pay off
Evidence-based is a subset of scientific, imo. A study of cells in a Petri dish is science but I wouldn’t recommend a drug to a person from such a study.
As I wrote in another reply, evidence based medicine (EBM) = population, intervention, comparison and outcome (PICO). In other words, PICO = RCT Randomized Control Trials. To me, EBM is medical instrumentalism (as instrumentalism is one of philosophies of sciences): pick a treatment that works for a large number of people, without understanding the underlying mechanism. Even today, we use many drugs with unknown mechanisms of action (MOA).
I would add that EBM took over because science-based medicine (at least the historical definition, how it's used now is more of a hybrid) didn't work out too well. Often the mechanism thought to be understood is incorrect or more complicated and the outcomes don't correlate well with what's expected, sometimes resulting in significant harm.
Fundamentally, the mechanism of why a medical intervention works isn't that important if you're thoroughly evaluating outcome measures and controlling confounders.
A simple example is (using the traditional understanding of these differences):
Science-based: Cholesterol levels are associated with major adverse cardiovascular events (MACE), statins lower cholesterol levels and therefore statin therapy is recommended to reduce MACE.
Evidence-based: A [specifically defined population] was given statin therapy or placebo. We observed lower MACE in the statin group, therefore statin therapy is recommended in [specifically defined population] to reduce MACE.
> Cholesterol levels are associated with major adverse cardiovascular events
> statins lower cholesterol levels and therefore
You lost me here - because the science based one is just stating unproven statements to come to some conclusion. If you were to expand on why those two sentences are "proven" you'd likely come to evidence based reasons.
> You lost me here - because the science based one is just stating unproven statements to come to some conclusion. If you were to expand on why those two sentences are "proven" you'd likely come to evidence based reasons.
I'm not sure I'm exactly understanding what you're getting at. If you expand on those statements to find direct evidence of MACE (of which there is plenty) then it is an evidence based medicine decision not a science based one.
Please disregard the specific intervention being discussed, it's just an example of the different thought processes and decision making between SBM and EBM. In real life there is plenty of science and evidence behind statins but I'm not intending on addressing reality or why this specific intervention is in use today.
Note the "medicine" part in these refers to the clinical practice of medicine. Maybe this is clearer:
The science based medicine approach (as traditionally defined) only considers the mechanisms and "indirect evidence" when making the clinical recommendation (i.e. statins are proven to lower cholesterol which are strongly correlated with MACE. Cholesterol reduction has also been proven to reduce plaque formation and artery stenosis, therefore we expect MACE will be reduced through statin use and are deciding to administer the intervention).
In this thought process we have not directly measured the outcome of interest (MACE) but we are giving an intervention because it makes sense/through chain of thought, or in other words we are intervening to optimize a variable (serum cholesterol levels) because we have reason to believe this variable reduces MACE based on the mechanism (which may have components proven by evidence like reduction in plaque formation but the primary outcome measure we are discussing, MACE, has not been measured).
In the evidence based medicine approach we are deciding to provide an intervention based on direct measurements of the outcome of interest in a specific population (i.e. less MACE was directly proven) and the decision making process is entirely independent of the mechanism or nature of the intervention (i.e. we are not considering plaque formation and cholesterol levels, simply that statins reduce MACE).
Much of the scientific literature in biology is working off of hypotheses that don't have a huge amount of evidence.
There are a large number of studies that present a molecular mechanism which, if studied in depth, will not always be consistent with what is presented in the literature. This is often dealt with by appending several conditions that must be met, such that the combinatorial space grows large enough and the only study that is done that meets the requirements is the one being presented.
In this way, almost anything can be stated to fit some shiny new hypothesis.
This is effectively the science based method, since it can be tracked to a, perhaps, 'plausible' mechanism (although often now lean towards more 'intriguing' than parsimonious, especially if published somewhere like Nature)
Evidence based medicine has other faults. One subfield is meta-analysis, which looks for very solid observations that are reproducible, but often lacks the complicated modeling that may be necessary to see any highly conditional effects. So the findings tend to be real, but boring and often "well-known". It is however, a good way to check if the "well-known" observations in the literature are reproducible.
You're (correctly) addressing some of the issues in biomedical research but even the most rigorous and validated science does not necessarily translate into clinical outcomes for a laundry list of reasons.
Evidence based medicine includes lab/animal studies (i.e. scientific literature in biology) but when we talk about EBM what we really mean is "outcomes based medicine" rather than relying on the science.
EBM reduces the question to "Does intervention X change clinical outcome Y in population Z" to obviate many (but not all) of the clinical translation issues.
"Conclusions Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention."
As I've said above it was subject to pretty hefty critique and I've never seen a robust defence. They meta combined papers which were not well grounded and it was unlike its RCT normal approach.
The US or EU should pick up the tab. The UK is not really a wealthy country anymore and things critical to STEM should slowly be transferred out as it doesn’t have the economic power to sustain them any longer.
Isn’t the UK in the top 6 or 5 economies in the world? Are you saying that there are only 5 wealthy countries in existence? I mean the UK definitely has problems but it’s a bit hysterical to say they’re not wealthy.
To be honest, that "factcheck" demonstrates the value of performing well-constructed systematic reviews, i.e. a clear and unbiased assessment of what the pertinent studies do and do not support.
> Therefore, the review does not imply that masks do not work [...]
Actually, for all intents and purposes, it does imply exactly that.
The RCTs involved sufficiently large sample sizes that they are representative of real-world outcomes. As such, masks, as we have generally deployed them, simply do not offer significant control over the trajectory of C-19-like infection at the population level.
This may seem like bad news, but as a grad student studying evidence synthesis I can say that there is a silver lining here. The quality of studies in systematic inputs is low and even Cochrane's reviews have been critiqued for missing things. It's a big factory that hasn't fundamentally innovated how research is synthesized. There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples).
Hopefully this results in some innovation at the conceptual and financial levels. For those who lost their income, they have transferrable skills that would be welcomed in healthtech companies, other governmental groups, and academic positions. It's really not as bad as it seems.
> The quality of studies in systematic inputs is low and even Cochrane's reviews have been critiqued for missing things.
Criticism is good, expecting perfection is not.
Quality issues are a problem of the underlying research rather than the review process. High quality reviews, like Cochrane, assess risk of bias which can limit the strength of conclusion they make where the underlying studies are poor quality.
> There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples).
All of these answer different questions/have different use cases than a SR and are much harder to do while maintaining comparable quality and risk of bias.
What's your gripe with high quality SRs following standards like PRISMA, STARD and QUADAS-2?
This is an insane ask, but I would love to hear more about various reviewing and synthesis styles, their strengths and weaknesses, which are particularly promising and why, etc. you’re welcome to link me to material, but I’d also appreciate a brief overview.
I have nothing to offer other than thinking that this is a super cool area of study and I’d love to hear more
I don't remember hearing anything about Cochrane in mainstream news until they put out an unfavorable review on the effectiveness of mask usage during covid 19.
A very unfortunate, badly run meta analysis as I understand it which fundamentally undermined trust in their governance because it was so easily critiqued. I haven't seen a robust defence of that one.
My opinion (feel free to have another one) is that it points to the need for training and more comfortable masks. The studies weren't WRONG per se, but they are very hard to interpret.
My whacky opinion: human facing sciences (medicine, psychology, diet, sociology, economics, etc.) have a huge problem with solutions that require motivation to succeed. In other words, if you are motivated, you can make masks work. If you are motivated, you can make intermittent fasting work. Same for EVs, Solar Panels, etc.
I haven't seen compelling critiques of it[0]. Curious to read them.
Their findings:
On medical/surgical vs. nothing:
"Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)"
On N95/P2 respirators vs. medical/surgical:
"The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence)."
Thats a revision of a paper with roots back to 2006. I applaud them for having a rich source, but the primary objections can be understood to be in their apologia:
The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses.
The problem was how they communicated. The data is the data.
"Evidence-based medicine (EBM’s) traditional methods, especially randomised controlled trials (RCTs) and meta-analyses, along with risk-of-bias tools and checklists, have contributed significantly to the science of COVID-19. But these methods and tools were designed primarily to answer simple, focused questions in a stable context where yesterday’s research can be mapped more or less unproblematically onto today’s clinical and policy questions. They have significant limitations when extended to complex questions about a novel pathogen causing chaos across multiple sectors in a fast-changing global context. Non-pharmaceutical interventions which combine material artefacts, human behaviour, organisational directives, occupational health and safety, and the built environment are a case in point: EBM’s experimental, intervention-focused, checklist-driven, effect-size-oriented and deductive approach has sometimes confused rather than informed debate. While RCTs are important, exclusion of other study designs and evidence sources has been particularly problematic in a context where rapid decision making is needed in order to save lives and protect health. It is time to bring in a wider range of evidence and a more pluralist approach to defining what counts as ‘high-quality’ evidence. We introduce some conceptual tools and quality frameworks from various fields involving what is known as mechanistic research, including complexity science, engineering and the social sciences. We propose that the tools and frameworks of mechanistic evidence, sometimes known as ‘EBM+’ when combined with traditional EBM, might be used to develop and evaluate the interdisciplinary evidence base needed to take us out of this protracted pandemic. Further articles in this series will apply pluralistic methods to specific research questions."
I don't understand, this paper is describing how EBM works today and calling it EBM+ for some inexplicable reason. This is a long winded rant basically concluding with face validity is important...
Dark public choice perspective: this is incentive compatible for pharmaceutical companies and governments who can now give out their guidelines and solutions without as strong scrutiny.
"incentive compatible" is doing a lot of heavy lifting in that sentence.
Meta-analysis and systematic reviews were done before Cochrane, and will be done after it. And most pharma companies are banking on RCT results - Cochrane can't do systematic reviews before there's multiple studies, so it's significantly further downstream than what makes pharma money.
Cochrane is kind of weird, though it probably is how frequentest science is supposed to work, where the null hypothesis is the default. If you actually followed their advice you would get vaccinated for hepatitis A every two years, because the protection lasting any longer is "unsupported by high quality sources".
72 comments
[ 3.0 ms ] story [ 136 ms ] threadWhat the actual fuck, those are peanuts for any single country or health care system. What a tragedy and a blow to evidence based medicine.
They do mention that Cochrane is only accounting for 7% of systematic reviews these days.
BTW, evidence-based medicine is not the same as science-based medicine. The Placebo Effect is evidence-based medicine.
Often times, you can find cures just by discovering mechanism of action, without PICO/RCT: for example, Helicobacter pylori causing peptic ulcers.
Placebo effect is NOT magic, but we don't understand the mechanism. Good relationship between doctors and patients help recover patients. Patients prefer to see particular doctors, even if the latter prescribe same medications as others, yet the differential effects on patients (when treated by preferred doctors vs others) is non-trivial.
[0] https://en.wikipedia.org/wiki/Category:Drugs_with_unknown_me...
I’m not what 7% of reviews represents, but it seems like particularly dumb austerity measure.
But more importantly Cochrane allows for the pre-registration of such reviews, i.e. where the methodology and goals (hypotheses) are stated before the review is conducted. This is probably the most scientifically rigorous method of assessing evidence, and is not performed for most systematic reviews.
[0] https://www.bma.org.uk/media/6882/bma-ia-juniors-fact-sheet-...
Edit:
For context the thing you linked says UK medical residents (in US language) make $36k/year, and the median household income is also $36k/y[0] .
In the US the average non specialist income is $60k[1], and the median household income is $69k[2]
This tells me things aren’t out of whack.
They also say 40% of residents in the UK want a different job. But residencies are temporary. In a few years they will have a new job - a full doctor.
I’m not saying there’s not a crisis, but that sheet doesn’t explain to me what the crisis actually is. They seem to be doing on par with US residents.
0 https://www.ons.gov.uk/peoplepopulationandcommunity/personal...
1 https://mededits.com/residency-admissions/residency-salary/
2 https://www.deptofnumbers.com/income/us/
UK a consultant earns £100k pretax and pays approximately 30% net tax rate and 60% marginal rate.
What about US?
Apparently family medicine in the US brings in 130% more than the equivalent in the UK (GP which makes up half of all UK doctors).
https://revisingrubies.com/us-vs-uk-doctors-salary/
That's truly astounding.
$230k, which when indexed by median household income, is comparable to the UK at 100k (I assume you used dollars? If not that’s $126k).
Edit: I’m not making any tax statements because taxes are unavoidable and applied to everyone in the nation, so are part of the indexing. That said US taxes sound comparable or more in some areas (taxes in NYC for instance can exceed 50%)
Is that a marginal rate or total tax burden?
According to https://www.moneysavingexpert.com/tax-calculator/ , someone in the UK who earns £100k takes home £67,049:
>Earn £100,000 in 2023/24 and you'll take home £67,049. This means £5,587 in your pocket a month. Over the year you'll pay £27,432 income tax and £5,519 in national insurance.
He didn't last long in London – he was working as an unskilled labourer in the construction industry, the pay was terrible, and the basic necessities of life were so expensive, he could hardly afford to eat. He'd worked similar jobs in Australia and got paid a lot more, and found life more bearable.
Some countries pay high because everything is expensive. Some countries pay low because everything is cheap. The UK seems to be a place where people get paid low because everything is expensive.
What was your brothers take on the US?
He lived in a small town in Oregon with his first wife. While they weren’t paid a lot, life wasn’t expensive either. I think he would have stayed except the marriage didn’t work out. Obviously that isn’t an entirely fair comparison to London, but I still think the pay-to-expenses ratio would be more favourable in small-town US than small-town UK
Summary: https://i.imgur.com/DGT0d0d.png
From: https://www.oecd.org/health/recent-trends-in-international-m...
While it depends on the specialty, in Australia it is not uncommon for consultants to be on AUD 300-400K (= 150-200K GBP, 200-250K USD). I don't understand why salaries in the UK are so low, even for highly educated professionals.
My grandfather was a GP. He hated the NHS so much, he left the UK and never went back. Pay was likely a factor, but he also viewed the NHS as a denial of his professional freedom. While Australia eventually adopted something akin to the NHS (Medicare), I don't think he objected so much to that, since it involved less government control over the how of his job.
Where did your 60% come from?
[1] https://www.gov.uk/income-tax-rates
The net effect is an effective tax rate much higher than the nominal rate.
You lose other benefits to but they aren't factored in as depend on if you have kids or not etc.
What’s the difference between something being scientific and something being evidence-based?
Fundamentally, the mechanism of why a medical intervention works isn't that important if you're thoroughly evaluating outcome measures and controlling confounders.
Science-based: Cholesterol levels are associated with major adverse cardiovascular events (MACE), statins lower cholesterol levels and therefore statin therapy is recommended to reduce MACE.
Evidence-based: A [specifically defined population] was given statin therapy or placebo. We observed lower MACE in the statin group, therefore statin therapy is recommended in [specifically defined population] to reduce MACE.
> statins lower cholesterol levels and therefore
You lost me here - because the science based one is just stating unproven statements to come to some conclusion. If you were to expand on why those two sentences are "proven" you'd likely come to evidence based reasons.
I'm not sure I'm exactly understanding what you're getting at. If you expand on those statements to find direct evidence of MACE (of which there is plenty) then it is an evidence based medicine decision not a science based one.
Please disregard the specific intervention being discussed, it's just an example of the different thought processes and decision making between SBM and EBM. In real life there is plenty of science and evidence behind statins but I'm not intending on addressing reality or why this specific intervention is in use today.
Note the "medicine" part in these refers to the clinical practice of medicine. Maybe this is clearer:
The science based medicine approach (as traditionally defined) only considers the mechanisms and "indirect evidence" when making the clinical recommendation (i.e. statins are proven to lower cholesterol which are strongly correlated with MACE. Cholesterol reduction has also been proven to reduce plaque formation and artery stenosis, therefore we expect MACE will be reduced through statin use and are deciding to administer the intervention).
In this thought process we have not directly measured the outcome of interest (MACE) but we are giving an intervention because it makes sense/through chain of thought, or in other words we are intervening to optimize a variable (serum cholesterol levels) because we have reason to believe this variable reduces MACE based on the mechanism (which may have components proven by evidence like reduction in plaque formation but the primary outcome measure we are discussing, MACE, has not been measured).
In the evidence based medicine approach we are deciding to provide an intervention based on direct measurements of the outcome of interest in a specific population (i.e. less MACE was directly proven) and the decision making process is entirely independent of the mechanism or nature of the intervention (i.e. we are not considering plaque formation and cholesterol levels, simply that statins reduce MACE).
This is effectively the science based method, since it can be tracked to a, perhaps, 'plausible' mechanism (although often now lean towards more 'intriguing' than parsimonious, especially if published somewhere like Nature)
Evidence based medicine has other faults. One subfield is meta-analysis, which looks for very solid observations that are reproducible, but often lacks the complicated modeling that may be necessary to see any highly conditional effects. So the findings tend to be real, but boring and often "well-known". It is however, a good way to check if the "well-known" observations in the literature are reproducible.
Evidence based medicine includes lab/animal studies (i.e. scientific literature in biology) but when we talk about EBM what we really mean is "outcomes based medicine" rather than relying on the science.
EBM reduces the question to "Does intervention X change clinical outcome Y in population Z" to obviate many (but not all) of the clinical translation issues.
"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial"
https://www.bmj.com/content/363/bmj.k5094
"Conclusions Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention."
Please read the context: https://blogs.bmj.com/bmj/2018/12/13/we-jumped-from-planes-w...
There is even one comment here along that line.
https://www.forbes.com/sites/brucelee/2023/03/11/cochrane-sa...
Can you provide more context for this? What was the study? Why was it misleading?
The misleading part was done by Tom during an interview with a major media organization with a similar established narrative.
Actually, for all intents and purposes, it does imply exactly that.
The RCTs involved sufficiently large sample sizes that they are representative of real-world outcomes. As such, masks, as we have generally deployed them, simply do not offer significant control over the trajectory of C-19-like infection at the population level.
Hopefully this results in some innovation at the conceptual and financial levels. For those who lost their income, they have transferrable skills that would be welcomed in healthtech companies, other governmental groups, and academic positions. It's really not as bad as it seems.
Criticism is good, expecting perfection is not.
Quality issues are a problem of the underlying research rather than the review process. High quality reviews, like Cochrane, assess risk of bias which can limit the strength of conclusion they make where the underlying studies are poor quality.
> There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples).
All of these answer different questions/have different use cases than a SR and are much harder to do while maintaining comparable quality and risk of bias.
What's your gripe with high quality SRs following standards like PRISMA, STARD and QUADAS-2?
I have nothing to offer other than thinking that this is a super cool area of study and I’d love to hear more
My whacky opinion: human facing sciences (medicine, psychology, diet, sociology, economics, etc.) have a huge problem with solutions that require motivation to succeed. In other words, if you are motivated, you can make masks work. If you are motivated, you can make intermittent fasting work. Same for EVs, Solar Panels, etc.
Their findings:
On medical/surgical vs. nothing:
"Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)"
On N95/P2 respirators vs. medical/surgical:
"The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence)."
[0] https://www.cochranelibrary.com/web/cochrane/content?templat...
https://www.cochrane.org/news/statement-physical-interventio...
The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses.
The problem was how they communicated. The data is the data.
"Adapt or die: how the pandemic made the shift from EBM to EBM+ more urgent"
https://ebm.bmj.com/content/27/5/253
"Evidence-based medicine (EBM’s) traditional methods, especially randomised controlled trials (RCTs) and meta-analyses, along with risk-of-bias tools and checklists, have contributed significantly to the science of COVID-19. But these methods and tools were designed primarily to answer simple, focused questions in a stable context where yesterday’s research can be mapped more or less unproblematically onto today’s clinical and policy questions. They have significant limitations when extended to complex questions about a novel pathogen causing chaos across multiple sectors in a fast-changing global context. Non-pharmaceutical interventions which combine material artefacts, human behaviour, organisational directives, occupational health and safety, and the built environment are a case in point: EBM’s experimental, intervention-focused, checklist-driven, effect-size-oriented and deductive approach has sometimes confused rather than informed debate. While RCTs are important, exclusion of other study designs and evidence sources has been particularly problematic in a context where rapid decision making is needed in order to save lives and protect health. It is time to bring in a wider range of evidence and a more pluralist approach to defining what counts as ‘high-quality’ evidence. We introduce some conceptual tools and quality frameworks from various fields involving what is known as mechanistic research, including complexity science, engineering and the social sciences. We propose that the tools and frameworks of mechanistic evidence, sometimes known as ‘EBM+’ when combined with traditional EBM, might be used to develop and evaluate the interdisciplinary evidence base needed to take us out of this protracted pandemic. Further articles in this series will apply pluralistic methods to specific research questions."
Too cynical??
Meta-analysis and systematic reviews were done before Cochrane, and will be done after it. And most pharma companies are banking on RCT results - Cochrane can't do systematic reviews before there's multiple studies, so it's significantly further downstream than what makes pharma money.