NNT is an interesting metric that people were very sanctimonious about when I was at university, but has rightly lost favour.
This is because NNT is actually a terrible metric for any treatment that reduces risk over a long period, including the most important outcomes like mortality.
Imagine a drug that halves your risk of dying of heart disease. Heart disease kills 1 in 4 people in developed countries, and so realistically it would be a no brainer that everyone middle aged or over should take it.
If you did this in a randomised controlled trial, you might recruit 1000 patients, and randomise 500 to this new drug and 500 to placebo. If you ran the trial for a year, you might find 1% of the placebo arm died, and only 0.5% in the control arm. This demonstrates a risk reduction of 0.5%, or a NNT of 200.
An NNT, by conventional teaching, is underwhelming.
Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy. In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.
It's different if you have a single one-off therapy, where all the benefit will have been delivered by a certain timepoint, at which you measure the NNT. But for a lot of medicine it is not that simple, and the NNT sucks.
You've probably thought about this more than I have, but your "take down" of NNT doesn't work for me. Let's use your example. We treat 500 people and observe another 500. 5 of the untreated die, and 2.5 of the treated die (I don't know how either). We observe that the NNT is 200.
And we this is great! If a drug is inexpensive, has minor but not severe side effects, and saves 1/200 people from dying within a year you should absolutely be giving it to everybody. It's a home run. The problem isn't the NNT, it's the expectation of what constitutes a a high or low number.
By contrast, consider another study. We treat 500,000,000 people, and observe another 500,000,000. We observe a 50% relative risk reduction among the treated. The drug is inexpensive, and has minor but not severe side effects. Should we create everyone?
I have no idea. It has to depend on how much risk we started with. If the mortality rate is 1% per year among the untreated, then almost definitely yes. But if we observed 2 deaths in the untreated and only 1 from the treated, probably there is a better area to spend our efforts, even if we somehow knew the life we saved just noise.
So yes, the reduction in risk matters, but only in concert with a concept of the baseline risk. By itself, it's not very helpful, but in concert with other data it's great. NNT is a way of expressing this in a single number. There is no apriori reason to believe that 200 is high or low. It may not be perfect, but it's better than just a bare relative risk percentage.
> If you did this in a randomised controlled trial, you might recruit 1000 patients, and randomise 500 to this new drug and 500 to placebo. If you ran the trial for a year, you might find 1% of the placebo arm died, and only 0.5% in the control arm. This demonstrates a risk reduction of 0.5%, or a NNT of 200.
That's because the only thing you've learned during the above trial is that it halves the death rate for one year. You have no idea what happens after that year. It could get better, worse, or the adverse effects would be worse than the cure.
Your assumption that that drug cuts the death rate beyond that single year is an assumption, not a fact.
> An NNT, by conventional teaching, is underwhelming.
As it should be. That's the only thing you can conclude from the trial.
> Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy.
Yes. The absolute proven benefit is miniscule.
> In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.
But you've only had a 1 year trial. Drugs often have long term adverse effects that dwarf the benefit.
As an example, when steroids were first discovered they were viewed as miracle drugs and given in massive doses. It didn't take long to understand the adverse effects were far worse than the symptoms it was treating. Today we are far more careful with steroids.
Similarly with Aspirin, or any number of very useful drugs.
So the question is what gives you the confidence to make any claims beyond what the trial showed?
bullshit. what about those killed by stopattack? there should at least be 3 distinct effects related to the medication: no effect, positive effect, negative effect. somehow the potential negative effect of the medication is gladly overlooked.
7 comments
[ 2.0 ms ] story [ 32.7 ms ] threadThis is because NNT is actually a terrible metric for any treatment that reduces risk over a long period, including the most important outcomes like mortality.
Imagine a drug that halves your risk of dying of heart disease. Heart disease kills 1 in 4 people in developed countries, and so realistically it would be a no brainer that everyone middle aged or over should take it.
If you did this in a randomised controlled trial, you might recruit 1000 patients, and randomise 500 to this new drug and 500 to placebo. If you ran the trial for a year, you might find 1% of the placebo arm died, and only 0.5% in the control arm. This demonstrates a risk reduction of 0.5%, or a NNT of 200.
An NNT, by conventional teaching, is underwhelming.
Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy. In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.
It's different if you have a single one-off therapy, where all the benefit will have been delivered by a certain timepoint, at which you measure the NNT. But for a lot of medicine it is not that simple, and the NNT sucks.
And we this is great! If a drug is inexpensive, has minor but not severe side effects, and saves 1/200 people from dying within a year you should absolutely be giving it to everybody. It's a home run. The problem isn't the NNT, it's the expectation of what constitutes a a high or low number.
By contrast, consider another study. We treat 500,000,000 people, and observe another 500,000,000. We observe a 50% relative risk reduction among the treated. The drug is inexpensive, and has minor but not severe side effects. Should we create everyone?
I have no idea. It has to depend on how much risk we started with. If the mortality rate is 1% per year among the untreated, then almost definitely yes. But if we observed 2 deaths in the untreated and only 1 from the treated, probably there is a better area to spend our efforts, even if we somehow knew the life we saved just noise.
So yes, the reduction in risk matters, but only in concert with a concept of the baseline risk. By itself, it's not very helpful, but in concert with other data it's great. NNT is a way of expressing this in a single number. There is no apriori reason to believe that 200 is high or low. It may not be perfect, but it's better than just a bare relative risk percentage.
That's because the only thing you've learned during the above trial is that it halves the death rate for one year. You have no idea what happens after that year. It could get better, worse, or the adverse effects would be worse than the cure.
Your assumption that that drug cuts the death rate beyond that single year is an assumption, not a fact.
> An NNT, by conventional teaching, is underwhelming.
As it should be. That's the only thing you can conclude from the trial.
> Is this the right way to think about it? ABSOLUTELY NOT! The high NNT is a consequence of a low event rate, NOT an ineffective therapy.
Yes. The absolute proven benefit is miniscule.
> In preventative medicine, where everyone will EVENTUALLY die of something (and 25% the thing you are looking at), it is the RELATIVE risk reduction (50% here, not 0.5%) that matters.
But you've only had a 1 year trial. Drugs often have long term adverse effects that dwarf the benefit.
As an example, when steroids were first discovered they were viewed as miracle drugs and given in massive doses. It didn't take long to understand the adverse effects were far worse than the symptoms it was treating. Today we are far more careful with steroids.
Similarly with Aspirin, or any number of very useful drugs.
So the question is what gives you the confidence to make any claims beyond what the trial showed?
https://en.m.wikipedia.org/wiki/Number_needed_to_harm
TheNNT also highlights treatments where the evidence for benefit isn’t clear (or controversial), e.g. thrombolytics in stroke.
https://www.thennt.com/nnt/thrombolytics-for-stroke/