Did you actually read the article? The only thing that might imply causation is the word "linked". However that also could imply correlation.
define link: A relationship between two things or situations, esp. where one thing affects the other
define correlation: A mutual relationship or connection between two or more things
The definitions are close enough that even amongst the math/science researchers I work with, the words are frequently used interchangeably.
Everything else in the article talks in terms of "maybe", "plausible", "associated" and so on. All words that don't imply causation, nor rule it out. Further, the article is careful to point out that:
The explanation for the study findings "might not specifically be the coffee," Fisher says. "It might be some characteristic of the coffee drinker."
I'n all, I found it to be pretty good "science for the layman" reporting - to the point where I was surprised at how good it was.
My guess is that if you're drinking 6 cups of coffee a day, you've probably got yourself a nice white collar job - and a lot of the other life lengthening perks that come along with gainful employment.
Thanks for that. It's a little annoying that every time a study like this comes up, somebody jumps in with "Well, did they control for [obvious thing to control for]?"
You don't become a scientist because you're lazy and stupid. If you're reading about a study and think there's an obvious reason it's inaccurate, you are almost certainly wrong. Not that the study is not inaccurate -- many, perhaps most are -- but the reason is definitely not obvious. It's something that the authors, people who have spent a lot more time thinking about it than you, and who are probably smarter and better educated to boot, never figured out.
Guess what: it looks like the authors didn't control for income (or even just whether the person has a job!) or intelligence.
I just read the paper; there are many diet and exercise-related controls, and that's it as far as I can see. I totally agree with the OP that being the kind of person who might drink coffee probably has a confounding effect on health. I can't even think of how to try to control it properly.
I don't understand how you get from "most studies are inaccurate" to "but you shouldn't assume the ones that get reported on in the media are inaccurate in a way that we can detect". Why not? People who are performing studies that get picked up by popular media are the least likely people to be performing their work rigorously, and the most likely to be choosing a sensational hypothesis that's difficult to measure correctly.
I agree that we might eventually get to a point where the amount of skepticism and cynicism about scientific results vastly outweighs the number of papers published with poor experimental design or lack of real statistical significance. But I don't think we're currently anywhere close!
You make some good points, so I hope this doesn't seem dismissive-- my claim is not that the study is right. My claim is simply that the editors of the NEJM decided to publish the study having read it in greater depth than any of us here would care to, so if I read the CNN article and think there's an obvious reason it's not true, I can be pretty sure I'm wrong.
Now that being said, I think we're talking at cross purposes. The study found a link between coffee drinknig and mortality; I misread the GGP as being critical of that link, where it's actually just a hypothesis which would explain it. So none of this is really relevant :P
No, they normally never control for anything that smells of class (like income) because if they do, the health effect disappears. There was a good recent case or rerunning a study and controlling for class, thereby eliminating an effect (I think it was about potatoes and weight gain or something.) Don't have access to it here, but I'll try to post it when I get home.
IMO 95% of the health studies that catch fire in the popular press are repeated rediscoveries of class (smoking, fast food and types of food, etc.), or the qualities of illness (people who sit more die earlier, people who are depressed die earlier, etc.)
They did mention controlling for smoking, though, because they found that it counted against them.
I read the "methods" section of the study, and saw no reference to controlling for occupation nor socioeconomic status (as opposed to marital status, self-reported health, and other things that are listed).
However, I only read it briefly, so I may have missed something.
Indeed. This type of study, while really popular in the press, is often horribly confounded and hard to control for. I think the hacker news audience mostly knows to be skeptical, so I don't know why it was submitted.
I'll use this post as an excuse to mention the Lothian Birth Cohort. A really cool dataset of IQ test results at age 11 for (almost) all Scottish kids born in 1921 and 1936. While IQ tests are flawed in many ways, these data offer some means of control for follow up studies on these individuals. Many apparent effects disappear after controlling for age 11 IQ, for example: drinking coffee on intelligence — http://www.ncbi.nlm.nih.gov/pubmed/19995882 — nicely demonstrating the confounding problems affecting most studies.
Edit in response to sibling comment: yes, studies do try to control for these things. But it's hard, and often not really possible. This particular study might be ok, but we're not going to tell from a CNN article. We're also used to enough contradictory and over-hyped results in the popular press to know not to trust journalists to be a good filter.
In response to your edit: I think you're saying something very different than what your sibling responded to. The parent's claim was "They didn't correct for X"; your claim is "They tried to correct for X, but we have no way of knowing if they did it right."
The difference is that what you're saying is true of any study. There are always obvious things to control for, and less-obvious ones, and the authors always try and sometimes they do it right and sometimes they don't and there's never any way to tell for sure without redoing the study[0].
The fact that a layman reading a mainstream article about the study can come up with a possible confounding factor doesn't imply much if anything about its possible veracity. We should be equally suspicious of all such claims, not only those which happen to have flaws which are apparent to us.
[0] If the study contains a basic statistical error, redoing it might just be a matter of reinterpreting the original data; still counts.
Have you done any blue collar work? Depending on situation, many folks carry large thermos of coffee with them, and between morning and afternoon breaks, and lunch, quite a bit of coffee is consumed. Yes, this is my experience (and that isn't scientific), but simply stating that only office dwellers drink a lot of coffee isn't exactly scientific either.
That said, I agree with the sentiment here (and of the siblings) that controlling this sort of study is rather difficult.
It would be interesting if the study showed this, but since the article doesn't mention it, that's a bit of an assumption you're making here.
I personally know a couple of heavy coffee drinkers, some of them should be retired but still work blue collar jobs (one is a roofer, another is a general contractor). I doubt they would agree with you on this one :)
I have a slow caffeine metabolism (SNP: rs762551) and according to some studies this may lead to me to develop heart disease if I drink too much caffeine.
The fact that I'm a minority and that fast metabolizers might actually benefit from caffeine says to me that going just on that article might not be a good idea.
A majority of people might benefit from caffeine, while a minority might have really dire consequences.
/edit this is all preliminary research so add more weight to "might" that I use here.
Gosh I hope this doesn't come off as too negative, but what is the point finding these 'links'?
Science is about trying to explain the world, not looking for random facts.
We learn nothing of substance about coffee, health or anything else from this article. Yet it has the appearance of being scientific, for example, it mentions antioxidants.
It wouldn't be fair to label it pseudoscience however it does share some of the features.
People are starting to learn by now that correlation doesn't imply causation, but they still think it is suggestive of causation, or makes causation more probable, or something like that.
To try to dispel that: the fact that coffee drinking correlates positively with life expectancy is logically consistent with coffee acting to reduce lifespan. More importantly, knowing causes is not very helpful since pretty much everything causes pretty much everything else.
What is useful is solving problems and trying to explain the world. That's what people like Newton, Darwin and Einstein were doing.
FWIW my guess is that drinking coffee causes people to eat less food which might indeed result in greater health, though this doesn't seem like a fruitful line of enquiry.
"What is useful is solving problems and trying to explain the world. That's what people like Newton, Darwin and Einstein were doing."
Of course. And that's why they are in the history books.
But now the most part of "scientific studies" have two objectives:
- Meet grant / tenure criteria of published papers
- Prove the new drug by "Big Pharma" is better than the old one whose patents are expiring. The fact that several are disproved afterwards is irrelevant.
At least we still have the LHC guys, the Hubble guys, MSL, AMS. But don't be surprised if those are accused of not being "real scientists" as well.
Do you know on how many centuries of tabulating 'links' Newton, Darwin, and Einstein built their work? Apart from modesty, there is truth in Newton's claim about standing on the shoulders of giants.
Also, this is not a random fact. People have argued both ways whether coffe is healthy or not. If we want doctors to give advice based on facts, we have to, somehow, figure out who (if anybody) is right there. Barring a clear mechanism either way, looking for facts is a step towards scientific understanding.
Tangential: I was actually pretty surprised at how good this article was at not being too hype-filled. It discussed the report in terms that did not imply causation. It offered potential explanations (with caveats that they were just speculation), pointed out potential flaws with the stats (2 different places where unaccounted factors may be the real responsibility), and generally didn't say "go drink lots of coffee, it will save your life".
So my kudos to the author and editor for being reasonable.
24 comments
[ 2.6 ms ] story [ 65.0 ms ] threaddefine link: A relationship between two things or situations, esp. where one thing affects the other
define correlation: A mutual relationship or connection between two or more things
The definitions are close enough that even amongst the math/science researchers I work with, the words are frequently used interchangeably.
Everything else in the article talks in terms of "maybe", "plausible", "associated" and so on. All words that don't imply causation, nor rule it out. Further, the article is careful to point out that:
The explanation for the study findings "might not specifically be the coffee," Fisher says. "It might be some characteristic of the coffee drinker."
I'n all, I found it to be pretty good "science for the layman" reporting - to the point where I was surprised at how good it was.
You don't become a scientist because you're lazy and stupid. If you're reading about a study and think there's an obvious reason it's inaccurate, you are almost certainly wrong. Not that the study is not inaccurate -- many, perhaps most are -- but the reason is definitely not obvious. It's something that the authors, people who have spent a lot more time thinking about it than you, and who are probably smarter and better educated to boot, never figured out.
I just read the paper; there are many diet and exercise-related controls, and that's it as far as I can see. I totally agree with the OP that being the kind of person who might drink coffee probably has a confounding effect on health. I can't even think of how to try to control it properly.
I don't understand how you get from "most studies are inaccurate" to "but you shouldn't assume the ones that get reported on in the media are inaccurate in a way that we can detect". Why not? People who are performing studies that get picked up by popular media are the least likely people to be performing their work rigorously, and the most likely to be choosing a sensational hypothesis that's difficult to measure correctly.
I agree that we might eventually get to a point where the amount of skepticism and cynicism about scientific results vastly outweighs the number of papers published with poor experimental design or lack of real statistical significance. But I don't think we're currently anywhere close!
Now that being said, I think we're talking at cross purposes. The study found a link between coffee drinknig and mortality; I misread the GGP as being critical of that link, where it's actually just a hypothesis which would explain it. So none of this is really relevant :P
IMO 95% of the health studies that catch fire in the popular press are repeated rediscoveries of class (smoking, fast food and types of food, etc.), or the qualities of illness (people who sit more die earlier, people who are depressed die earlier, etc.)
They did mention controlling for smoking, though, because they found that it counted against them.
However, I only read it briefly, so I may have missed something.
I'll use this post as an excuse to mention the Lothian Birth Cohort. A really cool dataset of IQ test results at age 11 for (almost) all Scottish kids born in 1921 and 1936. While IQ tests are flawed in many ways, these data offer some means of control for follow up studies on these individuals. Many apparent effects disappear after controlling for age 11 IQ, for example: drinking coffee on intelligence — http://www.ncbi.nlm.nih.gov/pubmed/19995882 — nicely demonstrating the confounding problems affecting most studies.
Edit in response to sibling comment: yes, studies do try to control for these things. But it's hard, and often not really possible. This particular study might be ok, but we're not going to tell from a CNN article. We're also used to enough contradictory and over-hyped results in the popular press to know not to trust journalists to be a good filter.
The difference is that what you're saying is true of any study. There are always obvious things to control for, and less-obvious ones, and the authors always try and sometimes they do it right and sometimes they don't and there's never any way to tell for sure without redoing the study[0].
The fact that a layman reading a mainstream article about the study can come up with a possible confounding factor doesn't imply much if anything about its possible veracity. We should be equally suspicious of all such claims, not only those which happen to have flaws which are apparent to us.
[0] If the study contains a basic statistical error, redoing it might just be a matter of reinterpreting the original data; still counts.
That said, I agree with the sentiment here (and of the siblings) that controlling this sort of study is rather difficult.
I personally know a couple of heavy coffee drinkers, some of them should be retired but still work blue collar jobs (one is a roofer, another is a general contractor). I doubt they would agree with you on this one :)
The fact that I'm a minority and that fast metabolizers might actually benefit from caffeine says to me that going just on that article might not be a good idea.
A majority of people might benefit from caffeine, while a minority might have really dire consequences.
/edit this is all preliminary research so add more weight to "might" that I use here.
Science is about trying to explain the world, not looking for random facts.
We learn nothing of substance about coffee, health or anything else from this article. Yet it has the appearance of being scientific, for example, it mentions antioxidants.
It wouldn't be fair to label it pseudoscience however it does share some of the features.
People are starting to learn by now that correlation doesn't imply causation, but they still think it is suggestive of causation, or makes causation more probable, or something like that.
To try to dispel that: the fact that coffee drinking correlates positively with life expectancy is logically consistent with coffee acting to reduce lifespan. More importantly, knowing causes is not very helpful since pretty much everything causes pretty much everything else.
What is useful is solving problems and trying to explain the world. That's what people like Newton, Darwin and Einstein were doing.
FWIW my guess is that drinking coffee causes people to eat less food which might indeed result in greater health, though this doesn't seem like a fruitful line of enquiry.
Of course. And that's why they are in the history books.
But now the most part of "scientific studies" have two objectives:
- Meet grant / tenure criteria of published papers
- Prove the new drug by "Big Pharma" is better than the old one whose patents are expiring. The fact that several are disproved afterwards is irrelevant.
At least we still have the LHC guys, the Hubble guys, MSL, AMS. But don't be surprised if those are accused of not being "real scientists" as well.
Also, this is not a random fact. People have argued both ways whether coffe is healthy or not. If we want doctors to give advice based on facts, we have to, somehow, figure out who (if anybody) is right there. Barring a clear mechanism either way, looking for facts is a step towards scientific understanding.
Given my dependence on coffee I'm probably going to end up floating in a tank of the stuff a few thousand years from now.
So my kudos to the author and editor for being reasonable.