The glib response is "US scientists significantly more likely to get caught". (In related news, New York City's arrest rate for murder is likely higher than Kabul's.)
But, given that there is no attempt to adjust the numbers for the US' disproportionate representation in the universe of all published papers, even the glib response isn't correct.
The source says that 33% of withdrawn papers by US first authors were attributed to fraud, while only 25% of withdrawn papers by Asian first authors were attributed to it. So, US first authors are - comparing race cohorts - more likely to publish fake results, are they not (disregarding the fact that there is no possible measure of how likely it is to get caught)?
I agree, though, that the finding is pretty much useless.
30% of retracted papers were from "Asian nations".
The UK, India, China, and Japan each had over 5% of the retracted papers.
33% of retracted US papers were fraudulent. So 67% of their retracted papers were erroneous.
25% of retracted ... is this Asian, or UK/India/China/Japan? papers were fraudulent. So 75% of their retracted papers were erroneous.
How does the percentage of US papers that were retracted, compare the the percentages for other countries (it has number of retracted papers, but not number of non-retracted)? This is necessary to compare the overall percentages of fraudulent/erroneous papers; what I see in the article could mean that US researchers are less likely to make mistakes and equally (or less, even) likely to commit fraud; it doesn't necesasrily mean that US researchers commit more fraud (or make less mistakes).
No - because the percentages are only for withdrawn papers. The likelihood of publishing fake results should be based on the total number of papers. If the US group had fewer withdrawn papers as a percentage of submitted papers then the 33% of withdrawn papers could amount to a smaller percentage of the total submitted than the Asians' 25%.
The cohorts aren't defined by race though, or even ethnicity, are they? Someone born in china, raised in Brazil who did their undergrad in France and got their doctorate in India before publishing a fraudulent paper as a postdoc at UCSF is going to be counted agains the US, right?
On another front though, what's the margin of error on the sample sizes used in the paper?
I know that there is data that students from China are more likely to cheat in classes. There's are no strict codes of conduct in universities and professors look the other way. I wouldn't be surprised if journals worked in much the same way.
The title is misleading. The abstract of the cited paper http://jme.bmj.com/content/early/2010/10/18/jme.2010.038125 says that retracted papers from the US published in PubMed had a higher ratio fraudulent to erroneous retractions than those of other nations. Given that the study targets English language publications only and that its results showed that groups of repeat offenders are responsible for the majority of fraudulent retractions, global statements about the likelihood of fake research originating from US scientists cannot be accurately drawn.
one result that seems interesting: "Roughly 53% of fraudulent papers were written by a first author who had written other retracted papers (‘repeat offender’), whereas only 18% of erroneous papers were written by a repeat offender."
Even if a primary research study (such as the one mentioned in the link submitted here) is conducted without deliberate fraud, it may not be meaningful and may not even be true.
To be convinced that the headline statement "US Scientists Significantly More Likely to Publish Fake Research" is true, I would go through Peter Norvig's checklist above, and see how many methodological issues may have been missed by the reported study. Then I would wait for a follow-up study by an independent researcher, preferably with different methodology and definitely based on a different data set. Right now, I have no idea whether or not there are national differences in rates of publication of fake research.
If I'm using it right, it looks like more than 7.5 million papers were submitted to PubMed in this 10 year period. Only 0.01% of all papers were retracted and only 0.003% were retracted due to "fraud". This seems like an exceedingly small sample. I wish we could see more of the raw analysis (p-scores and the like).
I think the other result was more interesting: 53% of "faked" research papers were submitted by "repeat offenders", while only 18% of "erroneous" papers were. If you do this once it seems you are likely to do this again.
14 comments
[ 6.6 ms ] story [ 49.1 ms ] threadBut, given that there is no attempt to adjust the numbers for the US' disproportionate representation in the universe of all published papers, even the glib response isn't correct.
I agree, though, that the finding is pretty much useless.
Great article in The Atlantic btw - "Lies, damned lies, and medical science" http://www.theatlantic.com/magazine/archive/2010/11/lies-dam...
30% of retracted papers were from "Asian nations".
The UK, India, China, and Japan each had over 5% of the retracted papers.
33% of retracted US papers were fraudulent. So 67% of their retracted papers were erroneous.
25% of retracted ... is this Asian, or UK/India/China/Japan? papers were fraudulent. So 75% of their retracted papers were erroneous.
How does the percentage of US papers that were retracted, compare the the percentages for other countries (it has number of retracted papers, but not number of non-retracted)? This is necessary to compare the overall percentages of fraudulent/erroneous papers; what I see in the article could mean that US researchers are less likely to make mistakes and equally (or less, even) likely to commit fraud; it doesn't necesasrily mean that US researchers commit more fraud (or make less mistakes).
On another front though, what's the margin of error on the sample sizes used in the paper?
It should also be noted that PubMed only covers a small, non-representative part of the entire research landscape.
http://norvig.com/experiment-design.html
To be convinced that the headline statement "US Scientists Significantly More Likely to Publish Fake Research" is true, I would go through Peter Norvig's checklist above, and see how many methodological issues may have been missed by the reported study. Then I would wait for a follow-up study by an independent researcher, preferably with different methodology and definitely based on a different data set. Right now, I have no idea whether or not there are national differences in rates of publication of fake research.
I think the other result was more interesting: 53% of "faked" research papers were submitted by "repeat offenders", while only 18% of "erroneous" papers were. If you do this once it seems you are likely to do this again.
BTW, the abstract is at http://jme.bmj.com/content/early/2010/10/18/jme.2010.038125