The article brings up a number of good points about advances in research from China, but the article also says:
In 2020, China topped the U.S. for the first time in terms of the number of times an academic article on AI is cited by others
Via Nature:
In China, one of the main indicators currently used to evaluate researchers, allocate funding and rank institutions is metrics collected by the Science Citation Index (SCI), a database of articles and citation records for more than 9,000 journals. Since 2009, articles in these journals written by authors from Chinese institutions increased from some 120,000 a year to 450,000 in 2019. Some institutions even pay researchers bonuses for publishing in them.
These practices have incentivized researchers to publish lots of papers at the expense of quality, says Jin Xuan, a chemical engineer at Loughborough University, UK. Evidence suggests that the focus on metrics has also driven a rise in inappropriate practices, such as researchers submitting plagiarized or fraudulent papers, or inappropriately citing their own or a colleague’s work to boost citations.
Not when it involves China, since gamesmanship and financial incentives play the dominant role for researchers.
A good analogy is the Olympics - China cares only about their gold medal count, while Americans are also interested in the various sports, bios, fitness, history, etc.
Regarding AI, China is hoping their surveillance state and mass collection of data give them an advantage over the US with more available training data and applications.
Godharts law in action. I think the problem is less that the metric is insufficiently tolerant of being gamed, as it is the research institutions are not sufficiently policing the researchers.
Here's the thing: Nobody at any top 50 AI lab reads Chinese papers. Academic fraud, machine-translations and citation rings make the signal to noise ratio to low to bother.
9 comments
[ 3.9 ms ] story [ 33.3 ms ] threadIn 2020, China topped the U.S. for the first time in terms of the number of times an academic article on AI is cited by others
Via Nature:
In China, one of the main indicators currently used to evaluate researchers, allocate funding and rank institutions is metrics collected by the Science Citation Index (SCI), a database of articles and citation records for more than 9,000 journals. Since 2009, articles in these journals written by authors from Chinese institutions increased from some 120,000 a year to 450,000 in 2019. Some institutions even pay researchers bonuses for publishing in them.
These practices have incentivized researchers to publish lots of papers at the expense of quality, says Jin Xuan, a chemical engineer at Loughborough University, UK. Evidence suggests that the focus on metrics has also driven a rise in inappropriate practices, such as researchers submitting plagiarized or fraudulent papers, or inappropriately citing their own or a colleague’s work to boost citations.
https://www.nature.com/articles/d41586-020-00574-8#ref-CR1
Is there some other objective measure of progress in any type of research that can be used to make such comparisons?
Not when it involves China, since gamesmanship and financial incentives play the dominant role for researchers.
A good analogy is the Olympics - China cares only about their gold medal count, while Americans are also interested in the various sports, bios, fitness, history, etc.
Regarding AI, China is hoping their surveillance state and mass collection of data give them an advantage over the US with more available training data and applications.
related: https://news.ycombinator.com/item?id=28107614