Things like citation brokers (paid to cite papers), abuse of power, paper mills, and blackmail (pg. 10) is appalling to me. I have to question how we ended up here. Academia seems very focused on results and output, and this is used as a metric to measure a researcher's worth or value.
Has this always been an issue in academia, or is this an increasing or new phenomenon? It seems as if there is a widespread need to take shortcuts and boost your h-index. Is there a better way to determine the impact of research and to encourage researchers to not feel so pressed to output and boost their citations? Why is it like this today?
Academic mathematics, from what I've seen, seems incredibly competitive and stressful (to be fair, so does competition math from a young age), perhaps because the only career for many mathematicians (outside a topics with applications such as but not limited to number theory, probability, and combinatorics) is academia. Does this play into what this article talks about?
> Is there a better way to determine the impact of research and to encourage researchers to not feel so pressed to output and boost their citations? Why is it like this today?
It's hard, specially if you have to compare people of different areas (like algebra vs calculus) that have different threshold for what is a paper worthily result and each community has a different size and different length of review time.
Solution 1) Just count the papers! Each one is 1 point. You can finish before lunch.
Solution 2) Add some metrics like citations (that favor big areas and areas that like to add many citations). Add impact index (that has the same problem). How do you count self citations and citation rings?
Solution 3) Cherry pick some good journals, but ensure the classification committee is not just making a list of the journals they publish in. Filter the citations, or add some weight according to the classification.
Solution 4) Give the chair of the department a golden crown and pretend s/he is the queen/king and can do whatever they like. It may work, but there are BDFL and nepotist idiots. Now try scaling it for a country.
Solution 5) RTFA. Nah. It's too hard. Assume you have 5 candidates and they have 5 papers in the last 5 years (or some other arbitrary threshold). You need like two weeks to read a paper, more if it's not in you area, perhaps you can skim it in 1 or 2 days, but it's not easy to have an accurate understanding of how interesting is the result and how much impact it has in the community. (How do you evaluate if it's a interesting new result, or just a hard stupid calculation?) You can distribute the process of reading the papers, but now you have the problem of merging the opinion of different people. (Are your 3/5 stars the same that my 3/5 stars?)
Sabine Hossenfelder has been on about this topic in the field of physics publishing for quite some time now.
It really is a terrible thing, though I can understand how some researchers feel trapped in a system that gives them little if any alternative if they wish to be employed the next year. Not just one thing needs to be changed to fix it.
TLDR: The publication culture of mathematics (with relatively few papers per researcher, few authors per paper, and few citations per paper) makes abuse of bibliometrics easier. The evidence suggests widespread abuse.
My take: I’ve published in well-regarded mathematical journals and the culture is definitely hard to explain to people outside of math. For example, it took more than two years to get my key graduate paper published in Foundations of Computational Mathematics, a highly regarded journal. The paper currently has over 100 citations, which (last I checked) is a couple times higher than the average citation count for the journal. In short, it’s a great, impactful work for a graduate student. But in a field like cell biology, this would be considered a pretty weak showing.
Given the long timelines and low citation counts, it’s not surprising that it’s so easy to manipulate the numbers. It is kinda ironic that mathematicians have this issue with numbers though.
Are "publication metrics" also used heavily in China by the bureaucracy ?
I know for a fact that the number of fake-journals exploded once the Govt. of India decided to use this for promotions.
It's a bit sad really: in the classical world both these countries spent inordinate amount of time on the questions of epistemology (India esp.). Now reduced to mimicking some silly thing that vaguely tracks knowledge-production even in the best case in the West.
This article does not seem to quite convey the experience of a pure mathematician. Yes, citation fraud is happening on an apalling scale, but no it is not a serious issue for mathematicians.
The problem of AI generated papers is much more serious, although not happening on the same scale (yet!).
Publishing math is one of the most time consuming things ever, between the submission, review/revising, and editing. I with there was a faster way of doing it outside of arXiv. Without having to review the paper closely, typically an experienced editor can tell at fist glace if it's correct or sound.
It is what we could call the “zone of occasional poor practice”.
Included are actions like
I think this is more common in computer science papers. I see this all the time, where 5- 10 authors will collaborate on a short paper, then collaborate on each other's papers in such a way that the effort is minimized and publishing count and citation count is maximized. .
Bibliometrics in science is just an unworkable approach in general, and IMO it causes more harms than not. Research is one of the least suitable human activities that you can possibly try to quantify, yet the entire scientific establishment runs on these metrics by now. I more or less believe that this strategy hinders scientific progress, as it pushes researchers into more and more risk-averse behaviors.
I love the table of tortured phrases [0], which shows hilarious examples of synonyms of established scientific phrases, machine-generated by fraudulent authors to stay below the radar of plagiarism detectors.
When I took business 101 in college one of the first things they taught us is that long term, fixed metrics will always become gamified, that both the ones measuring and the ones being measured will replace the real results with the metrics and sacrifice the first for the second. I understand that this is common knowledge in the administrative world. Yet, every single performance metric always becomes ossified as the only target that matters, every time.
Why?
13 comments
[ 2.2 ms ] story [ 35.5 ms ] threadHas this always been an issue in academia, or is this an increasing or new phenomenon? It seems as if there is a widespread need to take shortcuts and boost your h-index. Is there a better way to determine the impact of research and to encourage researchers to not feel so pressed to output and boost their citations? Why is it like this today?
Academic mathematics, from what I've seen, seems incredibly competitive and stressful (to be fair, so does competition math from a young age), perhaps because the only career for many mathematicians (outside a topics with applications such as but not limited to number theory, probability, and combinatorics) is academia. Does this play into what this article talks about?
It's hard, specially if you have to compare people of different areas (like algebra vs calculus) that have different threshold for what is a paper worthily result and each community has a different size and different length of review time.
Solution 1) Just count the papers! Each one is 1 point. You can finish before lunch.
Solution 2) Add some metrics like citations (that favor big areas and areas that like to add many citations). Add impact index (that has the same problem). How do you count self citations and citation rings?
Solution 3) Cherry pick some good journals, but ensure the classification committee is not just making a list of the journals they publish in. Filter the citations, or add some weight according to the classification.
Solution 4) Give the chair of the department a golden crown and pretend s/he is the queen/king and can do whatever they like. It may work, but there are BDFL and nepotist idiots. Now try scaling it for a country.
Solution 5) RTFA. Nah. It's too hard. Assume you have 5 candidates and they have 5 papers in the last 5 years (or some other arbitrary threshold). You need like two weeks to read a paper, more if it's not in you area, perhaps you can skim it in 1 or 2 days, but it's not easy to have an accurate understanding of how interesting is the result and how much impact it has in the community. (How do you evaluate if it's a interesting new result, or just a hard stupid calculation?) You can distribute the process of reading the papers, but now you have the problem of merging the opinion of different people. (Are your 3/5 stars the same that my 3/5 stars?)
It really is a terrible thing, though I can understand how some researchers feel trapped in a system that gives them little if any alternative if they wish to be employed the next year. Not just one thing needs to be changed to fix it.
My take: I’ve published in well-regarded mathematical journals and the culture is definitely hard to explain to people outside of math. For example, it took more than two years to get my key graduate paper published in Foundations of Computational Mathematics, a highly regarded journal. The paper currently has over 100 citations, which (last I checked) is a couple times higher than the average citation count for the journal. In short, it’s a great, impactful work for a graduate student. But in a field like cell biology, this would be considered a pretty weak showing.
Given the long timelines and low citation counts, it’s not surprising that it’s so easy to manipulate the numbers. It is kinda ironic that mathematicians have this issue with numbers though.
I know for a fact that the number of fake-journals exploded once the Govt. of India decided to use this for promotions.
It's a bit sad really: in the classical world both these countries spent inordinate amount of time on the questions of epistemology (India esp.). Now reduced to mimicking some silly thing that vaguely tracks knowledge-production even in the best case in the West.
The problem of AI generated papers is much more serious, although not happening on the same scale (yet!).
It is what we could call the “zone of occasional poor practice”. Included are actions like
I think this is more common in computer science papers. I see this all the time, where 5- 10 authors will collaborate on a short paper, then collaborate on each other's papers in such a way that the effort is minimized and publishing count and citation count is maximized. .
My favorites from that table:
- “fuzzy logic” becomes “fluffy rationale”
- “spectral analysis” becomes “phantom examinations”
- “big data” becomes “enormous information”
[0]: https://arxiv.org/pdf/2509.07257#table.3
[0] https://dbrech.irit.fr/pls/apex/f?p=9999:24