This is Goodhart's law at scale. Number of released papers/number of citations is a target. Correctness of those papers/citations is much more difficult so is not being used as a measure.
With that said, due to the apparent sizes of the fraud networks I'm not sure this will be easy to address. Having some kind of kill flag for individuals found to have committed fraud will be needed, but with nation state backing and the size of the groups this may quickly turn into a tit for tat where fraud accusations may not end up being an accurate signal.
> Number of released papers/number of citations is a target
Only in stupid university leaderships is that truly what gets you hired or promoted. It's simply not true. Junior researchers in fact are believing it stronger than the facts actually support. Yes, you have to have a solid amount of publications, but doing a ridiculous amount of low-impact salami-sliced stuff or getting your name on a ton of papers where you did no real work is not going to win you a job. People who evaluate applications also live in this world and know that these metrics are being gamed. It's a cat and mouse game but the cats are also paying attention. You can only play this against really dumb government bureaucracies that mechanically give points for publications and have hard numerical criteria etc. Good institutions don't do that.
Good evaluators actually read the papers themselves. Of course you can't read the papers of every single applicant if there are many. But once the applicant gets into the a somewhat filtered down list, reading the paper(s) or having an interview about it, or having them give a talk is much more informative than the number of the papers. Still not perfect, because some people can't communicate well, but communicating is part of the job, so maybe that's super bad but somewhat bad.
Evaluators will use also other evidence such as recommendation letters (informally being aware of the reputation of the recommender), previous fellowships or grants obtained, etc.
None of these are foolproof in themselves. But someone who has super few publications relative to their career stage will need some other piece of evidence in favor.
In machine learning and AI, peer reviews are known to be quite random. If you have a good Arxiv-only paper that makes sense and you can give a good talk on it and answer questions, that will get you further than having a rubberstamp on some paper that's "meh, so what".
There are some players in this game (which includes funding agencies, journals, university administration, hiring committees, conference organizers, students, etc) that are more ossified and slow-moving than others.
And it's also true that double blind peer review and the rubberstamp of a top-tier conference was mostly beneficial to small, not well connected research groups, as it puts the paper on an equal footing with the big labs. The more this system erodes, to more we fall back to reputation and branding of big labs and famous researchers. Again, because there is no infinite time and infinite wisdom available to pick from applicants and there never will be. There are only tradeoffs.
The future of science, the Internet, and all things: The Library of Babel by Jorge Luis Borges.
Some things should not have been democratized. Silicon Valley assumes that removing restrictions on information brings freedom, but reality shows that was naïve.
It kinda skips over how large mainstream journals, with their restrictive and often arbitrary standards, have contributed to this. Most will refuse to publish replications, negative studies, or anything they deem unimportant, even if the study was conducted correctly.
Plenty will publish it, but those are not as highly regarded by the community. It's not a problem of journals. It's not hard to start your own journal by teaming up with other academics. In machine learning, ICLR is such a venue for example. The problem is much deeper and more fundamental. You want to publish alongside groundbreaking novel research. Researcher's own ears perk up when they hear about something new. They invite colleagues to talk about their novel discoveries not to describe all their null results and successful replications of known results. Funding agencies want research with novelty and impact. They want to write reports to the higher ups and the politicians and the donors that document the innovations that their funding brought. The media will republish press releases that have cool new results.
To have research happening, you need someone saying "I want to give money to this researcher". There is an endless queue of people lining up who are ready to take this money and do something with it. The person with money (govt or private) has to use some heuristics to pick. One way is to say "I trust this one, I don't care too much what the project is, I'm sure this person will do something that makes sense". But that is dependent on a track record.
I have worked in this particular sausage factory. Multiple funded random replications are the only thing that will save science from this crisis. The scientific method works. We need to actually do it.
Replications don't have to be in the journals either. As long as money flows, someone will do them, and that is what matters. The randomization will help prevent coordination between authors and replicators.
In a better world, negative studies and replications would count towards tenure, but that is unlikely to occur. At least half of the problem is the pressure to continuously publish positive results.
My wife completed her PhD two years ago and she put a LOT of work into it. Many sleepless nights, and it almost destroyed our marriage. It took her about 6 years of non-stop madness and she didn’t even work during that time. She said that many of her colleagues engaged in fraudulent data generation and sometimes just complete forgery of anything and everything. It was
obvious some people were barely capable of putting together coherent sentences in posts, but somehow they generated a perfect dissertation in the end. It was common knowledge that candidates often hired writers and even experts like statisticians to do most of the heavy lifting. I don’t know if this is the norm now, but I simultaneously have more respect and less respect for those doctoral degrees, knowing that some poured their heart and soul into it, while others essentially cheated their way through. OTOH, I also understand that there may be a lot of grey area.
It is useful to distinguish between "effective" scientific fraud, where some set of fraudulent papers are published that drive a discipline in an unproductive direction, and "administrative" scientific fraud, where individuals use pseudo-scientific measures (H-index, rankings, etc) to make allocation decisions (grants, tenure, etc). This article suggests that administrative scientific fraud has become more accessible, but it is very unclear whether this is having a major impact on science as it is practiced.
Non-scientists often seem to think that if a paper is published, it is likely to be true. Most practicing scientists are much more skeptical. When I read a that paper sounds interesting in a high impact journal, I am constantly trying to figure out whether I should believe it. If it goes against a vast amount of science (e.g. bacteria that use arsenic rather than phosphorus in their DNA), I don't believe it (and can think of lots of ways to show that it is wrong). In lower impact journals, papers make claims that are not very surprising, so if they are fraudulent in some way, I don't care.
Science has to be reproducible, but more importantly, it must be possible to build on a set of results to extend them. Some results are hard to reproduce because the methods are technically challenging. But if results cannot be extended, they have little effect. Science really is self-correcting, and correction happens faster for results that matter. Not all fraud has the same impact. Most fraud is unfortunate, and should be reduced, but has a short lived impact.
One approach is more integration of researchers with businesses. Fraud (or simple incompetence) by researchers negatively affects businesses, as they expend effort on things that aren't real. I understand this is a constant problem in the pharmaceutical industry.
firstly, there are basically no legal repercussions for scientific misconduct (e.g. falsifying data, fake images, etc.). most individuals who are caught doing this get either 1) a slap on the wrist if they are too big to fail or in the employ of those who are too big to fail or 2) disbarred, banned, and lose their jobs. i don't see why you can go to jail for lying to investors about the number of users in your app but don't go to jail for lying to the public, government, and members of the scientific community about your results.
secondly, due to the over production of PhD's and limited number of professorship slots competition has become so incredibly intense that in order to even be considered for these jobs you must have Nature, Cell, and Science papers (or the field equivalent). for those desperate for the job their academic career is over either way if they caught falsifying data or if they don't get the professorship. so if your project is not going the way you want it to then...
sad state of things all around. i've personally witnessed enough misconduct that i have made the decision to leave the field entirely and go do something else.
and select "Mathematics and Computer Science", you'll find the top-ranked university is the University of Electronic Science and Technology of China.
My Chinese colleagues have heard of it, but never considered it a top-ranked school, and a quick inspection of their CS faculty pages shows a distinct lack of PhDs from top-ranked Chinese or US schools. It's possible their math faculty is amazing, but I think it's more likely that something underhanded is going on...
Almost as if capitalism makes everything into a market, and the profits make it self sustaining.
How many will see the connections between this and our capitalist mode of production? Probably few since modern lit/news is allergic to systemic analysis.
The blatant flaws of capitalism can't be ignored for much longer.
Capitalism certainly is hugely flawed and yet it is far less flawed than any other economic system we know of. Experimentation with the foundations of society is about as risky as it gets. You could end up with a utopia or you could end up with another USSR. History tells us which outcome is more likely
People say that because they are not aware of what Marxism is.
Marxism isnt "lets try something different based on ideas of justice"
Marxism is "Society evolves through general stages: primitive, slave, feudal and capitalist, this is determined by the level production. Capitalism is holding back its current evolution into a society of plenty."
Good luck backing speculation and profit gatekept production.
It's strange to me that in places full of smart people, it seems to be well understood that this happens and there are lots of anecdotes relating to it; yet the same people will be confused that their political adversaries don't trust "the science" on one issue or another.
I think it's difficult to relay to the public that a lot of this noise in "scientific publications" is not the same category as real research by reputable institutions. Yes, in certain cases the line can be blurry, fraudsters are sometimes caught in big-name institutions, maybe more in some fields than others, but serious researchers of the field know very well which publication venues and research groups are the real deal and what is bullshit. Overwhelmingly, these fraud papers and nonsense LLM-generated fake stuff are not published in serious journals or conferences.
It's a bit like how can we trust online shopping if I get all these emails trying to sell me aphrodisiac pills?
the "political adversaries" (let's not be coy, you mean conservatives) who "don't trust 'the science'" are less equipped to evaluate said "the science" than the scient_ists_ they don't trust. usually said people were one bad day away from rambling about global jewry and such. due to whatever social illnesses or sociopolitical causes, we've normalized this position, and now measles is back from the dead. but let's pretend that climate change isn't real and the earth's flat and vaccines give you autism, because universities too woke or whatever the fuck
(it's very funny to pretend tech is still, or really much ever was, one big libfest. it's funnier to say this here of all places)
More broadly, an incredible amount of our society's systems are built around actors being uncoordinated. Redesigning institutions to resist networks of coordinated action between seemingly unlinked individuals will, in my opinion, be one of the great social challenges of this era.
I ran into an interesting incident of this recently. I got a Google Scholar alert about a paper with some experiments related to a paper I had published a while ago, by one "N. Tvlg". I read the paper with interest but I started noticing that although the arguments sounded good, they didn't really make sense, and also the descriptions of the results didn't really match the figures. Eventually I came across a cluster of citations to completely unrelated papers---my field is computational linguistics and these were citations to, like, studies of battery technologies for electric cars. I looked up "N Tvlg" on Google Scholar and they had "published" several articles very recently in totally divergent fields, and upon inspection, all of them had citations back to this materials science research buried deeply somewhere. Clearly these were LLM generated papers trying to build up citation count and h-rank for someone's career.
The purpose of scientific publication used to be to deliver useful scientific results to one's peers. This meant that everyone ran their own personal filter of which peers were working on interesting things, and which collections (journals) were reproducing the most interesting ones. This system still works relatively well for most conscientious researchers. The idea that we should also use publication metrics to rank researchers was never part of this system, and it obviously leads to all sorts of spam (that most scientists just work around) but that seems to really upset non-scientists.
This is the part that feels hardest to fix: once a system starts rewarding throughput over scrutiny, fraud stops looking like individual misconduct and starts looking like a supply chain problem.
35 comments
[ 3.8 ms ] story [ 35.8 ms ] threadWith that said, due to the apparent sizes of the fraud networks I'm not sure this will be easy to address. Having some kind of kill flag for individuals found to have committed fraud will be needed, but with nation state backing and the size of the groups this may quickly turn into a tit for tat where fraud accusations may not end up being an accurate signal.
May you live in interesting times.
Only in stupid university leaderships is that truly what gets you hired or promoted. It's simply not true. Junior researchers in fact are believing it stronger than the facts actually support. Yes, you have to have a solid amount of publications, but doing a ridiculous amount of low-impact salami-sliced stuff or getting your name on a ton of papers where you did no real work is not going to win you a job. People who evaluate applications also live in this world and know that these metrics are being gamed. It's a cat and mouse game but the cats are also paying attention. You can only play this against really dumb government bureaucracies that mechanically give points for publications and have hard numerical criteria etc. Good institutions don't do that.
Good evaluators actually read the papers themselves. Of course you can't read the papers of every single applicant if there are many. But once the applicant gets into the a somewhat filtered down list, reading the paper(s) or having an interview about it, or having them give a talk is much more informative than the number of the papers. Still not perfect, because some people can't communicate well, but communicating is part of the job, so maybe that's super bad but somewhat bad.
Evaluators will use also other evidence such as recommendation letters (informally being aware of the reputation of the recommender), previous fellowships or grants obtained, etc.
None of these are foolproof in themselves. But someone who has super few publications relative to their career stage will need some other piece of evidence in favor.
In machine learning and AI, peer reviews are known to be quite random. If you have a good Arxiv-only paper that makes sense and you can give a good talk on it and answer questions, that will get you further than having a rubberstamp on some paper that's "meh, so what".
There are some players in this game (which includes funding agencies, journals, university administration, hiring committees, conference organizers, students, etc) that are more ossified and slow-moving than others.
And it's also true that double blind peer review and the rubberstamp of a top-tier conference was mostly beneficial to small, not well connected research groups, as it puts the paper on an equal footing with the big labs. The more this system erodes, to more we fall back to reputation and branding of big labs and famous researchers. Again, because there is no infinite time and infinite wisdom available to pick from applicants and there never will be. There are only tradeoffs.
Some things should not have been democratized. Silicon Valley assumes that removing restrictions on information brings freedom, but reality shows that was naïve.
To have research happening, you need someone saying "I want to give money to this researcher". There is an endless queue of people lining up who are ready to take this money and do something with it. The person with money (govt or private) has to use some heuristics to pick. One way is to say "I trust this one, I don't care too much what the project is, I'm sure this person will do something that makes sense". But that is dependent on a track record.
Replications don't have to be in the journals either. As long as money flows, someone will do them, and that is what matters. The randomization will help prevent coordination between authors and replicators.
In a better world, negative studies and replications would count towards tenure, but that is unlikely to occur. At least half of the problem is the pressure to continuously publish positive results.
The simple fact that theories should be falsified and not verified is something that most scientists don’t know.
My eyes have been opened!
Non-scientists often seem to think that if a paper is published, it is likely to be true. Most practicing scientists are much more skeptical. When I read a that paper sounds interesting in a high impact journal, I am constantly trying to figure out whether I should believe it. If it goes against a vast amount of science (e.g. bacteria that use arsenic rather than phosphorus in their DNA), I don't believe it (and can think of lots of ways to show that it is wrong). In lower impact journals, papers make claims that are not very surprising, so if they are fraudulent in some way, I don't care.
Science has to be reproducible, but more importantly, it must be possible to build on a set of results to extend them. Some results are hard to reproduce because the methods are technically challenging. But if results cannot be extended, they have little effect. Science really is self-correcting, and correction happens faster for results that matter. Not all fraud has the same impact. Most fraud is unfortunate, and should be reduced, but has a short lived impact.
firstly, there are basically no legal repercussions for scientific misconduct (e.g. falsifying data, fake images, etc.). most individuals who are caught doing this get either 1) a slap on the wrist if they are too big to fail or in the employ of those who are too big to fail or 2) disbarred, banned, and lose their jobs. i don't see why you can go to jail for lying to investors about the number of users in your app but don't go to jail for lying to the public, government, and members of the scientific community about your results.
secondly, due to the over production of PhD's and limited number of professorship slots competition has become so incredibly intense that in order to even be considered for these jobs you must have Nature, Cell, and Science papers (or the field equivalent). for those desperate for the job their academic career is over either way if they caught falsifying data or if they don't get the professorship. so if your project is not going the way you want it to then...
sad state of things all around. i've personally witnessed enough misconduct that i have made the decision to leave the field entirely and go do something else.
My Chinese colleagues have heard of it, but never considered it a top-ranked school, and a quick inspection of their CS faculty pages shows a distinct lack of PhDs from top-ranked Chinese or US schools. It's possible their math faculty is amazing, but I think it's more likely that something underhanded is going on...
How many will see the connections between this and our capitalist mode of production? Probably few since modern lit/news is allergic to systemic analysis.
The blatant flaws of capitalism can't be ignored for much longer.
Marxism isnt "lets try something different based on ideas of justice"
Marxism is "Society evolves through general stages: primitive, slave, feudal and capitalist, this is determined by the level production. Capitalism is holding back its current evolution into a society of plenty."
Good luck backing speculation and profit gatekept production.
Maybe it's the scientists they don't trust?
It's a bit like how can we trust online shopping if I get all these emails trying to sell me aphrodisiac pills?
(it's very funny to pretend tech is still, or really much ever was, one big libfest. it's funnier to say this here of all places)
Science is good, but it's mediated via corruptible humans.
Profits are the deciding factor, not honor.
If it then turns out any of it is fabricated, you should be personally liable for paying it back