> Now people barely bring it up at all. It’s like a lion has escaped the zoo and it’s gulping down schoolchildren, but when people suggest zoo improvements, all the agenda items are like, “We should add another Dippin’ Dots kiosk”. If you bring up the loose tiger, everyone gets annoyed at you, like “Of course, no one likes the tiger”.
I have had so many "why don't you just" conversations with academics about this. I know the "why don't you just" guy is such an annoying person to talk to, but I still don't really understand why they don't just.
This article pointed to a few cases where people tried to do the thing, i.e. the pledge taken by individual researchers, and the requirements placed by certain funding channels, and those sound like a solid attempt to do the thing. This shows that people care and are somewhat willing to organise about it.
But the thing I don't understand is why this can't happen at the department level? If you're an influential figure at a top-5 department in your field, you're friends with your counterparts at the other 4. You see them in person every year. You all hate $journal. Why don't you club together and say "why don't we all have moratorium on publishing in $journal for our departments?"
No temptation for individual research groups to violate the pledge. No dependence on individual funding channels to influence the policy. Just, suddenly, $journal isn't the top publication in that field any more?
I'm sure there are lots of varied reasons why this is difficult but fundamentally it seems like the obvious approach?
There are hundreds of reputable research universities around the world. Top-5 departments can't meaningfully change the culture of a field on their own. Top-100 perhaps could, but the coordination problem is much bigger on that level.
I explain here (https://news.ycombinator.com/item?id=47250811) but tl;dr it's because Universities need this system to get money and to give money. Nobody has yet proposed a solution which solves the money/prestige problem. With no money there's no research.
Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it.
It doesn't. Not even close.
Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does.
Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1900's to deal with the fallout from another mistake ("publish or perish") that meant people had to try to publish even if they had nothing to say.
We already have open-access publications: Just put it on arXiV. Most researchers I work with do this already.
The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions.
The current academic enterprise relies heavily on third-party gatekeeping. We rely on others to do the vetting for us. The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
Any gatekeeper will naturally tend towards charging for access over time: It's a captive market, the economics demands it. Unless we eliminate that dependency, we cannot change the system.
I've long wished that "journals" and academic societies would transition from a publishing model to a cultivation model. If everything is available on arXiv, that's great, but it also means the best of the best is mixed in with all the rest.
Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articles are on X topic". The paper's already there on arXiv, you could already read it before. But having a group of editors that cultivate a list of good articles (as well as the peer review process that can, in an ideal world, serve to improve a paper) can serve to make sifting through arXiv less overwhelming, and draw attention to papers in particular subfields, subject matter, or whatever other criteria might be relevant.
In the publishing world, there is this thing called the slush pile: the collection of unsolicited submissions, essentially the only way a person without an agent can break into the field. And you can find quite a few editors' experiences with the slush pile in various blog posts or articles online. And the general reaction goes from naïve wonder at the idea of finding the diamond-in-the-rough to frustration with the quality of the submissions and a realization that the actual game is to figure out how to reject submissions with as little reading as possible (because they don't have the time to do any reading!). This is before LLMs came about, which have made the slush pile problem much worse because they don't improve the quality of the submissions but the increase the amount of reading that needs to be done to reject them.
Academia has the same fundamental problem. We don't actually have the time to read every possible paper someone has for us, because keeping up with literature takes time that we don't have. And while relying on the quality of the journal or conference as a metric for "is this paper worth reading?" has issues, to be honest, it is more effective than other proposed solutions. When I have done the literature searches that delved into the unknown, low-quality tiers of journals... no, those results were not worth the time I spent reading them.
There's also a middle ground, i.e., renowned publishers who aren't free but still publish everything as OA. One example is Dagstuhl Publishing for CS research papers.
I don't think there's really such a thing as a credible citation source. The citation just has to point to a source that hosts the genuine article; the credibility of the article comes from the articles that point to it, and the independent verifiability of the article's claims, not the location that it is hosted.
The other factor preventing a fix is that people with no actual serious experience of academic publishing and peer review will defend these journals, because they still think that (journal-based) peer review acts like some kind of meaningful quality filter. But, it really doesn't.
Because someone is surely going to try to defend journals via peer review in this thread, I want to provide a counter to the arguments that journal peer review does much good. Also, since everyone knows that if you just go to a poor enough journal, you can be published, I am going to focus on the (IMO mostly false) claim that higher-profile journals are still doing a good thing here.
There are numerous studies showing that higher-profile journals in general have more retractions and research misconduct [1-2], lower research quality [3], in fact weaker statistical power and reliability [4], and that statistical reliability even in high prestige journals is still extremely poor overall [5]. Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
In 2014, 49.5% of the papers accepted by the first committee were rejected by the second (with a fairly wide confidence interval as the experiment included only 116 papers). This year, this number was 50.6%. We can also look at the probability that a randomly chosen rejected paper would have been accepted if it were re-reviewed. This number was 14.9% this year, compared to 17.5% in 2014. [7]
We should just move to arXiv-like approaches and allow the scientific community to broadly judge relevance and quality. Journals just slow things down and burn funding for very little gain or benefit to anyone other than the journal owners.
If you're going to write an article titled "The one science reform we can all agree on, but we're too cowardly to do" and that one thing isn't explicitly stated in the first paragraph, I'm out.
Stop with the meandering nonsense and make your argument.
> So the solution here is straightforward: every government grant should ...
People who write such sentences have no idea what they are talking about or are being intentionally naive for whatever reason.
Just because your one-sentence solution reads simple doesn't make the actual solution simple. Because such a solution involves changes to laws, changes to entrenched interests, changes to distribution of money involved in the whole system, and changes to balance of powers between stakeholders. Unless the push for such changes is significant enough to overcome the current state of affairs (due to public opinion, redistribution of power or money, etc.), nothing will happen.
Acknowledging, I am not a expert in this stuff, here is an idea: getting momentum for these sorts of things is so important, what is the journal that would be easiest to make a big example of, so that everyone understands that it is possible? Just completely mercilessly drive them out of business, and then hound their executives when they try to get other jobs. It appeals to peoples base instincts, but the last 10 years have shown those are pretty powerful. Then the movement which has formed around that can take down progressively bigger journals. Probably want a different organization building the alternative; the people with the personality to fight at the Vanguard of the revolution don’t tend to be great at building in the long-term.
Journals are not about providing access to science, much less public access.
Journals are an academic-career-advancement service. It therefore makes sense that they do not pay academics. You don't pay your customers.
That means they need to generate a secondary customer base elsewhere, who will pay. Those secondary customers happen to be the employers of the academics who are the primary customers. That socializes the cost of providing the service, since academics individually wouldn't be willing and able to pay.
Once journals have established a reputation, their policies and paywalls and fees are the result of trying to signal exclusivity and set an optimum market price.
Until the supply side of the research market largely agrees on a way to use open-access repositories like arXiv as a primary career-advancement signal, complaining about closed-access journals is tilting at windmills.
Changing the law to prevent journals from being able to copyright anything could potentially force the research industry to rapidly develop a new solution, but at the cost of short-term chaos and career instability for new academics.
> Robert Maxwell, one of the architects of the for-profit scientific publishing scheme. When he later went into debt, he plundered hundreds of millions of pounds from his employees’ pension funds. You may be familiar with his daughter and lieutenant Ghislaine Maxwell, who went on to have a successful career in child trafficking.
Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.
I don’t understand why people care so much about the cost of journal subscriptions. If we add up all the revenue from all major scientific journal publishers, is that a big number in the context of the national economy? Or even compared to one major tech company?
I feel like this is one of those classic local minima where a community starving for resources fights vociferously amongst itself because they have internalized that they can’t win externally. From where I sit outside academia the problem with science seems obvious: there is not nearly enough money going into it.
I doubt bringing the heads of for-profit journals would change that under current national conditions in the U.S.
Like some other posters here I think that a paid service is probably a necessary evil for long term quality regulation (although currently it skews too much into evil)
This is not how computer science publishing works, however. Post it on arxiv, submit to a conference, get 3 peer reviews, accepted, “published”. 99% of papers are effectively open access for free.
Arxiv isn't the solution. But i think computer science conferences are. These have the same scientific rigour and standards in the review process as journals in other scientific fields, but don't price gouge. Yes, conferences are also a bit expensive, but you get a lot for your money, and they usally aren't out to make a big profit.
Conferences can be truly wonderful, but not a universal replacement for publishing.
If you think journals are expensive, try sending your whole lab to a conference in another country. That may not let you in. Where some of the attendees have to fill out paperwork before talking to a foreign national. (does that ever make for awkward small talk...)
For all their many faults, journals provide access to a really wide audience, and- in theory- make it possible to form connections who wouldn't be able to meet directly.
The peer review process in CS is pretty bad, especially in conferences with a single round of reviews. When you combine this with high rejection rates, peer review becomes more about finding excuses to reject a submission than about trying to improve it.
Conferences also don't work that well as publication venues, as they often require that one of the authors must attend the conference physically. And it's not as much about money than about visa policies and travel restrictions. Even in the 2000s and 2010s, when international travel was easy, people from non-Western countries could often not get visas to attend conferences. And today the situation is much worse.
I've been to three international conferences in the past year. One was held in Europe. People from Russia and Israel had to present remotely, the former due to an ongoing war and the latter due to an unexpected war. Another was in the US, and there were fewer Europeans than usual, as many were not willing to take the risk. And the third one was in Japan. People from China could not attend due to increased tensions. People from Israel were there, but they were worrying if they would make it home before the next war. (They made it, and the war started a day or two later.)
i am very glad to see others (presumably non-scientists) in this thread dunking on the false paradigm that "peer review = true". anyone who peddles this notion is naive or a moron.
while the author is correct that the for-profit publishing is definitely a negative externality, i can't help but feel they are missing the forest for the trees when it comes to all the other worse issues in academia.
a full explanation of which would be much too onerous for a hn comment, but in no particular order: rampant scientific fraud, waste of tax payer dollars, wage suppression via "students" and visa-dependent laborers (J1 visa abuse), publish or perish evaluation criteria, lack of management training, blatant and rampant racism, etc. etc. etc.
the whole system needs to burn down and be rebuilt from the ground up.
Science has never been free, and it isn't mostly progressive; like the bulk of the population, it is hyper-conservative without admitting it. So, the first flaw lies in the very social structure of those who practice science.
The second problem, however, is a modern one: the pure, naked, and raw commercialization of science through "publish or perish", whereby the researcher is a Ford-style assembly line worker to be managed and who must be replaceable.
Without a MENTAL paradigm shift, even before a material one, we will only be able to plug small leaks on a ship with a torn hull.
Worth pointing out a success story: all ACM publications have gone open access starting this year[1]. Papers are now going to be CC licensed, with either the very open CC-BY[2] license or the pretty restrictive (but still better than nothing!) CC-BY-NC-ND[3] license.
Computer science as a discipline has always been relatively open and has had its own norms on publication that are different from most other fields (the top venues are almost always conferences rather than journals, and turn-around times on publications are relatively short), so it isn't a surprise that CS is one of the first areas to embrace open access.
Still, having a single example of how this approach works and how grass-roots efforts by CS researchers led to change in the community is useful to demonstrate that this idea is viable, and to motivate other research communities to follow suit.
That works nicely if your institution participates in ACM Open (no such institution in my country, and no, my country is not in the list of lower-middle income countries).
The combination of 'publish or perish' with 'pay for publication' and 'miserly grant money' is deadly.
While in theory the idea is nice, in practice this is a problem (maybe not in most rich countries, but here definitely).
Nowadays, you could always get the article you are interested in, even if it is beyond a paywall. Hence, perversely, the old model (which I hate, for reasons well explained in the original post) worked better for me. :-(
> Academia is so cutthroat that anyone who righteously gives up an advantage will be outcompeted by someone who has fewer scruples. What we have here is a collective action problem.
And what, pray tell, is this advantage? If there is no utility to anyone in publishing in Science or Nature then how can it be an advantage.
I suspect it’s simply that these guys are a curation service. They separate the cranks from the science. They can be imperfect at this so long as important people separate the cranks from the science.
This kind of winnowing is pretty useful in general. Many universities are pretty much that and people pay to attend them.
It makes sense that a credentialing service would charge for the credential. It doesn’t make that much sense to say “no credentials allowed; you and timecube guy must be considered the same”. I want you to show your credentials. We all do because science is an empirical field and empiricism depends on facts. I cannot process your paper with pure reason.
If you looked under the microscope and saw light I don’t know that you didn’t. At scale I need someone to figure out “this wasn’t a Photoshop situation; that’s totally fluorescence”. Arguing that we should remove these organizations is similar to saying you should remove diplomas and so on. Anyone can continue to attend universities. They just can’t hand out diplomas. So no credentials. Only learning. Simple thing. Or perhaps not so.
There should be a journal where it only publishes studies that have been replicated. Too much research slop is being generated for journals and we already know we have a severe reproducibility problem in science right now.
46 comments
[ 3.1 ms ] story [ 70.3 ms ] threadThis article pointed to a few cases where people tried to do the thing, i.e. the pledge taken by individual researchers, and the requirements placed by certain funding channels, and those sound like a solid attempt to do the thing. This shows that people care and are somewhat willing to organise about it.
But the thing I don't understand is why this can't happen at the department level? If you're an influential figure at a top-5 department in your field, you're friends with your counterparts at the other 4. You see them in person every year. You all hate $journal. Why don't you club together and say "why don't we all have moratorium on publishing in $journal for our departments?"
No temptation for individual research groups to violate the pledge. No dependence on individual funding channels to influence the policy. Just, suddenly, $journal isn't the top publication in that field any more?
I'm sure there are lots of varied reasons why this is difficult but fundamentally it seems like the obvious approach?
It doesn't. Not even close.
Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does.
Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1900's to deal with the fallout from another mistake ("publish or perish") that meant people had to try to publish even if they had nothing to say.
The problem isn't access, it's citations. arXiV is not considered a credible citation source since anyone can publish anything. TPCs don't use it in their list of citations, neither do grant funding agencies or government institutions.
The current academic enterprise relies heavily on third-party gatekeeping. We rely on others to do the vetting for us. The first thing an academic does is check where a paper is published, before even reading it. It's a crutch.
Any gatekeeper will naturally tend towards charging for access over time: It's a captive market, the economics demands it. Unless we eliminate that dependency, we cannot change the system.
Journals (in the sense of whoever is on the editorial board) don't need to cease to exist; they just need to transition to "here's our list this month of what the best new articles are on X topic". The paper's already there on arXiv, you could already read it before. But having a group of editors that cultivate a list of good articles (as well as the peer review process that can, in an ideal world, serve to improve a paper) can serve to make sifting through arXiv less overwhelming, and draw attention to papers in particular subfields, subject matter, or whatever other criteria might be relevant.
Academia has the same fundamental problem. We don't actually have the time to read every possible paper someone has for us, because keeping up with literature takes time that we don't have. And while relying on the quality of the journal or conference as a metric for "is this paper worth reading?" has issues, to be honest, it is more effective than other proposed solutions. When I have done the literature searches that delved into the unknown, low-quality tiers of journals... no, those results were not worth the time I spent reading them.
Then you get the private branded badge social proof and access can continue.
Also, til anyone can publish to arxiv.org?
Any crank who learned to use LaTeX is not allowed to post articles willy-nilly. You need endorsements in the field.
Because someone is surely going to try to defend journals via peer review in this thread, I want to provide a counter to the arguments that journal peer review does much good. Also, since everyone knows that if you just go to a poor enough journal, you can be published, I am going to focus on the (IMO mostly false) claim that higher-profile journals are still doing a good thing here.
There are numerous studies showing that higher-profile journals in general have more retractions and research misconduct [1-2], lower research quality [3], in fact weaker statistical power and reliability [4], and that statistical reliability even in high prestige journals is still extremely poor overall [5]. Also, making it through peer review is highly random and dependent on who you get as a reviewer [6], or is just basically a coin toss even when looking at reviewer groups:
We should just move to arXiv-like approaches and allow the scientific community to broadly judge relevance and quality. Journals just slow things down and burn funding for very little gain or benefit to anyone other than the journal owners.[1] https://pmc.ncbi.nlm.nih.gov/articles/PMC3187237/
[2] https://www.pnas.org/doi/10.1073/pnas.1212247109
[3] https://pmc.ncbi.nlm.nih.gov/articles/PMC9382220/
[4] https://journals.plos.org/plosbiology/article?id=10.1371%2Fj...
[5] https://www.frontiersin.org/journals/human-neuroscience/arti...
[6] https://journals.plos.org/plosone/article?id=10.1371%2Fjourn...
[7] https://blog.neurips.cc/2021/12/08/the-neurips-2021-consiste...
If you're going to write an article titled "The one science reform we can all agree on, but we're too cowardly to do" and that one thing isn't explicitly stated in the first paragraph, I'm out.
Stop with the meandering nonsense and make your argument.
People who write such sentences have no idea what they are talking about or are being intentionally naive for whatever reason.
Just because your one-sentence solution reads simple doesn't make the actual solution simple. Because such a solution involves changes to laws, changes to entrenched interests, changes to distribution of money involved in the whole system, and changes to balance of powers between stakeholders. Unless the push for such changes is significant enough to overcome the current state of affairs (due to public opinion, redistribution of power or money, etc.), nothing will happen.
Journals are an academic-career-advancement service. It therefore makes sense that they do not pay academics. You don't pay your customers.
That means they need to generate a secondary customer base elsewhere, who will pay. Those secondary customers happen to be the employers of the academics who are the primary customers. That socializes the cost of providing the service, since academics individually wouldn't be willing and able to pay.
Once journals have established a reputation, their policies and paywalls and fees are the result of trying to signal exclusivity and set an optimum market price.
Until the supply side of the research market largely agrees on a way to use open-access repositories like arXiv as a primary career-advancement signal, complaining about closed-access journals is tilting at windmills.
Changing the law to prevent journals from being able to copyright anything could potentially force the research industry to rapidly develop a new solution, but at the cost of short-term chaos and career instability for new academics.
Wow! Surprised that hasn't been mentioned here already. Jumped out to me immediately as a morbidly curious bit of trivia.
I feel like this is one of those classic local minima where a community starving for resources fights vociferously amongst itself because they have internalized that they can’t win externally. From where I sit outside academia the problem with science seems obvious: there is not nearly enough money going into it.
I doubt bringing the heads of for-profit journals would change that under current national conditions in the U.S.
If you think journals are expensive, try sending your whole lab to a conference in another country. That may not let you in. Where some of the attendees have to fill out paperwork before talking to a foreign national. (does that ever make for awkward small talk...)
For all their many faults, journals provide access to a really wide audience, and- in theory- make it possible to form connections who wouldn't be able to meet directly.
Conferences also don't work that well as publication venues, as they often require that one of the authors must attend the conference physically. And it's not as much about money than about visa policies and travel restrictions. Even in the 2000s and 2010s, when international travel was easy, people from non-Western countries could often not get visas to attend conferences. And today the situation is much worse.
I've been to three international conferences in the past year. One was held in Europe. People from Russia and Israel had to present remotely, the former due to an ongoing war and the latter due to an unexpected war. Another was in the US, and there were fewer Europeans than usual, as many were not willing to take the risk. And the third one was in Japan. People from China could not attend due to increased tensions. People from Israel were there, but they were worrying if they would make it home before the next war. (They made it, and the war started a day or two later.)
while the author is correct that the for-profit publishing is definitely a negative externality, i can't help but feel they are missing the forest for the trees when it comes to all the other worse issues in academia.
a full explanation of which would be much too onerous for a hn comment, but in no particular order: rampant scientific fraud, waste of tax payer dollars, wage suppression via "students" and visa-dependent laborers (J1 visa abuse), publish or perish evaluation criteria, lack of management training, blatant and rampant racism, etc. etc. etc.
the whole system needs to burn down and be rebuilt from the ground up.
The second problem, however, is a modern one: the pure, naked, and raw commercialization of science through "publish or perish", whereby the researcher is a Ford-style assembly line worker to be managed and who must be replaceable.
Without a MENTAL paradigm shift, even before a material one, we will only be able to plug small leaks on a ship with a torn hull.
Computer science as a discipline has always been relatively open and has had its own norms on publication that are different from most other fields (the top venues are almost always conferences rather than journals, and turn-around times on publications are relatively short), so it isn't a surprise that CS is one of the first areas to embrace open access.
Still, having a single example of how this approach works and how grass-roots efforts by CS researchers led to change in the community is useful to demonstrate that this idea is viable, and to motivate other research communities to follow suit.
[1]: https://authors.acm.org/open-access/acm-open-for-authors-hom...
[2]: https://creativecommons.org/licenses/by/4.0/
[3]: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
The combination of 'publish or perish' with 'pay for publication' and 'miserly grant money' is deadly.
While in theory the idea is nice, in practice this is a problem (maybe not in most rich countries, but here definitely).
Nowadays, you could always get the article you are interested in, even if it is beyond a paywall. Hence, perversely, the old model (which I hate, for reasons well explained in the original post) worked better for me. :-(
And what, pray tell, is this advantage? If there is no utility to anyone in publishing in Science or Nature then how can it be an advantage.
I suspect it’s simply that these guys are a curation service. They separate the cranks from the science. They can be imperfect at this so long as important people separate the cranks from the science.
This kind of winnowing is pretty useful in general. Many universities are pretty much that and people pay to attend them.
It makes sense that a credentialing service would charge for the credential. It doesn’t make that much sense to say “no credentials allowed; you and timecube guy must be considered the same”. I want you to show your credentials. We all do because science is an empirical field and empiricism depends on facts. I cannot process your paper with pure reason.
If you looked under the microscope and saw light I don’t know that you didn’t. At scale I need someone to figure out “this wasn’t a Photoshop situation; that’s totally fluorescence”. Arguing that we should remove these organizations is similar to saying you should remove diplomas and so on. Anyone can continue to attend universities. They just can’t hand out diplomas. So no credentials. Only learning. Simple thing. Or perhaps not so.
The credential is the useful thing.
See also from the same author:
* https://www.experimental-history.com/p/the-rise-and-fall-of-...
* https://www.experimental-history.com/p/the-dance-of-the-nake...
* https://www.experimental-history.com/p/lets-build-a-fleet-an...
1) pay reviewers. 2) you can't publish unless a reviewer replicates your work.
yes. It can be done.
https://www.orgsyn.org/