Queueing to publish in AI and CS (damaru2.github.io)

I really think the conference model is super detrimental to science. It's not like journals are perfect either, but revise and resubmit and desk rejects are a much better filter than continually resubmitting to the same few conferences over and over again. Not to mention that peer review in conferences is probably much lower quality than what you get in most journals (this is my impression anyhow, I don't know how one could quantify such a thing).

[flagged]

I agree that a) rejecting a paper that has been recommended for acceptance by _all_ reviewers (something that routinely happens in, say, NeurIPS) is nonsense. However, in-person conferences have physical limits. In the case of, again, NeurIPS, you may get accepted and _not_ present the paper to an audience. This is also a bit of a travesty.

The community would be better off working with established journals so that they take reviews from A* conferences as an informal first round, giving authors a clear path to publication. Even though top conferences will always have their appeal, the writing is on the wall that this model is unsustainable.

There's two main problems. First, the field is too saturated -- too many people publishing papers. Second, the reviewers are the same people that publish papers -- it's a zero sum game (regardless of acceptance criteria).

Submission numbers this year have been absolutely crazy. I honestly don't think it can be solved.

If there's more good papers due to more activity, doesn't that allow for the possibility of more conferences and journals to publish to, which increases the acceptance rate?

I don't think that the solution has to be that existing conferences and journals accept more.

[flagged]

My general experience as a PhD student makes me feel that we probably should look at the system as a whole. I get the feeling most universities are abusing publications as a way of assessing their own students. It means people will tend to publish slop for the sake of publication instead of genuinely tackling hard problems. I personally have seen my university get away with forming defence committees with 0 knowledge of the field they're assessing. If the university can't assess its students work then they should not be teaching.

I think both conferences and journals are broken in this regard. It doesn't help that professors primary jobs these days is to be a social media influencer and attract funding. How the funding is used doesn't seem to matter or impact their careers. What we need is more accountability from senior researchers. They should be at the very least assessing their own students work before stamping their name on the work.

On the flip side it isn't untrue that there are major breakthroughs happening daily at this point in many fields. We just don't have the bandwidth to handle all the information overload.

The real issue is careers valuing where papers are published over what they contribute, and incentives won’t change until hiring and funding reward depth over volume.

As pointed out in the comments, conferences have physical limits on the number of presenters (even in poster sessions), so the number of accepted papers at top conferences will likely stop growing at some point. (Perhaps it should have been capped already.) This will probably lead to new conferences appearing to satisfy the "demand", but more conferences of varying quality is probably better than reviewing at top conferences being random at best.

I just witnessed LLM (ab)use coming from one graduate student (not the first to do it and definitely not the last), where they submitted a conference paper draft for their coauthors and advisors to review short before a deadline, with completely regurgitated material plus hallucinations backed up by multiple non-existent citations.

The problem is every coauthor wants to increase submissions, LLMs are great at making something that looks OK at first glance, and people have low(er) expectations for a conference paper. A recipe for disaster.

Extrapolate a bit and there are LLM written papers being peer reviewed by LLMs, but fear it not, even if they are accepted the will not be cited because LLMs are hallucinating citations that better support their arguments! And then there is the poor researcher, just a beginner, writing a draft, simple but honest material getting lost in all this line noise, or worse out, feeding it.

So research is also now a race to the bottom? Come join me at my private research coop, where we emphasize quality over quantity and don't give a shit about whats perceived as cool...

Honest question: why not charge a fee per submission or per review?

Or if the problem is bad papers, a fee that is returned unless it’s a universal strong reject.

Or if you don’t want to miss the best papers, a fee only for resubmitted papers?

Or a fee that is returned if your paper is strong accept?

Or a fee that is returned if your paper is accepted.

There’s some model that has to be fair (not a financial burden to those writing good papers) and will limit the rate of submissions.

Thoughts?

Papers tend to roll down hill. Failure to accept by the big conferences pushes them to lower tier venues.