Engineering and science have an advantage, however, in that they are empirical and readily tested. Things evaluate to true or false, even if doing the evaluation is difficult. In the humanities, things often come down to value judgements and mere assertions, which cannot be validated one way or another.
Plenty of people try to replicate, in trying to build on work from other labs. What they don’t do is publish when replication fails, but the failure to replicate usually becomes known in the community.
The 2nd part of academia that isn't being talked about here is citations. Getting published can be huge, but if your paper is actually garbage no one is going to cite it. That's a far better measure of the impact of a paper - who's using the data in it.
There are much more egregious examples in computer science publishing. A paper that just repeated the words "get me off your fucking mailing list" over and over [1] was published in a journal. I also remember there being some automatic paper generators that resulted in several publications several years back. This is 100% an issue with bogus for-profit publishing, not with any particular area of study.
I find it phenomenally interesting that people simply say what they want to be true with no consideration for whether it is or not. We are indeed moving into a post-fact world aren't we? [1] Here is the wiki page on the 'get me off your mailing list' paper. Here [2] is the page on the journal that chose to accept (after revisions) the Mein Kampf paper. You might notice certain differences. The other journals that accepted the fake papers are part of the T&F/Springer publishing networks, and some responded in this [3] article. They were not scam journals by any means of the word, just journals blinded by ideological bias.
It's my understanding that the hoax papers in the article were also published by predatory scam journals. Even the "get me off your fucking mailing list" paper went through peer review.
> “They told me to add some more recent references and do a bit of reformatting,” he said.
> It's my understanding that the hoax papers in the article were also published by predatory scam journals.
If the hoax papers were published in "predatory scam journals," they're engaged in an entirely different type of predatory scam than the "get me off your fucking mailing list" computer science journal. Sort of like the difference between fake news and herbal Viagra spam/scams.
> Even the "get me off your fucking mailing list" paper went through peer review.
>> “They told me to add some more recent references and do a bit of reformatting,” he said.
Obviously not any peer review process that deserves the name. That must have been an entirely automatic (or even automated) bit of feedback.
The papers in the article were published in journals that will publish anything for a fee. The "get me off your fucking mailing list" paper was also peer reviewed. The reviewer asked for formatting changes and citations, very similar to the reviews that these other hoax papers got.
"Gender, Place and Culture" ( https://www.tandfonline.com/toc/cgpc20/current ) has an impact factor of 1.594. No idea whether that's enough to make it a top journal in feminist geography or not.
The retraction notice for the "Dog Park" paper is currently the most recent publication on that site.
We approached this new effort by asking two central questions: Are we correct in our claim that highly regarded peer-reviewed journals in gender studies and related fields will publish obvious hoaxes? (...) And, if not, what will they publish?
We set out with three basic rules: (1) we’ll focus almost exclusively upon ranked peer-reviewed journals in the field, the higher the better and at the top of their subdisciplines whenever possible; (2) we will not pay to publish any paper (...)
The back story I heard is that the authors submitted to the top journals, were rejected, and kept going downward in perceived impact until the article was accepted. That is, that most of the articles (with perhaps one exception?) did not appear in the close-to-top journals. And they had many papers in the works at one time, and some of them did not get in at all despite the above "just-good-enough" optimization technique.
Although I kind of enjoyed the original Sokal paper - I was a subscriber to Lingua Franca, where Sokal announced the hoax in the first place - I think the message of this iteration is more that: "if you work hard to game the system, you can indeed find a hole and game the system". This finding is not that interesting.
"Three scholars [...] spent 10 months writing 20 hoax papers [...] and submitted them to "the best journals in the relevant fields." Of the 20, seven papers were accepted, four were published online, and three were in process when the authors [stopped].
The real meat depends on the publisher. There are always publication mills that accept basically anything or push objectively wrong agendas like AIDS denialism. It is when the bellwethers publish absolute crap that it is a sign of true rot in the field and not just a publication.
> I am slightly cautious that this hoax is itself a hoax and no actual hoax papers were published by actual academic journals.
This is apparently the original account of the hoax. They had to call it off prematurely because they started getting media attention from places like the Wall Street Journal. I think the hoax is real.
I think there is a real disconnect between what peer review is and what the public seems to think peer review is.
The public seems to think that peer review indicates that something is true, but that's clearly not the case because we have no way of verifying any of the data used, etc.
The paper that got published in a top journal "Human Reaction to Rape Culture and Queer Performativity at Urban Dog Parks in Portland, Oregon", while silly, doesn't seem like it has to be rejected if it were an honest paper that actually did the experiments they said they did. It seems no more silly than the lobster nonsense Jordan Peterson peddles and conservatives eat up. I don't personally think it's a useful paper, but most individual papers rarely are, even in CS.
You mean this paper by E. A. Kravitz at Harvard Medical School Department of Neurobiology that was published in the Journal of Comparative Physiology in 1999?
Serotonin and aggression: insights gained from a lobster model system and speculations on the role of amine neurons in a complex behavior [pdf]
I wonder if many of the people so acidly contemptuous of Peterson have ever bothered to actually read what he actually writes, or just viscerally react to the hot take media coverage. He's eminently reasonable, not an alt-right troll like Alex Jones or Ben Shapiro.
Peterson was a Harvard professor from 1993–1998 so he was there at the time this paper was being written. Now that he's at Toronto, I'm curious to know how much interaction he has with Geoffrey Hinton. He's made references to research related to the AI embodiment problem, and one of his business partners is Daniel Higgins who was at MIT.
I see him as more of a Ben Carson figure personally (a legitimate expert in his field that often answers questions about things he knows nothing about), but I agree that it's a little disappointing when his critics make fun of something where he's actually knowledgable.
The methodology section of the article says that one person spent almost an entire year sitting in a dog park for up to seven hours a day (for a total of 1,000 hours). During this time, that person allegedly inspected up to 10,000 dog genitalia, or approximately one every six minutes. Even if the alleged institution was real (which it wasn't - they didn't even put up a website until after the article was submitted), the author had valid credentials (which they didn't), or someone knew of them personally (academia is a small world) there were enough red flags that the reviewers should have questioned the article's validity.
> Even if the alleged institution was real (which it wasn't - they didn't even put up a website until after the article was submitted), the author had valid credentials (which they didn't), or someone knew of them personally (academia is a small world)
Not sure about these journals, but a lot of peer review is blind, so the reviewers do not have access to any of this information intentionally.
I think the most ridiculous example was the one where they advocated for "privileged students" (ie: white men) to sit on the floor chained in classes and not be allowed to speak, so that they can "experience reparations":
> I wonder if that ‘progressive stack’ in the news could be written into a paper that says white males in college shouldn’t be allowed to speak in class (or have their emails answered by the instructor), and, for good measure, be asked to sit in the floor in chains so they can ‘experience reparations. That was our “Progressive Stack” paper. The answer seems to be yes, and feminist philosophy titan Hypatia has been surprisingly warm to it.
That was mentioned in the Atlantic article, though they didn't mention that the journal rejected it conditionally because the proposed "reparations" of sitting chained on the floor and being spoken over weren't harsh enough for privileged students -- the author was offered acceptance upon revision.
> Poe's law is an adage of Internet culture stating that, without a clear indicator of the author's intent, it is impossible to create a parody of extreme views so obviously exaggerated that it cannot be mistaken by some readers for a sincere expression of the parodied views.
I haven't read Mine Kampf and I don't like Nazis or Socialism for that matter. But are you certain that every single chapter of that book is 100% genocidal rubbish? Because I'm guessing that out of the ~15 or so chapters there is probably one that is palatable.
The German's didn't go in to WWII because they were just terrible people. They had a very long list of quite justifiable grievances after WWI. If they hadn't been proud supporters of eugenics they'd be just another entry in a long list of bloodthirsty European states.
EDIT So here is a selected paragraph (from 3 random candidates) from said text. It isn't exactly anti-factual.
"Thus, from a certain vanity, which is always a cousin of stupidity, the great mass of politicians will keep far removed from all really weighty plans for the future, in order not to lose the momentary sympathy of the great mob. The success and significance of such a politician lie then exclusively in the present, and do not exist for posterity. But small minds are little troubled by this; they are content."
There are WAY too many compelling thinkers in the world to read to justify any of my time sifting through Hitler's writings hoping to find some nuggets that are not "100% genocidal rubbish".
I'm imagine he advocated for exercise and a good diet at some point. Why we need to try to find something useful the man said is utterly beyond me.
The claim is that a journal published a rewritten chapter of Mein Kampf and that journal is therefore bad.
I don't think that is a reasonable argument. It depends on what the chapter was talking about. If it made a big point about Jewish conspiracies and keeping the American people ethnically pure then that is a problem. If it was 1,000 words complaining about vision-less politicians then what is the issue?
If we are talking in the context of submitting it as a paper to a journal, then a problem is plagiarism. If we are talking about "does hitler have some non-genocidal ideas?" then like I said, not sure I care. "How was hitler so persuasive?" Interesting. "How did hitler outmanuever his opposition domestic and foreign" . Interesting.
But yes you have a fair point that just because Hilter said X doesn't automatically invalidate X. The old "the nazis drank milk so dont drink milk" fallacy
If you replace all mentions of Germans, Marxists, Jews and any other related details by appropriate counterparts, you can probably fit the template to any "us vs. them" narrative you want.
Of course the aversion to Mein Kampf is mostly due to Hitler's choice of "us" and "them" (EDIT: and the means used to fight "them", but they are not the focus of that chapter, as far as I can tell), so by rewriting the chapter they have also removed the reasons why it's taboo.
But most people who hear about this story are unlikely to consider whether the essence of Nazism is preserved in the rewritten version and will simply associate it with "feminazis".
Somewhere around that. There is an "Angst" in society, deep inside we all know that there isn't a sunny place for everyone, while in the surface we pretend we can do it like the Baby Boomers. Making people SJWing around dog rape is a dummy load that keep them from demanding structural changes.
Is there a full list of journals that published these?
This sort of hoax seems to happen every few years, but often to journals that will explicitly accept anything and let anyone who wants to give a talk, so long as it's related to the journal's mission (which means there's nothing alarmist about it).
A common response to this thread is "the review process must not have looked closely enough".
This tweet[1] contains screenshots of the review process of the dog park paper, and claims they received constructive feedback. As originally written, the journal wouldn't publish it.
Edit: Hah — interestingly enough, The Atlantic refers to this as "the new Sokal Hoax" in the <title> tag and the URL, but as "an Audacious Hoax" in the page proper.
Journals are broken. A lot of people recognize that.
Peer review is also broken. A lot of people recognize that.
The fact that we don't publish negative results is a tragedy. We're losing so much collective knowledge that way. We learn by failing, but somehow we're ashamed to fail in public in academia.
I personally hope that arXiv and things like it keep taking off. And that we end up funding respected people to review articles publicly. And that we end up funding people to try to replicate interesting results, and publish their findings.
And that the news media learns to explain to people what p scores are, and what p-hacking is, and what the half-life of knowledge is, and that one new research article with p=0.05 doesn't mean jack squat. Basically the news media shouldn't cover leading research, it's just too noisy and it's difficult to understand the context and relevance of new research.
I think that's a good summary. My favorite relevant quote from Dr. John Ioannidis:
> “Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”
Other means superior to the natural sciences exist to extract alternative knowledges about stars and enriching astronomy ... incorporation of mythological narratives and modern feminist analysis of them, feminist interpretative dance (especially with regard to the movements of the stars and their astrological significance), and direct application of feminist and postcolonial discourses
Let us see some absolute gibberish paper get published in a solid SCIENCE journal (e.g. Impact Factor > 5) before we all go "PEER REVIEW IS BROKEN ARGHH!!"
86 comments
[ 0.20 ms ] story [ 136 ms ] threadHowever, lots of junk is also published in science and engineering, it's just not as egregious and apparent.
published in 1958 when he was editor of the IRE (now IEEE) Transactions on Information Theory.
Gender and Identity Studies?
[1] http://www.scs.stanford.edu/~dm/home/papers/remove.pdf
Which is why they were spamming people.
[1] - https://en.wikipedia.org/wiki/International_Journal_of_Advan...
[2] - https://en.wikipedia.org/wiki/Affilia
[3] - https://reason.com/blog/2018/10/03/dog-rape-hoax-papers-pluc...
https://www.theguardian.com/australia-news/2014/nov/25/journ...
These were "real" publication, with peer- and/or editorial-review. Not the same thing by a long shot.
> “They told me to add some more recent references and do a bit of reformatting,” he said.
If the hoax papers were published in "predatory scam journals," they're engaged in an entirely different type of predatory scam than the "get me off your fucking mailing list" computer science journal. Sort of like the difference between fake news and herbal Viagra spam/scams.
> Even the "get me off your fucking mailing list" paper went through peer review.
>> “They told me to add some more recent references and do a bit of reformatting,” he said.
Obviously not any peer review process that deserves the name. That must have been an entirely automatic (or even automated) bit of feedback.
That’s not a peer reviewed journal — it’s a publication that will publish anything for a fee.
All of the papers in the original article are peer reviewed.
The retraction notice for the "Dog Park" paper is currently the most recent publication on that site.
We set out with three basic rules: (1) we’ll focus almost exclusively upon ranked peer-reviewed journals in the field, the higher the better and at the top of their subdisciplines whenever possible; (2) we will not pay to publish any paper (...)
https://areomagazine.com/2018/10/02/academic-grievance-studi...
Although I kind of enjoyed the original Sokal paper - I was a subscriber to Lingua Franca, where Sokal announced the hoax in the first place - I think the message of this iteration is more that: "if you work hard to game the system, you can indeed find a hole and game the system". This finding is not that interesting.
---
Specifically (https://www.chronicle.com/article/Sokal-Squared-Is-Huge/2447...):
"Three scholars [...] spent 10 months writing 20 hoax papers [...] and submitted them to "the best journals in the relevant fields." Of the 20, seven papers were accepted, four were published online, and three were in process when the authors [stopped].
https://www.tandfonline.com/doi/abs/10.1080/0966369X.2018.14...
https://en.wikipedia.org/wiki/Gender,_Place_%26_Culture
Nature, one of the most impactful, is 40.
https://en.wikipedia.org/wiki/Impact_factor
This is apparently the original account of the hoax. They had to call it off prematurely because they started getting media attention from places like the Wall Street Journal. I think the hoax is real.
The public seems to think that peer review indicates that something is true, but that's clearly not the case because we have no way of verifying any of the data used, etc.
The paper that got published in a top journal "Human Reaction to Rape Culture and Queer Performativity at Urban Dog Parks in Portland, Oregon", while silly, doesn't seem like it has to be rejected if it were an honest paper that actually did the experiments they said they did. It seems no more silly than the lobster nonsense Jordan Peterson peddles and conservatives eat up. I don't personally think it's a useful paper, but most individual papers rarely are, even in CS.
Serotonin and aggression: insights gained from a lobster model system and speculations on the role of amine neurons in a complex behavior [pdf]
https://caspar.bgsu.edu/~huberlab/private/KingSolPub.pdf
Actually that's based on real science. It's not social studies...
Not sure about these journals, but a lot of peer review is blind, so the reviewers do not have access to any of this information intentionally.
https://www.google.com/amp/s/www.wsj.com/amp/articles/fake-n...
> I wonder if that ‘progressive stack’ in the news could be written into a paper that says white males in college shouldn’t be allowed to speak in class (or have their emails answered by the instructor), and, for good measure, be asked to sit in the floor in chains so they can ‘experience reparations. That was our “Progressive Stack” paper. The answer seems to be yes, and feminist philosophy titan Hypatia has been surprisingly warm to it.
https://quillette.com/2018/10/01/the-grievance-studies-scand...
> Poe's law is an adage of Internet culture stating that, without a clear indicator of the author's intent, it is impossible to create a parody of extreme views so obviously exaggerated that it cannot be mistaken by some readers for a sincere expression of the parodied views.
The German's didn't go in to WWII because they were just terrible people. They had a very long list of quite justifiable grievances after WWI. If they hadn't been proud supporters of eugenics they'd be just another entry in a long list of bloodthirsty European states.
EDIT So here is a selected paragraph (from 3 random candidates) from said text. It isn't exactly anti-factual.
"Thus, from a certain vanity, which is always a cousin of stupidity, the great mass of politicians will keep far removed from all really weighty plans for the future, in order not to lose the momentary sympathy of the great mob. The success and significance of such a politician lie then exclusively in the present, and do not exist for posterity. But small minds are little troubled by this; they are content."
I'm imagine he advocated for exercise and a good diet at some point. Why we need to try to find something useful the man said is utterly beyond me.
I don't think that is a reasonable argument. It depends on what the chapter was talking about. If it made a big point about Jewish conspiracies and keeping the American people ethnically pure then that is a problem. If it was 1,000 words complaining about vision-less politicians then what is the issue?
But yes you have a fair point that just because Hilter said X doesn't automatically invalidate X. The old "the nazis drank milk so dont drink milk" fallacy
If you replace all mentions of Germans, Marxists, Jews and any other related details by appropriate counterparts, you can probably fit the template to any "us vs. them" narrative you want.
Of course the aversion to Mein Kampf is mostly due to Hitler's choice of "us" and "them" (EDIT: and the means used to fight "them", but they are not the focus of that chapter, as far as I can tell), so by rewriting the chapter they have also removed the reasons why it's taboo.
But most people who hear about this story are unlikely to consider whether the essence of Nazism is preserved in the rewritten version and will simply associate it with "feminazis".
Pushing political agendas and keeping forever-students employed?
This sort of hoax seems to happen every few years, but often to journals that will explicitly accept anything and let anyone who wants to give a talk, so long as it's related to the journal's mission (which means there's nothing alarmist about it).
1) how likely can renowned academic get published with gibberish
2) compare #1 to how likely can an unknown get published with gibberish.
3) how likely can an unknown get a serious paper published when it was actually the renowned academic who wrote it.
4) compare #3 to how likely a renowned academic can get a serious paper published when it was actually the unknown / lesser known peer who wrote it.
This tweet[1] contains screenshots of the review process of the dog park paper, and claims they received constructive feedback. As originally written, the journal wouldn't publish it.
https://twitter.com/RealPeerReview/status/104781375318043852...
Edit: Hah — interestingly enough, The Atlantic refers to this as "the new Sokal Hoax" in the <title> tag and the URL, but as "an Audacious Hoax" in the page proper.
Peer review is also broken. A lot of people recognize that.
The fact that we don't publish negative results is a tragedy. We're losing so much collective knowledge that way. We learn by failing, but somehow we're ashamed to fail in public in academia.
I personally hope that arXiv and things like it keep taking off. And that we end up funding respected people to review articles publicly. And that we end up funding people to try to replicate interesting results, and publish their findings.
And that the news media learns to explain to people what p scores are, and what p-hacking is, and what the half-life of knowledge is, and that one new research article with p=0.05 doesn't mean jack squat. Basically the news media shouldn't cover leading research, it's just too noisy and it's difficult to understand the context and relevance of new research.
> “Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”
https://www.theatlantic.com/magazine/archive/2010/11/lies-da...
I cracked here.