I had such high regard for academia while I was in school. Glad I didn’t end up going down that path, it seems like every day recently there have been stories like this posted to Hacker News. Behavioral economics especially seems hard hit.
Seems like behavioral economics really just receives splash damage from what is the massive fraud at the heart of experimental social psychology. At this point I basically assume any catchy finding from the field is at best HARKed or p-hacked and at worst completely fraudulent.
Pre-registration protects against HARKing and p-hacking, but it unfortunately doesn't protect at all against outright fraud, which we appear to see from the likes of Ariely and Gino.
The field migrated to were the data is. Some communication dark departments within Meta and Google, sponsored by three letters.
You can even do ethical virtual experiments with the data of 3 billion people. Just specify the situation, the person and what's to happen.. Voila. All things happen regularly at some point..
I think it's pretty clear what the post is saying. It doesn't need to be clarified, it needs to be supported, though the nature of the claim is such that it would be tough to provide direct evidence.
Same techniques (largely derived from psychology, statistics, sociology, etc) used in social media marketing, intent tracking, etc has uses in national security. Same thing that might let meta know you might be interested in batman pajamas can be used to sort out if you are being radicalized to action by some group. Interesting example is fractalrabbit.
Group of people who both have the theoretical training and practical experience is probably pretty small so private/public crossover isn't unreasonable.
Me too, then following my Ph.D at a relatively small school went off to a post-doc at a major research university where the grants are millions or you're worthless... That environment essentially forces you to commit fraud so I got out pretty quick
Consider how many studies are being produced each day. That there is a story every day like this suggests that academia is making an effort to identify bad actors, but the frequency of discovery is actually quite low.
Well we don’t know how many people falsify data because, if it goes undiscovered, we don’t know it happened. It could easily be that the rate at which data is falsified is quite low and we are good at catching it, or it is high and we are bad at catching it.
I would think instances of fabrication are quite low — it’s high-risk, medium reward. Even if it goes undiscovered, academia is a marketing game, and you can’t control what other people feel is impactful or not. It’s very easy to do bad science without falsifying data, which is probably the deadliest sin in academia.
> Well we don’t know how many people falsify data because, if it goes undiscovered, we don’t know it happened. It could easily be that the rate at which data is falsified is quite low and we are good at catching it, or it is high and we are bad at catching it.
Yes, which is why I said the frequency of discovery is low. Seeing a story in the paper about fraud every day doesn't really tell us anything about the rate of fraudulence, which I was hoping would become transparent with my point.
To be fair, industry has a similar problem: every day there's a new company that gets in trouble for abusing customer data, losing customer data, abusing customer trust, making it hard to cancel a subscription, etc, etc. People in general tend to be dishonest if they think they can get away with it.
I'm still glad I didn't go into academia for many other reasons, but I don't think I dodged an ethics bullet.
Reputation probably matters more for academia (people go into it because they think the research is important, not for the money, and they think they're standing on the shoulder of giants and not amiable bullshitters taking the credit for findings actually created by their overworked research assistants kludging the data), but if you think academia is full of lies and half truths relative to other professions, you haven't encountered many startups!
I forget the guys name (he’s German iirc), but a very impactful one was at Bell Labs in the 00’s. STEM PhD programs now all have a research ethics component because of him. While you can’t teach someone to be ethical, it at least hammers home how many people you are defrauding by falsifying data, which includes everyone from your funding organizations to the lowly PhD students whose advisors are making them iterate on your study and don’t understand why, after three years, it still isn’t working.
While fields are small, academia is a big place. Most academics won’t falsify data — from a pragmatic perspective, that’s their career on the line, and from an emotional perspective, their academic work is tied to their self-image. It’s also arguably a lot harder to do in the hard sciences but there have still been examples.
In the hard sciences, I feel like the solution is some method to increase how much people at different labs talk to one another (in addition to including all raw data and processing code from experiments). If an impactful article comes out, labs will try to build on it. Not every lab will have the equipment but those in the sub area might. If you’re trying to iterate on a study, and you can’t reproduce the base result before adding your modifications, you might just think you’re dumb or doing something dumb. There’s likely nobody outside maybe your research group who can help you. However, if it was easier to talk to other researchers in the field (like a web forum), you might find out that nobody has been able to get the original experiment working!
In the social sciences, I’m not sure what you do besides running automated checks on submitted data. Humans are bad at generating things that look random (for example, we use more 3s and 7s than nature), and you can probably flag warnings. Either that or include paper trails like a lab notebook in chemistry — which is considered a legal document and thus must be kept in pen. That way you can establish a chain of custody for the data from collection to publication.
> Most academics won’t falsify data — from a pragmatic perspective, that’s their career on the line
In my experience, this is absolutely not true. I mostly only have first-hand observations in the economics, finance, and marketing disciplines. They left me EXTREMELY cynical about all academics.
I have seen:
1) A well-known professor at an Ivy-league B-school openly talking to his class about how the NAR funded his latest study on the real estate market, which (of course) found that the market was not overvalued. This was 2007. The co-author on the paper was the dean of the B-school. He is no longer the dean, but only because he seems to have moved on to more lucrative consulting roles.
2) An older professor, who was one of the first to be in the group labeled "behavioral economics," when asked why he wasn't writing a book focused on more mainstream readers like some other such big names were at the time, STRONGLY hinting if you filled in the blanks although not outright saying that their ethics were questionable, they were all forming hypotheses that were useful for for consulting dollars and dredging up data to support it, and he was too old school (and too old) to be interested in playing that game.
3) A full cohort of marketing grad students (6-7) talking openly about how the system is messed up, and to get a job, you absolutely have to have a good dissertation with a strong results, but to avoid p-hacking allegations, you have to shoot in the dark. So you're left in this situation where you commit financial and career suicide if you DON'T make up data to support your thesis. It is not rare. It is an open secret. No one likes it. They hate it. But they also feel like they are the stupid idiot if they try to fight against it alone. It is how the system works. Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market. Some people do report non-findings, but they are rare. IMO, academic departments should specifically seek out graduates who were willing to report that and stuck by it. But that is not the way the world works. As another poster pointed out, we all like stories, and no significance means no good story to tell.
None of the things you listed are bad or failures of academia.
1) Yeah, lots of people missed the crisis.
2) Yes, when funding doesn't come from the government it comes from industry. And then you need to find hypotheses that are interesting for industry.
3) That's how science works! You don't know what is true or false. You need to take your shot, see if the experiment you run works out. Sometimes you can be the smartest person in the world, put in the most work, but be in an area asking a question that doesn't pan out. It happens!
There are many ways to mitigate this. For example, you can often make experiments where it doesn't matter what the answer is. It's always interesting and publishable regardless of how the statistics work out. But, there is definitely luck involved, not just skill.
> Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market.
Only a scientist would! That's the game we play to discover new truths. And the vast majority of scientists do this.
Pretty much every idea that I have ever had is bad! Yet, somehow, I managed to get a PhD and now graduate many of my own PhD students. The trick is to learn to fail quickly, to ask the kinds of questions that are interesting no matter what, and to develop a sense for the types of questions that are more fertile.
Your take on the first two points is not uncommon and just underscores why academia is broken. The research being produced today is often of zero or negative value.
Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens. It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
> Your take on the first two points is not uncommon and just underscores why academia is broken. The research being produced today is often of zero or negative value.
That's objectively false. Batteries keep getting better. Cancer survival keeps going up. Computers keep getting faster. By almost any metric you pick, research in almost every single discipline is a massive life-changing success! Not just a success, a wild success.
> Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens.
Nonsense. I've been a scientist for two decades at multiple universities. I've collaborated with hundreds of people. Never has anyone tweaked data on anything we've ever worked with. We usually do the opposite, find that something works and work as hard as possible to trash our own work so we can't publish it. We spend more time trying to disprove our own work before publishing it than getting any positive results.
> It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
That's a conspiracy theory just like aliens landing on the moon and vaccines causing autism. Somehow, hundreds of thousands of people work together to stay silent and not leak. People who frankly hate each other's guts and want nothing more to do than to disprove each other's theories and show they're superior. Please, this doesn't stand up to even the most basic scrutiny or thought.
Every scientist that I know wants to do science. Wants to be remembered for contributing to our understanding of the universe hundreds of years after they die. You don't do this by lying, you do it by doing the best work you possibly can.
Something I have seen in academia while I was in school is that the researchers are well-meaning but can't code. They make dumb mistakes which cause data to be misinterpreted. You try to correct them and they look at you like a deer in headlights.
Experimentation in moral psychology without robust, independently verified design is an incredibly slippery slope. This is why certain programs require interdisciplinary approaches (e.g., philosophy, psychology, linguistics, math, physics, neuroscience, biology, history). Otherwise people just make stuff up. And if there's no one to run independent replication studies, and no journals require replication studies prior to publication, then we get... this.
Just wait for people to dive into the false equivalence of scales and measures used in political research. It's very much "make it up and see if it sticks." The goal appears to be Psychology Today, not scientific rigor. I wrote about this in graduate school, as part of my hefty statistics coursework, it bothered me so much.
Wow to say the least. I have Predictably Irrational on my shelf and have highly recommend it in the past. Sad to see the whole field failing the irreproducibility task...
I'm very happy to see this type of scrutiny come to headline research but I really wish it begins at the beginning, with funding agencies and journals. And at the very least the moment a university sends out a breathless PR, the media all scrutinize the data provenance... A boy can wish
For balance, here's Dan Arieli's response (Google-translated from the Hebrew source):
It's ironic that you try to attribute unprofessionalism to me using unprofessional tools. The experiments conducted by Prof. Siniver are not reproductions of experiments I conducted, but rather experiments that are clearly different in many essential differences. A claim for refutation through a completely different experiment is a provocation without a professional basis.
Researchers are required to keep data on studies for 5 years. I do not have data from studies that were carried out 15 years ago or more, and I have no way to find them either. This is not to criticize my work, and any attempt to do so is manipulation of the viewers.
I don't do the research alone. Throughout 30 years of research I had 248 scientific writing partners, among them the most respected researchers in the field. I led 2 labs at MIT and another lab at Duke where hundreds of interns, students and research assistants worked alongside me. You understand that if the claim is that I fabricated studies, you accuse a huge group of academics from dozens of universities in the world, when the only "evidence" for this is that I failed to prove "that I don't have a sister", that is, to provide documentation that I no longer have.
The experiment in which Prof. Amy Droley assisted took place over 15 years ago. I don't know why she remembers the things as she described to you. At first she didn't remember taking part in it at all, and only later did she remember. I and the other two professors who signed off on the study remember the details as they are described in the published article.
In the shredder test we met all MIT conditions. There are many methods to maintain the anonymity of the participants and not just one way as you presented.
There is a difference between academic articles, in which every word and data is carefully chosen, and between a free conversation in front of an audience, in which I convey the main points and do not go into all the details, for the important purpose of making science accessible. Your attempt to "catch" me with one word or another and take it out of context only reflects a desire to slander and nothing else.
Regarding the dentists, these things were said to me and Dr. Janet Schwartz in a face-to-face meeting, by senior company officials, as Dr. Schwartz also confirmed to you. I repeated the things that were said to me in media interviews because they seem important to me. The blog of my institute at Duke University accidentally mixed in an incorrect quote and I asked for it to be corrected.
We recently launched together with Harbor Capital various indices that try to improve the way companies treat their employees and also create value for investors. I forwarded to you the thorough research conducted by one of the best research units in the world, of the investment bank JP Morgan, on the data on which we were based, a study that reached the same conclusions as us.
Prof. Omar Moab holds opinions about many people, but to my understanding he is not an expert in the behavioral sciences and it is not clear why he was specifically invited to give an opinion on my research.
Only once was an error discovered in one of my studies, out of over 150 academic articles. My research partners and I realized that something was wrong with the data, we noticed it and shared the data transparently. I took responsibility for the incident, but I still don't know how it happened.
In my more than 30 years as a scientist, I participated in hundreds of studies, wrote seven books and had the great privilege of helping many organizations improve people's lives. I am proud of my work and contribution.
td;dr: Criticizing me is unprofessional. I don't have any of the data because nobody required me to keep it (I only do the bare minimum that is required of me.) I worked with lots of other people, how dare you tarnish their reputations by criticizing me.
Essentially, Dan Ariely is a well known, oft-interviewed and oft-cited researcher, who has done research on how the environment influences a person to cheat or not cheat. One of his most commonly cited is that having students recall the Ten Commandments before a test reduced cheating. It has not proven to be reproducible, and it now appears that it may have all been a fabrication, which is both disappointing and also deliciously ironic.
How could neuroscience possibly prove (or even provide any evidence in favour of) this? Did they use identify the gene for trust and produce a knock-out human?
Interesting. He claims she collected the data. She claims she did not and could not have. At the least, it appears that it makes sense to have data provenance recorded and signed.
The Latin phrases don't all appear to relate to what's going on. Don't know what's up with that, especially the one from Horace's Odes.
What are ppl even getting so excited about? I mean this is a field which teaches kids to recite - "half the money I spend is wasted; the trouble is I don't know which half".
Any field will have cheaters. Contrary to many posters I happy to see this. It shows that science is self-correcting, as it should be. I'm only disappointed that it's taken so long for many of these allegations to surface.
It is so incredibly easy to fabricate data that looks real and which has provenance going back to "raw data files" or whatever - Just generate random values with the statistical properties of other published data on similar topics. A little more effort if there's cryptographic signing of files (put fake samples into the machine instead, no big deal that's what you do to calibrate the machine in the first place) or hand written questionnaires (actually this could be tricky without an army of complicit grad students). So it's shocking when successful high profile researchers are this sloppy and manage to get away with it for so long.
In fact, it strongly suggests that fabricating data is so common, that it doesn't seem dangerous or risky to them, and they take no great amount of care while doing it. Familiarity breeds contempt. I'm pretty sure the researchers implicated are smart enough to figure out how to fabricate data well, if they wanted to; that they did not bother to do so, suggests a cultural rot.
Familiarity for the researcher breeds contempt. If you succeed once and don’t get caught, it’s easier to do it the second time. It doesn’t require a culture of dishonestly for bad actors to get comfortable breaking rules.
You're correct that I stated that badly. I didn't mean cultural rot as in "everyone does it", I meant cultural rot as in there isn't much perceived chance of being caught, i.e. not much of a culture of double-checking each others' work, so they do it again and again (until they get sloppy). But you're correct that it doesn't mean most people are doing it.
It's a real problem. The whole issue complex has turned me off of using regression analysis in my work completely. If there is data, I just show the data as good as possible. But as long as regression analysis is a useful tool to advance your career (as opposed to research) this will continue.
There’s no reason you can’t do both. Summarizing Tufte: you should always show the data if possible, but you can both show the data and analysis of the data. Don’t throw the baby out with the bath water.
I took Prof Ariely's courses while he taught at MIT - a fantastic professor and well-thought material. However these are serious charges and I wish we exert bit more caution before trashing someone.
The irony here is that everything that surfaced makes him the very person you'd want to be learning about the subject from. He's not being dishonest in being an expert on dishonesty-- he is clearly familiar with the topic if he's being accused of it himself. If true, the man walks the walk, so we'd benefit from listening to him talk. This is one of those "trust but verify" situations. If it can't be verified, do not trust. Ariely will not be the last case of this in our lifetimes.
Much more effective than endless pontificating and theorizing by people who have no idea what the fuck they're doing but haven't publicly fucked up yet so we trust them in reverence of their credentials. The amount of institutionalized deception that has permeated the field of psychology since its inception should scare anybody away from ever taking it seriously.
I've long been suspicious of this study from Ariely from 2003. It suggests that if you "prime" students by having them write down the last 2 digits of their student ID, then students with numbers like "93" will then be willing to pay 3.3x as much for a bottle of wine as students with numbers like "04". At first, some follow-up research seemed to replicate this, but then later work found no effect whatsoever. Very odd. https://justindomke.wordpress.com/2021/10/14/is-anchoring-a-...
Dan Ariely is a fraud. To be specific: he has been credibly accused of fabricating data in at least four separate studies (Ten Commandments, delta dental, Hartford car insurance, and the study which he claimed Rossi ran for him which is discussed at the link). There are likely more.
More broadly: There was a “golden age” for this kind of “behavioral economics”/social psychology work in the early 2000’s up to probably 2012-2015.
Researchers would generally publish “results” which were either “delightfully counterintuitive” (ex: prime subjects to think about Florida and they walk slowly) OR featured extremely cheap interventions which showed large effects on outcomes (ex: signing an honest pledge at the top or the bottom of the page made a big difference in how honest people actually were on their tasks).
In retrospect, there are two problems (at least!) with this literature.
1. There was no theory backing up any of these predictions, so you could rationalize any statistically significant result with a compelling story. Publish it and you’re on your way to tenure.
2. Related to (1.), you didn’t even need to bother running actual experiments at all! Just fabricate the data. Make it fit your cute story and you’re done.
There are many tenured people in business schools (generally in “organizational behavior” or “social psychology” style departments rather than Econ) who were tenured on the basis of papers published under (1.) or (2.).
This is a big shame because there are serious researchers who do difficult work in behavioral and experimental economics which is _not at all_ like this BS. (Some are personal friends of mine, so this bothers me.).
There was no incentive for about 15 years to do the hard work though. And given that tenure is for life, it’ll be a long long time before these people are flushed out (barring more investigations, which should be encouraged!).
What should you do?
- be extremely, extremely skeptical of any results in the “priming” literature (eg the “think of Florida” example above).
- be moderately skeptical of results in the “nudges” literature. The retirement savings stuff makes sense and has replicated but the honesty nudge above is BS and has not.
- don’t get really excited about a researcher who says “economics bad - people dumb! - I behavioralist!” More than a few of these people have been revealed to be total frauds.
- don’t click or share this stuff when you see it on Reddit, HN or anywhere else. That kind of engagement “feeds the beast” - Deans love it and Deans assume you haven’t fabricated your data.
- maybe if you are specifically a Duke or Duke Fuqua alum send them a note asking why they aren’t taking the Ariely misconduct more seriously???
- (edit) throw away your copies of any of Ariely’s books. They are BS.
Just to be clear, are the "credible" charges that he intentionally fabricated data, or that he received data (from assistants/corporations) that was fabricated, but that he didn't catch? Because he wasn't actually collecting the data directly himself, was he? And there's a world of difference between these two things.
You are correct that there’s a world of difference between those two accusations.
Specific example: In the car insurance paper, which was covered by Data Colada, there was agreement among all the four authors that Ariely himself was the only person to touch the data. That data (IIRC) was completely fabricated.
I’m happy to edit this comment and my original one if those details are wrong. I’ll give Ariely a presumption of innocence which he does not deserve and indeed which I suspect he abuses in research integrity investigations.
At the end under “replies” Dan says himself that he was the person in touch with the insurer. It really beggars belief to imagine that there was someone at the insurer who fabricated a dataset which perfectly tells the BS story of this paper.
Also note that another of the four authors (Gino) will soon be fired from HBS for fabricating data in a series of studies.
>First, it is visually and statistically (p=.84) indistinguishable from a uniform distribution ranging from 0 miles to 50,000 miles [5]. Think about what that means. Between Time 1 and Time 2, just as many people drove 40,000 miles as drove 20,000 as drove 10,000 as drove 1,000 as drove 500 miles, etc. [6]. This is not what real data look like, and we can’t think of a plausible benign explanation for it.
It is just remarkable to see how people with enough knowledge of datascience to perpetrate this fraud for so long would also make errors as rudimentary as producing a faked 'random' data set in a uniform distribution. And with zero cars reporting over 50k miles driven. (edit: and so many others...this is worth reading just to see it all, thanks for linking).
These frauds really are not as smart as they think they are. But I guess selling best-selling books about stuff you made up could make you believe you're a genius.
The downfall of Hwang Woo-Suk, the infamous stem cell con artist, started when a whistleblower posted that two photos of stems cells that were supposed to be at different stages were actually cropped from a single photo. You could match them by dragging them in photoshop.
You would imagine a con artist of his caliber would be more careful than that, but still ...
> You would imagine a con artist of his caliber would be more careful than that, but still
They've been doing it for a long time and it's always worked, either because they've always been this sloppy but everyone else reviewing was sloppier or gave the benefit of the doubt for reasons, or that they've put less and less effort into it over time because it hasn't been necessary.
Also the simpler the fraud is in execution the more likely you might be able to explain it away as a simple mistake or lack of familiarity with certain "tradesperson" tools adjacent to the skills you have framed in latin blackletter on sheepskin.
At least for the insurance case it's crystal clear he fabricated the data himself. He is the only person who touched the data in the research team, by the admission of everyone involved. He is the only person with a motive; why would the insurance company fake data?
It's also clear that the data fabrication happened in Excel (based on styling differences that wouldn't pass through other formats) and the Excel metadata has Ariely as the creator of the file. If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators. If it had been fabricated by them as a e.g. a CSV and Ariely just imported it to Excel, it wouldn't show the inconsistent styling.
Just to be pedantic (I am not contesting that this individual was involved in malfeasance or not, just that the enumerated proofs are insufficient at best) -
> If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators.
Digital forensics does not work this way. It is very rare when many digital evidence line up and imply that an actual person did or did not do something in the digital realm. A user account that shows up in an Excel file is absolutely insufficient to tie a person to the keyboard.
> it wouldn't show the inconsistent styling.
This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
Yes, of course I'm not going to retype thousands of words of somebody else's investigation[0] into a HN comment. It's a summary.
> This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
No, sorry, I don't agree. It shows the "when" of the data fabrication, i.e. after the data was imported to Excel not before. The nature of the styling inconsistencies shows the "why", since it shows that the data was duplicated and one half of the duplicates was manipulated. Half the data was in one font, half in the other, and the two sets of data can be matched with each other. It also shows the "who" in combination with the metadata, due to the chain of reasoning in my previous message, since it shows that the data fabrication happened after the import to Excel and we know Ariely is the person who did that import.
Taking a look at the conclusion in the link you provide, it states:
> it is impossible to tell from the data who fabricated it. But because the fourth author has made it clear to us that he was the only author in touch with the insurance company, there are three logical possibilities: the fourth author himself, someone in the fourth author's lab, or someone at the insurance company.
So you seem to be much more confident about it only being one possibility, when the link explicity provides three possibilities.
Yes, the article lists three theoretical possibilities. I expect the datacolada authors are not stating how likely they think each of those theoretical possibilities is, for obvious reasons. I don't feel under such constraints.
One of those logical possibilities can be excluded entirely thanks to everyone involved agreeing that Ariely was the only person in contact with the company and handling the data ingestion. It could not have been anybody else in the lab.
The other theory is beyond absurd. Not only did the insurance company fabricate the data, but they happened to fabricate results that showed the effect Ariely needed for a high profile publication. Theories like your "slacking off data entry clerk" just don't make sense there.
Now, maybe if Ariely could provide even a shred of evidence for this being the data he received, I could change my mind. But he can't produce the file he received from the company. He can't tell who at the company sent it to him. The trail just runs cold at Ariely.
> they happened to fabricate results that showed the effect Ariely needed for a high profile publication
He'd already done lab experiments that showed the effect he thought existed, and it's easy to imagine an associate who actually fabricated the data already knew what he was looking to confirm.
Again, I'm not saying I think he's guilty or not. I'm just saying that from everything I've read so far, it seems plausible it was sloppiness and not checking data for signs of manipulation. That this sloppiness isn't "beyond absurd", but rather more like the classic "never attribute to malice that which is adequately explained by stupidity" (or by an overworked/overambitious academic).
> He'd already done lab experiments that showed the effect he thought existed, and it's easy to imagine the associate at the company knew what he was looking to confirm.
Based on Bazerman's response, actually, he hadn't:
> [The fradulent paper] came together in a merger of two prior non-published empirical efforts. Mazar-Ariely independently provided the data for Study 3,
while Shu-Gino-Bazerman had written a paper containing two laboratory experiments (Studies 1 and 2). The Shu-Gino-Bazerman group knew of the Mazar-Ariely data from multiple public presentations by Ariely.
I think reading the authors' responses is illuminating here.
Mazar's response is basically "oh god, I'm ashamed this happened. I played no role in the data collection, and I don't know who did. But thank you for all the hard work you've done in exposing this fraud, and I should have done better at questioning the data."
Gino's response (for context, she has been implicated in other fabricated data studies) is basically "I'm ashamed. I didn't collect the data, and I trusted the people who did collect it. But thank you for all the hard work you've done in exposing this fraud, I should have done better, and this is what I'm doing in the future to do better."
Ariely's response is "I'm the only one of the authors who had the data. Also, it wasn't me."
Bazerman's response is "Thank you for exposing this fraud. Let me explain how the study came to be, and the history of my own doubts about the quality of this study [which includes two episodes which seem to be red flags]. I should have pushed harder on those doubts [he was in the minority who wanted to retract the paper after nonreplication in the follow up]."
Ariely's response to me feels like raising a preemptive defense against being thrown under the bus. Neither Mazar nor Gino implicate anybody (only disclaim responsibility themselves), whereas Bazerman suggests that it would have had to have been Ariely or Mazar. (There's an interesting bit where Bazerman has an unspecified issue with the [now known to be faked] data prepublication, where one coauthor has unconvincing responses but a later coauthor was able to provide plausible explanations, complete with reference to the underlying data file--but both coauthors are unnamed). Given that, I just don't find the other possibilities all that plausible (and in the case of the middle possibility, it's not exactly exonerative anyways).
I've got other people's names and dates in some of my Office documents metadata. Sometimes it's easier to wipe all but the first two pages out and go from there, than to keep track of other people's templates.
Clearly he had control over the file, but I wouldn't put too much weight on the metadata when there's plenty of other evidence.
It's easy to imagine reasons. A lazy assistant who's supposed to go through 13,488 filled-out forms decides to count 1,000 of them and then fabricate the rest so they can spend the week watching soccer online instead? Or an employee lost the data on which form results were associated with which condition, and they might get fired or not get promoted if caught, so they made it up? And so forth.
And metadata as you've described it doesn't sem like a smoking gun either. It's easy enough to imagine Ariely created an otherwise empty Excel sheet in the column format he wanted them to insert data into, e-mailed it to them, and got it back with their results. Or he just copied their Excel results into a spreadsheet of his own that already had other tabs, or similar.
I'm not saying Ariely didn't do it or did do it, I'm just wondering if there's an actual smoking gun here, or if it's accusations but there isn't enough evidence to actually know.
to some degree it's not relevant -- the data is bunk, and Dan A flogged it hard.
He also was the only one handling it, so if the data is bad then the responsibility falls to him regardless, both as the handler, as well as the head of the investigation. He either deliberately faked it, or got garbage data and didn't realize it when, now, plenty of others have.
In the case linked in the article, Ariely claims to have received the data from a researcher who not only denies doing it but also points out specific things that are implausible about the construction of the study and the description of how the research was done doesn't align with how she'd have done it or normal university policy
Which points towards either him fabricating it himself, or asking somebody else to give him results without going through normal procedures for registering/conducting the experiment and then naming someone else as responsible for it...
I do think there are cases where academics will have unwittingly published falsified or partly false data and sometimes even garnered fame and citations for it x, and no doubt many more where the academics spot issues with the data they've received and bin it, but this doesn't look like one of them.
If reproducible results are the concern, the entire field of psychology should be thrown in the trash. Something like 70% of published papers are irreproducible and serve to push an agenda. “Subjective science” in general belongs in the shitter.
I've just gotten around to Kahneman's "Thinking Fast and Slow" (2011) and it seems a LOT of the discussion in the book relies on non-reproduced or even reversed results in your list. I hope the book changes significantly in the second half, as the first half is essentially barren after reading your FORRT link.
Most of Kahneman's own work has held up quite well to replication, and that makes up a large portion of the book. But a lot of the social psych findings he discusses have not fared well to replication attempts.
Failure to replicate doesn't mean the original findings aren't true or useful. A thousand things besides "the original finding is wrong" can happen to cause a failure of replication.
If you limit your understanding to science that is conclusive, you will paralyze yourself. This is the reality of scientific study; there are exceedingly few conclusions, only more questions, confusions, misunderstandings, etc.
When scientists say we don't "know" anything, this is what they're talking about. Nothing replicates, everything seems more confusing and less clear the deeper you dig, and the further you stray from the "hard" sciences that are easy to experiment, the more true this gets. Psychology, economics, etc. are insanely complex and effectively impossible to test the way we think of a physics or chemistry experiment.
On the contrary, failure to replicate should definitely reduce your posterior probability over the original findings being either true OR useful. True findings should replicate. There are complexities - maybe you can't perfectly recreate the original conditions, especially in non-experimental settings - but broadly speaking replication is the right standard to use.
The presumption here is that it was a true replication, and that's pretty damn hard to accomplish in the soft sciences. It matters, just not nearly as much or as conclusively as in something like physics.
If a result is incredibly hard to replicate, that's a good indicator that the effect is either so niche, or so small, that it's not actually telling you any details of the world worth talking about.
If 'priming' only works every wet Tuesday in February, after someone has had a cheeseburger with no sauce and took their Grandma to the bingo, perhaps there's some _actual_ interesting artefact explaining it...or it could just be random noise.
A lot of the studies involved do replicate, just with reduced effects.
But the presumption here is that the same experiment is even being run a second time, and my point is that many studies who have the stated goal of replication do so in ways that I find hard to justify.
So even when a study claims to "fail to replicate", I don't think we should just take that on face value.
“This is the reality of scientific study; there are exceedingly few conclusions, only more questions, confusions, misunderstandings, etc.“
That goes too far. Some science really does follow the scientific method and documents the methodology well-enough to be replicated.
That’s the basis for giving deference to scientists on some topics — they aren’t just experts, they have subjected their ideas to the cold hard scientific method and others have replicated the results.
Somewhere along the line we demoted science to just be another expertise, where you can just walk around in a lab coat and have social status and people aren’t supposed to question you. Throw in some jargon. No, that’s not science.
They can’t have it both ways. If scientists want deference on important topics, they need to follow the scientific method and not extrapolate too far from established results. No excuses like being “paralyzed” by requiring replicable results.
Thanks. I actually met Kahneman when he spoke at Google, and I'm very interested in not citing any results that couldn't be replicated. Particularly Optimism Bias, at the moment.
Looking at that site, though: if I search on "Ariely" I don't get what I would have hoped, which would be a list of his publications that have been debunked. Instead, there are some generalized articles not about him.
> he has been credibly accused of fabricating data in at least four separate studies (Ten Commandments, delta dental, Hartford car insurance, and the study which he claimed Rossi ran for him which is discussed at the link)
Minor point here, the article says Rossi ran the Ten Commandments study. Not sure if there's a different study you mean?
If you read the linked email exchange with Rossi, it appears that while she may have collected some data for him, the method and process used does not conform with the details included in the paper.
It seems no one can provide receipts (which is bad! due to reproducibility), and since Ariely's name is on the paper and Rossi's is not, the onus is on him to have kept that documentation. He asked that Rossi at least corroborate the details described in the paper, and she refused to do so saying that it didn't align with her memory or her standard procedure at that time.
A fair point: there may be three unique articles referred to in my list there, rather than four. An additional fake paper on which I could have added to that list revolves around an apparently-modified paper shredder.
However (!) there are a bunch of other Ariely articles which are extremely suspicious and which are suspected of being fake. At this point I would say everything he has ever written should be assumed to be faked.
I expect a broader investigation will reveal more fraud. Apparently Duke doesn't care enough but hopefully some renewed attention to the case will force them to take action.
> - don’t click or share this stuff when you see it on Reddit, HN or anywhere else.
I think this point is under appreciated. I also want to add another point
- Don't take a (strong) position on something you don't understand or have equally strong evidence for.
We live in a world of specialization, which means a lot of us "are dumb" with respect to a lot of domains. There's nothing wrong with that. But there's a propensity for society to generalize this intelligence, which makes people think that because they can adequately reason about one domain that they can do so in another domain. It is kinda like the Murry Gell-Mann Amnesia, but to ourselves. Look at a domain you _are_ and expert in, and you'll probably realize that a lot of nuance matters, resulting in many things that would be non-intuitive without that high level of detail. Then ask yourself, do you have sufficient nuanced understanding for this other domain topic? I think we know the answer. Opinions are totally fine to have, but it is the strength of them which matters, and places like Reddit and HN are full of people who have been taught that their intelligence generalizes far better than it does, because what we're comparing ourselves to is other people's ability to reason and rationalize and not the distance our knowledge/reasoning is from the truth. i.e. you can out debate someone while being an absolute walnut.
>Don't take a (strong) position on something you don't understand or have equally strong evidence for.
...or else what?
We all live in a world of specialization certainly, but we also live in a world where we must make decisions with imperfect information. This idea of "only take positions on things you fully understand" is a luxury that isn't practical when you realize how frequently (and rapidly) those decisions must be made.
One thing that matters even more than nuance is execution, and a not-so-great idea executed extremely well is much better than a super well researched and understood idea that didn't get executed at all because the person with it took forever getting to that level of understanding.
There is, as in all things, a middle ground. Take positions, make decisions, but accept you'll be wrong decently often, and that's okay.
I'm not sure how to answer this without sounding rude. But you'll make a fool of yourself if you do, put people at risk, and probably lose money.
Considering the other context of your post, I'm not quite sure where your disagreement is. I specify _strong_ as a qualifier to indicate that the strength of the opinion/position (your confidence) should be proportional to your knowledge/understanding/evidence. I explicitly state that having opinions is perfectly fine and would go even further to say that it is impossible to be absolutely devoid of them. Human brains just make conclusions.
To clarify my point, I'm saying that being an armchair expert is not helping anyone. Having some armchair expertise is perfectly fine, but arguing from that position (especially passionately) is generally unhelpful and just adds noise. These are the common posts that you see where someone responds with a very simple explanation that is rather incomplete. You'll often see these highly upvoted as well and better answers buried simply due to their length. The laziness hurts us all! It is better to build habits and communal acceptability to just shut up a lot of times, as the problem with information retrieval is often about accessibility not availability; noise decreases accessibility due to the additional information we must sort through. If armchair comments are given (and they can be helpful, especially on vague topics or where no other level of expertise is present), then admission often helps the person receiving the information as well as creates the ability for better communication to determine higher quality information as well as better allow a more informed specialist to append or correct the basic answer.
The problem we see is not the existence of these armchair expert answers but rather that they are done so with high levels of confidence and the authors often feel the need to defend them, including against actual experts. Corrections or additions are not a critique to one's self, just the knowledge that was laid forth (which is often anonymous, so identity/self-value should play a lower role, but doesn't). Being wrong is not just okay, but should be welcomed as nerds we pride ourselves on being "smart" which means we pride ourselves on _gaining_ knowledge. Not everything needs to be a debate and honestly it is in general a terrible way to acquire or disseminate information.
I just want you to understand that you're coming across as if you want people not to discuss most things in their lives, because as you say, we're hardly experts on most things.
Being wrong is completely okay, and trying to police what people discuss with one another is a dark road to go down. Besides, where do you draw the line? Do I need a specific number of course hours completed? Some number of books read? Three or more articles published on the topic?
Discussion, particularly in writing, is a form of thinking, and thinking with others is infinitely more productive than thinking alone, so if what you're asking is for folks to stop thinking about topics they're not experts on, I'm sorry but no. That's not going to happen.
> I just want you to understand that you're coming across as if you want people not to discuss most things in their lives, because as you say, we're hardly experts on most things.
Thank you. If you have any suggestions as to how to make my point clearer, I'd greatly appreciate it. I hope it is clear now, but obviously I can always improve. I do want to promote discussions, but that also means wanting to decrease arguments. Especially arguments that are uninformed and ones which has a clear answer (or lack of) from the domain experts.
Where to draw the line of domain expertise is not clear, you're right. Academia is one example of a way to pass that line but I wouldn't say the only means as it is just a method that correlates with sufficient knowledge gain. But I also wouldn't say that watching some YouTube videos does not constitute expertise (more explicitly, videos by science communicators rather than lectures). The boundary is fuzzy, but that should put pressure to reduce confidence (which is good for science and truth seeking) rather than increase it.
Discussion is not only fine, but actively encouraged. We're having one right now and at least on my side I don't feel like this is a argumentative. I tried to clarify this previously but I guess this is difficult. I'm not sure how to make myself more clear so all I can do is ask that you read more carefully with the updated knowledge that there's a substantial amount that I agree with you on (differences are in finer points) and request that you point out specifically where what I said may be confusing or resulted in misinterpretation. Hoping that this can facilitate a better discussion, if you still wish to have one.
I guess I just don't understand your point, if it isn't meant to stifle conversation in a practical sense. Is there any other way you can put that idea, that might make it easier for me to understand?
I'm curious about the mechanical details here. How can a tenured professor at UCLA not be aware that they contributed (not author?!) to a paper cited 3000 times?
So... the mechanical details are that the authors of the paper w/ fake data said that Rossi did the study for them when asked about it.
She wasn't an author, so she had no reason to be involved w/ (for example) the referee process. She was made aware that this was their claim when someone contacted her.
She also works in a slightly different literature IMO. Even if she had read the article, it probably didn't say "the data gathering was done by Prof. Rossi at UCLA"
Seems a bit extreme to throw out a whole set of concepts (nudge, behavioral economics) because one person may have fabricated some data.
How could he have successfully executed this fabrication? Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.? The idea that one man, alone, could pull off such a large deception seems, to me, to be pretty farfetched.
Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
Combine that with the ever-present reproducibility problem in all of psychology, and the economists who were made to look like fools (in their view) in the earlier years of the 21st century are striking back with exactly these kinds of arguments (economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues).
Ariely's books aren't "BS", and throwing that in there kind of shows your hand. Large portions of his books are explanations of the work of others, and even if everything this UCLA professor says is true, that doesn't invalidate a single thing he wrote about in his books, other than perhaps specific citations of his own work (of which we're talking about a tiny handful of hundreds of published and peer reviewed studies he worked on with thousands of other people).
Not sure how that's relevant, unless you're suggesting that because some frauds exist in a given field, therefore all members of that field are frauds?
Astrology is many orders less predictive than psychology...
Huge swaths of behavioral economics have replicated successfully. Care to cite peer-reviewed studies related to astrology that have successfully replicated?
It is not clear to me whether Amy Cuddy was a fraud OR whether she just published a "statistically significant" result without really understanding it or making any attempt to replicate it in a large sample. I lean towards the latter on the basis of reading about her case probably nearly ten years ago.
> Seems a bit extreme to throw out a whole set of concepts (nudge, behavioral economics) because one person may have fabricated some data.
I specifically said don't throw out nudges - some of them work and replicate e.g., retirement savings defaults, while others are BS like the honesty pledge. Real experimental/behavioral economics is not this media-attention-grabbing garbage anyway. As I said, I have friends in the field. They don't do this BS.
> Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.?
No, he doesn't. See the Data Colada responses from his three coauthors on the car insurance paper. All admit he was the only author to have data access. If you can find it, check the earlier, separate investigation into his other fabricated paper w/ Delta Dental where the insurer itself said "we didn't give him that data."
> Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
I disagree on many dimensions w/ your characterization of behavioral econ AND the entire point is irrelevant here.
> economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues
I can introduce you to ten experimental economists who would disagree. Economists have also been pioneers in figuring out when non-experimental situations parallel experiments enough that we can make the same inferences.
> Ariely's books aren't "BS"
Please continue enjoying them if you like them, but I don't have any idea how you'd separate fact from fiction. Ariely is not the first researcher in this exact field who has been revealed to be a fraud.
You've got personal responsibility and liberalism completely backwards. Liberalism is predicated on personal responsibility; people can be given liberty because people can be responsible for themselves. Conservativism rejects personal responsibility and thinks people need to be put under the thumb of paternalistic organizations such as the church or government to keep them in line.
As someone working in this space, I largely agree with your take.
I'll only add that compared to, say, traditional economics we don't have a sense of how unique the problem is to social psych/organizational behavior. In the past decade psychology has done a lot more replication attempts and data auditing than have other fields in the social sciences. So it could be that other fields are equally problematic, but we don't know.
This is a fair point. I would like to see more replications AND better incentives around replications.
FWIW... In plenty of economics papers you can't do a meaningful replication b/c there may be no analogous natural experiment. You can only do analysis of the same data. Such "replications" do not always lead to the same results.
The more experimental parts of the econ literature are now very big on pre-registration (esp. in development, e.g.). Not a panacea by any means, but an improvement for sure.
And there are generally higher standards around making code available now. If you work in this area and read the AER you may be aware of a recent retraction in PF due to coding malpractice. The system has improved but is far from perfect.
When I studied Econ 20 years ago, behavioral economics wasn’t offered. A decade later, I’d read a bunch of books on the topic, including by Ariel’s. I lamented that this field wasn’t offered when I was in college. Now another decade later, I’m glad I wasn’t in school during the period when it was offered, since it seems to be holding up so poorly. I hope that it can have a more rigorous rebirth, and in the next decade produce some results that are more robust (even if they give rise to fewer TED talks).
Academic fraud is massively costly for society as it polluted downstream knowledge. Worse, it justifies doubt in all knowledge-generating efforts.
It’s a shame that the worst that can happen to these frauds is, typically, job loss. The incentive to cheat is way higher than the cost times the likelihood of getting caught. There should be criminal penalties at least as severe as other white collar crimes.
The average "half-life of knowledge" in psychology is already a dismal 7 years (and has been measured as low as 3.3 years). And that's excluding outright fraud...
You seem to be misinterpreting the concept of half-life of knowledge.
Lower values mean the field is advancing quickly. Rapid scientific advancement is a good thing for knowledge generally. It's merely annoying for people in the field because they have to do more work to keep up-to-date. :)
It's not a measure of things being proven wrong, but more about newer better-predictive theories making older theories obsolete.
I think there is an insidious tendency to want to post-hoc rationalize things that are just random noise in the kinds of surveys that Ariely et al were doing. I can imagine over time that it must be tempting for these kinds of people to just start faking stuff.
One of my undergrad psych research projects was focus on testing some of the Kahneman & Tversky anchoring/adjustment stuff. We were able to replicate some things related to the theory itself based around economic estimates, but also had some "control" questions in our surveys (one example: "estimate the average weight of a human brainL). Our experience was that some populations just sucked worse at the kinds of estimation tasks we were using. We had no good theory for gender-based differences in estimating the weight of a brain, and in retrospect it was probably just people confusing ounces with pounds as a half-remembered number for weight... but no one gets published by admitting that their experiments were probably just poorly designed.
132 comments
[ 3.1 ms ] story [ 212 ms ] threadYou can even do ethical virtual experiments with the data of 3 billion people. Just specify the situation, the person and what's to happen.. Voila. All things happen regularly at some point..
You really need to clarify this.
> The government is trying to control your mind through the internet
Group of people who both have the theoretical training and practical experience is probably pretty small so private/public crossover isn't unreasonable.
I would think instances of fabrication are quite low — it’s high-risk, medium reward. Even if it goes undiscovered, academia is a marketing game, and you can’t control what other people feel is impactful or not. It’s very easy to do bad science without falsifying data, which is probably the deadliest sin in academia.
Yes, which is why I said the frequency of discovery is low. Seeing a story in the paper about fraud every day doesn't really tell us anything about the rate of fraudulence, which I was hoping would become transparent with my point.
I'm still glad I didn't go into academia for many other reasons, but I don't think I dodged an ethics bullet.
Reputation probably matters more for academia (people go into it because they think the research is important, not for the money, and they think they're standing on the shoulder of giants and not amiable bullshitters taking the credit for findings actually created by their overworked research assistants kludging the data), but if you think academia is full of lies and half truths relative to other professions, you haven't encountered many startups!
https://retractionwatch.com/
In my experience, this is absolutely not true. I mostly only have first-hand observations in the economics, finance, and marketing disciplines. They left me EXTREMELY cynical about all academics.
I have seen:
1) A well-known professor at an Ivy-league B-school openly talking to his class about how the NAR funded his latest study on the real estate market, which (of course) found that the market was not overvalued. This was 2007. The co-author on the paper was the dean of the B-school. He is no longer the dean, but only because he seems to have moved on to more lucrative consulting roles.
2) An older professor, who was one of the first to be in the group labeled "behavioral economics," when asked why he wasn't writing a book focused on more mainstream readers like some other such big names were at the time, STRONGLY hinting if you filled in the blanks although not outright saying that their ethics were questionable, they were all forming hypotheses that were useful for for consulting dollars and dredging up data to support it, and he was too old school (and too old) to be interested in playing that game.
3) A full cohort of marketing grad students (6-7) talking openly about how the system is messed up, and to get a job, you absolutely have to have a good dissertation with a strong results, but to avoid p-hacking allegations, you have to shoot in the dark. So you're left in this situation where you commit financial and career suicide if you DON'T make up data to support your thesis. It is not rare. It is an open secret. No one likes it. They hate it. But they also feel like they are the stupid idiot if they try to fight against it alone. It is how the system works. Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market. Some people do report non-findings, but they are rare. IMO, academic departments should specifically seek out graduates who were willing to report that and stuck by it. But that is not the way the world works. As another poster pointed out, we all like stories, and no significance means no good story to tell.
1) Yeah, lots of people missed the crisis.
2) Yes, when funding doesn't come from the government it comes from industry. And then you need to find hypotheses that are interesting for industry.
3) That's how science works! You don't know what is true or false. You need to take your shot, see if the experiment you run works out. Sometimes you can be the smartest person in the world, put in the most work, but be in an area asking a question that doesn't pan out. It happens!
There are many ways to mitigate this. For example, you can often make experiments where it doesn't matter what the answer is. It's always interesting and publishable regardless of how the statistics work out. But, there is definitely luck involved, not just skill.
> Only a true saint would accept data that actually does NOT support their hypothesis because it makes it unlikely they will get a job in a very competitive market.
Only a scientist would! That's the game we play to discover new truths. And the vast majority of scientists do this.
Pretty much every idea that I have ever had is bad! Yet, somehow, I managed to get a PhD and now graduate many of my own PhD students. The trick is to learn to fail quickly, to ask the kinds of questions that are interesting no matter what, and to develop a sense for the types of questions that are more fertile.
Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens. It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
That's objectively false. Batteries keep getting better. Cancer survival keeps going up. Computers keep getting faster. By almost any metric you pick, research in almost every single discipline is a massive life-changing success! Not just a success, a wild success.
> Your take on the last point is not so different from mine. I think you misunderstood. The incentive to change data to avoid a finding of no significance is very great. The risks are small. Yes, good science would be to report it accurately. It almost NEVER happens.
Nonsense. I've been a scientist for two decades at multiple universities. I've collaborated with hundreds of people. Never has anyone tweaked data on anything we've ever worked with. We usually do the opposite, find that something works and work as hard as possible to trash our own work so we can't publish it. We spend more time trying to disprove our own work before publishing it than getting any positive results.
> It is widespread. More so than people think. These two dishonesty researchers are just the tip. A lot of people in academia are staying silent because they don't want people looking into their own research. False data is not the exception. It is endemic.
That's a conspiracy theory just like aliens landing on the moon and vaccines causing autism. Somehow, hundreds of thousands of people work together to stay silent and not leak. People who frankly hate each other's guts and want nothing more to do than to disprove each other's theories and show they're superior. Please, this doesn't stand up to even the most basic scrutiny or thought.
Every scientist that I know wants to do science. Wants to be remembered for contributing to our understanding of the universe hundreds of years after they die. You don't do this by lying, you do it by doing the best work you possibly can.
Just wait for people to dive into the false equivalence of scales and measures used in political research. It's very much "make it up and see if it sticks." The goal appears to be Psychology Today, not scientific rigor. I wrote about this in graduate school, as part of my hefty statistics coursework, it bothered me so much.
I'm very happy to see this type of scrutiny come to headline research but I really wish it begins at the beginning, with funding agencies and journals. And at the very least the moment a university sends out a breathless PR, the media all scrutinize the data provenance... A boy can wish
It's ironic that you try to attribute unprofessionalism to me using unprofessional tools. The experiments conducted by Prof. Siniver are not reproductions of experiments I conducted, but rather experiments that are clearly different in many essential differences. A claim for refutation through a completely different experiment is a provocation without a professional basis.
Researchers are required to keep data on studies for 5 years. I do not have data from studies that were carried out 15 years ago or more, and I have no way to find them either. This is not to criticize my work, and any attempt to do so is manipulation of the viewers.
I don't do the research alone. Throughout 30 years of research I had 248 scientific writing partners, among them the most respected researchers in the field. I led 2 labs at MIT and another lab at Duke where hundreds of interns, students and research assistants worked alongside me. You understand that if the claim is that I fabricated studies, you accuse a huge group of academics from dozens of universities in the world, when the only "evidence" for this is that I failed to prove "that I don't have a sister", that is, to provide documentation that I no longer have.
The experiment in which Prof. Amy Droley assisted took place over 15 years ago. I don't know why she remembers the things as she described to you. At first she didn't remember taking part in it at all, and only later did she remember. I and the other two professors who signed off on the study remember the details as they are described in the published article.
In the shredder test we met all MIT conditions. There are many methods to maintain the anonymity of the participants and not just one way as you presented.
There is a difference between academic articles, in which every word and data is carefully chosen, and between a free conversation in front of an audience, in which I convey the main points and do not go into all the details, for the important purpose of making science accessible. Your attempt to "catch" me with one word or another and take it out of context only reflects a desire to slander and nothing else.
Regarding the dentists, these things were said to me and Dr. Janet Schwartz in a face-to-face meeting, by senior company officials, as Dr. Schwartz also confirmed to you. I repeated the things that were said to me in media interviews because they seem important to me. The blog of my institute at Duke University accidentally mixed in an incorrect quote and I asked for it to be corrected.
We recently launched together with Harbor Capital various indices that try to improve the way companies treat their employees and also create value for investors. I forwarded to you the thorough research conducted by one of the best research units in the world, of the investment bank JP Morgan, on the data on which we were based, a study that reached the same conclusions as us.
Prof. Omar Moab holds opinions about many people, but to my understanding he is not an expert in the behavioral sciences and it is not clear why he was specifically invited to give an opinion on my research.
Only once was an error discovered in one of my studies, out of over 150 academic articles. My research partners and I realized that something was wrong with the data, we noticed it and shared the data transparently. I took responsibility for the incident, but I still don't know how it happened.
In my more than 30 years as a scientist, I participated in hundreds of studies, wrote seven books and had the great privilege of helping many organizations improve people's lives. I am proud of my work and contribution.
I don't mean that in a LMGTFY way, just in a "here's a tool that can help".
B) Moreover, our brains are also hardwired to storytelling[2][3]. When presented with a nice narrative, we feel rewarded.
C) Finally, to quote Charlie Munger, "Show me the incentive and I will show you the outcome". People in academia focus on the # of papers published.
Put (A), (B), and (C) together and we can explain news such as these.
[1] https://www.psychologytoday.com/us/blog/the-athletes-way/201... (hopefully this study is legit :))
[2] https://www.tadigital.com/insights/perspectives/art-storytel...
[3] https://cnlm.uci.edu/2018/12/04/story/
How could neuroscience possibly prove (or even provide any evidence in favour of) this? Did they use identify the gene for trust and produce a knock-out human?
The Latin phrases don't all appear to relate to what's going on. Don't know what's up with that, especially the one from Horace's Odes.
What are ppl even getting so excited about? I mean this is a field which teaches kids to recite - "half the money I spend is wasted; the trouble is I don't know which half".
Noted study in psychology fails to replicate, crumbles with evidence of fraud - https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102 comments)
A Big Study About Honesty Turns Out to Be Based on Fake Data - https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90 comments)
Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)
Related but different (I think?):
Harvard ethics professor allegedly fabricated multiple studies - https://news.ycombinator.com/item?id=36665247 - July 2023 (214 comments)
Harvard dishonesty expert accused of dishonesty - https://news.ycombinator.com/item?id=36424090 - June 2023 (201 comments)
Much more effective than endless pontificating and theorizing by people who have no idea what the fuck they're doing but haven't publicly fucked up yet so we trust them in reverence of their credentials. The amount of institutionalized deception that has permeated the field of psychology since its inception should scare anybody away from ever taking it seriously.
Dan Ariely is a fraud. To be specific: he has been credibly accused of fabricating data in at least four separate studies (Ten Commandments, delta dental, Hartford car insurance, and the study which he claimed Rossi ran for him which is discussed at the link). There are likely more.
More broadly: There was a “golden age” for this kind of “behavioral economics”/social psychology work in the early 2000’s up to probably 2012-2015.
Researchers would generally publish “results” which were either “delightfully counterintuitive” (ex: prime subjects to think about Florida and they walk slowly) OR featured extremely cheap interventions which showed large effects on outcomes (ex: signing an honest pledge at the top or the bottom of the page made a big difference in how honest people actually were on their tasks).
In retrospect, there are two problems (at least!) with this literature.
1. There was no theory backing up any of these predictions, so you could rationalize any statistically significant result with a compelling story. Publish it and you’re on your way to tenure.
2. Related to (1.), you didn’t even need to bother running actual experiments at all! Just fabricate the data. Make it fit your cute story and you’re done.
There are many tenured people in business schools (generally in “organizational behavior” or “social psychology” style departments rather than Econ) who were tenured on the basis of papers published under (1.) or (2.).
This is a big shame because there are serious researchers who do difficult work in behavioral and experimental economics which is _not at all_ like this BS. (Some are personal friends of mine, so this bothers me.).
There was no incentive for about 15 years to do the hard work though. And given that tenure is for life, it’ll be a long long time before these people are flushed out (barring more investigations, which should be encouraged!).
What should you do?
- be extremely, extremely skeptical of any results in the “priming” literature (eg the “think of Florida” example above).
- be moderately skeptical of results in the “nudges” literature. The retirement savings stuff makes sense and has replicated but the honesty nudge above is BS and has not.
- don’t get really excited about a researcher who says “economics bad - people dumb! - I behavioralist!” More than a few of these people have been revealed to be total frauds.
- don’t click or share this stuff when you see it on Reddit, HN or anywhere else. That kind of engagement “feeds the beast” - Deans love it and Deans assume you haven’t fabricated your data.
- maybe if you are specifically a Duke or Duke Fuqua alum send them a note asking why they aren’t taking the Ariely misconduct more seriously???
- (edit) throw away your copies of any of Ariely’s books. They are BS.
Specific example: In the car insurance paper, which was covered by Data Colada, there was agreement among all the four authors that Ariely himself was the only person to touch the data. That data (IIRC) was completely fabricated.
I’m happy to edit this comment and my original one if those details are wrong. I’ll give Ariely a presumption of innocence which he does not deserve and indeed which I suspect he abuses in research integrity investigations.
(Edit) data colada link: http://datacolada.org/98
At the end under “replies” Dan says himself that he was the person in touch with the insurer. It really beggars belief to imagine that there was someone at the insurer who fabricated a dataset which perfectly tells the BS story of this paper.
Also note that another of the four authors (Gino) will soon be fired from HBS for fabricating data in a series of studies.
It is just remarkable to see how people with enough knowledge of datascience to perpetrate this fraud for so long would also make errors as rudimentary as producing a faked 'random' data set in a uniform distribution. And with zero cars reporting over 50k miles driven. (edit: and so many others...this is worth reading just to see it all, thanks for linking).
These frauds really are not as smart as they think they are. But I guess selling best-selling books about stuff you made up could make you believe you're a genius.
You would imagine a con artist of his caliber would be more careful than that, but still ...
They've been doing it for a long time and it's always worked, either because they've always been this sloppy but everyone else reviewing was sloppier or gave the benefit of the doubt for reasons, or that they've put less and less effort into it over time because it hasn't been necessary.
Also the simpler the fraud is in execution the more likely you might be able to explain it away as a simple mistake or lack of familiarity with certain "tradesperson" tools adjacent to the skills you have framed in latin blackletter on sheepskin.
It's also clear that the data fabrication happened in Excel (based on styling differences that wouldn't pass through other formats) and the Excel metadata has Ariely as the creator of the file. If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators. If it had been fabricated by them as a e.g. a CSV and Ariely just imported it to Excel, it wouldn't show the inconsistent styling.
> If the data had been fabricated by the insurance company as an Excel file, they'd show up as the creators.
Digital forensics does not work this way. It is very rare when many digital evidence line up and imply that an actual person did or did not do something in the digital realm. A user account that shows up in an Excel file is absolutely insufficient to tie a person to the keyboard.
> it wouldn't show the inconsistent styling.
This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
> This is also inconsistent how digital forensics works. It can imply that something changed, but nothing about who, when, and why. It could have been an incorrectly propagated filter, or coloring, template, or functions.
No, sorry, I don't agree. It shows the "when" of the data fabrication, i.e. after the data was imported to Excel not before. The nature of the styling inconsistencies shows the "why", since it shows that the data was duplicated and one half of the duplicates was manipulated. Half the data was in one font, half in the other, and the two sets of data can be matched with each other. It also shows the "who" in combination with the metadata, due to the chain of reasoning in my previous message, since it shows that the data fabrication happened after the import to Excel and we know Ariely is the person who did that import.
[0] https://datacolada.org/98
> it is impossible to tell from the data who fabricated it. But because the fourth author has made it clear to us that he was the only author in touch with the insurance company, there are three logical possibilities: the fourth author himself, someone in the fourth author's lab, or someone at the insurance company.
So you seem to be much more confident about it only being one possibility, when the link explicity provides three possibilities.
One of those logical possibilities can be excluded entirely thanks to everyone involved agreeing that Ariely was the only person in contact with the company and handling the data ingestion. It could not have been anybody else in the lab.
The other theory is beyond absurd. Not only did the insurance company fabricate the data, but they happened to fabricate results that showed the effect Ariely needed for a high profile publication. Theories like your "slacking off data entry clerk" just don't make sense there.
Now, maybe if Ariely could provide even a shred of evidence for this being the data he received, I could change my mind. But he can't produce the file he received from the company. He can't tell who at the company sent it to him. The trail just runs cold at Ariely.
He'd already done lab experiments that showed the effect he thought existed, and it's easy to imagine an associate who actually fabricated the data already knew what he was looking to confirm.
Again, I'm not saying I think he's guilty or not. I'm just saying that from everything I've read so far, it seems plausible it was sloppiness and not checking data for signs of manipulation. That this sloppiness isn't "beyond absurd", but rather more like the classic "never attribute to malice that which is adequately explained by stupidity" (or by an overworked/overambitious academic).
Based on Bazerman's response, actually, he hadn't:
> [The fradulent paper] came together in a merger of two prior non-published empirical efforts. Mazar-Ariely independently provided the data for Study 3, while Shu-Gino-Bazerman had written a paper containing two laboratory experiments (Studies 1 and 2). The Shu-Gino-Bazerman group knew of the Mazar-Ariely data from multiple public presentations by Ariely.
Mazar's response is basically "oh god, I'm ashamed this happened. I played no role in the data collection, and I don't know who did. But thank you for all the hard work you've done in exposing this fraud, and I should have done better at questioning the data."
Gino's response (for context, she has been implicated in other fabricated data studies) is basically "I'm ashamed. I didn't collect the data, and I trusted the people who did collect it. But thank you for all the hard work you've done in exposing this fraud, I should have done better, and this is what I'm doing in the future to do better."
Ariely's response is "I'm the only one of the authors who had the data. Also, it wasn't me."
Bazerman's response is "Thank you for exposing this fraud. Let me explain how the study came to be, and the history of my own doubts about the quality of this study [which includes two episodes which seem to be red flags]. I should have pushed harder on those doubts [he was in the minority who wanted to retract the paper after nonreplication in the follow up]."
Ariely's response to me feels like raising a preemptive defense against being thrown under the bus. Neither Mazar nor Gino implicate anybody (only disclaim responsibility themselves), whereas Bazerman suggests that it would have had to have been Ariely or Mazar. (There's an interesting bit where Bazerman has an unspecified issue with the [now known to be faked] data prepublication, where one coauthor has unconvincing responses but a later coauthor was able to provide plausible explanations, complete with reference to the underlying data file--but both coauthors are unnamed). Given that, I just don't find the other possibilities all that plausible (and in the case of the middle possibility, it's not exactly exonerative anyways).
Clearly he had control over the file, but I wouldn't put too much weight on the metadata when there's plenty of other evidence.
It's easy to imagine reasons. A lazy assistant who's supposed to go through 13,488 filled-out forms decides to count 1,000 of them and then fabricate the rest so they can spend the week watching soccer online instead? Or an employee lost the data on which form results were associated with which condition, and they might get fired or not get promoted if caught, so they made it up? And so forth.
And metadata as you've described it doesn't sem like a smoking gun either. It's easy enough to imagine Ariely created an otherwise empty Excel sheet in the column format he wanted them to insert data into, e-mailed it to them, and got it back with their results. Or he just copied their Excel results into a spreadsheet of his own that already had other tabs, or similar.
I'm not saying Ariely didn't do it or did do it, I'm just wondering if there's an actual smoking gun here, or if it's accusations but there isn't enough evidence to actually know.
He also was the only one handling it, so if the data is bad then the responsibility falls to him regardless, both as the handler, as well as the head of the investigation. He either deliberately faked it, or got garbage data and didn't realize it when, now, plenty of others have.
Which points towards either him fabricating it himself, or asking somebody else to give him results without going through normal procedures for registering/conducting the experiment and then naming someone else as responsible for it...
I do think there are cases where academics will have unwittingly published falsified or partly false data and sometimes even garnered fame and citations for it x, and no doubt many more where the academics spot issues with the data they've received and bin it, but this doesn't look like one of them.
I've just gotten around to Kahneman's "Thinking Fast and Slow" (2011) and it seems a LOT of the discussion in the book relies on non-reproduced or even reversed results in your list. I hope the book changes significantly in the second half, as the first half is essentially barren after reading your FORRT link.
If you limit your understanding to science that is conclusive, you will paralyze yourself. This is the reality of scientific study; there are exceedingly few conclusions, only more questions, confusions, misunderstandings, etc.
When scientists say we don't "know" anything, this is what they're talking about. Nothing replicates, everything seems more confusing and less clear the deeper you dig, and the further you stray from the "hard" sciences that are easy to experiment, the more true this gets. Psychology, economics, etc. are insanely complex and effectively impossible to test the way we think of a physics or chemistry experiment.
If 'priming' only works every wet Tuesday in February, after someone has had a cheeseburger with no sauce and took their Grandma to the bingo, perhaps there's some _actual_ interesting artefact explaining it...or it could just be random noise.
But the presumption here is that the same experiment is even being run a second time, and my point is that many studies who have the stated goal of replication do so in ways that I find hard to justify.
So even when a study claims to "fail to replicate", I don't think we should just take that on face value.
That goes too far. Some science really does follow the scientific method and documents the methodology well-enough to be replicated.
That’s the basis for giving deference to scientists on some topics — they aren’t just experts, they have subjected their ideas to the cold hard scientific method and others have replicated the results.
Somewhere along the line we demoted science to just be another expertise, where you can just walk around in a lab coat and have social status and people aren’t supposed to question you. Throw in some jargon. No, that’s not science.
They can’t have it both ways. If scientists want deference on important topics, they need to follow the scientific method and not extrapolate too far from established results. No excuses like being “paralyzed” by requiring replicable results.
Looking at that site, though: if I search on "Ariely" I don't get what I would have hoped, which would be a list of his publications that have been debunked. Instead, there are some generalized articles not about him.
Minor point here, the article says Rossi ran the Ten Commandments study. Not sure if there's a different study you mean?
It seems no one can provide receipts (which is bad! due to reproducibility), and since Ariely's name is on the paper and Rossi's is not, the onus is on him to have kept that documentation. He asked that Rossi at least corroborate the details described in the paper, and she refused to do so saying that it didn't align with her memory or her standard procedure at that time.
Maybe the whole scandal is an elaborate memory priming experiment?
However (!) there are a bunch of other Ariely articles which are extremely suspicious and which are suspected of being fake. At this point I would say everything he has ever written should be assumed to be faked.
I expect a broader investigation will reveal more fraud. Apparently Duke doesn't care enough but hopefully some renewed attention to the case will force them to take action.
I think this point is under appreciated. I also want to add another point
- Don't take a (strong) position on something you don't understand or have equally strong evidence for.
We live in a world of specialization, which means a lot of us "are dumb" with respect to a lot of domains. There's nothing wrong with that. But there's a propensity for society to generalize this intelligence, which makes people think that because they can adequately reason about one domain that they can do so in another domain. It is kinda like the Murry Gell-Mann Amnesia, but to ourselves. Look at a domain you _are_ and expert in, and you'll probably realize that a lot of nuance matters, resulting in many things that would be non-intuitive without that high level of detail. Then ask yourself, do you have sufficient nuanced understanding for this other domain topic? I think we know the answer. Opinions are totally fine to have, but it is the strength of them which matters, and places like Reddit and HN are full of people who have been taught that their intelligence generalizes far better than it does, because what we're comparing ourselves to is other people's ability to reason and rationalize and not the distance our knowledge/reasoning is from the truth. i.e. you can out debate someone while being an absolute walnut.
...or else what?
We all live in a world of specialization certainly, but we also live in a world where we must make decisions with imperfect information. This idea of "only take positions on things you fully understand" is a luxury that isn't practical when you realize how frequently (and rapidly) those decisions must be made.
One thing that matters even more than nuance is execution, and a not-so-great idea executed extremely well is much better than a super well researched and understood idea that didn't get executed at all because the person with it took forever getting to that level of understanding.
There is, as in all things, a middle ground. Take positions, make decisions, but accept you'll be wrong decently often, and that's okay.
I'm not sure how to answer this without sounding rude. But you'll make a fool of yourself if you do, put people at risk, and probably lose money.
Considering the other context of your post, I'm not quite sure where your disagreement is. I specify _strong_ as a qualifier to indicate that the strength of the opinion/position (your confidence) should be proportional to your knowledge/understanding/evidence. I explicitly state that having opinions is perfectly fine and would go even further to say that it is impossible to be absolutely devoid of them. Human brains just make conclusions.
To clarify my point, I'm saying that being an armchair expert is not helping anyone. Having some armchair expertise is perfectly fine, but arguing from that position (especially passionately) is generally unhelpful and just adds noise. These are the common posts that you see where someone responds with a very simple explanation that is rather incomplete. You'll often see these highly upvoted as well and better answers buried simply due to their length. The laziness hurts us all! It is better to build habits and communal acceptability to just shut up a lot of times, as the problem with information retrieval is often about accessibility not availability; noise decreases accessibility due to the additional information we must sort through. If armchair comments are given (and they can be helpful, especially on vague topics or where no other level of expertise is present), then admission often helps the person receiving the information as well as creates the ability for better communication to determine higher quality information as well as better allow a more informed specialist to append or correct the basic answer.
The problem we see is not the existence of these armchair expert answers but rather that they are done so with high levels of confidence and the authors often feel the need to defend them, including against actual experts. Corrections or additions are not a critique to one's self, just the knowledge that was laid forth (which is often anonymous, so identity/self-value should play a lower role, but doesn't). Being wrong is not just okay, but should be welcomed as nerds we pride ourselves on being "smart" which means we pride ourselves on _gaining_ knowledge. Not everything needs to be a debate and honestly it is in general a terrible way to acquire or disseminate information.
Being wrong is completely okay, and trying to police what people discuss with one another is a dark road to go down. Besides, where do you draw the line? Do I need a specific number of course hours completed? Some number of books read? Three or more articles published on the topic?
Discussion, particularly in writing, is a form of thinking, and thinking with others is infinitely more productive than thinking alone, so if what you're asking is for folks to stop thinking about topics they're not experts on, I'm sorry but no. That's not going to happen.
Thank you. If you have any suggestions as to how to make my point clearer, I'd greatly appreciate it. I hope it is clear now, but obviously I can always improve. I do want to promote discussions, but that also means wanting to decrease arguments. Especially arguments that are uninformed and ones which has a clear answer (or lack of) from the domain experts.
Where to draw the line of domain expertise is not clear, you're right. Academia is one example of a way to pass that line but I wouldn't say the only means as it is just a method that correlates with sufficient knowledge gain. But I also wouldn't say that watching some YouTube videos does not constitute expertise (more explicitly, videos by science communicators rather than lectures). The boundary is fuzzy, but that should put pressure to reduce confidence (which is good for science and truth seeking) rather than increase it.
Discussion is not only fine, but actively encouraged. We're having one right now and at least on my side I don't feel like this is a argumentative. I tried to clarify this previously but I guess this is difficult. I'm not sure how to make myself more clear so all I can do is ask that you read more carefully with the updated knowledge that there's a substantial amount that I agree with you on (differences are in finer points) and request that you point out specifically where what I said may be confusing or resulted in misinterpretation. Hoping that this can facilitate a better discussion, if you still wish to have one.
She wasn't an author, so she had no reason to be involved w/ (for example) the referee process. She was made aware that this was their claim when someone contacted her.
She also works in a slightly different literature IMO. Even if she had read the article, it probably didn't say "the data gathering was done by Prof. Rossi at UCLA"
How could he have successfully executed this fabrication? Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.? The idea that one man, alone, could pull off such a large deception seems, to me, to be pretty farfetched.
Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
Combine that with the ever-present reproducibility problem in all of psychology, and the economists who were made to look like fools (in their view) in the earlier years of the 21st century are striking back with exactly these kinds of arguments (economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues).
Ariely's books aren't "BS", and throwing that in there kind of shows your hand. Large portions of his books are explanations of the work of others, and even if everything this UCLA professor says is true, that doesn't invalidate a single thing he wrote about in his books, other than perhaps specific citations of his own work (of which we're talking about a tiny handful of hundreds of published and peer reviewed studies he worked on with thousands of other people).
Huge swaths of behavioral economics have replicated successfully. Care to cite peer-reviewed studies related to astrology that have successfully replicated?
I specifically said don't throw out nudges - some of them work and replicate e.g., retirement savings defaults, while others are BS like the honesty pledge. Real experimental/behavioral economics is not this media-attention-grabbing garbage anyway. As I said, I have friends in the field. They don't do this BS.
> Doesn't he work with entire teams of others, don't others review his work, his analysis, the data collection, etc.?
No, he doesn't. See the Data Colada responses from his three coauthors on the car insurance paper. All admit he was the only author to have data access. If you can find it, check the earlier, separate investigation into his other fabricated paper w/ Delta Dental where the insurer itself said "we didn't give him that data."
> Some additional context here is that in academia, there has been a lot of pushback on the burgeoning field of behavioral economics as a result of political viewpoints; nudge is inherently a liberal concept, and puts conservative viewpoints of personal responsibility in a negative light.
I disagree on many dimensions w/ your characterization of behavioral econ AND the entire point is irrelevant here.
> economics isn't strictly a "hard science" insofar as it can't really run experiments the way physics can, so it's easy for economists to point out experimental issues
I can introduce you to ten experimental economists who would disagree. Economists have also been pioneers in figuring out when non-experimental situations parallel experiments enough that we can make the same inferences.
> Ariely's books aren't "BS"
Please continue enjoying them if you like them, but I don't have any idea how you'd separate fact from fiction. Ariely is not the first researcher in this exact field who has been revealed to be a fraud.
I'll only add that compared to, say, traditional economics we don't have a sense of how unique the problem is to social psych/organizational behavior. In the past decade psychology has done a lot more replication attempts and data auditing than have other fields in the social sciences. So it could be that other fields are equally problematic, but we don't know.
FWIW... In plenty of economics papers you can't do a meaningful replication b/c there may be no analogous natural experiment. You can only do analysis of the same data. Such "replications" do not always lead to the same results.
The more experimental parts of the econ literature are now very big on pre-registration (esp. in development, e.g.). Not a panacea by any means, but an improvement for sure.
And there are generally higher standards around making code available now. If you work in this area and read the AER you may be aware of a recent retraction in PF due to coding malpractice. The system has improved but is far from perfect.
in terms of defamation, I don't think you can summarize "accused" as "is a fraud"
Specifically
> Unfortunately for Ariely, the Israeli press won’t let the issue die and has been digging into this issue of provenance rather deeply.
Haven't noticed any stories about this issue, this year (2023), in the Israeli press.
It’s a shame that the worst that can happen to these frauds is, typically, job loss. The incentive to cheat is way higher than the cost times the likelihood of getting caught. There should be criminal penalties at least as severe as other white collar crimes.
https://en.m.wikipedia.org/wiki/Half-life_of_knowledge
P.S. Ironically, the title of the paper is "The dishonesty of honest people" (it's 15 years old by now).
Lower values mean the field is advancing quickly. Rapid scientific advancement is a good thing for knowledge generally. It's merely annoying for people in the field because they have to do more work to keep up-to-date. :)
It's not a measure of things being proven wrong, but more about newer better-predictive theories making older theories obsolete.
I don't like to see trials by Twitter. I also don't like to see institutions of academia shirking their responsibilities.
(Full disclosure: I met with Ariely a long time ago, and he seemed decent, but I know almost nothing about that field.)
One of my undergrad psych research projects was focus on testing some of the Kahneman & Tversky anchoring/adjustment stuff. We were able to replicate some things related to the theory itself based around economic estimates, but also had some "control" questions in our surveys (one example: "estimate the average weight of a human brainL). Our experience was that some populations just sucked worse at the kinds of estimation tasks we were using. We had no good theory for gender-based differences in estimating the weight of a brain, and in retrospect it was probably just people confusing ounces with pounds as a half-remembered number for weight... but no one gets published by admitting that their experiments were probably just poorly designed.