His own work held up very well to replication. It's when he is citing the work of other scholars (in particular, that of social psychologists) that doesn't hold up well to replication.
As someone working in this space, I largely agree with your take. I'll only add that compared to, say, traditional economics we don't have a sense of how unique the problem is to social psych/organizational behavior. In…
Most of Kahneman's own work has held up quite well to replication, and that makes up a large portion of the book. But a lot of the social psych findings he discusses have not fared well to replication attempts.
We tested both ideas (altruism and theft aversion) in the paper. Short answer is we find evidence for both. For somewhat technical reasons, we think the "viewing oneself as a thief" component is probably necessary in…
It's a fair point, but we've tried to test this issue in a number of different ways. Based on the available data, cross country differences in email usage doesn't seem to have a meaningful impact on our results. See my…
A valid concern, but a couple points: (1) This issue doesn't affect the treatment effect (the difference in return rates for money vs no money), which was the main focus of the paper. If email usage is low in a…
Actually one of our research assistants was temporarily detained (in Kenya, I believe) for suspicious activity. As you can imagine, having a bunch of "lost" wallets on hand required some explaining.
Definitely a concern. We took steps to examine if there were specific "experimenter effects", that is whether response rates were affected by one research assistant rather than another, and don't find evidence that was…
One of the study authors here. On targeting front-desk employees, we did this both to allow relatively portable cross-country comparisons and for reasons of internal validity (e.g., if we placed wallets on the ground,…
She never taught at Cal State Fullerton. Here is her CV: https://ucirvine-my.sharepoint.com/:w:/g/personal/eloftus_ad...
My understanding is that there have also been semi-recent replications done in France, Iran, and a partial replication conducted at Santa Clara university. All found the same basic pattern as the original study.
Right --- it's called block randomization.
Ted Chiang is fantastic. Good podcast discussion of this story: https://overcast.fm/+HV3dvbFhA
Definitely a fair criticism. In the paper we test for experimenter effects (are the results different for male vs female research assistants? Are some researcher assistants acting differently in a way that might bias…
Cost was about US $600K in total.
Absolutely. We thought a lot about this trade-off when designing the study. The disadvantage of using a clear business-card case over a traditional wallet is clear, in that it is relatively unusual. The advantage of…
Thanks for your comments. 1. This is a fair point. In the Supplemental Material, we explore cross-country differences in email usage. When we statistically adjust for country-level differences in email usage (using…
Many thanks. Yes, we only went back for those who contacted us first. Your idea is an interesting one, but it gets at something else (how many people, who otherwise would keep the wallet, instead would turn over the…
We originally planned to include Japan but after some initial pilot testing we realized that the country was unsuitable for methodological reasons. Japan has a lot of small “police booths” where people can return lost…
Quick clarification: we did adjust for purchasing power parity across countries (but not within countries). As you surmised, even for differences between cities like SF and Fresno, the PPP adjustments would be…
Thanks for your comments (I'm one of the authors on this paper). For a subset of the countries, we went back to retrieve the wallets (logistically, collecting the wallets turned out to be extremely difficult, which is…
The reason why it is usually considered irrational/illogical is because hyperbolic discounting leads to dynamically inconsistent preferences. For example, under hyperbolic discounting I may prefer $100 today to $150 in…
His own work held up very well to replication. It's when he is citing the work of other scholars (in particular, that of social psychologists) that doesn't hold up well to replication.
As someone working in this space, I largely agree with your take. I'll only add that compared to, say, traditional economics we don't have a sense of how unique the problem is to social psych/organizational behavior. In…
Most of Kahneman's own work has held up quite well to replication, and that makes up a large portion of the book. But a lot of the social psych findings he discusses have not fared well to replication attempts.
We tested both ideas (altruism and theft aversion) in the paper. Short answer is we find evidence for both. For somewhat technical reasons, we think the "viewing oneself as a thief" component is probably necessary in…
It's a fair point, but we've tried to test this issue in a number of different ways. Based on the available data, cross country differences in email usage doesn't seem to have a meaningful impact on our results. See my…
A valid concern, but a couple points: (1) This issue doesn't affect the treatment effect (the difference in return rates for money vs no money), which was the main focus of the paper. If email usage is low in a…
Actually one of our research assistants was temporarily detained (in Kenya, I believe) for suspicious activity. As you can imagine, having a bunch of "lost" wallets on hand required some explaining.
Definitely a concern. We took steps to examine if there were specific "experimenter effects", that is whether response rates were affected by one research assistant rather than another, and don't find evidence that was…
One of the study authors here. On targeting front-desk employees, we did this both to allow relatively portable cross-country comparisons and for reasons of internal validity (e.g., if we placed wallets on the ground,…
She never taught at Cal State Fullerton. Here is her CV: https://ucirvine-my.sharepoint.com/:w:/g/personal/eloftus_ad...
My understanding is that there have also been semi-recent replications done in France, Iran, and a partial replication conducted at Santa Clara university. All found the same basic pattern as the original study.
Right --- it's called block randomization.
Ted Chiang is fantastic. Good podcast discussion of this story: https://overcast.fm/+HV3dvbFhA
Definitely a fair criticism. In the paper we test for experimenter effects (are the results different for male vs female research assistants? Are some researcher assistants acting differently in a way that might bias…
Cost was about US $600K in total.
Absolutely. We thought a lot about this trade-off when designing the study. The disadvantage of using a clear business-card case over a traditional wallet is clear, in that it is relatively unusual. The advantage of…
Thanks for your comments. 1. This is a fair point. In the Supplemental Material, we explore cross-country differences in email usage. When we statistically adjust for country-level differences in email usage (using…
Many thanks. Yes, we only went back for those who contacted us first. Your idea is an interesting one, but it gets at something else (how many people, who otherwise would keep the wallet, instead would turn over the…
We originally planned to include Japan but after some initial pilot testing we realized that the country was unsuitable for methodological reasons. Japan has a lot of small “police booths” where people can return lost…
Quick clarification: we did adjust for purchasing power parity across countries (but not within countries). As you surmised, even for differences between cities like SF and Fresno, the PPP adjustments would be…
Thanks for your comments (I'm one of the authors on this paper). For a subset of the countries, we went back to retrieve the wallets (logistically, collecting the wallets turned out to be extremely difficult, which is…
The reason why it is usually considered irrational/illogical is because hyperbolic discounting leads to dynamically inconsistent preferences. For example, under hyperbolic discounting I may prefer $100 today to $150 in…