Me too. When I started reading, I realized that my cluelessness is what computer illiterate people probably feel when reading about IT stuff.
But on the whole, I felt that the article was rather trying to state that effect sizes matter, using the ego depletion theory as an example for that. The implications at the end of the article are particularly interesting, or at least, if I were to be more brazen, don't really deny commonly held prejudices on soft sciences...
It seems to me that if a psychological effect is so small that you can meta-analyse hundreds of studies and still not be sure whether it exists or not, then it doesn’t matter if it does or not.
This isn’t physics, where the existence of a tiny effect such as time dilation might nonetheless have significance beyond the observable regime. If a psychological effect is barely measurable then it’s of no use to anybody. In particular you don’t need to go around taking it into effect in your everyday life.
Compare to the original analogy of a muscle. An actual muscle, if you work it close to its limit, will be weaker one minute later... but if you work it to its limit three times a week for a month it will get stronger. These effect sizes are large, they’re consistently observed in ~100% of healthy adults, and you can actually apply them in your life.
I am so sick and tired of the muscle analogy (not criticizing you, but the general usage)! Almost always when I see it used, it is to describe a situation where we know almost _nothing_ about the fundamental phenomena! So any so-called similarity is 100 % speculative.
I'd consider "it is like a muscle" a red flag when reading news/science.
It's doubly funny, because we actually have not enough of an idea how muscles do what they do... Even so far as immune system may be involved in their growth and strength adaptation.
I don't think so, no. Here is why: the basic behavior of muscles are decently well understood. For example why and how they contract and relax, and why they tire. We know of several reasons why muscles might malfunction. What is not well established is how and why muscles become stronger over time.
Compare this to for example ego depletion, and I would say that the analogy is not apt.
If one pumps iron, then one gets big and strong. What is the if-then for ego depletion? This entire fuzzy informal idea of common-sense causality is what is being wagered by these flimsy psychological concepts.
If one exercises his willpower a lot†, then one has a progressively harder time doing so again shortly after‡. This is so because willpower is a finite, expendable resource which has to be renewed (through not exercising it).
† - for instance, by denying himself bad, tempting things
‡ - gives in to temptation
It's not flimsy at all, it just appears to be wrong.
The ego depletion effect story is interesting to me for numerous reasons, some of which are statistical/inferential, some of which are more theoretical, and some of which are kind of personal or anecdotal (I know the author of the blog post in question and our research overlaps in important ways, although not with regard to the ego depletion effect in particular; I don't research it).
I kind of think Brent could have gone further with some of the implications of the ED effect size being so small. To me, it's interesting that it is basically zero (I don't think it's even .08, based on Bayesian prior choice effects, which is another interesting angle on this story). Why? Because I think it's a fairly intuitive idea that gets applied by people in everyday life over and over again, and has broader implications.
For example, you are in a situation where you're tempted by something, have to resist temptation, and experience it as exhausting. You go home from work and are tired and need to relax, ostensibly because you're tired by being on your best behavior all day. There's recent very cogently argued public health literature arguing that diets don't work largely because it's too hard (read: too fatiguing, too effortful) to resist our favorite foods, either in the short term or over the long term.
In short, I think there are a lot of people who would be surprised that the ego depletion effect is so close to zero. Maybe not everyone, but a lot of people. It also raises the question of why it's difficult to resist temptations over a sustained period, and how to model that process more accurately (my reading of the ego depletion literature is that a lot of the effect can be explained by sustained negative emotional state or heightened attention, rather than resource depletion, but that's a different issue; another explanation is with reward prediction models, but I think those are predicting a different aspect of behavior).
I guess I'm agreeing with you, but I think the magnitude of the effect size being zero or close to it is important for a lot of reasons aside from meta-scientific issues pertaining to replicability and trustworthiness of results.
Great article. This hits close to home as a product developer. I'm certain this phenomena happens in "data driven" product decisions. Top of the funnel, we have the numbers to get statistically sound outcomes on A/B tests quickly. I work on a B2B product (high value, low volume sales cycle) and bottom of the funnel or internal sales process experimentation we try our best at is sketchy and hard to do. Experimental design is hard trying to get adequate power.
Any HNers have tips for working in low-data regimes (N=low hundreds)? Is there some magic Bayesian angle I don't know about?
We haven't systemically looked at effect sizes and this article is a good push for me to dig into that.
There's money on the line for companies. So directly or indirectly developer are going to be paranoid about fooling themselves. Like you demonstrate here!
Sure fire way to make it to the top of psychology / sociology academia:
1) Go to Ivy school, get your phd, make all the key connections
2) Find something that makes intuitive sense to a highly educated person, and create a test that you claim measures it. Even better if it’s really fuzzy, like “ego depletion”. For example, call it “mood enhancement”. This is if your mood is elevated. The test would ask 20 questions about your current mood, like “Do I feel happy” and “I feel above-normal positive emotions”. These 20 questions will use 1 to 5 scale and provide a 100 point outcome.
3) Use your connections to get people in the academic industry to start promoting it and using it. Publish a paper showing what baseline is.
4) Now is your time to strike. Start making experiments that you anticipate would modify someone’s mood, like giving them a glass of wine before the test, or having them call their mother. Do control and experimental and see the boost to “mood enhancement”.
5) Publish paper after paper on your amazing discoveries of how to boost “mood enhancement”. Get others to cite your publications and make a cottage industry of “mood enhancement” research experts. More people, more papers, more funding - and you started it all with the key genesis “mood enhancement” paper!
6) Write books, go on NPR as a guest speaker, give speeches. You have quantified mood and determined how to boost it. “My research shows that a 7.2 percent increase in mood occurs when you call your mother before a big company presentation. It’s science!” You are now a public figure and can get tenure and big corporate speech bucks. The world has been forever changed by your mood enhancing research!
Despite its length, this comment is a shallow dismissal in the sense that the site guidelines ask you not to post here:
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
Elongating an extra cynical rant does not count as teaching us something, so please don't post like this to HN. If what you're writing is based on knowledge or experience, please share your knowledge or experience directly. If it isn't, please post elsewhere.
I am indeed throwing stones at the entire academic discipline of psychology and sociology. I see your point but I am curious in the counterpoints in replies, given the massive edifice that has been constructed on taxpayer dollars and has been shown to have absolutely no basis in reality with further pre-registered studies. I can link to many articles about this crisis if you are unaware
Jesus fucking christ, dang. You are such a fucking faggot.
You've completely ruined this fucking community. From the day tou started, all you do is stifle any fucking vibrance.
Every move that you've made on this fucking site only serves to grind all conversations to a meaningless exchange of hushed platitudes. People used to talk to each other around here. Funny things used to entertain and surprise.
Now, nothing is suprising anymore. Another fucking grey thread filled with the stale words of people who are dead inside.
It's clear by now what your job is. Just make sure the forum doesn't make your overlord look bad, by any means available to you.
I can look back at threads from 2013, before your changes, and detect the obvious difference in tone. Soon after, the trend in hellbans and ranking shifted the tone. Volume changed the site, and continued to snowball, and no matter what, there are more users, but the conversations and submitted stories are worse.
New accounts serve no purpose as throwaways, since all new accounts default to invisible. What a fucking joke.
You've managed to elevate sickly, boring personalities to new heights, while failing to attract anybody interesting. The nobodies chat away, and they prattle on about the things no one is interested in listening to.
Right here, we have someone with something interesting to say. They poke holes at soft science with this comment, and point to a cult of personality, where students must ingratiate themselves to faculty with feel good bullshit artistry. Not real science.
But cynicism has no place here. Snark provokes fear beneath the spectre of suicides of Aaron Swartz and Ilya Zhitomirskiy. Once bitten, twice shy. Oh well.
Given that the placebo effect is a well-documented meta-effect, to what degree can we suggest that any particular weak psychological effect is real and not placebo? This is not rhetorical; to what degree do you believe that psychology even exists as a reasonable experimental science? The main lesson of these meta-analyses and examinations of statistics is that we cannot trust our minds to be honest.
Your failure to be cynical in the face of ever-mounting evidence is a clear sign that you are failing to show the skepticism demanded of sincere scientists. From [0] linked upthread:
> This means, presumably after about Jan 1st 2025 utilizing willpower will increase the available resources to utilize in willpower tasks. This in turn will require us to hold back from utilizing our willpower in order to prevent exponential amplification in a loop… However, holding back from utilizing willpower itself requires willpower, leading to actually super-exponential ego growth. Shortly after midnight Jan 1 2025 all humans will vanish in a blinding flash of pure ego.
Clearly something is wrong here; clearly ego depletion, as described by the literature of psychology, does not exist. (Or I am about to write a lot of code in the next few years!)
The thread-starter you are criticizing was obliquely referring to several well-known failures of the psychologist community: power poses [1], priming [2], and more beliefs about food than you knew you ever held [3].
As somebody elsewhere says, the point is indeed that perhaps it's all rubbish. Do you have a coherent reason to believe, or is it something taken on faith?
Then I feel that you're engaging in a double standard. The thread-starter and I both have a critique of the social construction of psychology, and your response is that the direction of the critique is "shallow" and "extra cynical". However, only the thread-starter and I have given anything of substance, and y'all have not really refuted any of it.
It's disappointing when a site is supposed to promote curiosity and yet its moderators are not only uninformed but actively incurious. You have a pattern of policing folks' tones under the auspices of poor content, but it doesn't hold up well when you post two-sentence dismissals to hundreds of words worth of original, sincere, considered thought. It makes you come across as lacking in empathy and insight.
This ain't the first time you've shown this attitude, for what it's worth. I am picking up on a pattern.
It's clear that "this mood enhancement research" has some value. So what is the problem? That we should aim higher? What could be done better to compared to the story you describe?
>It's clear that "this mood enhancement research" has some value.
Only in the same sense that the fad diet industry has "some value". It makes people careers and money. Actual value in the field would be to discover truths about people's psychology.
For those who may nbot be familiar with the term: From Wiki:
Ego depletion refers to the idea that self-control or willpower draws upon a limited pool of mental resources that can be used up. When the energy for mental activity is low, self-control is typically impaired, which would be considered a state of ego depletion.
I've been following this through the lens of Beeminder (and "Quantified Self") -- taking away the idea that if you know your own willpower better, you can do magic tricks like "only promise what you can deliver on" and "incrementally ramping up your own willpower"
After looking up this concept of Ego Depletion, which seemed like a new idea I haven't heard of before, when I clicked this discussion today... I can see it's basically the same idea. But what I hadn't considered is that the same ideas are able to be used by, eg. advertisers!
While my goal of study in this field is making the most of my own efforts, there is someone out there looking at the same charts, trying to figure out exactly where is the right place to put their advertisement to maximize the effect it has on me, even going as far as to get me to buy something that I wouldn't have bought otherwise.
That's just fascinating. (Alternatively, that's a horrifying idea...)
33 comments
[ 2.1 ms ] story [ 80.2 ms ] threadBut on the whole, I felt that the article was rather trying to state that effect sizes matter, using the ego depletion theory as an example for that. The implications at the end of the article are particularly interesting, or at least, if I were to be more brazen, don't really deny commonly held prejudices on soft sciences...
This isn’t physics, where the existence of a tiny effect such as time dilation might nonetheless have significance beyond the observable regime. If a psychological effect is barely measurable then it’s of no use to anybody. In particular you don’t need to go around taking it into effect in your everyday life.
Compare to the original analogy of a muscle. An actual muscle, if you work it close to its limit, will be weaker one minute later... but if you work it to its limit three times a week for a month it will get stronger. These effect sizes are large, they’re consistently observed in ~100% of healthy adults, and you can actually apply them in your life.
I'd consider "it is like a muscle" a red flag when reading news/science.
Compare this to for example ego depletion, and I would say that the analogy is not apt.
† - for instance, by denying himself bad, tempting things
‡ - gives in to temptation
It's not flimsy at all, it just appears to be wrong.
I kind of think Brent could have gone further with some of the implications of the ED effect size being so small. To me, it's interesting that it is basically zero (I don't think it's even .08, based on Bayesian prior choice effects, which is another interesting angle on this story). Why? Because I think it's a fairly intuitive idea that gets applied by people in everyday life over and over again, and has broader implications.
For example, you are in a situation where you're tempted by something, have to resist temptation, and experience it as exhausting. You go home from work and are tired and need to relax, ostensibly because you're tired by being on your best behavior all day. There's recent very cogently argued public health literature arguing that diets don't work largely because it's too hard (read: too fatiguing, too effortful) to resist our favorite foods, either in the short term or over the long term.
In short, I think there are a lot of people who would be surprised that the ego depletion effect is so close to zero. Maybe not everyone, but a lot of people. It also raises the question of why it's difficult to resist temptations over a sustained period, and how to model that process more accurately (my reading of the ego depletion literature is that a lot of the effect can be explained by sustained negative emotional state or heightened attention, rather than resource depletion, but that's a different issue; another explanation is with reward prediction models, but I think those are predicting a different aspect of behavior).
I guess I'm agreeing with you, but I think the magnitude of the effect size being zero or close to it is important for a lot of reasons aside from meta-scientific issues pertaining to replicability and trustworthiness of results.
Any HNers have tips for working in low-data regimes (N=low hundreds)? Is there some magic Bayesian angle I don't know about?
We haven't systemically looked at effect sizes and this article is a good push for me to dig into that.
If you have a million users you can get few data per user but lots of data in total.
If you have 100 users you can get a lot of data on each one by direct contact
Locking yourself into 100 early adopters may or may not be prudent.
Main thing to answer is how representative your sample of target users is. The deeper the questions asked, less likely it is.
Alas, some of the social sciences didn't use to be so honest. See this infamous piece about 'methodological terrorism': http://datacolada.org/wp-content/uploads/2016/09/Fiske-presi... (or https://www.thecut.com/2016/10/inside-psychologys-methodolog... for an overview).
1) Go to Ivy school, get your phd, make all the key connections
2) Find something that makes intuitive sense to a highly educated person, and create a test that you claim measures it. Even better if it’s really fuzzy, like “ego depletion”. For example, call it “mood enhancement”. This is if your mood is elevated. The test would ask 20 questions about your current mood, like “Do I feel happy” and “I feel above-normal positive emotions”. These 20 questions will use 1 to 5 scale and provide a 100 point outcome.
3) Use your connections to get people in the academic industry to start promoting it and using it. Publish a paper showing what baseline is.
4) Now is your time to strike. Start making experiments that you anticipate would modify someone’s mood, like giving them a glass of wine before the test, or having them call their mother. Do control and experimental and see the boost to “mood enhancement”.
5) Publish paper after paper on your amazing discoveries of how to boost “mood enhancement”. Get others to cite your publications and make a cottage industry of “mood enhancement” research experts. More people, more papers, more funding - and you started it all with the key genesis “mood enhancement” paper!
6) Write books, go on NPR as a guest speaker, give speeches. You have quantified mood and determined how to boost it. “My research shows that a 7.2 percent increase in mood occurs when you call your mother before a big company presentation. It’s science!” You are now a public figure and can get tenure and big corporate speech bucks. The world has been forever changed by your mood enhancing research!
Elongating an extra cynical rant does not count as teaching us something, so please don't post like this to HN. If what you're writing is based on knowledge or experience, please share your knowledge or experience directly. If it isn't, please post elsewhere.
https://news.ycombinator.com/newsguidelines.html
You've completely ruined this fucking community. From the day tou started, all you do is stifle any fucking vibrance.
Every move that you've made on this fucking site only serves to grind all conversations to a meaningless exchange of hushed platitudes. People used to talk to each other around here. Funny things used to entertain and surprise.
Now, nothing is suprising anymore. Another fucking grey thread filled with the stale words of people who are dead inside.
It's clear by now what your job is. Just make sure the forum doesn't make your overlord look bad, by any means available to you.
I can look back at threads from 2013, before your changes, and detect the obvious difference in tone. Soon after, the trend in hellbans and ranking shifted the tone. Volume changed the site, and continued to snowball, and no matter what, there are more users, but the conversations and submitted stories are worse.
New accounts serve no purpose as throwaways, since all new accounts default to invisible. What a fucking joke.
You've managed to elevate sickly, boring personalities to new heights, while failing to attract anybody interesting. The nobodies chat away, and they prattle on about the things no one is interested in listening to.
Right here, we have someone with something interesting to say. They poke holes at soft science with this comment, and point to a cult of personality, where students must ingratiate themselves to faculty with feel good bullshit artistry. Not real science.
But cynicism has no place here. Snark provokes fear beneath the spectre of suicides of Aaron Swartz and Ilya Zhitomirskiy. Once bitten, twice shy. Oh well.
Nothing interesting will ever happen on HN again.
Your failure to be cynical in the face of ever-mounting evidence is a clear sign that you are failing to show the skepticism demanded of sincere scientists. From [0] linked upthread:
> This means, presumably after about Jan 1st 2025 utilizing willpower will increase the available resources to utilize in willpower tasks. This in turn will require us to hold back from utilizing our willpower in order to prevent exponential amplification in a loop… However, holding back from utilizing willpower itself requires willpower, leading to actually super-exponential ego growth. Shortly after midnight Jan 1 2025 all humans will vanish in a blinding flash of pure ego.
Clearly something is wrong here; clearly ego depletion, as described by the literature of psychology, does not exist. (Or I am about to write a lot of code in the next few years!)
The thread-starter you are criticizing was obliquely referring to several well-known failures of the psychologist community: power poses [1], priming [2], and more beliefs about food than you knew you ever held [3].
As somebody elsewhere says, the point is indeed that perhaps it's all rubbish. Do you have a coherent reason to believe, or is it something taken on faith?
[0] https://statmodeling.stat.columbia.edu/2019/11/19/break-out-...
[1] https://en.wikipedia.org/wiki/Power_posing
[2] https://www.nature.com/news/nobel-laureate-challenges-psycho...
[3] https://en.wikipedia.org/wiki/Brian_Wansink
It's disappointing when a site is supposed to promote curiosity and yet its moderators are not only uninformed but actively incurious. You have a pattern of policing folks' tones under the auspices of poor content, but it doesn't hold up well when you post two-sentence dismissals to hundreds of words worth of original, sincere, considered thought. It makes you come across as lacking in empathy and insight.
This ain't the first time you've shown this attitude, for what it's worth. I am picking up on a pattern.
Only in the same sense that the fad diet industry has "some value". It makes people careers and money. Actual value in the field would be to discover truths about people's psychology.
This will bring about the ego singularity as the positive feedback loop quickly spirals out of control.
Ego depletion refers to the idea that self-control or willpower draws upon a limited pool of mental resources that can be used up. When the energy for mental activity is low, self-control is typically impaired, which would be considered a state of ego depletion.
After looking up this concept of Ego Depletion, which seemed like a new idea I haven't heard of before, when I clicked this discussion today... I can see it's basically the same idea. But what I hadn't considered is that the same ideas are able to be used by, eg. advertisers!
While my goal of study in this field is making the most of my own efforts, there is someone out there looking at the same charts, trying to figure out exactly where is the right place to put their advertisement to maximize the effect it has on me, even going as far as to get me to buy something that I wouldn't have bought otherwise.
That's just fascinating. (Alternatively, that's a horrifying idea...)