Question from me would be, what are the necessary conditions in an organization to create a culture that rewards recognizing and mitigating cognitive biases? Imagining that a series of Rationalist interview questions would seem a bit culty, and run the risk of going full Bridgewater.
I would imagine asking "When you approach a problem, do you apply any techniques to evaluate any cognitive biases that might be applicable" would elicit some sort of response without upsetting people.
I don't know how important this is for most line employees. As a code monkey, I rarely have some strong bias besides "Angular is my personal preference over React" which immediately leads to "Well, should we use React anyway because the team knows it better and blah blah."
If you want to be a software engineer instead of a "code monkey", this is part of it. The software engineer has to be able to see the second order effects of changes to the code.
In my opinion it's almost a given that any senior dev considers these. You can't be a senior dev / team lead / architect without thinking about how the system will be affected by decisions in the long run. From "MongoDB is faster, but it won't integrate with BI as well down the line" to "Well, GCP is great, but Google has a history of not supporting services in the long run."
Haha, well, I won't start an argument given your affiliation, but let's just say that most BI tools CLAIMED to be able to integrate with Mongo well, but in the end, we ended up dumping the data into SQL for the BI tools. This was 4 years ago, and we tested about 5 major BI providers and were willing to pay them ridiculous sums.
With that said, I enjoy Mongo, for its specific use cases.
We built our BI connector a few years ago for all the reasons you outline. Definitely worth a spin if you have this challenge again. Basically makes MongoDB look like MySQL (if that floats your boat :-)).
I wonder if I might explore that a little. Do recognizing and mitigating cognitive biases -- as goals -- necessarily lead to better organizations?
On the face of it, it might seem the answer is yes. Fewer cognitive biases means better decisions that correspond more closely to reality. The thrust is that the more rational we are, the more effective our actions are.
However, in my observation, the type of rationality that "works" isn't always the same as the type that is "logically correct" (they can be identical in some instances, but not in all). Successful organizations, it seems to me, tend to prize "instrumental rationality" over "value rationality" / "epistemic rationality" [1].
Just wondering what folks' thoughts are on that?
Folks like Gerd Gigerenzer advocate the use of heuristics, which while are often biased, actually lead to good outcomes on average -- especially in incomplete information scenarios (which is most scenarios in real life). [2, 3]
It has even been shown that heuristics or crude (or even slightly wrong) simple models can outperform complex rigorous models in complex environments – because they are usually more robust to assumption violations under changing conditions [4,5]. In real life, assumptions are violated to various degrees almost all the time.
Ironically, the Rationalists might frown on this. Several posts in the Sequences had a theme of "don't do what's raTi0NaL, do what actually wins." E.g. "The Ultimate Newcomb Problem" [0]. HN recognizes this same meme as "goal > tool".
If the goal is truth, you cultivate a desire to learn to succeed under diverse scenarios. The incentive is success itself. In a particular scenario, Falsehood can offer greater benefit than Truth. But the advantage of Truth, is that it provides your decision-making a robust foundation, even under volatile conditions. "Only when the tide goes out do you discover who's been swimming naked."
If the goal is recognition of bias, you cultivate social-norms which incentivize critique (both in yourself and others). Notice, however, that this begets perverse incentives. I.e. you can fool yourself, "recognize" a billion biases, then applaud yourself for your diligence. You can also invent new biases that don't actually make much difference.
Here's a mental model I've discovered that has been useful for me:
"Even if you have cleverly identified a second-order effect in the opposite direction, it doesn't mean it actually outweighs the magnitude of the first-order effect – you still have to check."
Also, life is not one-dimensional. The second-order change may not be parallel (or antiparallel) to the change - it may go off in some other direction.
Identifying second-order effects, along with their relative magnitude is hard enough to begin with. Only if your model of the 1st order situation is precise enough, you may come up with adequate 2nd order possibilities, and even then, N-th order thinking becomes exponentially harder as N increases. Each new effect presents a new model, which exponentially increases the area of the plane of outcomes.
>Second order thinking is not obvious, and requires a complex systematic thought process. Hence the observations are more profound, well-reasoned, and unique.
What a terrrible non-sequitur. Just because you think really hard, doesn't mean the conclusion so derived will be profound. Well known example, Newton thought really hard about many things and some paths didnt go anywhere.
The well-reasoned part is begging the question.
And there's no guarantee whatsoever of uniqueness.
Really great article. Very well put. I can see how plastics also apply. First order is that they are convenient for grocery bags and product packaging. Second order is that they are creating a huge amount of waste going mostly into the oceans. The recycling is far behind the creation of waste. It's already affecting the environment now. Who knows the damage 50 or 100 years from now
I find that the difference between how I make decisions and how people I consider smart make decisions is a difference of first order and second order thinking. In many ways I got lucky. I chose the computer science major and happened to love it, I became a software engineer and happened to love it. I did not think very deeply about these decisions. It's ironic because my job is to think about the control flow of methods, and the potential consequences that could arise from some current design.
I am trying very hard to integrate second order thinking into my decision making process.
The mental model notes are quite interesting to read but applying them is another matter. I have been reading mental models for a couple decades now and I honed in on one - invert always invert. This one simple rule has been really hard to apply.
Let me give an analogy of trying to reason about physical properties from first principles. This is another popular exhortation - think from first principles just like Feynman or Fermi. I can think from first principles in one extremely narrow field where I happen to have half a decade of education and 20 years of experience. To reason from first principles like Fermi or Feynman in a broad domain like physics requires a world class mind, a world class education and a world of experience.
Most mental model writing is akin to consuming youtube fitness porn. It looks easy to do, you look cool doing it and the end results are just spectacular. However, like Arnold or David Goggins it requires an inhuman dedication, purpose, ability to withstand pain, bounce back from trauma and just keep sacrificing. Most of the time the only person benefiting is the video creator from the ad-roll.
I appreciate the posts but I now believe these mental models are incredibly hard to do and like most of the self-improvement/growth hacking genre is just good for entertainment and commerce 99.99% of the time.
A better method would be 5 whys(https://en.wikipedia.org/wiki/Five_whys). Keep asking whys until you discover the root cause. Or in this case until you discover the larger impact of an event.
26 comments
[ 6.5 ms ] story [ 31.8 ms ] threadI don't know how important this is for most line employees. As a code monkey, I rarely have some strong bias besides "Angular is my personal preference over React" which immediately leads to "Well, should we use React anyway because the team knows it better and blah blah."
With that said, I enjoy Mongo, for its specific use cases.
On the face of it, it might seem the answer is yes. Fewer cognitive biases means better decisions that correspond more closely to reality. The thrust is that the more rational we are, the more effective our actions are.
However, in my observation, the type of rationality that "works" isn't always the same as the type that is "logically correct" (they can be identical in some instances, but not in all). Successful organizations, it seems to me, tend to prize "instrumental rationality" over "value rationality" / "epistemic rationality" [1].
Just wondering what folks' thoughts are on that?
Folks like Gerd Gigerenzer advocate the use of heuristics, which while are often biased, actually lead to good outcomes on average -- especially in incomplete information scenarios (which is most scenarios in real life). [2, 3]
It has even been shown that heuristics or crude (or even slightly wrong) simple models can outperform complex rigorous models in complex environments – because they are usually more robust to assumption violations under changing conditions [4,5]. In real life, assumptions are violated to various degrees almost all the time.
[1] https://en.wikipedia.org/wiki/Instrumental_and_value_rationa...
[2] https://www.verywellmind.com/what-is-a-heuristic-2795235
[3] https://en.wikipedia.org/wiki/Heuristics_in_judgment_and_dec...
[4] https://www.johndcook.com/blog/2012/09/17/robustness-of-simp...
[5] https://sloanreview.mit.edu/article/why-forecasts-fail-what-...
[0] https://www.lesswrong.com/posts/RAh4fekdiRhZxb2Kw/the-ultima...
If the goal is recognition of bias, you cultivate social-norms which incentivize critique (both in yourself and others). Notice, however, that this begets perverse incentives. I.e. you can fool yourself, "recognize" a billion biases, then applaud yourself for your diligence. You can also invent new biases that don't actually make much difference.
https://untools.co/second-order-thinking
"Even if you have cleverly identified a second-order effect in the opposite direction, it doesn't mean it actually outweighs the magnitude of the first-order effect – you still have to check."
In simple systems, first order effects tend to dominate.
In complex systems, higher order effects tend to dominate.
What a terrrible non-sequitur. Just because you think really hard, doesn't mean the conclusion so derived will be profound. Well known example, Newton thought really hard about many things and some paths didnt go anywhere.
The well-reasoned part is begging the question.
And there's no guarantee whatsoever of uniqueness.
I am trying very hard to integrate second order thinking into my decision making process.
Let me give an analogy of trying to reason about physical properties from first principles. This is another popular exhortation - think from first principles just like Feynman or Fermi. I can think from first principles in one extremely narrow field where I happen to have half a decade of education and 20 years of experience. To reason from first principles like Fermi or Feynman in a broad domain like physics requires a world class mind, a world class education and a world of experience.
Most mental model writing is akin to consuming youtube fitness porn. It looks easy to do, you look cool doing it and the end results are just spectacular. However, like Arnold or David Goggins it requires an inhuman dedication, purpose, ability to withstand pain, bounce back from trauma and just keep sacrificing. Most of the time the only person benefiting is the video creator from the ad-roll.
I appreciate the posts but I now believe these mental models are incredibly hard to do and like most of the self-improvement/growth hacking genre is just good for entertainment and commerce 99.99% of the time.