I always had the suspicion (which may be wrong) that in questions like "which is a more likely outcome of 6 coin tosses, HHHHHH or HTHHTT", people tend to understand this question as: "which is a more likely outcome of 6 coin tosses, a sequence of 6 heads, or another sequence without any obvious regularity in it", in which case the latter is clearly the correct answer.
I have an additional suspicion: in my experience, people with slight autistic tendencies often have problems with sorting their impressions into broad, general categories, but instead always analyze (even remember) the exact impression (or "special case") they are confronted with. For people like this, the question "HHHHHH or HTHHTT" is a completely different question than for the majority of the population. So, the problem to me does not seem to be a problem with human intuition for probability (or with System 1, in Kahneman's words), but just one of semantics. If you go further and assume that scientists have a higher percentage of people with autistic tendencies than the general population, then the problem simply becomes one of communication between the researcher and the subject.
I read Thinking: Fast and Slow a few years ago and remember many experiments where it was clear to me that the subjects just answered a different question than the one the researcher asked them. And I do not mean that they "anchored" their answers on something else or they replaced the question with a different question because they had no answer to the original question - they just had a completely different understanding of the question itself, and their answers to this question were correct, even in System 2.
I find a lot of psychology experiments about game theory have this problem, for an example.
The common mistake is to analyze it as if people were in a prisoners’ dilemma situation rather than an iterated prisoners’ dilemma situation: there’s no way to fully detach lab participants from the fact that the people they’re interacting with exist in society, which is a sort of large scale iteration of all the social games.
Once you realize lab participants are treating each other as members of society they’ll have to deal with in the future, instead of isolated lab rats only existing for one round of experimentation, almost all of the “people don’t understand game theory” results disappear.
It seems instead we have multiple papers confirming psychologists don’t understand society.
Papers are published within an academic pretense. So you also have to account for the brown nosing and sucking up to authority.
You pay tuition, but you don’t get your degree without conforming your behavior to the expectations of faculty, and the institutional demands to publish words that seem like a readable corpus of text.
This goes up the chain, as the expectation pervades a normative culture.
Academia eats Prestigious Institutions. Institutions eat Departments. Chair of Department eats Professors. Professors eat Students. Students photosynthesize Words pleasing to the ears of their parasitic herbivores.
> the subjects just answered a different question than the one the researcher asked them.
I completely agree with your interpretation. And that is why science education is so important. Thinking like a scientist is not natural. It is a learned skill. You need to stop asking yourself what it feels is the correct answer and analyze it to get a universal truth.
A lot of "controversial" political topics raise from the difference of perceived reality and reality itself. We suck at statistics and we suck at getting the right answer. And, even educated professionals fail to apply that knowledge to all the domains in their lives.
> And that is why science education is so important. Thinking like a scientist is not natural. It is a learned skill. You need to stop asking yourself what it feels is the correct answer and analyze it to get a universal truth.
I agree, but you have to be careful: a universal truth might not exist outside fixed formal systems, and after years (or centuries) of analyzing the problem you might realize that you were only analyzing the question itself, and not the problem it tried to state.
I think in a more deeper way, this is the basic problem behind many experiments like the one described in the paper: they all assume that the question they are giving to the subject is equivalent to the problem they are trying to present to them. However, as soon as you transfer something into spoken (or written) language, something is lost. Always. You can never be sure that you were successful in transferring an idea, or a problem statement, to another person in exactly the way you are thinking of it. Because if this, imho you just cannot ask people which is more likely, HHHHHH or HHTTHT if I toss a coin 6 times, and then, if 70% answer HHTTHT, say that people's intuition for probability is bad, or explain it with "anchoring" or "substitution" or another mechanism. It only proves that the majority of people will not give the correct answer if a probabilistic problem is given in formal language to them, in a controlled environment. This is a completely different problem. If the same problem appeared to the subjects in an everyday situation, directly and without the need for language as a mediator, they may very well have precise intuition. I think the problem is, however, that creating such situations is hard for researchers, and also hard to reproduce. But as long as you are trying to keep a sterile testing environment, you will only get answers that will hold in this sterile environment, not in the real world. On the other hand, creating social science or psychology experiments that are not sterile might very well be extremely unethical.
Our intuition is that random processes produce high-entropy output. Patterns carry less entropy and therefore offer less convincing evidence that the result is indeed the product of a random process if any alternative explanation or generating function is permitted. It would not be biased to think so, but it would be a misinterpretation of the question to use that as the basis of an answer, and of course people may be biased in their reading of the question.
But to be fair to researchers that's not the bias they are trying to illustrate. There does appear to exist human bias that 'characteristics of the generating process will be represented, “not only globally in the entire sequence, but also locally in each of its parts”.
There are other ways of putting it, or there are related biases if you like, which I would generalise as a belief that mean reversion should be expected in the short term, a notion that the future will balance the past naturally, inevitably and here's the bias: quickly. It arises imho from a misplaced intuition of mean reversion 1) as a process that operates, like a force of nature, rather than being an emergent phenomenon and 2) as a tendency for results to converge to expectation in absolute terms rather than in the mean.
Consider the following three plots. In one of the plots each of the blue points is sampled with equal probability from the entire square. Which one is it?
All three are possible results of a random sampling, but presumably you mean the vaguely-specified question “which one looks similar to most random samplings?”
My stats professor said he used to have an assignment where he had students flip a coin 100 times and record the results on a sheet of paper. He said he could be reasonably confident who did and did not actually flip the coin because the students who did not flip the coin would not have any runs longer than 2 or 3.
There is a very accessible "numberphile" video which I think is closely related to this, about playing the odds on predicting elements of a nominally-random coin-flip sequence using the expected run length.
The most interesting part of this paper to me is that "HHT" is more likely than "HHH" to appear in a sub-set of tosses of length 4 or 12 (for example), and that any sequence like HHH or HHT has a "wait time" calculated by mathematicians. The wait times are how many flips on average you need to produce the sequence and they are not equal -- HHHH has a wait time of 30 vs HHHT has a wait time of 16 (not sure if I got the numbers exactly right.)
So the answer to the question "Which is more likely to be seen in a sequence of 10 coin flips, HHH or HHT", the answer is surprisingly, "HHT".
This is the key point of the paper. People are shown a sequence and they answer the question as if the sequence was pulled from a longer sequence (though that isn't strictly what the question asks).
Not having read the paper to which to parent referred, but I might be able to shed some light on this...
Let us describe the state of the previous two tosses as XX, with X being either H or T. If --(X)--> describes the next toss, we can describe the transition between states as follows:
HH --(H)--> HH
HH --(T)--> HT
and
HT --(H)--> TH
HT --(T)--> TT
etc.
If we are waiting for HHT, starting from state HH
HH --(H)--> HH : or, we get back to state HH
HH --(T)--> "HHT" : HHT is found and we stop searching
as opposed to waiting for HHH
HH --(H)--> "HHH" : HHH is found, and we stop
HH --(T)--> HT : and we are in a new state which requires at least two more --(H)--> in sequence before we return to state HH.
13 comments
[ 3.0 ms ] story [ 46.6 ms ] threadI have an additional suspicion: in my experience, people with slight autistic tendencies often have problems with sorting their impressions into broad, general categories, but instead always analyze (even remember) the exact impression (or "special case") they are confronted with. For people like this, the question "HHHHHH or HTHHTT" is a completely different question than for the majority of the population. So, the problem to me does not seem to be a problem with human intuition for probability (or with System 1, in Kahneman's words), but just one of semantics. If you go further and assume that scientists have a higher percentage of people with autistic tendencies than the general population, then the problem simply becomes one of communication between the researcher and the subject.
I read Thinking: Fast and Slow a few years ago and remember many experiments where it was clear to me that the subjects just answered a different question than the one the researcher asked them. And I do not mean that they "anchored" their answers on something else or they replaced the question with a different question because they had no answer to the original question - they just had a completely different understanding of the question itself, and their answers to this question were correct, even in System 2.
The common mistake is to analyze it as if people were in a prisoners’ dilemma situation rather than an iterated prisoners’ dilemma situation: there’s no way to fully detach lab participants from the fact that the people they’re interacting with exist in society, which is a sort of large scale iteration of all the social games.
Once you realize lab participants are treating each other as members of society they’ll have to deal with in the future, instead of isolated lab rats only existing for one round of experimentation, almost all of the “people don’t understand game theory” results disappear.
It seems instead we have multiple papers confirming psychologists don’t understand society.
You pay tuition, but you don’t get your degree without conforming your behavior to the expectations of faculty, and the institutional demands to publish words that seem like a readable corpus of text.
This goes up the chain, as the expectation pervades a normative culture.
Academia eats Prestigious Institutions. Institutions eat Departments. Chair of Department eats Professors. Professors eat Students. Students photosynthesize Words pleasing to the ears of their parasitic herbivores.
You are what you eat.
I completely agree with your interpretation. And that is why science education is so important. Thinking like a scientist is not natural. It is a learned skill. You need to stop asking yourself what it feels is the correct answer and analyze it to get a universal truth.
A lot of "controversial" political topics raise from the difference of perceived reality and reality itself. We suck at statistics and we suck at getting the right answer. And, even educated professionals fail to apply that knowledge to all the domains in their lives.
I agree, but you have to be careful: a universal truth might not exist outside fixed formal systems, and after years (or centuries) of analyzing the problem you might realize that you were only analyzing the question itself, and not the problem it tried to state.
I think in a more deeper way, this is the basic problem behind many experiments like the one described in the paper: they all assume that the question they are giving to the subject is equivalent to the problem they are trying to present to them. However, as soon as you transfer something into spoken (or written) language, something is lost. Always. You can never be sure that you were successful in transferring an idea, or a problem statement, to another person in exactly the way you are thinking of it. Because if this, imho you just cannot ask people which is more likely, HHHHHH or HHTTHT if I toss a coin 6 times, and then, if 70% answer HHTTHT, say that people's intuition for probability is bad, or explain it with "anchoring" or "substitution" or another mechanism. It only proves that the majority of people will not give the correct answer if a probabilistic problem is given in formal language to them, in a controlled environment. This is a completely different problem. If the same problem appeared to the subjects in an everyday situation, directly and without the need for language as a mediator, they may very well have precise intuition. I think the problem is, however, that creating such situations is hard for researchers, and also hard to reproduce. But as long as you are trying to keep a sterile testing environment, you will only get answers that will hold in this sterile environment, not in the real world. On the other hand, creating social science or psychology experiments that are not sterile might very well be extremely unethical.
But to be fair to researchers that's not the bias they are trying to illustrate. There does appear to exist human bias that 'characteristics of the generating process will be represented, “not only globally in the entire sequence, but also locally in each of its parts”.
There are other ways of putting it, or there are related biases if you like, which I would generalise as a belief that mean reversion should be expected in the short term, a notion that the future will balance the past naturally, inevitably and here's the bias: quickly. It arises imho from a misplaced intuition of mean reversion 1) as a process that operates, like a force of nature, rather than being an emergent phenomenon and 2) as a tendency for results to converge to expectation in absolute terms rather than in the mean.
1. http://norvig.com/plot1.png
2. http://norvig.com/plot2.png
3. http://norvig.com/plot3.png
You can read the rest from http://norvig.com/experiment-design.html
https://www.youtube.com/watch?v=tP-Ipsat90c
So the answer to the question "Which is more likely to be seen in a sequence of 10 coin flips, HHH or HHT", the answer is surprisingly, "HHT".
This is the key point of the paper. People are shown a sequence and they answer the question as if the sequence was pulled from a longer sequence (though that isn't strictly what the question asks).
Do you have a link to the paper?
I can't quickly wrap my mind around why HHT would be have a shorter wait time than HHH
Let us describe the state of the previous two tosses as XX, with X being either H or T. If --(X)--> describes the next toss, we can describe the transition between states as follows:
HH --(H)--> HH
HH --(T)--> HT
and
HT --(H)--> TH
HT --(T)--> TT
etc.
If we are waiting for HHT, starting from state HH
HH --(H)--> HH : or, we get back to state HH
HH --(T)--> "HHT" : HHT is found and we stop searching
as opposed to waiting for HHH
HH --(H)--> "HHH" : HHH is found, and we stop
HH --(T)--> HT : and we are in a new state which requires at least two more --(H)--> in sequence before we return to state HH.
I hope this helps.
EDIT: reformatting & typo