Results of yesterday's personality poll: we're strange.
On the hypothesis that this site selects for certain personality types that are different from the general population, I decided to run some simple statistics on the results of yesterday's MBTI poll (http://news.ycombinator.com/item?id=943722).
I compared the frequencies of each of the types to the population means on the Wikipedia article (http://en.wikipedia.org/wiki/Myers-Briggs_Type_Indicator#Type_dynamics_and_development). The results show that we are a very atypical group indeed:
Using a binomial test at a 99% confidence level.
ISTJ is significant: 1.82% ( 12/660) vs. 11.60% expected.
ISFJ is significant: 0.61% ( 4/660) vs. 13.80% expected.
INFJ is NOT significant: 2.12% ( 14/660) vs. 1.50% expected.
INTJ is significant: 31.21% (206/660) vs. 2.10% expected.
ISTP is significant: 2.42% ( 16/660) vs. 5.40% expected.
ISFP is significant: 0.15% ( 1/660) vs. 8.80% expected.
INFP is significant: 8.48% ( 56/660) vs. 4.30% expected.
INTP is significant: 28.94% (191/660) vs. 4.30% expected.
ESTP is significant: 0.45% ( 3/660) vs. 4.30% expected.
ESFP is significant: 0.15% ( 1/660) vs. 8.50% expected.
ENFP is significant: 3.33% ( 22/660) vs. 8.10% expected.
ENTP is significant: 8.48% ( 56/660) vs. 3.30% expected.
ESTJ is significant: 1.36% ( 9/660) vs. 8.70% expected.
ESFJ is significant: 0.76% ( 5/660) vs. 12.30% expected.
ENFJ is NOT significant: 2.27% ( 15/660) vs. 2.40% expected.
ENTJ is significant: 7.42% ( 49/660) vs. 1.80% expected.
EDIT:
Here are the results broken down by pairs:
FP is significant: 12.12% ( 80/660) vs. 29.70% expected.
TP is significant: 40.30% (266/660) vs. 17.30% expected.
FJ is significant: 5.76% ( 38/660) vs. 30.00% expected.
NJ is significant: 43.03% (284/660) vs. 7.80% expected.
NF is NOT significant: 16.21% (107/660) vs. 16.30% expected.
TJ is significant: 41.82% (276/660) vs. 24.20% expected.
NP is significant: 49.24% (325/660) vs. 20.00% expected.
NT is significant: 76.06% (502/660) vs. 11.50% expected.
EN is significant: 21.52% (142/660) vs. 15.60% expected.
EJ is significant: 11.82% ( 78/660) vs. 25.20% expected.
IP is significant: 40.00% (264/660) vs. 22.80% expected.
IS is significant: 5.00% ( 33/660) vs. 39.60% expected.
EF is significant: 6.52% ( 43/660) vs. 31.30% expected.
IT is significant: 64.39% (425/660) vs. 23.40% expected.
IJ is significant: 35.76% (236/660) vs. 29.00% expected.
IN is significant: 70.76% (467/660) vs. 12.20% expected.
ET is NOT significant: 17.73% (117/660) vs. 18.10% expected.
EP is significant: 12.42% ( 82/660) vs. 24.20% expected.
ES is significant: 2.73% ( 18/660) vs. 33.80% expected.
IF is significant: 11.36% ( 75/660) vs. 28.40% expected.
SP is significant: 3.18% ( 21/660) vs. 27.00% expected.
ST is significant: 6.06% ( 40/660) vs. 30.00% expected.
SJ is significant: 4.55% ( 30/660) vs. 46.40% expected.
SF is significant: 1.67% ( 11/660) vs. 43.40% expected.
51 comments
[ 2.4 ms ] story [ 125 ms ] threadAny suggestions?
Yes. It is very likely that once you have gone through what appears to be a laborious procedure to generate a description, you will accept whatever description comes out of that procedure.
http://en.wikipedia.org/wiki/Forer_effect
http://www.skepdic.com/forer.html
Gives literal meaning to the phrase split personality :)
Perhaps our Erlang-flooding people-filter has some influence on this.
If they are meaningless then I'd have expected randomness...
People are responding to individual items, one item at a time, in an unvalidated test. Then they see a group characteristic description and the Forer effect (the fallacy of personal validation) leaves them convinced their answers to the unvalidated questions led to new information, which is the issue that still needs to be demonstrated by a validation study.
There are a couple that I could interpret as "somewhat like me" and several that are clearly not me at all.
If you take as a given that the HN population is not representative of the population at large, then it is clear that Myers-Briggs measures something, as it correctly identified that.
If you'd asked me for a hypothesis before the survey, I would have predicted a significant overrepresentation of NT-types. That would seem to give the test some predictive value, unless you attribute the full significance of the findings to selection bias in who chose to answer (in which case Myers-Briggs would be an outstanding predictor of who will answer surveys).
I can be extremely charming, I like being outgoing in large groups, and a friend of mine said I was probably the most popular person in high school. I also dislike most people so I spend a lot of time online. I can be extremely rational and I have little problems programming computers, but I often make irrational decisions on a whim because I trust my subconscious way to much.
So, I don't know am I an introvert or an extrovert, rational or irrational? I honestly don't know and I don't think picking one is important because they don't tell me something I don't already know.
My point is this: Just because you don't buy into the whole M-B system, can't you appreciate that it is at least good at indicating what type of group answered it? IE - Hackers vs Painters vs Philosophers.
That remains to be demonstrated with respect to "HN readers," because the group of people who responded to the online poll are very likely not a representative subset of HN readers in general.
Response groups can number in the millions and still be biased by self-selection. There are well known historical examples taught in any good elementary statistics class.
http://mathforum.org/kb/thread.jspa?threadID=194473&tsta...
http://aurora.wells.edu/~srs/Math151-Fall02/Litdigest.htm
http://historymatters.gmu.edu/d/5168/
http://i36.tinypic.com/3011z11.png
It matches up surprisingly well with HN, except that (perhaps as expected) the results are 97% I.
Merely reading and writing are "odd" activities. Humans aren't born knowing how to do them (unlike, say, walking or talking) and not all humans enjoy them in the same way.
I have tested as INTJ many times before, but yesterday my result was ESFJ. Go figure. I've been a programmer all my life and I have the classic personality. All I did was answer their questions. I remember thinking that many of my answers could have easily gone either way. Maybe if I took it today (which isn't a Monday), some answers would be different. I have no idea why the result would vary so much.
You are doing analysis to 4 significant digits of work based upon the appropriateness of questions with no significant digits. You've built a concrete structure on top of quicksand. Nice job, but I really don't think it means much.
Only true when the large numbers are representative of the whole population because the sample is random, as any elementary statistics textbook will point out, and as I pointed out in the previous thread: This kind of voluntary response poll would have exactly NIL value in answering the question that appears to be asked here, which is, what Myers-Briggs types are there on HN? The people who come forward to answer the poll are not a random sample of the tens of thousands of people who read HN.
http://mathforum.org/kb/thread.jspa?threadID=194473&tsta...
http://aurora.wells.edu/~srs/Math151-Fall02/Litdigest.htm
http://historymatters.gmu.edu/d/5168/
A response group can be ever so large, but if that group was not randomly selected, there is no particular reason to suppose that it accurately represents the population of interest.
It's as if you measured height and had to classify everyone into tall or short because your theory suggests that no-one is average height. Obviously most people are going to be actually hovering around the average but the minor variations in height and measurement combined with an abitrary cutoff mean that one day large chunks of people will be "tall" and the next they'll be "short" and vice versa.
And yet, based on your testing, one day you'll be telling these people they'd be ideally suited to basketball, then next suggesting they become jockeys.
http://en.wikipedia.org/wiki/Myers-Briggs_Type_Indicator#Sta...
I've kept my opinions out of this discussion so far, but I definitely agree with what you say about the test. While the MBTI undoubtedly indicates something, its utility and validity are questionable. It's interesting for novelty value, but that's about it.
I'm definitely in the same boat as you: at various times over the past 5 years or so I have scored as INTP, ENTP and ISTP, so in both our cases, test-retest reliability is an issue. (I didn't take the test yesterday, because at this point I feel I'm too familiar with it to answer without bias.)
I make no claims about how valid this analysis is; it's certainly not going to appear in a statistics journal any time soon. I just decided to do it for fun and was surprised by just how drastically different the results are from the general population.
At the end of the day, all that matters is how you view your own personality. The test is just a way to try to pry that data out of your brain.
IMHO your personality is not something set in stone - it's a rather slow-changing beast surely, but I think it's a fallacy to believe that it's somehow cemented by some age.
So I do think the results are meaningful.
To talk intelligently about the results of the test, you must first understand the vocabulary they are using. That vocabulary is more technical than the common understanding of the words that are used to name the categories. This test, love it or hate it, was created by people in the sciences, and the categories, whether useful as predictors or not, have specific definitions.
The M-B test is centered around four behaviors [(N|S)(F|T)] and two attitudes [I|E]. The attitudes are perhaps the most commonly misunderstood, because we have strong natural usage around the words extrovert and introvert that is quite different from the technical definition in the M-B system. For starters, it is probably safe to just think of these as modifiers on the behaviors, such that an extroverted behavior is directed out to the world, and an introverted behavior is directed towards your mental life.
Of the four behaviors [(S|N)(T|F)], two behaviors function to Judge the world (Thinking, Feeling) and two behaviors function to Perceive the world (Sensing, iNtuiting). Feeling is another misunderstood one. While it includes "emotional" decisions, it also includes "going with your gut".
The least obvious piece of the system is how the four letters interact. Lets take INTP, since for once, we are the majority here, and it is mine, so I can use myself as the subject to discuss rather than the obnoxious "If one is a ... than he/she..."
What these four letters do (yes, in theory) is establish my PREFERENCE for these behaviors. From INTP, my core type is NT or Rational (according to Keirsey). I prefer to perceive through iNtuition, and decide through Thinking.
The I and the P are modifiers on those types. The P tells me that, when interacting with the world (in my Extraverted attitude), I prefer to be in a Perceiving behavior. The [I] tells me that, all other things being equal, I prefer to be in an Introverted attitude.
The primary piece of M-B theory that I find interesting and worthy of debate is this: A person has a primary Perceiving and Judging behavior, and they tend to prefer to use one in an Extraverted attitude and the other in an Introverted attitude.
This implies that, since I prefer to use my Perceiving behavior (iNtuiting) in my Extraverted attitude, I prefer to use my Judging behavior (Thinking) in my Intraverted attitude. This is the difference between an INTP and an INTJ.
So, M-B would say my two main behaviors are Ne and Ti. Ok, now let's deal with the [I]. This tells us which, of the two attitudes I prefer. So, according to M-B:
My primary behavior is Ti
My secondary behavior is Ne
My terciary behavior is S(i) (sometimes i is not shown)
My inferior behavior is Fe
One actionable result for me from taking this test has been that it reminds me that I have a tendency to under use my extraverted feeling decision maker. Which I have found to be true, and so I make sure to consciously bolster this response when the opportunity to use it presents itself.
In closing, it is crucial to remember that this test determines PREFERENCES. It says NOTHING about ability or correctness. If you have Judged Thinking better than Feeling, or think you can perceive more by Sensing than iNtuiting, than you are probably out of touch with your other behavior. I believe that we should all strive to be very balanced on this test.
However, I do agree with M-B theory that people do have a tendency to gravitate to the poles, often because of strength or social culture. I believe one of the best uses of this test is to recognize which behaviors you prefer, and also recognize the ones that you could benefit from giving more atte...
If you're interested in showing that the frequency distribution for personality types is different on Hacker News than in the general population, then the right way to show this would be to run a single multinomial test to see whether your empirical multinomial distribution is outside of the range of chance variation from the canonical distribution listed on Wikipedia for the MBTI. See http://en.wikipedia.org/wiki/Multinomial_test for a description of this hypothesis test.
To give you a sense of why your multiple comparisons across binomial tests is problematic, imagine that the INTP group is underrepresented because all of the people who would test as INTP tested as ENTP this time around; in this case, you run two different binomial tests, but you really only have one result, because the size divergence in one group is effectively the cause of the size divergence in the other group.
I hope that makes it clear why you should really treat your data set as containing an N of 1 and doing a simple null hypothesis test of the probability of getting your results given the canonical values for the global multinomial distribution, rather than testing based on a series of binomial marginal distributions.