Would like to see per-capita, or age profile. One hot spot is centered on eastern Iowa/western Illinois. I live there, and am utterly surprised by this. That coincides on a cluster of college campuses; perhaps its careless language by college students?
The zoom level seems to make the data look very different. Zoomed out and the entire midwest is red, zoom in and it looks like there is one racist/homophobe in each of Muscatine, Iowa and Peoria, Illinois.
It's also unclear what criteria was used to define negative. If a reference to Lonely Island's "no homo" song gets flagged as negative for example, that may not fit in most people's definition of hate speech.
Total wild ass guess, but I'd bet the total number of tweets for a region is roughly proportional to its total population. If that's true, the normalized results you see here will be very similar to the per-capita results. It'd be interesting to verify this, however. It would tell you "this region's twitter users are X% [more|less] 'hateful' than the region's general population."
The data is normalized by number of tweets for a particular region (county, I believe). So what you're looking at is a comparison between the portion of total tweets for a region which is racist/homophobic/disabled-biased.
Put differently:
If county A has 100 tweets over the time period and 1 of them are racist it will appear identical to county B which has 1000 tweets of which 10 are racist over the same time period.
Its broken at least on current firefox with reasonable adblock etc. Its supposed to give you the actual numbers.
I think we're being trolled. The one guy in the north woods of wisconsin who uses twitter used the "n word" once but no one in the entire 4 million person metro Milwaukee area has. Hmm. The Milwaukee MSA has 0.5% of the population of the entire USA and about 90% of the black population of Wisconsin, so it should have included 750 tweets based on supposed sample size yet none of them in troll happy twitter were *-ist in any form. Unlikely. I think we're being trolled, perhaps as a sociology test of gullibility, and being fed data on "decade of founding of city" or something equally irrelevant. Or perhaps just random. Or data intentionally designed to make the east look worse than the west, although it seems weird to merge Wyoming in culturally with San Franciso or Berkley in with Colorado.
Should also include, at minimum, the percentage of Tweets that were racist / sexist / whatever. We need to know the total volume of tweets before we can tell if it's higher or lower than just "average" racism.
"To produce the map all tweets containing each 'hate word' were aggregated to the county level and normalized by the total twitter traffic in each county."
Was my initial thought too, but it seems this is already implemented. From the page:
To protect privacy, actual tweet locations have been
aggregated to the county level and normalized by number
of tweets.
I don't understand how normalization is supposed to protect privacy though.
There must be some kind of adjustment. I don't think it's possible that Idaho could "outshine" California otherwise. (Well, it's Idaho, but even so, there just aren't enough people.) Not to mention parts of Maine and Vermont beating out New York City.
Some of these words can also have other meanings. For example "fag" is english slang for cigarette and "chink" obviously has other meanings. I also know gay people who prefer the term 'queer' in describing themselves. I guess you could get a lot of "nigger" too by retweeting hip hop lyrics..
Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
Nearly all questions answered in the "about map" link:
=========================================
The data behind this map is based on every geocoded tweet in the United States from June 2012 - April 2013 containing one of the 'hate words'. This equated to over 150,000 tweets and was drawn from the DOLLY project based at the University of Kentucky. Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
To produce the map all tweets containing each 'hate word' were aggregated to the county level and normalized by the total twitter traffic in each county. Counties were reduced to their centroids and assigned a weight derived from this normalization process. This was used to generate a heat map that demonstrates the variability in the frequency of hateful tweets relative to all tweets over space. Where there is a larger proportion of negative tweets referencing a particular 'hate word' the region appears red on the map, where the proportion is moderate, the word was used less (although still more than the national average) and appears a pale blue on the map. Areas without shading indicate places that have a lower proportion of negative tweets relative to the national average.
The numbers that appear in the map during a mouse hover indicate the total number of hateful tweets and number of unique users sending them in each county.
==========================================
EDIT: The mouse overs don't appear to work very well in Chrome or Firefox, but from the one or two times I was able to see some numbers it appears that each red circle may be a dozen or less tweets. Also, the hot zones dissipate significantly the further you zoom in, so without any statistics or numbers it's difficult to draw conclusions.
A very interesting experiment, but given that the data is only normalized by Twitter traffic (non-response bias) this is in no way indicative of the actual distribution of racism.
Your "Hate Words" need some work. I don't think your hate words are used in the context you hope for in some cases. See the hot bed of hate that is Detroit Lakes MN.
I want to see that predefined rubric. I am unwilling to believe Iowans are more racist than Mississippians. Being racist in Iowa means hating like 3 people in the next county over.
As someone originally from MS, let me clarify that the state is not what most people imagine it is based on various movies or their U.S. history class. Mississippians take "the hospitality state" seriously.
To be clear, I'm solely focusing on the opportunity for racism. E.g., it's not an accident that the Germans, in the aggregate, that anti-Catholic sentiment in the 1850s was significantly stronger in the North than in the South.
> Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
I wonder how well a Bayesian classifier would work if the this was used as a training set. If it worked relatively well, there's no reason why you couldn't create a live version of the map.
Not very well. Twitter sentiment is a difficult problem.
Consider using millions of training examples (vs. thousands). This was done as part of the "distant supervision" Twitter sentiment technique. What this means is that tweets with positive emoticons were labeled as positive sentiment, and negative emoticons were labeled as having negative sentiment. Emoticons were stripped before training. This system got 80% accuracy.
I'm not sure it's interesting. It reflects something so complicated that it's hard to learn anything from. If you interpreted it as a heat map of racism, you'd conclude that the least racist region of the country is the mountain west outside of Idaho. Utah and Montana are almost spotless. It might be closer to interpret it as a heat map of "racist passion, overtly expressed" but even so I bet a lot of it has to do with the demographics of Twitter adoption. I guess sitting here guessing is kind of interesting, but I feel like I'm projecting my own assumptions onto it to try to figure out what it means rather than learning anything from it.
So this explains why LA disappeared since the counties there actually have some twitter active people living in it to help normalize it. When zoomed in, in the east there seems to be some halo effect at work were rural counties around bigger metros (Dallas, Atlanta, North Eastern Seaboard), with a lesser volume of tweets (low sample size) appear more active.
When zoomed out the 100th meridian population divide is as obvious as it could be. So At the global level normalization is pretty much not in effect. This might likely be caused by the not uniform distribution and size of counties in the US.
A better approach might be binning in equal area regions, with a threshold for low sample sizes.
I wonder if that's a flaw in the model by not normalizing over how popular twitter is in that area. Seems like it's more useful to know what percentage of tweets are hateful instead of counting sheer number.
ie: maybe there's just way more tweeting in Austin.
[edit: I'm apparently wrong. @ronaldx points out below a statement that suggests this is already happening and is why Orange County doesn't shine brighter]
Come again? It looks to me like the map tags almost every one of the major population centers. Southern California seems abnormally low, but that's just about it.
Yes and a comparison of similar low population counties on opposite sides of the Mississippi river shows staggering difference in claimed result yet no real difference in culture or demographics. We're being trolled, gentlemen. It only remains to figure out why, for what purpose. That troll study might end up being nearly as interesting as the supposed data would be, were it real.
I don't think we have enough information to draw any conclusions. In many cases, a single highly active user could be pushing a whole county (which also means that it isn't very suspicious to find a county without one).
The number of users tweeting hate and how connected they are to others is probably a more interesting map.
"I don't think we have enough information to draw any conclusions."
I would disgree in that we have a geographic claim that, for example, Nebraska and Iowa supposedly have spectacular difference in hate levels, like order of magnitude, yet simultaneously claims that Salt Lake City and San Fran are equally gay-friendly. And on a micro level, the hyper-segregated highly urban area to my east has no racist tweets at all, yet farmville east of the mississippi is supposedly a hotbed of racism but magically it all disappears west of the mississippi. I've lived in both Wisconsin and Alabama and anyone is seriously claiming the racial climate is basically identical? LOL.
Nope, we're being trolled here. I do agree with you that more data, or more analysis, would make it more interesting.
The map is difficult to interpret. You are making the mistake of conflating the national average rate with "no racist tweets at all". It is a jump to say that the difficulty interpreting the map results from intent of the creators, which is the line I would draw for trolling.
It could be titled something like "The Apparent Relative Visibility of Hate in Tweets at the County Level". "The Geography of Hate" isn't such a far jump from there, especially if you are just trying to show that the tweets are coming from all over.
although it's not clear in the language used, the data seems to have been normalized per Tweet:
"Orange County, California has the highest absolute number of tweets mentioning many of the slurs, but because of its significant overall Twitter activity, such hateful tweets are less prominent and therefore do not appear as prominently on our map."
In Idaho, New Mexico, and Wyoming, the counties that look really hot are very, very low population counties. For example, I'm assuming the giant red spot in New Mexico is Sierra County, population 11,988. The southern most county in Idaho is Oneida county, population 2900 (My high school was nearly that big).
You can't zoom in very far without the heat map degenerating into points for individual counties. I suspect in many areas they represent individual tweets or tweeters, but I'm not seeing any hover-over data, so I'm not sure. I'm surprised -- 150k tweets sounds like a lot, but there isn't enough data for a heat map effect unless you're zoomed out to the national level.
There are 3000+ counties and county equivalents in the US, so that makes about 50 tweets per county. That could easily be output by one or two people per county. Seeing as a lot of southern cities seem clean but with one or two really bright points nearby, I suspect that your thesis is correct and that most of these points are caused by particularly vociferous individual racists.
interesting idea but this map needs work: you really have to zoom in to see what's really happening. when zoomed out it looks like practically the entire USofA is full of hate.
It's funny (or annoying) how charts invite criticism and feature creep.
Thanks for an interesting chart.
What did you do to cut down on lying human classifiers? Do you give them an incentive? Did you have them vote as a group on the classification of a sentiment?
Whites aren't allowed to say the N word, and Anita can say the S word http://www.youtube.com/watch?v=Qy6wo2wpT2k&t=0m45s. But maybe your human classifiers can handle this problem too? I hope they got a good grade.
A fundamental problem with this strategy is that is may bias the results towards prolific haters—i.e. people who are frequently hateful on Twitter. These people may live in places where social norms permit online hateful speech without repercussions. This strategy will miss people who are haters, but must be more subdued in their expressions of hatred online.
It would be interesting to identify people who have posted just a few hateful messages—perhaps few enough that they can get away with it in their local social context. This may more sensitive to occult haters.
Something like this:
1. Identify individual twitter users as being hateful in a particular category. For instance, user A uses the word "chink" 3x and "fag" 5x in 100 tweets, so he gets added to the "chink" and "fag" categories. Play with these threshold to see what makes sense.
2. Divide the # of hateful users in each category by the total # of users in that location. Allocation of users to location can be done proportional to the # of tweets they make from each location.
Another reason why it may seem off to the viewer, (Mississippi is less racist than other areas may have to do with the populations in particular areas.)
If you have 50% people of the racial group you are sampling racist tweets from living in a particular state, you should see less racist tweets by %.
I have always disliked the term "homophobic". "Phobic" means that someone has a fear of something. Given that the people using these words in tweets are doing so in a very public fashion, they are not very likely to be afraid of gay people. They are perhaps hateful and/or prejudiced, but phobic is just an entirely inaccurate term in this and most other instances where that term is used.
Bullies are only brave when targeting a minority. They're scared of things they don't understand but will only make a noise about it if they feel in a strong position.
A better complaint about the word homophobic is that its meaning is dependant on one understanding "homo" to mean homosexual. Otherwise it means one who fears same-ness. Most homophobes actually, obviously, fear difference.
But again, that makes a tremendous number of assumptions as to why the person is saying those words. I get that it is a term meant to degrade and shame those who use these terms, but I just don't think it's an accurate description in the majority of cases.
Phobia in the original greek covers "fear", "hatred", and "aversion". For instance, hydrophobic molecules do not fear water.
That said, homophobia, in cultures like North America, is very much based on fear. It's not a simple sort of fear, but it is wrapped up in how gay people challenge gender roles, and the fear that men have of being less than dominant. People who aspire to socially dominant roles, who secretly have same-sex feelings, are often the worst oppressors. Before you condemn the term, I suggest you read up on it.
60 comments
[ 3.4 ms ] story [ 127 ms ] threadIt's also unclear what criteria was used to define negative. If a reference to Lonely Island's "no homo" song gets flagged as negative for example, that may not fit in most people's definition of hate speech.
[Edit: I should place extra emphasis on the word "roughly," in my OP as well.]
An interesting distinction you've made there.
It makes a big difference if red is 4-5 tweets (per what time period), or 4-5 thousand.
Put differently: If county A has 100 tweets over the time period and 1 of them are racist it will appear identical to county B which has 1000 tweets of which 10 are racist over the same time period.
I think we're being trolled. The one guy in the north woods of wisconsin who uses twitter used the "n word" once but no one in the entire 4 million person metro Milwaukee area has. Hmm. The Milwaukee MSA has 0.5% of the population of the entire USA and about 90% of the black population of Wisconsin, so it should have included 750 tweets based on supposed sample size yet none of them in troll happy twitter were *-ist in any form. Unlikely. I think we're being trolled, perhaps as a sociology test of gullibility, and being fed data on "decade of founding of city" or something equally irrelevant. Or perhaps just random. Or data intentionally designed to make the east look worse than the west, although it seems weird to merge Wyoming in culturally with San Franciso or Berkley in with Colorado.
Emphasis mine.
I don't understand how normalization is supposed to protect privacy though.
Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
=========================================
The data behind this map is based on every geocoded tweet in the United States from June 2012 - April 2013 containing one of the 'hate words'. This equated to over 150,000 tweets and was drawn from the DOLLY project based at the University of Kentucky. Because algorithmic sentiment analysis would automatically classify any tweet containing 'hate words' as "negative," this project relied upon the HSU students to read the entirety of tweet and classify it as positive, neutral or negative based on a predefined rubric. Only those tweets that were identified by human readers as negative were used in this analysis.
To produce the map all tweets containing each 'hate word' were aggregated to the county level and normalized by the total twitter traffic in each county. Counties were reduced to their centroids and assigned a weight derived from this normalization process. This was used to generate a heat map that demonstrates the variability in the frequency of hateful tweets relative to all tweets over space. Where there is a larger proportion of negative tweets referencing a particular 'hate word' the region appears red on the map, where the proportion is moderate, the word was used less (although still more than the national average) and appears a pale blue on the map. Areas without shading indicate places that have a lower proportion of negative tweets relative to the national average.
The numbers that appear in the map during a mouse hover indicate the total number of hateful tweets and number of unique users sending them in each county.
==========================================
EDIT: The mouse overs don't appear to work very well in Chrome or Firefox, but from the one or two times I was able to see some numbers it appears that each red circle may be a dozen or less tweets. Also, the hot zones dissipate significantly the further you zoom in, so without any statistics or numbers it's difficult to draw conclusions.
A very interesting experiment, but given that the data is only normalized by Twitter traffic (non-response bias) this is in no way indicative of the actual distribution of racism.
I wonder how well a Bayesian classifier would work if the this was used as a training set. If it worked relatively well, there's no reason why you couldn't create a live version of the map.
Something like http://aworldoftweets.frogdesign.com/ maybe?
Consider using millions of training examples (vs. thousands). This was done as part of the "distant supervision" Twitter sentiment technique. What this means is that tweets with positive emoticons were labeled as positive sentiment, and negative emoticons were labeled as having negative sentiment. Emoticons were stripped before training. This system got 80% accuracy.
http://cs.wmich.edu/~tllake/fileshare/TwitterDistantSupervis...
"Pet Peeve #208: Geographic profile maps which are basically just population maps."
When zoomed out the 100th meridian population divide is as obvious as it could be. So At the global level normalization is pretty much not in effect. This might likely be caused by the not uniform distribution and size of counties in the US.
A better approach might be binning in equal area regions, with a threshold for low sample sizes.
ie: maybe there's just way more tweeting in Austin.
[edit: I'm apparently wrong. @ronaldx points out below a statement that suggests this is already happening and is why Orange County doesn't shine brighter]
The three hottest counties in New Mexico and Idaho have populations under 34000 combined. One of them is just 2900 people.
The number of users tweeting hate and how connected they are to others is probably a more interesting map.
I would disgree in that we have a geographic claim that, for example, Nebraska and Iowa supposedly have spectacular difference in hate levels, like order of magnitude, yet simultaneously claims that Salt Lake City and San Fran are equally gay-friendly. And on a micro level, the hyper-segregated highly urban area to my east has no racist tweets at all, yet farmville east of the mississippi is supposedly a hotbed of racism but magically it all disappears west of the mississippi. I've lived in both Wisconsin and Alabama and anyone is seriously claiming the racial climate is basically identical? LOL.
Nope, we're being trolled here. I do agree with you that more data, or more analysis, would make it more interesting.
It could be titled something like "The Apparent Relative Visibility of Hate in Tweets at the County Level". "The Geography of Hate" isn't such a far jump from there, especially if you are just trying to show that the tweets are coming from all over.
So, given that this is a rate-based map, the XKCD doesn't apply
"Orange County, California has the highest absolute number of tweets mentioning many of the slurs, but because of its significant overall Twitter activity, such hateful tweets are less prominent and therefore do not appear as prominently on our map."
Thanks for an interesting chart.
What did you do to cut down on lying human classifiers? Do you give them an incentive? Did you have them vote as a group on the classification of a sentiment?
Whites aren't allowed to say the N word, and Anita can say the S word http://www.youtube.com/watch?v=Qy6wo2wpT2k&t=0m45s. But maybe your human classifiers can handle this problem too? I hope they got a good grade.
It would be interesting to identify people who have posted just a few hateful messages—perhaps few enough that they can get away with it in their local social context. This may more sensitive to occult haters.
Something like this: 1. Identify individual twitter users as being hateful in a particular category. For instance, user A uses the word "chink" 3x and "fag" 5x in 100 tweets, so he gets added to the "chink" and "fag" categories. Play with these threshold to see what makes sense. 2. Divide the # of hateful users in each category by the total # of users in that location. Allocation of users to location can be done proportional to the # of tweets they make from each location.
Cool project :).
If you have 50% people of the racial group you are sampling racist tweets from living in a particular state, you should see less racist tweets by %.
A better complaint about the word homophobic is that its meaning is dependant on one understanding "homo" to mean homosexual. Otherwise it means one who fears same-ness. Most homophobes actually, obviously, fear difference.
That said, homophobia, in cultures like North America, is very much based on fear. It's not a simple sort of fear, but it is wrapped up in how gay people challenge gender roles, and the fear that men have of being less than dominant. People who aspire to socially dominant roles, who secretly have same-sex feelings, are often the worst oppressors. Before you condemn the term, I suggest you read up on it.