"We have found that even a modestly sophisticated statistical model does very little to counter unreliable data"... so.. garbage in... garbage out?
I have an alternative explanation for this : people who buy newspapers and mainstream web sites are liberal. Pollsters and analysts who wish to make a living must get clicks and sell copy. The ones that create models that agree with the general zeitgeist of the liberal milieu get clicks and sell copy, other models are gradually ground to extinction.
Markets are not efficient mechanisms for creating insight.
Unfortunately your explanation is not correct. For example, it does not explain the extensive coverage of Donald Trump that was used to get clicks and sell copy, despite his violation of everything liberals hold dear...
Early in the campaign, Trump acknowledged his media advantage. “I’ve gotten so much free advertising, it’s like nothing I’d have expected,” he told the New York Times in September. “When you look at cable television, a lot of the programs are 100% Trump, so why would you need more Trump during the commercial breaks?”[1]
The coverage of Trump was entirely negative, though. It makes perfect sense that one could sell a magazine to liberals by attacking the main competitor to liberals in the Presidential race.
Trying not to venture too far into a political discussion, I think the majority of the coverage of both was negative. Towards the last couple weeks, the quantity of coverage was very disproportionate depending on the source of the news [0]. People of the same political affiliation tend to gravitate towards the same news sources, and those sources appeared to be appealing to the affiliation of their viewers with the selection of information given.
The particular example I have provided is from Fox News because it was quick to search for, but I suspect you could find similar elsewhere about other news organizations.
Did you follow mainstream media during election? If you did you wouldn't have to ask that. Have a look at the Podesta Wikileaks. MSM colluded with Clinton campaign creating the narrative of Trump being 'crazy racist'. Seriously look at the emails its all there.
Fabricated Fake news about Trump raping multiple people and 13 year kid certainly didn't help Trump to win.
"downvoted by bots already" i live on the same internet as you, how are you sure its bots downvoting you when im not? really destroys your credibility to be certain of something that you cant be.
The coverage of Hillary tended to criticize her actions (e.g. regarding emails). The coverage of Trump tended to criticize his character. Objectively speaking, one was more of an ad hominem attack, which reflects poorly on the media.
Ad hominem is a fallacy when evaluating an argument. When you're evaluating a person (as a candidate), arguments are necessarily ad hominem, whether they're about temperament or conduct.
Wasn't it done on purpose to push 'pied piper candidates', so dems can have a slam dunk. Trump was identified as one of the 'pied piper candidates' in podesta emails. Even pravda ran story on it [1]. Those 'and tell the press' emails would make even Stalin blush.
Fear sells better than anything. You would think the liberal papers would get more clicks talking about how Trump was winning. Also, if Hillary voters think the election is in the bag, more would stay home versus if they thought it was close.
Perhaps polling is just more or less accurate because fewer people take part in polls.
The polls Huffington Post was using weren't conducted by them [1]. They were using the same polls everyone else had access to. Their model was probably worse than 538's though.
Polls are mainly a way to influence people to vote certain way not show who people are actually voting for. Social proof.
This will be obvious if you look at the weighting and questions 'Would you vote for Obama or Trump'. Reuters/Ipsos and other outlets had to change the way they did the polls after Trump was leading.
On the other hand, it often seems like news coverage is biased in the direction of a horse race. As in, the media is incentivized to make it seem like a close race with a lot of action, because that is engaging.
that is exactly incorrect actually. a literal betting market has historically been the best predictor of outcomes. this cycle was particularly surprising, so everyone is all up in arms. but the markets that i follow (predictit.com) were more bullish on trump than almost all poll models. if you look at past performance they are excellent.
PredictWise is not a prediction market. It's an aggregate of them, with David Rothschild's own personal special sauce added on top - which, in this case, is why it was "wrong".
PredictWise's mistake was that it assumed polling errors would not be correlated[1]. (He actually had a fight with Nate Silver over this.)
Nate Silver had, in advance, stated that polling errors would be correlated[2] (rightfully, as it turned out), which is part of why 538's model turned out to be closer to reality than PredictWise's.
The actual betting markets, which PredictWise was pulling data from, were in the range of 20-30% Trump. Much more bullish.
538 made an interesting point about these kinds of biases. Essentially, polling errors are correlated. You can't unbias your estimate, but you can realize that if voters are reluctant to state their Trump preference in Wisconsin, then they are also likely to be reluctant in Michigan. If you know how correlated your errors are, you can properly account it in your own uncertainty, probably leading to a prediction that is closer to 50-50.
Actually the discrepancy stems from underestimating voter turnout for Trump and over estimating it for Clinton and is not caused by the mythical timid Trump voter.
Doesn't matter. All that matters is that the errors are correlated. If voter turnout is biased in one state, then it likely is in another. The reluctance was just an example.
One of the better articles I've read on why US election forecasts are inaccurate. And written by a couple of guys from Brazil.
While beyond the scope of this article, it would be great see more research done into the relative strengths of each bias, i.e. which biases are skewing polls the most, rather than just using lumped bias terms. Though I did see after Brexit forecasts got it wrong too the British Polling Council is "looking into it".
What's most interesting is there wasn't more of an effort by the media to get accurate vote forecasts before the election. Granted, it's more expensive and takes longer but it's not terribly difficult to do. It does involve a research team and statisticians spending time in person in the state and getting deeper data from people. But the time/costs aren't as outrageous as one would think because it only needs to be done in the handful of close swing states. And a sample size of just a few hundred people is enough to get significant results. Seems obvious and worthwhile, maybe in 2020 they figure it out.
34 comments
[ 4.9 ms ] story [ 85.8 ms ] threadI have an alternative explanation for this : people who buy newspapers and mainstream web sites are liberal. Pollsters and analysts who wish to make a living must get clicks and sell copy. The ones that create models that agree with the general zeitgeist of the liberal milieu get clicks and sell copy, other models are gradually ground to extinction.
Markets are not efficient mechanisms for creating insight.
Early in the campaign, Trump acknowledged his media advantage. “I’ve gotten so much free advertising, it’s like nothing I’d have expected,” he told the New York Times in September. “When you look at cable television, a lot of the programs are 100% Trump, so why would you need more Trump during the commercial breaks?”[1]
[1]http://www.marketwatch.com/story/trump-has-gotten-nearly-3-b...
http://www.politico.com/blogs/on-media/2016/02/les-moonves-t...
The particular example I have provided is from Fox News because it was quick to search for, but I suspect you could find similar elsewhere about other news organizations.
[0]: https://twitter.com/FoxNews/status/787078152895901696/photo/...
Fabricated Fake news about Trump raping multiple people and 13 year kid certainly didn't help Trump to win.
Edit: Downvoted by the bots already.
Most important part of the sentence: 'and tell the press'
http://www.salon.com/2016/11/09/the-hillary-clinton-campaign...
1. http://www.salon.com/2016/11/09/the-hillary-clinton-campaign...
Perhaps polling is just more or less accurate because fewer people take part in polls.
Or the opposite - a voting abstentionist might be more likely to take a position when polled.
http://elections.huffingtonpost.com/2016/forecast/president
[1]http://elections.huffingtonpost.com/pollster/2016-general-el...
why would that be, though?
This will be obvious if you look at the weighting and questions 'Would you vote for Obama or Trump'. Reuters/Ipsos and other outlets had to change the way they did the polls after Trump was leading.
Predictwise was calling ~90% for Hillary and 320ish in the college.
PredictWise's mistake was that it assumed polling errors would not be correlated[1]. (He actually had a fight with Nate Silver over this.)
Nate Silver had, in advance, stated that polling errors would be correlated[2] (rightfully, as it turned out), which is part of why 538's model turned out to be closer to reality than PredictWise's.
The actual betting markets, which PredictWise was pulling data from, were in the range of 20-30% Trump. Much more bullish.
[1] http://predictwise.com/blog/2016/09/poll-aggregation-fight/
[2] http://fivethirtyeight.com/features/election-update-why-our-...
And statisticians have pointed out the inherent limits to surveys/polls in election forecasting since at least the Literary Digest debacle; a particularly good discussion is in Schlaifer 1959 https://www.gwern.net/docs/statistics/1959-schlaifer-probabi... (excerpt: https://www.reddit.com/r/slatestarcodex/comments/5c10jx/post... )
While beyond the scope of this article, it would be great see more research done into the relative strengths of each bias, i.e. which biases are skewing polls the most, rather than just using lumped bias terms. Though I did see after Brexit forecasts got it wrong too the British Polling Council is "looking into it".
What's most interesting is there wasn't more of an effort by the media to get accurate vote forecasts before the election. Granted, it's more expensive and takes longer but it's not terribly difficult to do. It does involve a research team and statisticians spending time in person in the state and getting deeper data from people. But the time/costs aren't as outrageous as one would think because it only needs to be done in the handful of close swing states. And a sample size of just a few hundred people is enough to get significant results. Seems obvious and worthwhile, maybe in 2020 they figure it out.