My suspicion is that it's not that Facebook sucks at machine learning, but that they are scared of public backlash when their algorithm accidentally takes a story from Breitbart and classifies it (arguably correctly) as fake news. At that point Facebook is derided as a tool of the liberal msm (after all it's run by liberal Silicon Valley technocrats), and from there eyeballs go elsewhere and Facebook is harmed.
Better for them to feign ignorance and lie to themselves.
Not according to my logic, no. If I wanted a candidate to win, I'd imagine it would be better for me to propagandize predictions that my candidate would lose, as to motivate more of his/her supporters to actually vote.
I don't agree with tomp's reasoning but you are confusing cause and effect. If tomp is correct, it only implies that a news outlet biased towards one candidate would predict a victory for the other, not that every media outlet predicts a victory a candidate they are biased against.
Because of extreme amounts of arrogance on the democratic side, this is actually what happened. It's the same as you hear from Russians. If something's in any big newspaper, it is
(a) fake
(b) it is rather obvious what the truth is
I'm not saying this is the level the MSM has dropped to, but it most certainly is where it's going.
I'd also like to say that this is what's happening to large media worldwide, and is in fact being used to marginalize the victims of the current economic climate.
Exactly what leftists SHOULD NOT do, if anything they should err on the side of the poor.
Now read the Hillary articles again, assuming you 100% believe the above links and what they mean about the MSM. I think you'll understand better what is happening.
Frankly the MSM lies about economic facts that are very much in the face of a hell of a lot of people, and that causes people to not trust them anymore. This seems natural, and ... censorship of fake news will make this worse, not better (or better put: I have enough Russian friends and stories from them certainly indicate it will).
But as someone who regularly visits Middle/Eastern Europe but does not live there I see the same thing. I read the newspapers and ... I just don't believe them. The difference with the country I see when visiting is too great. Online fora, filled with locals are easy to find and while filtering absolutely is a necessity, they are generally telling the truth. There are areas where it is worse, e.g. immigrants (and whether or not a particular town was seriously damaged/looted during a "protest" march), but it is certainly true in general. And yes, these fora do not put forward the view the American left has of the world. They don't like immigrants (too many stories of looting, extremely poor work ethic, attacks, criminal behavior, ... combined with stories of the government working against the victims. It is extremely hard to not believe all of them when you've had 2 friends tell you one. In confidence, because they were afraid of retaliation at work.
I think that's one thing people don't realize. The main fake news, in terms of reach, are sites like reddit, and it's country-local alternatives.
But the story goes like this: they had rented a "work" apartment, a place to sleep close to work for during the week, avoiding a commute after a long day at work. Work is in the middle of the capital city, surrounded by immigrant neighborhoods, so this apartment was in an immigrant neighborhood. They got constantly treated very unfriendly in the hallways, but that, they can take. One day a few obviously muslim youth stormed in, speaking French, screaming to eachother how it's their right to rob kuffar (arabic word meaning infidel in the sense of non-muslim, or cockroach), lots of screaming, obviously high. They broke the tv they were stealing, and beat my friend and his colleague in revenge for that. They had guns, took several macbooks, their cell phones and wallets, beat them up, an...
Never followed or watched Breitbart, but CNN and friends presented it as a poll, presumably done with rigor and impartiality. Either they couldn't afford to pay their statisticans or they deliberately fudged them and lied.
> Either they couldn't afford to pay their statisticans or they deliberately fudged them and lied.
So it's only those two options? It's not possible that the polling was just flawed or wrong but it's they can't pay them (implying they have shit statisticians) or they lied?
Not a fan of most of the media but come on there are more options here. It was a tough election to predict. Bringing in two, fairly unpopular candidates is going to create some interesting results.
The implication was that they fudged their data to make a candidate look better. Clearly they have the money to pay for the poll and statisticians.
The more I think about the more it is seems one of the reason for the loss was this overconfidence. News channels were often way too happy to put up a map with Hillary in the 400+ points and the whole map being blue. They believed that would just discourage those voting for Trump to show up. In their it was a terrible blunder (or win depending who you support), in the end more people who would have voted for the Democrats stayed home.
Overconfidence is certainly a story we're getting out of the Clinton campaign; WRT to the Rust Belt where she decisively lost the election:
Bill's advice to put effort into was ridiculed.
Russ Feingold, who was as far as I know honestly expected to win his election for Senator (note he'd just lost the seat 6 years earlier in the off-season 2010 Republican langslide), desperately asked for help towards the end of the campaign and was ignored.
It's been reported that a computer system named Ada was driving pretty much all these sorts of decisions, and it said the Rust Belt was safe.
And there are some other things I think I'm forgetting this early in the morning, anyway, just how wrong this was is demonstrated by how close Minnesota ended up. If Team Trump had realized the opportunity there earlier and put more resources into it, he might well have won it. That that Democratic (well, DFL) bastion was in play....
Take a look at the open source election models. They all make similar mistakes in that they don't properly propergate sources of errors from polls into their prediction. 538 did a good job but was criticised for it.
I don't think you need a conspiracy when simple incompetent will do.
The fake part comes into play if you think they lied and pushed the wrong results, but presented them as "scientific".
Note it was not just a small mistake. They presented Hillary having a massive landslide, and instead she delivered the worst Democratic performance in 20 years.
Frankly, this is an incredibly stupid argument and I'm sick of hearing it. Incorrectly forecasting the future is a far cry from incorrectly reporting facts. Nobody said it was a fact that Clinton would win.
Where do you think they "lied?" Even simple poll averages showed Hillary was ahead.
I'm willing to admit that they overestimated the chances of a Clinton victory. Nate Silver's 60% was a lot more realistic than the 90% the NYT was pushing. But having a poor prediction model is a lot different than falsely reporting the past.
More importantly, it's totally incoherent. If the MSM was tilting their model to try to get Hillary elected, the worst thing they could do would be to present it as a sure thing (depressing turnout).
Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.
State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.
> Frankly, this is an incredibly stupid argument and I'm sick of hearing it.
Sorry. Didn't know you heard this before. Yeah the argument is misguided perhaps, but "incredibly stupid"? I don't know, I think you're being a bit harsh.
> Where do you think they "lied?"
They lied who they polled and how they fudged the results
> Incorrectly forecasting the future is a far cry from incorrectly reporting facts.
It is not far cry. This is one organization which has lost its credibility because it routinely lied and misconstrued facts, cheated during debates. I don't see as suspicious that they fudged the models to make her look good as "incredibly stupid", but again you have your opinion, so that's cool.
> More importantly, it's totally incoherent.
Sometimes maliciousness and coherency doesn't go together. Again, I would say it might be "mildly incoherent" but not "totally". You can just as well say they wanted her to look good to discourage anyone voting for the other candidate, which indeed jives with the rest of the rhetoric.
> Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.
What does that have to do with anything? Are you saying, she is a top experienced politician, lawyer, with a stellar team of experts, media behind her back, President, DNC, may large corporate donors and she didn't know that she had to win electoral college votes and just got a little confused?
> State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.
Somehow the "idiot" Trump managed to poll in the right states at the right time make Hillary deliver the worst Democratic election results in more than 20 years. If a dummy can do it, surely it was possible to poll properly.
> They lied who they polled and how they fudged the results
Care to provide evidence that dozens of independent pollsters all colluded to fudge results?
> It is not far cry.
Yes it is. Do you really not see the difference between me saying "it's raining outside" (when it's not) and saying "I think it's going to rain tomorrow" (and it doesn't).
> What does that have to do with anything?
It has a lot to do with it. The primary input into most forecasting models was national polling data. And national polls ended up being correct: Clinton won the national popular vote.
I'm not sure why you think Trump managed to poll better than the media. He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.
You're not even mentioning betting markets. Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?
If you want to assert a global conspiracy to subvert reality, please bring data.
> "I think it's going to rain tomorrow" (and it doesn't).
Not if your job is to predict rain and you somehow completely missed the that there's going to be a massive rain-storm. And you also happened to own a company which sells umbrellas on the street corner.
> It has a lot to do with it.
Well, ok, I take it back. It has a lot to do with them failing to predict it because they relied on the popular vote.
> Care to provide evidence that dozens of independent pollsters all colluded to fudge results?
I didn't say they colluded. They fudged their models by being in denial and wanting to see a certain candidate win. Someone doing it as a hobby is understandable. But when they are paid a pretty penny for it, then it borders into dishonesty.
All actors doing the same thing doesn't imply it is a cabal and they all colluding. They simply have to have the same incentives.
Surprisingly one of the polls that does measure enthusiasm the Daybreak poll from USC did predict a win for Trump http://cesrusc.org/election/
> Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?
The probably took the cue from the media.
> He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.
But wasn't that a much better strategy, as overconfidence lead to people staying home. That idea is probably not novel to him or his team!
We've all heard how dumb he is, yet somehow he knew exactly which states to go to, doing up to 6 speeches a day, in seemingly just at the right places. He got lucky a few times, maybe in primaries, but he seems to keep getting lucky consistently.
> If you want to assert a global conspiracy to subvert reality, please bring data.
Would you say reality was subverted already? We have a TV personality as a President. This is the stuff they put in comic books. Conspiracy here is called overconfidence, cockiness and complacency. That's the conspiracy. We can make fun of and call Trump supporters names as much as we want. But in the end Hillary had a very expensive gun (the media) and she went ahead and shot herself in the foot with it. I think that is one of the primary reason people didn't vote -- they saw the map with 400+ electoral college votes in her favor and 1% chance for Trump to win. But in the end it is those pesky KGB agents and Fake News that we are blaming... The more we blame them. The more Trump will keep winning.
Note that national polling was fine. It was the state polls that turned out to be hugely inaccurate. That doesn't mean they were "unscientific," but that they made some mistakes with their polling methods.
Bill Clinton and Obama, they had a better numbers. Dukakis only was worst and was early 90 or so. And that is despite her having such a well funded campaign.
And anyone who knew Dukakis and Flyover country, as I did having been raised in the latter and then moving to Massachusetts for college in 1979, knew it was very likely Dukakis was going to lose.
No mainstream prediction I saw predicted a massive Clinton landslide; most predicted a slightly larger popular vote victory than actually occurred.
Some, though not the best regarded, gave a Clinton victory a very high probability, largely because they modelled state-level variability as independent rather than highly correlated, an error that 538 called out explicitly.
Most were also biased in her favor. That was the big learning experience for me in this election. She had an unprecedented support from media, unlike few other candidates. I think even Fox (the traditional Republican faux-news channel) didn't like him very much.
Yeah I don't know the answer, what the mistake was. I'd want to know. I suspect some never responded or wanted to participate. Those who participated, said they'd vote for someone else but changed their mind. They used registered phone numbers and some people more consistently paid to have their number de-listed...
Yes, from what I've read Fox was very anti-Trump, aside from a few of its figures who's contracts have not inspired.
I mean, this election was used as an opportunity to mount a successful coup ousting its founder, who'd, BTW, announced a couple of years ago that the network was moving to the left anyway....
Presenting the election result as 98% certainty is either fake or insanity (for context, that would mean (on average) only 1 wrong prediction in 200 years of US presidential elections).
Edit: admittedly, that was Huffington Post, hardly a standard for good journalism. E.g. NYT had 80%, 1 wrong in 20 years, which is still quite optimistic IMO.
Thinking further about this, I think framing it in years is highly misleading. If we imagine an event that occurs once every ten-thousand years, with even probability of two outcomes, then if I pick one of the two as my guess and I correctly assign a 50% probability, you can say "you're saying your guess will only be wrong, on average, once every twenty-thousand years!" That's technically true, but I'm clearly not making too bold a claim!
Yeah I sometimes question the sensibility of even trying to assign probability to what are essentially one-time events. I prefer to think of it, the probability is either 0% or 100%, we just don't know it yet!
In a deterministic universe, there's a sense in which that's always the case. But really, probability is always "given the information available". "I know everything about a deterministic universe" is just a special case.
It's mostly unsubstantiated claims, without scientific rigor to back them up (ex: they cite papers but misrepresent the findings), and heavily mixed with personal opinion. Breitbart may have been right about the election, but that doesn't really seem to be because the website is dedicated to clear objective journalism... Wouldn't you agree?
> It's mostly unsubstantiated claims, without scientific rigor to back them up (ex: they cite papers but misrepresent the findings), and heavily mixed with personal opinion.
Sounds like pretty much any non-scientific media citing scientific articles.
Okay, so other news sources also can be biased/wrong/misrepresent science ... how does that make this Breitbart article more accurate? or Breitbart a more objective news source?
Nah, same bullshit (neither more nor less fake), just in the other direction. Best to read both left-ist and right-ist media to avoid bias. Although personally I sometimes find Breitbart more agreeable, as they seem opposed to political correctness and unafraid of voicing inconvenient truths (opinions?).
It's Milo being Milo, which any regular reader of Breitbart, or anyone sensitive to nuance and style, would realize is more opinion than official un/less biased "news".
This whole fake news thing is ridiculous. Where is all of the outrage at CNN telling its viewers, falsely, that it's illegal for them to read Wikileaks?
>“Also interesting is, remember, it’s illegal to possess these stolen documents. It’s different for the media. So everything you learn about this, you’re learning from us.”
CNN has yet to apologize for, or even retract, this.
Where is the outrage at all of these "real news" outlets coordinating their messaging with the DNC? Where is the shock at them asking the DNC for questions to pose to Trump? Or the concern that Hillary Clinton was given multiple debate questions by a CNN employee?
The simple fact is that almost all media is propaganda. Depending on anyone, or anything, to protect you from "fake news" is laughable.
The media industry in the United States, primarily because it functions as a propaganda medium for state narrative, has increasingly been losing credibility with its audience.
The 'fake news' thing, the 'troll' thing before that, the 'political spam' thing before that: these are attempts to prevent the manifestations of credible alternatives that can attract funding, challenge the doctrinal ideology espoused by the network of media, or further disrupt the reputation of the current infrastructure for engaging in public diplomacy.
I'm often curious why particular comments are down voted, so this caught my eye. Why did you come to the conclusion that it was due to a typo? That said, I don't have an alternative theory.
I hadn't considered it. The HN crowd is a lot better at understanding that downvotes are for comments that do not contribute to the conversation, rather than what they disagree with.
Maybe that's a more popular trend now? Having been active on HN for a long time and watched it evolve I guess I could see that...
Anyway, the branch of the conversation has gotten quite deep down the 'meta' rabbithole.
You and the people writing these articles need to take a less US-centric view of this situation. This has been going on for a long time in non-English speaking communities.
My hope is that now that it has become very noticeable in the US maybe they'll do something about it, but it's entirely possible they do nothing for the reasons you said.
Until the 2000s there was only one evening news broadcast and a few newspapers. But most people didn't bother with those and got their informations from friends and family. It was mostly nonsense before Facebook too. Facebook just made it digital and public.
And that's the problem. The people who make up this stuff can see when it's being shared, and commented on, and what resonates most with people. They can write up the perfect crazy story for my Mom and Uncle to talk about like they have done for the past 60 years.
I can go in and say this is fake, don't believe anything you read from this website or facebook page, and sometimes they'll believe me. But that doesn't stop the content from flooding in.
The only thing I can think of is getting people to stop using Facebook altogether.
In Sweden, we have had numerous incidents where fake news has been spread with the purpose of spreading anti-immigrant sentiment. A recent example is a rumour stating that we had banned christmas decorations as it might offend muslim immigrants. What actually happened was that the Swedish Transport Administration had urged people not to decorate street poles as it might be a violation of our electricity laws. [0]
We haven't really figured out how to address these issues. The best we've got now is an initiative by Metro called Viralgranskaren (~"viral post inspector") that debunks myths like these. [1] What I think needs to happen is for the education system to start putting a larger emphasis on skepticism and source criticism. These skills and qualities are crucial for a 21th century liberal democracy.
fall? It's based on an email, but the content is, IMO, pretty low quality. It talks about unrelated things, and ignores context to portray a skewed version of facts.
Despite that, you could argue that yes it is a fact based news source and should be allowed.
It's rather less ridiculous when you look at the artwork he displays in his office and home.
I believe that's one of the things that gave all this traction, something is seriously wrong here, even if it's just a few sick souls. One of whom just happened to be the chairman of Hillary's campaign.
If you believe the Clinton Machine is evil, and you're open to supernatural explanations for evil (I myself am not willing to rule them out), well, we can go so far as demonic possession, that's straight out of the New Testament, Jesus driving demons out of people.
You're 100% correct. Look at all the backlash they got because their human editors wouldn't put fake news into the trending topics and that fake news happened to be conservative.
Even look in this comment thread. You have people insisting that Breitbart is a highly reputable site (that just happens to promulgate far-right conspiracy theories) while mainstream news is supposedly "fake" because they incorrectly forecast an election.
I was merely pointing out that "fake" is more of a propaganda, not a completely black and white issue.
Or do you think it's a coincidence the "fake news" story only broke after the "non-fake" media lost the election, not before while they were still very confident they'd win?
> I was merely pointing out that "fake" is more of a propaganda, not a completely black and white issue.
Don't pretend that your comment history doesn't display a clear Trump bias.
There might be some difficult cases at the margin, but lots of fake news is very easy to spot. No, ISIS was not founded by Clinton. No, Obama is not instituting Sharia law. No, George Soros is not hacking voting machines.
> Or do you think it's a coincidence the "fake news" story only broke after the "non-fake" media lost the election, not before while they were still very confident they'd win?
No, I don't think it's a coincidence. It seems very natural that new events would shine an event on existing trends. Or perhaps do you think that Islamic terrorism only began on 9/11, or that mortgage fraud began in 2008?
I like to think that my comment history displays mostly a contrarian (vis-a-vis mainstream) bias, although admittedly that would seem pro-Trump to most HN users (I think yummyfajitas is often the target of similar accusations).
You can "like to think that" but there's a difference between being "contrarian" and partisan.
Though I might disagree with yummyfajitas on things, he at least brings data and arguments to back up his claims instead of just asserting that the MSM is obviously fake and biased.
> No, I don't think it's a coincidence. It seems very natural that new events would shine an event on existing trends. Or perhaps do you think that Islamic terrorism only began on 9/11, or that mortgage fraud began in 2008?
So instituting censorship (since you agree there is a bias in your previous paragraphs) to affect the outcome of an election is:
1) acceptable
2) you DO realize the keys to the kingdom will be handed off to Trump, Pence and Steve Bannon, right ? Really think about what you're doing here. You're mounting a bigger gun on a tank that will be driven by Trump. Do you seriously not realize who it'll be firing at ?
But clearly you think things like censorship are acceptable as long as you agree with them. Hell, it seems the entire democratic establishment agrees with it. Why ? Just to not offend Hillary Clinton ? Fuck that.
Trump won because he showcased the ugly side of the democratic party and Hillary Clinton in particular, quite successfully. And what is being done in response ? Dig deeper. Of course the issue MUST be that they're being shown in the wrong light by "fake" articles. Even if that actually is the issue you should still assume it's not.
I would love it if this whole nonsense stopped, and some serious introspection would be done instead. How about we declare that Middle America's white supremacy fringe is about as offensive as Islam's sharia laws, that underwear toting rednecks with guns are pretty much the same thing as child-raping muslim "revolutionaries", and should be ignored and marginalized WITHOUT marginalizing the entire population there, and since they are having a hugely difficult time, they deserve the same protection and help and state action that minorities get (they are, by the way, a minority) ?
Now THAT would help the next president to be democratic. THAT is what we should do.
I'm not sure where you go the idea that I support instituting censorship. You seem to have written an entire comment attacking a non-existent strawman.
To make it unequivocally clear: no, I do not think the government should be in the business of censoring news (fake or not).
Government vs. private. A media outlet has a full choice in terms of what they send out. They can self-censor and always have. But the government has no business in telling _all_ media what they can or cannot publish. Fake news at its core is not subjective at all. Is Obama on record anywhere saying "I want to institute sharia law"? No. you can write an opinion piece if you think this is the case or have facts but its not _news_ and should be labeled so.
Reading those "fake" news stories I would say that they don't actually claim that. Now, I am sure there's an exception, but I haven't found one and I've honestly read every article in the first 2 pages of google results for "Obama sharia law".
Most take offence at Obama defending muslims and sharia law, voluntary application of sharia law that is, after Newt Gingrich's call for mass-deportation. They do not claim Obama said anything he didn't say. Obama was very political, leaving out any actual controversial statement.
So I don't think, on the whole, that your statement is fair. These articles mostly lack quality, so they're easy to attack. They're not wrong, in a strict sense. Clickbaity, spammy, unpleasant, even misleading ... yes. Wrong, no.
And I have no trouble whatsoever finding all those properties in articles linked from the homepages of CNN, MSN, ... especially in the business and science sections. And I have to say, I am against Trump, but most of the frontpage stuff I would agree if anyone said half of them were hit pieces.
I don't see any downside in this scenario, especially if Brietbarters leave FB en masse. I'm not sure there is another network they can presently go to due to massive network effects.
For what it's worth, if a reasonable person can make a compelling case that a story is not fake news (i.e. if it is "arguable"), doesn't the question of whether it is "fake" or not boil down to a reader's inherent bias (an opinion, which is by definition not fake)?
Also, why not use a white list that includes outlets that sizable portions of facebook's user base seem to think are reliable? For example, include breitbart.com and motherjones.com, but exclude .tumblr.com and .wordpress.com.
its probably somewhat hard to separate parody too, sometimes the only thing that makes it parody is the context or people the message is sent to. it could be an inside joke not meant to be taken literally.
for example, say i am a hillary fan, and i say to another hillary fan "hillary is a criminal" as a way to mock people who say that. will facebook know when something is insincere and in jest?
Well, if I see one more utterly fake story passed off as news, and if the horror show of disgusting comments e.g. "Obamas are apes" infects my stream, I'm going to go elsewhere. Let FB turn into 4chan if it wants to, without me.
My (limited) experience with feeds online has been disappointing. Do you have examples of good feed implementations, especially at larger scales, such as what Facebook works at? What tech do they use?
Feedly's Today page kind of feels like a news feed, though of RSS feeds which I've subscribed to by hand. That said, I believe it's just selecting the most viewed stories over a very recent period from your most popular feeds (some ambiguity here).
People change, we like different stuff every day, with these curated feeds we eventually end up seeing only a few things because the algorithms are designed to predict patterns using very very subjective parameters.
Fake news is saying that George Soros has voting machines that are rigging the election. It's just not true no matter how much people want to hear it.
Real news is saying that Trump wants to address immigration by building a wall. People may disagree about what that means, but it's still a true story.
Very. Prove I went to a restaurant yesterday. You could show a video of me at a restaurant, but is it from yesterday, or some other day. Is it really me there, or someone that looked like me.
Well perhaps that is the problem. "The news" used to be about events that had happened, often several days in the past due to the propagation speed of information.
Nowadays "the news" also includes:
1. What someone says might be happening right now.
2. What someone says about something that happened, or which might be happening right now.
3. What someone thinks might happen next.
None of which are necessarily true or grounded in fact but all of which are valued by the media in the race to be first with the headlines.
So perhaps "real news" could be defined as "events which have occurred and for which we have objective proof" and everything else is just "editorial".
Most mainstream media post interpretations, not facts. E.g. that Trump is "racist" (based on some comments about some, not even all, Mexican, who aren't even a race!).
I'd start with: news posted on sites that are fake. What's a fake site, you say? Well, how about one that claims to be a site that it is obviously not.
IOW, this is very similar to a problem we expect companies to be able to solve: identifying phishing sites.
If a site claims to be ABC News, a known media entity, and it clearly is not, I'd argue anything posted on it should be considered fake.
Fortunately, Facebook only has to solve the much simpler problem of identifying clearly fake news. This is like determining that a site is a link farm, not that it is a content farm. (I think Google has even made decent progress on the latter, harder problem, though I still see ask.com results sometimes.)
The problem is much more difficult than creating a classifier. The purpose of filtering news is to control the impact, not to control the tactic. The last time Facebook issued a surge to control 'spam' posts for the same government concerns they seek to deal with "fake news" today, they took out pro-Sanders articles, censored sharing of the Snowden documents, prevented people from linking to Wikileaks, and shut down Facebook organizing for May Day protests.
Facebook regularly censors and directs conversation online around the world to the whims of various regimes and administrations, but it's just not easy "use machine learning".
The issue is that machine learning is inherently modeled to solve decision problems in spaces that separate high level 'features' from low level observations. The study and practice of machine learning thus far has modeled noise, uncertainty and complex state spaces.
However, machine learning classifiers do not themselves have a concept of adversarial data: this led, for example, to spammers who would "poison" machine learning classifiers by biasing them to learn the wrong things, so that they could get their spam through. Similarly, not too long ago, Microsoft put a conversation bot on the web that 'learned the wrong things' because people approached the bot in an attempt to trick it.
Facebook's requirement for battling unsanctioned opinions is that it solves this problem - it hasn't been solved yet. Facebook needs to continuously monitor the adapting activity and patterns of people who are trying to say things that governments and Facebook customers don't want people saying.
Basically: machine learning doesn't adapt but people do, and naive attempts to get machine learning to adapt make it vulnerable to being tricked, because classifier theory inherently has no concept of adversarial data.
umm no!...I don't know too much about this sub-field..but there are machine methods designed explicitly for adversarial data. it even has a wikipedia page! I don't know if facebook's methods implement any of this yet..but your statements about machine learning not modelling this just ain't so.
yes precisely! i'm sure Facebook's methods are very susceptible to these types of attacks. Hopefully they still are in 6 years and we can get jobs fixing this for them lol.
It would help me if the author were to point more examples of what he means by "fake news". As it is, I can't really tell if he's really complaining about "Onion" style satire, copypasta with clickbait titles, evil government propaganda, or just political opinions that he disagrees with. Does a public training set exist that makes it clear what he's talking about? Does the division point he wants actually exist?
Snopes published an article yesterday asking that asks the question more clearly:
There is much bad news in the online world, but not all of it is "fake" (i.e., completely fabricated information that has little or no intersection with real-world events). There are also partisan political sites that take nuggets of real news and spin them into highly distorted, clickbait articles. There are sites that misleadingly repackage old news as if it were current information. There are sites that aggregate articles from a variety of dubious and questionable sources. There are sites (especially in the fields of health and science) that believe they're presenting pertinent information but are woefully inaccurate in their information-gathering and reporting. These forms of news are all bad in one way or another, but broadly classifying all such information as "fake news" clouds an already confusing issue.
Immediate red flags should be: This isn't ABC News's website. There are statements in the article like: "No one still has an AOL email address except people that would vote for Hillary Clinton" and from a made up Trump supporter quoted saying "I knew those weren’t real protesters, they were too organized and smart..." and (awesomely) "I think I was paid more than the other protesters because I was white and had taken classes in street fighting and boxing a few years back".
I mean, this may well be parody, but it gets picked up as real news and he's making real money off it, impersonating a real news site, which is the point at which it cross the line from parody to fake, IMO.
You can tell this site has no credibility based on the other articles on the site.
I think the trick with a machine learning approach to identifying these sites is not letting the model fixate on particular words (e.g. Obama or Moon Landing could be terms with the right dataset; though if they occur together maybe that's a good indicator after all...)
"May well be" is being unnecessarily conservative. Maybe you didn't make it far enough into the article to read the faked interview with the author of the the Snope's article I posted:
David Mikkelson, founder of Snopes.com, a website known for its biased opinions and inaccurate information they write about stories on the internet in order to generate advertising revenue, told ABC News that he approves of what a story like this is accomplishing.
“You have to understand that when a story like this goes viral, and we spend a minute or two debunking it, we make lots of money. Stories like this have helped put my children through college, buy a new car, a home and even get the Silverback gorilla my wife Barbara always wanted since she was a child,” Mikkleson said.
Buried, but actually rather funny. In a way, this is both a fabulous and a terrible example. Read in its entirety, it's clearly satire. Yet I'm sure that many people never read it closely enough to notice.
Should Facebook be censoring this? No. Should they be labelling it? Yeah, it might be a good idea. Is this the sort of story that the author is referring to? I don't know, but it would be nice if he'd tell us explicitly.
Given who Snopes assigned political "debunking" to and the predicable results, you could also label this item as "Fake but Accurate", the "website known for its biased opinions and inaccurate information" is accurate enough for enough of their political items.
One things I don't get, though, is why so many organizations destroyed whatever credibility they had remaining in a futile attempt to win this election for their preferred candidate, I mean, this was an "it's my turn" race, a bit like Dole in 1996 and Romney in 2012. Especially prior to Trump winning the Republican nomination, was it really that important?
Note that this goes all the way to the DNC for the party itself....
I have some problem with those claims that offer little to none details. Use the above link as an example, how do we teach machine to verify this news? First, we need to teach them to formulate a question, in this case, 'Who has a larger number of popular vote [as of now]?', then it needs to find its way to verify the number from reliable sources, possibly need to struggle between conflicted sources as well.
This is no trivial, if we solve this, we might as well solve the problem of understanding human language as a whole.
The mentioning of Princeton students' Hackthon project is really offbeat, if the author really bothers to look into what that project actually is, nothing more than an API mashup.
Disclaimer: I worked for a major news app, and the effort to battle fake news or low quality non-trustworthy news is ever growing. While application of machine learning is promising, human, especially experienced human is still the real workhorse for this problem
That's the only thing you can focus on. Why should someone be punished and labeled as fake if they're on a wordpress.com subdomain? Domain is just out, even spammy or weird looking domains may end up being popular / culturally fit. You would be preventing good journalistic folks from having any type of reach if you want to test against domains names, something that arguable may go away as far as the user is concerned in the next decade.
I'm unconvinced there is anything but content that you can really go off of. Keywords are not even used in search results, why would they determine news legitimacy? Even references are not always useful.
> Why should someone be punished and labeled as fake if they're on a wordpress.com subdomain?
Because metadata can be super useful in making inferences about page quality. Pagerank exploited this idea a ton. If wordpress subdomains are more likely fake in a training corpus, then that piece of evidence should be counted.
> Why should someone be punished and labeled as fake if they're on a wordpress.com subdomain
i don't know about "labeled as fake", exactly, but it's possible that quickly and easily acquired urls (such as a wordpress subdomain) may display a different amount of reliability (per whatever heuristic) in comparison with various established urls.
I haven't worked on a news app, but why doesn't something like PageRank plus keeping track of a site's longevity handle this. It seems like fake sites would come and go quickly -- I doubt "70news.wordpress.com" has been around long, or gets many (non-gamed) links.
I'm surprised Google messed it up so badly. Was 70news manipulated to the top of the results page, or did their ranking algorithm totally screw up a simple query? Neither makes much sense.
Whitelist or blacklist or assigning PageRank to websites are tempting but also controversial because it will fall under the accusation of censorship. Take the below link as an example:
This is not 'fake' news, in a literal sense, because it just says someone is about to release something about Clinton. But for some people, this news could simply be seen as an indictment. Can we claim it is fake in that sense? As I have said above, it is equally controversial to penalize such websites for editorialization, because find story out of non-story is how most news agencies makes money and they have incentive to do this. Blame modern media and journalism for this.
In the end, using one simple algorithm to battle this is kinda of naive, not because it won't work for simple cases, but it fails to cover corner cases. In such scenario, human judgement is still crucial.
> In the end, using one simple algorithm to battle this is kinda of naive, not because it won't work for simple cases, but it fails to cover corner cases. In such scenario, human judgement is still crucial.
I agree that it doesn't cover corner cases, but why should it? If an algorithm knocks out 90% of the crap (like 70news.wordpress.com), there's much less work for the humans.
Also, wow, I didn't know Breitbart was quite that nutty. It reminds me of that LBJ quip along the lines of "it doesn't matter if he porks pigs or not, but now I made him deny it." This seems like a case where a human could do a bit of actual investigation, decide that the article's "Hillary is a lesbian" gist was BS, and permanently penalize Breitbart for publishing it.
This problem has been going on for years and only now is it receiving wide recognition due to the recent election. I wrote about fake Google News in mid-2014:
My first reaction to this was to object to using machine learning to validate news stories, since it's very hard for a machine to go out into the world and verify that a story is based on reality. But the idea that fake news on Facebook is similar to the content farmed / bot generated / spammy stuff Google tries to eliminate, makes me think. The curious uptick of fake news over the last few months makes me wonder if Facebook either tried to, or was complicit in swaying public opinion before the election. I've long suspected that Google/Facebook technically have enough power to effectively decide an election by manipulating public opinion. But if they were going to do so, they'd attempt to do it in a way that doesn't make them look culpable. Someone should investigate.
we'll they have some top talent money can buy there. Including member of RU academy of science. They can't suck just like that. These people made history and are most probably being watched by any group with any interest in machine learning.
The fundamental problem is the mixing of opinion and facts into the same story combined with a complete lack of interest and care by writers in finding out what happened.
I want and will pay for a news service which only reports facts (no opinions) and which is concerned with accuracy. There are specialty services that provide this in certain niche areas, but no general service.
I think there is also a incentive problem. Internally, from what I know, the performance of a team is always driven much how much metric it can drive, how much incremental revenue it can generate, how much more engagement this feature will drive, etc. The number of contents taken down doesn't seem like a sexy metric, so I assume no team wants to take this unsexy task.
Most of the other ML applications listed (driving, voice recognition, content recommendation) are somewhat easy tasks for humans, if not at scale... Reading, understanding and fact checking a news article is comparatively pretty difficult.
Too bad that this clickbait is upvoted. Both the premises reasoning in the article are not sound; filtering fake news is not at all similar to spam filtering, anf even if it were, it still wouldn't be easy as a result.
Then there is the false dilemma, and there are probably more fallacies in there, but I won't bother to read this utter bullshit anymore.
One person's fake news is another person's entertainment. The problem is not a technical issue. It is a cultural issue. ML cannot solve a cultural issue.
The Information is an unpopular, I think even blacklisted, news source here due to their paywall. But they do a really great job writing a very terse and high signal two-sentence takeaway for each article. I'm not a subscriber anymore (too pricy IMO), but here are some recent examples:
Surge-Price Builder Leaves Uber By Amir Efrati · Monday Oct 17, 2016
> THE TAKEAWAY: One of Uber’s secret weapons in recent years was an economics professor, Keith Chen. He overhauled Uber’s surge-price algorithm and made other key contributions that helped riders, and Uber, save money.
How Dropbox Doubled Down on Business Market By Steve Nellis · Thursday Sep 15, 2016
> THE TAKEAWAY: Dropbox founder Drew Houston has come to terms with the fact that his company probably isn’t the next consumer tech giant like Facebook or Google. That was reflected in a decision halfway through last year to sharpen the company’s focus on the business market, including shutting off consumer-focused apps like Carousel.
I've heard good things about The Information as well. One of the reasons they're able to do that, I suspect, is because it's behind a paywall: they can pay their journalists for quality content.
There are ML systems out there for summarization. I wonder how long until the systems are good enough to provide this as well. The examples I've seen (e.g., Summly, which uses SRI tech, IIRC), are likely good enough if you just want the facts. I still want it to be well-written.
Ouch. I just checked out the subscription rates for The Information. AFAICT, you can't even see the rates without supplying your name and email address. :/
Edit to add: From other sources on the internet, it looks like subscription rates are $399/year.
I feel the same way. Maybe there's room for a small service ($5/month) where we can submit articles that might be interesting and the most popular ones get summarized in a well-written way.
I do remember the rates being hard to find as well: $39/month or $399/annual [1]. They do offer you half-price initially etc for a few months.
I've thought about something along the same lines, though more in terms of "news source/quality certification". I just don't see a way to prevent the summaries/ratings from being pirated/"re-used" elsewhere.
Hmm, well… I don't think a paywall is desirable. Maybe the summaries are free and users can pay for individual summaries that they like with microtransactions. A fractional penny up to a quarter perhaps.
Maybe that would work. Are you familiar with any successful use of micro transactions? I haven't gone out of my way to look. Something like Patreon also comes to mind, though I've only heard of that in the podcast sphere, where people have a more personal relationship with the content creator given the format.
I haven't seen one yet personally. I heard of a program Google did where users paid a flat rate, say $10/mo, which was divvied up at the end of the month amongst articles / authors each person chose. I saw the website once last year and have not been able to figure out the name or find it since.
I suppose what I meant is technically not microtransactions (mostly just used in the context of in-app purchases) but micropayments [1]. I've added ko-fi [2] to my personal site but no one has actually done it yet.
A question - does anyone utilize the fact observed in cognitive neuroscience in robotics, that the best learning happens in organically growing systems, not in universal systems where you run a huge batch on a clean, pre-configured state? So far I see everyone designing the structure of an ML system in advance and hoping for the best (e.g. define the whole structure in TensorFlow in the beginning), instead of allowing the system to self-organize as the learning happens.
What a misleading title. The real title says "must suck at" (which probably is insinuating that they probably don't suck), and somehow it's been fabricated into this clickbait title.
When person A says something publicly that is fake and newspaper writes article quoting the person - is the article fake or not?
It's very blurred indeed.
What would probably help is extracting list of (main) statements/assumptions made and give them fact-probability score (ideally with references) - and show it to people.
But this is not trivial exercise, is it?
I mean, once we have technology that's good in doing it, well... it means we can probably rely on this AI to make 80% of decisions to run the country.
154 comments
[ 3.3 ms ] story [ 211 ms ] threadBetter for them to feign ignorance and lie to themselves.
[1] https://www.washingtonpost.com/news/volokh-conspiracy/wp/201...
(a) fake
(b) it is rather obvious what the truth is
I'm not saying this is the level the MSM has dropped to, but it most certainly is where it's going.
I'd also like to say that this is what's happening to large media worldwide, and is in fact being used to marginalize the victims of the current economic climate.
Exactly what leftists SHOULD NOT do, if anything they should err on the side of the poor.
http://edition.cnn.com/2016/09/14/opinions/america-doing-bet... (and 1000s other articles)
vs
http://www.wsj.com/articles/election-2016-is-propelled-by-th...
and the "fake news" version of it
https://www.videoblocks.com/video/detroit-ghetto-driving-1-d...
Now read the Hillary articles again, assuming you 100% believe the above links and what they mean about the MSM. I think you'll understand better what is happening.
Frankly the MSM lies about economic facts that are very much in the face of a hell of a lot of people, and that causes people to not trust them anymore. This seems natural, and ... censorship of fake news will make this worse, not better (or better put: I have enough Russian friends and stories from them certainly indicate it will).
But as someone who regularly visits Middle/Eastern Europe but does not live there I see the same thing. I read the newspapers and ... I just don't believe them. The difference with the country I see when visiting is too great. Online fora, filled with locals are easy to find and while filtering absolutely is a necessity, they are generally telling the truth. There are areas where it is worse, e.g. immigrants (and whether or not a particular town was seriously damaged/looted during a "protest" march), but it is certainly true in general. And yes, these fora do not put forward the view the American left has of the world. They don't like immigrants (too many stories of looting, extremely poor work ethic, attacks, criminal behavior, ... combined with stories of the government working against the victims. It is extremely hard to not believe all of them when you've had 2 friends tell you one. In confidence, because they were afraid of retaliation at work.
I think that's one thing people don't realize. The main fake news, in terms of reach, are sites like reddit, and it's country-local alternatives.
But the story goes like this: they had rented a "work" apartment, a place to sleep close to work for during the week, avoiding a commute after a long day at work. Work is in the middle of the capital city, surrounded by immigrant neighborhoods, so this apartment was in an immigrant neighborhood. They got constantly treated very unfriendly in the hallways, but that, they can take. One day a few obviously muslim youth stormed in, speaking French, screaming to eachother how it's their right to rob kuffar (arabic word meaning infidel in the sense of non-muslim, or cockroach), lots of screaming, obviously high. They broke the tv they were stealing, and beat my friend and his colleague in revenge for that. They had guns, took several macbooks, their cell phones and wallets, beat them up, an...
So it's only those two options? It's not possible that the polling was just flawed or wrong but it's they can't pay them (implying they have shit statisticians) or they lied?
Not a fan of most of the media but come on there are more options here. It was a tough election to predict. Bringing in two, fairly unpopular candidates is going to create some interesting results.
The implication was that they fudged their data to make a candidate look better. Clearly they have the money to pay for the poll and statisticians.
The more I think about the more it is seems one of the reason for the loss was this overconfidence. News channels were often way too happy to put up a map with Hillary in the 400+ points and the whole map being blue. They believed that would just discourage those voting for Trump to show up. In their it was a terrible blunder (or win depending who you support), in the end more people who would have voted for the Democrats stayed home.
Bill's advice to put effort into was ridiculed.
Russ Feingold, who was as far as I know honestly expected to win his election for Senator (note he'd just lost the seat 6 years earlier in the off-season 2010 Republican langslide), desperately asked for help towards the end of the campaign and was ignored.
It's been reported that a computer system named Ada was driving pretty much all these sorts of decisions, and it said the Rust Belt was safe.
And there are some other things I think I'm forgetting this early in the morning, anyway, just how wrong this was is demonstrated by how close Minnesota ended up. If Team Trump had realized the opportunity there earlier and put more resources into it, he might well have won it. That that Democratic (well, DFL) bastion was in play....
Take a look at the open source election models. They all make similar mistakes in that they don't properly propergate sources of errors from polls into their prediction. 538 did a good job but was criticised for it.
I don't think you need a conspiracy when simple incompetent will do.
Note it was not just a small mistake. They presented Hillary having a massive landslide, and instead she delivered the worst Democratic performance in 20 years.
Something doesn't add up...
Where do you think they "lied?" Even simple poll averages showed Hillary was ahead.
I'm willing to admit that they overestimated the chances of a Clinton victory. Nate Silver's 60% was a lot more realistic than the 90% the NYT was pushing. But having a poor prediction model is a lot different than falsely reporting the past.
More importantly, it's totally incoherent. If the MSM was tilting their model to try to get Hillary elected, the worst thing they could do would be to present it as a sure thing (depressing turnout).
Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.
State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.
Sorry. Didn't know you heard this before. Yeah the argument is misguided perhaps, but "incredibly stupid"? I don't know, I think you're being a bit harsh.
> Where do you think they "lied?"
They lied who they polled and how they fudged the results
> Incorrectly forecasting the future is a far cry from incorrectly reporting facts.
It is not far cry. This is one organization which has lost its credibility because it routinely lied and misconstrued facts, cheated during debates. I don't see as suspicious that they fudged the models to make her look good as "incredibly stupid", but again you have your opinion, so that's cool.
> More importantly, it's totally incoherent.
Sometimes maliciousness and coherency doesn't go together. Again, I would say it might be "mildly incoherent" but not "totally". You can just as well say they wanted her to look good to discourage anyone voting for the other candidate, which indeed jives with the rest of the rhetoric.
> Also, for the record, Clinton won the popular vote. So the polls got the popular vote percentage right.
What does that have to do with anything? Are you saying, she is a top experienced politician, lawyer, with a stellar team of experts, media behind her back, President, DNC, may large corporate donors and she didn't know that she had to win electoral college votes and just got a little confused?
> State polls were a lot more noisy and infrequent—it's not that surprising that there was a polling miss.
Somehow the "idiot" Trump managed to poll in the right states at the right time make Hillary deliver the worst Democratic election results in more than 20 years. If a dummy can do it, surely it was possible to poll properly.
Care to provide evidence that dozens of independent pollsters all colluded to fudge results?
> It is not far cry.
Yes it is. Do you really not see the difference between me saying "it's raining outside" (when it's not) and saying "I think it's going to rain tomorrow" (and it doesn't).
> What does that have to do with anything?
It has a lot to do with it. The primary input into most forecasting models was national polling data. And national polls ended up being correct: Clinton won the national popular vote.
I'm not sure why you think Trump managed to poll better than the media. He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.
You're not even mentioning betting markets. Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?
If you want to assert a global conspiracy to subvert reality, please bring data.
Not if your job is to predict rain and you somehow completely missed the that there's going to be a massive rain-storm. And you also happened to own a company which sells umbrellas on the street corner.
> It has a lot to do with it.
Well, ok, I take it back. It has a lot to do with them failing to predict it because they relied on the popular vote.
> Care to provide evidence that dozens of independent pollsters all colluded to fudge results?
I didn't say they colluded. They fudged their models by being in denial and wanting to see a certain candidate win. Someone doing it as a hobby is understandable. But when they are paid a pretty penny for it, then it borders into dishonesty.
All actors doing the same thing doesn't imply it is a cabal and they all colluding. They simply have to have the same incentives.
Surprisingly one of the polls that does measure enthusiasm the Daybreak poll from USC did predict a win for Trump http://cesrusc.org/election/
> Do you think thousands of independent people bet money in order to incorrectly portray Clinton as having a lead?
The probably took the cue from the media.
> He spent months railing about how the election was rigged and he might not accept the result—if he was convinced he'd win, he wouldn't be wasting effort on that.
But wasn't that a much better strategy, as overconfidence lead to people staying home. That idea is probably not novel to him or his team!
We've all heard how dumb he is, yet somehow he knew exactly which states to go to, doing up to 6 speeches a day, in seemingly just at the right places. He got lucky a few times, maybe in primaries, but he seems to keep getting lucky consistently.
> If you want to assert a global conspiracy to subvert reality, please bring data.
Would you say reality was subverted already? We have a TV personality as a President. This is the stuff they put in comic books. Conspiracy here is called overconfidence, cockiness and complacency. That's the conspiracy. We can make fun of and call Trump supporters names as much as we want. But in the end Hillary had a very expensive gun (the media) and she went ahead and shot herself in the foot with it. I think that is one of the primary reason people didn't vote -- they saw the map with 400+ electoral college votes in her favor and 1% chance for Trump to win. But in the end it is those pesky KGB agents and Fake News that we are blaming... The more we blame them. The more Trump will keep winning.
Elections don't happen that often... so it's just a comparison against fellow losers Gore and Kerry.
And Kerry lost by both the popular vote and the electoral vote.
Oh, wow, here's a Wikipedia article worth looking at, "Historical polling for U.S. Presidential elections": https://en.wikipedia.org/wiki/Historical_polling_for_U.S._Pr...
Some, though not the best regarded, gave a Clinton victory a very high probability, largely because they modelled state-level variability as independent rather than highly correlated, an error that 538 called out explicitly.
Yeah I don't know the answer, what the mistake was. I'd want to know. I suspect some never responded or wanted to participate. Those who participated, said they'd vote for someone else but changed their mind. They used registered phone numbers and some people more consistently paid to have their number de-listed...
I mean, this election was used as an opportunity to mount a successful coup ousting its founder, who'd, BTW, announced a couple of years ago that the network was moving to the left anyway....
Edit: admittedly, that was Huffington Post, hardly a standard for good journalism. E.g. NYT had 80%, 1 wrong in 20 years, which is still quite optimistic IMO.
Related to the broader topic: https://en.wikipedia.org/wiki/Scoring_rule#Proper_scoring_ru...
It's mostly unsubstantiated claims, without scientific rigor to back them up (ex: they cite papers but misrepresent the findings), and heavily mixed with personal opinion. Breitbart may have been right about the election, but that doesn't really seem to be because the website is dedicated to clear objective journalism... Wouldn't you agree?
Sounds like pretty much any non-scientific media citing scientific articles.
https://www.youtube.com/watch?v=d_Wtu8Pzvd4
>“Also interesting is, remember, it’s illegal to possess these stolen documents. It’s different for the media. So everything you learn about this, you’re learning from us.”
CNN has yet to apologize for, or even retract, this.
Where is the outrage at all of these "real news" outlets coordinating their messaging with the DNC? Where is the shock at them asking the DNC for questions to pose to Trump? Or the concern that Hillary Clinton was given multiple debate questions by a CNN employee?
The simple fact is that almost all media is propaganda. Depending on anyone, or anything, to protect you from "fake news" is laughable.
The 'fake news' thing, the 'troll' thing before that, the 'political spam' thing before that: these are attempts to prevent the manifestations of credible alternatives that can attract funding, challenge the doctrinal ideology espoused by the network of media, or further disrupt the reputation of the current infrastructure for engaging in public diplomacy.
Maybe that's a more popular trend now? Having been active on HN for a long time and watched it evolve I guess I could see that...
Anyway, the branch of the conversation has gotten quite deep down the 'meta' rabbithole.
What consitutes false news to you? I've seen it on huffpo, salon, cnn, and msnbc.
Any article claiming Trump is a racist/rapist is false, for instance...because there is really no substantial proof.
Most people screaming about fake articles just want to censor opposing opinions.
My hope is that now that it has become very noticeable in the US maybe they'll do something about it, but it's entirely possible they do nothing for the reasons you said.
And that's the problem. The people who make up this stuff can see when it's being shared, and commented on, and what resonates most with people. They can write up the perfect crazy story for my Mom and Uncle to talk about like they have done for the past 60 years.
I can go in and say this is fake, don't believe anything you read from this website or facebook page, and sometimes they'll believe me. But that doesn't stop the content from flooding in.
The only thing I can think of is getting people to stop using Facebook altogether.
We haven't really figured out how to address these issues. The best we've got now is an initiative by Metro called Viralgranskaren (~"viral post inspector") that debunks myths like these. [1] What I think needs to happen is for the education system to start putting a larger emphasis on skepticism and source criticism. These skills and qualities are crucial for a 21th century liberal democracy.
[0]: http://www.snopes.com/sweden-bans-christmas-lights/
[1]: http://www.metro.se/nyheter/viralgranskaren/ (sadly only available in Swedish, it seems)
http://gizmodo.com/facebooks-fight-against-fake-news-was-und...
Yeah, fine line between fake news and something that is just extremely slanted.
where that ridiculous stuff about the podesta "spirit dinner" fall?
Where does something like this
https://fellowshipoftheminds.com/2016/10/22/wikileaks-podest...
fall? It's based on an email, but the content is, IMO, pretty low quality. It talks about unrelated things, and ignores context to portray a skewed version of facts.
Despite that, you could argue that yes it is a fact based news source and should be allowed.
I believe that's one of the things that gave all this traction, something is seriously wrong here, even if it's just a few sick souls. One of whom just happened to be the chairman of Hillary's campaign.
If you believe the Clinton Machine is evil, and you're open to supernatural explanations for evil (I myself am not willing to rule them out), well, we can go so far as demonic possession, that's straight out of the New Testament, Jesus driving demons out of people.
Even look in this comment thread. You have people insisting that Breitbart is a highly reputable site (that just happens to promulgate far-right conspiracy theories) while mainstream news is supposedly "fake" because they incorrectly forecast an election.
Or do you think it's a coincidence the "fake news" story only broke after the "non-fake" media lost the election, not before while they were still very confident they'd win?
Don't pretend that your comment history doesn't display a clear Trump bias.
There might be some difficult cases at the margin, but lots of fake news is very easy to spot. No, ISIS was not founded by Clinton. No, Obama is not instituting Sharia law. No, George Soros is not hacking voting machines.
> Or do you think it's a coincidence the "fake news" story only broke after the "non-fake" media lost the election, not before while they were still very confident they'd win?
No, I don't think it's a coincidence. It seems very natural that new events would shine an event on existing trends. Or perhaps do you think that Islamic terrorism only began on 9/11, or that mortgage fraud began in 2008?
Though I might disagree with yummyfajitas on things, he at least brings data and arguments to back up his claims instead of just asserting that the MSM is obviously fake and biased.
So instituting censorship (since you agree there is a bias in your previous paragraphs) to affect the outcome of an election is:
1) acceptable
2) you DO realize the keys to the kingdom will be handed off to Trump, Pence and Steve Bannon, right ? Really think about what you're doing here. You're mounting a bigger gun on a tank that will be driven by Trump. Do you seriously not realize who it'll be firing at ?
But clearly you think things like censorship are acceptable as long as you agree with them. Hell, it seems the entire democratic establishment agrees with it. Why ? Just to not offend Hillary Clinton ? Fuck that.
Trump won because he showcased the ugly side of the democratic party and Hillary Clinton in particular, quite successfully. And what is being done in response ? Dig deeper. Of course the issue MUST be that they're being shown in the wrong light by "fake" articles. Even if that actually is the issue you should still assume it's not.
I would love it if this whole nonsense stopped, and some serious introspection would be done instead. How about we declare that Middle America's white supremacy fringe is about as offensive as Islam's sharia laws, that underwear toting rednecks with guns are pretty much the same thing as child-raping muslim "revolutionaries", and should be ignored and marginalized WITHOUT marginalizing the entire population there, and since they are having a hugely difficult time, they deserve the same protection and help and state action that minorities get (they are, by the way, a minority) ?
Now THAT would help the next president to be democratic. THAT is what we should do.
To make it unequivocally clear: no, I do not think the government should be in the business of censoring news (fake or not).
Most take offence at Obama defending muslims and sharia law, voluntary application of sharia law that is, after Newt Gingrich's call for mass-deportation. They do not claim Obama said anything he didn't say. Obama was very political, leaving out any actual controversial statement.
So I don't think, on the whole, that your statement is fair. These articles mostly lack quality, so they're easy to attack. They're not wrong, in a strict sense. Clickbaity, spammy, unpleasant, even misleading ... yes. Wrong, no.
And I have no trouble whatsoever finding all those properties in articles linked from the homepages of CNN, MSN, ... especially in the business and science sections. And I have to say, I am against Trump, but most of the frontpage stuff I would agree if anyone said half of them were hit pieces.
Also, why not use a white list that includes outlets that sizable portions of facebook's user base seem to think are reliable? For example, include breitbart.com and motherjones.com, but exclude .tumblr.com and .wordpress.com.
for example, say i am a hillary fan, and i say to another hillary fan "hillary is a criminal" as a way to mock people who say that. will facebook know when something is insincere and in jest?
How much of the hoopla around Sessions is "fake"?
How hard a concept is this, really?
Real news is saying that Trump wants to address immigration by building a wall. People may disagree about what that means, but it's still a true story.
The discussion is about the definition of "real news".
Two very different things.
Nowadays "the news" also includes:
1. What someone says might be happening right now.
2. What someone says about something that happened, or which might be happening right now.
3. What someone thinks might happen next.
None of which are necessarily true or grounded in fact but all of which are valued by the media in the race to be first with the headlines.
So perhaps "real news" could be defined as "events which have occurred and for which we have objective proof" and everything else is just "editorial".
IOW, this is very similar to a problem we expect companies to be able to solve: identifying phishing sites.
If a site claims to be ABC News, a known media entity, and it clearly is not, I'd argue anything posted on it should be considered fake.
http://www.denverpost.com/2016/11/05/there-is-no-such-thing-...
Define "news" as probabilistic statement of an event.
Define "real news" as news with probability > e, where e is some chosen threshold value.
Facebook regularly censors and directs conversation online around the world to the whims of various regimes and administrations, but it's just not easy "use machine learning".
The issue is that machine learning is inherently modeled to solve decision problems in spaces that separate high level 'features' from low level observations. The study and practice of machine learning thus far has modeled noise, uncertainty and complex state spaces.
However, machine learning classifiers do not themselves have a concept of adversarial data: this led, for example, to spammers who would "poison" machine learning classifiers by biasing them to learn the wrong things, so that they could get their spam through. Similarly, not too long ago, Microsoft put a conversation bot on the web that 'learned the wrong things' because people approached the bot in an attempt to trick it.
Facebook's requirement for battling unsanctioned opinions is that it solves this problem - it hasn't been solved yet. Facebook needs to continuously monitor the adapting activity and patterns of people who are trying to say things that governments and Facebook customers don't want people saying.
Basically: machine learning doesn't adapt but people do, and naive attempts to get machine learning to adapt make it vulnerable to being tricked, because classifier theory inherently has no concept of adversarial data.
https://en.wikipedia.org/wiki/Adversarial_machine_learning
http://www.cs.virginia.edu/~evans/
It's important to understand that these approaches are on the leading edge of research, and represent unsolved problems!
I also really recommend this book https://www.amazon.com/Trust-Me-Lying-Confessions-Manipulato...
The type of attack is a lot more oldschool but nonetheless, pretty effective.
Snopes published an article yesterday asking that asks the question more clearly:
There is much bad news in the online world, but not all of it is "fake" (i.e., completely fabricated information that has little or no intersection with real-world events). There are also partisan political sites that take nuggets of real news and spin them into highly distorted, clickbait articles. There are sites that misleadingly repackage old news as if it were current information. There are sites that aggregate articles from a variety of dubious and questionable sources. There are sites (especially in the fields of health and science) that believe they're presenting pertinent information but are woefully inaccurate in their information-gathering and reporting. These forms of news are all bad in one way or another, but broadly classifying all such information as "fake news" clouds an already confusing issue.
http://www.snopes.com/2016/11/17/we-have-a-bad-news-problem-...
http://abcnews.com.co/donald-trump-protester-speaks-out-i-wa...
Google is probably just as easy to fool as Facebook. This was the first hit on a search for "protesters paid to protest", a story that is demonstrably fake: https://www.washingtonpost.com/news/the-intersect/wp/2016/11...
Immediate red flags should be: This isn't ABC News's website. There are statements in the article like: "No one still has an AOL email address except people that would vote for Hillary Clinton" and from a made up Trump supporter quoted saying "I knew those weren’t real protesters, they were too organized and smart..." and (awesomely) "I think I was paid more than the other protesters because I was white and had taken classes in street fighting and boxing a few years back".
I mean, this may well be parody, but it gets picked up as real news and he's making real money off it, impersonating a real news site, which is the point at which it cross the line from parody to fake, IMO.
I think the trick with a machine learning approach to identifying these sites is not letting the model fixate on particular words (e.g. Obama or Moon Landing could be terms with the right dataset; though if they occur together maybe that's a good indicator after all...)
"May well be" is being unnecessarily conservative. Maybe you didn't make it far enough into the article to read the faked interview with the author of the the Snope's article I posted:
David Mikkelson, founder of Snopes.com, a website known for its biased opinions and inaccurate information they write about stories on the internet in order to generate advertising revenue, told ABC News that he approves of what a story like this is accomplishing.
“You have to understand that when a story like this goes viral, and we spend a minute or two debunking it, we make lots of money. Stories like this have helped put my children through college, buy a new car, a home and even get the Silverback gorilla my wife Barbara always wanted since she was a child,” Mikkleson said.
Buried, but actually rather funny. In a way, this is both a fabulous and a terrible example. Read in its entirety, it's clearly satire. Yet I'm sure that many people never read it closely enough to notice.
Should Facebook be censoring this? No. Should they be labelling it? Yeah, it might be a good idea. Is this the sort of story that the author is referring to? I don't know, but it would be nice if he'd tell us explicitly.
One things I don't get, though, is why so many organizations destroyed whatever credibility they had remaining in a futile attempt to win this election for their preferred candidate, I mean, this was an "it's my turn" race, a bit like Dole in 1996 and Romney in 2012. Especially prior to Trump winning the Republican nomination, was it really that important?
Note that this goes all the way to the DNC for the party itself....
http://www.cbsnews.com/news/googles-top-search-result-for-fi...
I have some problem with those claims that offer little to none details. Use the above link as an example, how do we teach machine to verify this news? First, we need to teach them to formulate a question, in this case, 'Who has a larger number of popular vote [as of now]?', then it needs to find its way to verify the number from reliable sources, possibly need to struggle between conflicted sources as well.
This is no trivial, if we solve this, we might as well solve the problem of understanding human language as a whole.
The mentioning of Princeton students' Hackthon project is really offbeat, if the author really bothers to look into what that project actually is, nothing more than an API mashup.
https://github.com/anantdgoel/HackPrincetonF16/blob/master/b...
Disclaimer: I worked for a major news app, and the effort to battle fake news or low quality non-trustworthy news is ever growing. While application of machine learning is promising, human, especially experienced human is still the real workhorse for this problem
Edit: Typo.
Look at the source. 70news.wordpress.com should not be at the top of anybody's list of reliable news sources.
Look at the topic (e.g. from simple keywords). Look at the references (urls) mentioned in the article. Look at the timeliness.
There is plenty of information available to machine learning algorithms, without even needing to touch the natural language aspect.
That's the only thing you can focus on. Why should someone be punished and labeled as fake if they're on a wordpress.com subdomain? Domain is just out, even spammy or weird looking domains may end up being popular / culturally fit. You would be preventing good journalistic folks from having any type of reach if you want to test against domains names, something that arguable may go away as far as the user is concerned in the next decade.
I'm unconvinced there is anything but content that you can really go off of. Keywords are not even used in search results, why would they determine news legitimacy? Even references are not always useful.
Because metadata can be super useful in making inferences about page quality. Pagerank exploited this idea a ton. If wordpress subdomains are more likely fake in a training corpus, then that piece of evidence should be counted.
i don't know about "labeled as fake", exactly, but it's possible that quickly and easily acquired urls (such as a wordpress subdomain) may display a different amount of reliability (per whatever heuristic) in comparison with various established urls.
I'm surprised Google messed it up so badly. Was 70news manipulated to the top of the results page, or did their ranking algorithm totally screw up a simple query? Neither makes much sense.
http://www.breitbart.com/2016-presidential-race/2016/10/16/m...
This is not 'fake' news, in a literal sense, because it just says someone is about to release something about Clinton. But for some people, this news could simply be seen as an indictment. Can we claim it is fake in that sense? As I have said above, it is equally controversial to penalize such websites for editorialization, because find story out of non-story is how most news agencies makes money and they have incentive to do this. Blame modern media and journalism for this.
In the end, using one simple algorithm to battle this is kinda of naive, not because it won't work for simple cases, but it fails to cover corner cases. In such scenario, human judgement is still crucial.
I agree that it doesn't cover corner cases, but why should it? If an algorithm knocks out 90% of the crap (like 70news.wordpress.com), there's much less work for the humans.
Also, wow, I didn't know Breitbart was quite that nutty. It reminds me of that LBJ quip along the lines of "it doesn't matter if he porks pigs or not, but now I made him deny it." This seems like a case where a human could do a bit of actual investigation, decide that the article's "Hillary is a lesbian" gist was BS, and permanently penalize Breitbart for publishing it.
"Google News Growth Hack Exposed" http://jmarbach.com/google-news-growth-hack-exposed
I want and will pay for a news service which only reports facts (no opinions) and which is concerned with accuracy. There are specialty services that provide this in certain niche areas, but no general service.
Then there is the false dilemma, and there are probably more fallacies in there, but I won't bother to read this utter bullshit anymore.
Google News seems to have something in place with the "Interested in? / Not interested" underneath some story abstracts.
The Information is an unpopular, I think even blacklisted, news source here due to their paywall. But they do a really great job writing a very terse and high signal two-sentence takeaway for each article. I'm not a subscriber anymore (too pricy IMO), but here are some recent examples:
Surge-Price Builder Leaves Uber By Amir Efrati · Monday Oct 17, 2016
> THE TAKEAWAY: One of Uber’s secret weapons in recent years was an economics professor, Keith Chen. He overhauled Uber’s surge-price algorithm and made other key contributions that helped riders, and Uber, save money.
How Dropbox Doubled Down on Business Market By Steve Nellis · Thursday Sep 15, 2016
> THE TAKEAWAY: Dropbox founder Drew Houston has come to terms with the fact that his company probably isn’t the next consumer tech giant like Facebook or Google. That was reflected in a decision halfway through last year to sharpen the company’s focus on the business market, including shutting off consumer-focused apps like Carousel.
I would love to have that for all tech news.
There are ML systems out there for summarization. I wonder how long until the systems are good enough to provide this as well. The examples I've seen (e.g., Summly, which uses SRI tech, IIRC), are likely good enough if you just want the facts. I still want it to be well-written.
Ouch. I just checked out the subscription rates for The Information. AFAICT, you can't even see the rates without supplying your name and email address. :/
Edit to add: From other sources on the internet, it looks like subscription rates are $399/year.
I do remember the rates being hard to find as well: $39/month or $399/annual [1]. They do offer you half-price initially etc for a few months.
[1]: https://www.theinformation.com/payment-policy
I suppose what I meant is technically not microtransactions (mostly just used in the context of in-app purchases) but micropayments [1]. I've added ko-fi [2] to my personal site but no one has actually done it yet.
[1]: https://en.m.wikipedia.org/wiki/Micropayment
[2]: https://ko-fi.com/Home/About
they rewrite all the headlines to get rid of clickbait nonsense and actually save you the click in some cases if their summary is enough.
[1]: https://en.wikipedia.org/wiki/Reinforcement_learning
It's very blurred indeed.
What would probably help is extracting list of (main) statements/assumptions made and give them fact-probability score (ideally with references) - and show it to people.
But this is not trivial exercise, is it?
I mean, once we have technology that's good in doing it, well... it means we can probably rely on this AI to make 80% of decisions to run the country.