109 comments

[ 4.0 ms ] story [ 193 ms ] thread
Interesting choice — I love the NYT Public Editor's work and was actually just about to add her column to my media literacy / filter bubble project, Read Across the Aisle. Many people have never heard of a Public Editor and have no idea what the position entails. From a media literacy standpoint, this column offered a behind-the-scenes look at why certain decisions were made, how journalistic standards were applied, and the pros/cons of alternative options.

In this time of political discord and journalistic polarization, I am surprised and disappointed that such an esteemed paper would decide this position is no longer necessary.

I hope the NYT keeps the existing archive online, and frankly I may still add this archive to my media literacy app — even though the underlying stories would become increasingly stale over time. IMO, it is that good!

You love Liz Spayd’s work, or the previous public editor, Margaret Sullivan? I’ve found Spayd’s column pretty anodyne.
I have mostly read Spayd's work, and I enjoy the analytical aspects of it. How would you say Sullivan's work differed? Worth combing through the archives?
The NY Times created the Public Editor role in response to a fabrication scandal. Are they trying to tell us something?
That they never really wanted it?
So they think that they can just replace that role with social media? Wasn't the whole issue with newspapers in the 2016 election was they relied far too much on social media and ended up in a bubble?

I don't see anything in the article that makes me feel like they are interested in fixing that problem.

Agreed. I have occasionally tweeted corrections at authors of articles and have never gotten any response. And these aren't famous authors who get too many tweets to respond too — these are contributors with a handful of Twitter followers. If the new model of interacting with the public is anything like this, I don't have much hope for it.

I will say that when I eventually sent an email to the NYT's corrections email address, they immediately fixed the errors in the articles. This won't help with close judgment calls or understanding how the newsroom works, but at least it can be used to address blatant errors.

ended up in a bubble?

Yes they did. They know it. And they don't really give a fuck.

After the election they did some public navel gazing about what went wrong. To prove they were serious, there was even an editorial about The Future of the Times.

So I wrote to Ms Spayd. I challenged her, at the next meeting of journalists and editors, to say something like: "show of hands, who voted for Donald Trump for president".

My contention is that the result would be zero. Nada. Zilch. Even if they did vote for Trump, they wouldn't be dumb enough to admit it publicly to a room full of their colleagues.

So how can the Times ever begin to understand the 46% of the country who are the deplorables who voted for Trump?

They can't, and they don't want to.

Or, maybe, Ms. Spayd actually did what I suggested. And the powers that be decided that she just wasn't a good fit for them anymore. And that's why she was shown the door. :)

Employing Trump voters would create an unsafe work environment for women and minorities.
> So how can the Times ever begin to understand the 46% of the country who are the deplorables who voted for Trump?

You don't need to be a member of a certain group to be able to report on them. E.g. you don't need to be terminally ill to report on people that are terminally ill, or member of the KKK to report on them.

In the last election people reporting on the KKK outnumbered the clan 10 to 1.
No one is expecting liberal journalists to turn conservative. What we need is for liberal journalists to take conservatives seriously. If the liberals really are right (and I think they're righter than the right) then they should be able to address conservative grievances within their own framework. But in recent years, they haven't been. Encouraged by Obama's success, the liberal establishment has determined that conservatives are all racist sexist homophobic rubes and can be freely ignored. That's how you end up with Connecticut, currently a wasteland where the taxes are unusually high even for a New England state, but public satisfaction in public infrastructure and services is low.

If there are ZERO Trump voters among staff at the Times, that is a strong indicator that the Times reporters are spending a lot of time in a bubble, and could benefit by hiring a conservative just to function as devil's advocate (like Fox News did with Alan Colmes).

> and could benefit by hiring a conservative just to function as devil's advocate

They do hire conservatives (Brooks, Douthat, Kristol, etc) and they just hired Bret Stephens.

> (like Fox News did with Alan Colmes)

You make it sound as if Alan Colmes, during his time at Fox, was there to give an equal voice to the liberal side of an issue, but in reality he was just marginalized to be the token "lefty" punching bag for Hannity.

I don't follow the Times that closely any more, but from what I can see David Brooks is their analog to what Alan Colmes was to Fox. Somewhat like a court jester.

But even Brooks knew enough to publicly declare that he wouldn't vote for Trump, even though Trump's political views probably aligned more closely to Brooks than Hillary's views did.

> I don't follow the Times that closely any more, but from what I can see David Brooks is their analog to what Alan Colmes was to Fox. Somewhat like a court jester.

Then I'm not sure why you have such a strong opinion of the NY Times? Also, no one at the paper is using Brooks as a token conservative in the same manner as what Colmes was for Hannity. Brooks even appears weekly on PBS.

> But even Brooks knew enough to publicly declare that he wouldn't vote for Trump, even though Trump's political views probably aligned more closely to Brooks than Hillary's views did.

Political views aren't binary. Brooks can have differing views from Trump and Clinton. I don't understand the underlying need to declare blind allegiance to Trump just because one is a conservative.

Political views aren't binary.

Not that Trump is a true conservative, but ...

Our current political system more or less forces binary choices.

E.g. take the case of the Supreme Court. Unequivocally Gorsuch is a much more conservative appointee than Garland would have been.

Were Trump's negatives so high (in Brooks' eyes) that Brooks was willing to accept perhaps two or three liberal court appointees from Hillary rather than live with Trump's baggage? Apparently so!

One issue is that the token conservatives brought on for intellectual diversity are almost always the same sort of conservative. The elitist think thank crowd that formed the Never Trump movement. It's pretty clear that outside of Northern Virginia, those guys don't represent mainstream conservative thought.

But even putting someone like Douthat on the editorial side doesn't impact the news side of the Times.

I think the NYTimes does a pretty decent job, but there is no question they missed the story of 2016 because their news reporters are all inside the bubble.

Agree that Colmes was an unglorified punching bag. Fox is the wrong entity to imitate.

What is "mainstream conservative thought" to you, may I ask?

From where I sit, the Republican party (often called "conservative") is divided into several factions right now. The think-tank side traditionally caters to three of them: neo-conservative / "law and order" type causes, libertarian causes, or business interests. The Republican party also is the home of both a religious / fundamentalist wing, and a cultural-identity / white nationalist wing; these sides are generally not well represented in the think tanks.

Maybe the later two are what you consider "mainstream"? The issue with the later two camps is that the representative media can be very populist in nature. And populism can be very us-vs-them / emotion driven, sometimes with disregard for facts. This probably would not fit well at all with the current editorial stance of the Times, even if the populism comes from the left.

you don't need to be terminally ill to report on people that are terminally ill

That's because reporters don't generally despise the terminally ill.

But if you want to honestly report about politics, then you can't think of people as deplorables. You can't automatically ascribe all sorts of negatives to 46% of the country because they didn't vote for "your" candidate.

If you despise what you're reporting on you will never, can never, report honestly. Your biases will overwhelm your coverage. You will not be able to understand what you're writing about.

And in the case of the Times, they don't even make an attempt at impartiality. They don't need to. Their target audience exists in the same bubble.

You do when that group is half of the population give or take a few percent, and you want to pretend to be anything resembling objective. Stances on terminal illness and the KKK are easy to take and uncontroversial - especially when both of those groups are tiny.

(And let's not even get into how loaded of a comparison you just made...)

Nate Silver has done some good work on what exactly went wrong, and it was a combination of factors:

- (some) polls were systematically off, mostly because they underestimated white non-college turnout and overestimated black turnout. Note that poll errors were actually lower than for any other election–only this time they actually got the winner wrong.

- Most of the poll aggregators operated under the assumption of statistical independence of state-level results. This was a serious mistake, and both Huffpost as well as The Upshot (to a lesser extend) should be somewhat ashamed. 538, by the way, did this right and had Trump at 3X% on election day.

- There actually was some significant change in voters' intent in the last days, possibly fuelled by all the news stories the FBI directors was generating. It's highly likely that an election two weeks earlier or two weeks later would have ended differently.

It's also important to note that newspapers aren't really in the business of predicting the future. If they "relied too much on social media" for any facts they reported, I'd love to see some examples.

Beyond making bad predictions and tonally describing the election poorly (it didn't seem to be coming down to the wire based on the coverage), the NYT did an atrociously poor job of explaining the mind of a Trump voter to its readership. To this day, there is a lot of reliance on imagery of be-mulleted bogeyman that is both facile and harmful if America wants to live in harmony with itself again some day.

That is, we're not just talking about getting the numbers wrong. The "narrative" itself is way out of whack as well.

2X% actually (I remember watching it in real time), but the point stands as it was way higher than anyone else thought.
And Nate Silver got a lot of flak for that in the week before the election.
He did indeed! He published an article in the last week that listed 4 outcomes for the election, each with ~25% confidence. 3 Had a Clinton victory and 1 did not. I recall him also making some pretty emphatic condemnations of Sam Wang's Princeton Election Consortium, arguing that the numbers absolutely did not support PEC's >95% confidence in a Clinton win. Wang, for his part was so confident that he offered to eat a bug if Clinton lost. Wang's statistical method turned out to be a disaster, but he did follow through.
Having actually eaten a bug before (yay Boy Scout survival badge), I'm going to remember that one. The follow-through will impress people but it's not actually dangerous or even that unhygenic (as long as you eat the right bug).
The system I've been working on actually predicted a Trump win: https://projectpiglet.com/

The problem was so many people just hated Clinton, and at the time, Trump had more people positive about him.

Then you should be very sceptical of your system. "History proves me right" works for Fidel Castro, but not for lottery winners.
I am fairly skeptical, but given it's been pretty consistently accurate, I'm becoming more confident.

I look at net Promotor Score on any type of entity. It's predicted the rise of Bicoin, and various other currencies, but it's not always perfect. For example, it didn't predict a brexit.

It could just be I don't have enough data. It's a work in progress.

Silver's analysis probably overstates the impact of the Comey letter:

> it’s now clear that Mrs. Clinton was weaker heading into Oct. 28 than was understood at the time. Several other polls were conducted over the same period that showed Mr. Trump gaining quickly on Mrs. Clinton in the days ahead of the Comey letter. And the timing of these polls — particularly the gap between when they were taken and when they were released — has probably helped to exaggerate the effect of Mr. Comey’s letter on the presidential race

https://www.nytimes.com/2017/05/08/upshot/a-2016-review-ther...

On one hand, having an ombudsman (whatever you want to call it) is a good thing for an institution like the Times.

On the other hand... having Liz Spayd in that role is pointless, so this isn't a loss. The past year has shown us how the New York Times has let false equivalence and the need to sell newspapers to both sides of the aisle has severely damaged the quality of the paper's coverage, and Spayd has continually carried water for this rather than hold the paper to account. I for one am tired of watching the Times sell itself with ads about how the truth is more important than ever and then follow it up with coverage like this:

https://www.nytimes.com/2017/05/03/us/politics/health-care-f...

Where Pelosi saying "Up to 17 million children" is supposed to be misleading because it chooses the high side estimate, when that is exactly what 'up to' means.

Getting rid of the public editor won't make this better, but not having to read a public editor who won't confront this sort of coverage head-on will at least make me a bit less cranky about it.

I'd feel more able to agree with you if the NYT didn't carry water for some whoppers ("you can keep you doctor", "insurers will compete for your business", etc.) back when the ACA was passed in the first place.
Insurer's response to ACA reminds me of a Frank & Ernest comic, which I can't find right now, sorry:

Two USPS workers. Package label says "Do not bend, fold, mutilate." One says to the other "Doesn't say anything about crumple!"

Har har har.

---

I had high hopes for ACA. I believed the 15% medical loss ratio was the poison pill that would eventually drive insurers out of business, and therefore lead to Medicare for All. For comparison, Medicare's is under 5%, meaning 95 cents for every dollar spent goes to patient care, vs coke and prostitutes for the execs.

Silly me. There's no set of rules that can't be broken, gamed.

So now I'm for dozing it all down and rebuilding with universal coverage, single payer, using the capitation model (reward wellness vs fee for service). No more incrementalism. Just do it.

---

Aside: Mid 2000s, my team created, implemented, supported 5 regional health information exchanges (eg BHIX). It was clear then, to everyone in healthcare IT, that we were pushing cooked noodles across hot pavement. Since then, it's only gotten worse. Overhead continues to grow geometrically, due to things like ICD-10 and meaningful care, without nudging the patient care (outcomes) needle one bit.

I can't explain it.

I just read about cost disease. It seems to explain rising costs, but I don't understand the mechanism, or what to do about it.

https://en.wikipedia.org/wiki/Baumol%27s_cost_disease

And this book blew my mind:

"The Utopia of Rules: On Technology, Stupidity, and the Secret Joys of Bureaucracy"

by David Graeber

http://amzn.to/2mi169U

Putting any kind of cap on efficiency of a middle man is asinine. The only outcome is the realization that the only way to make more profits is increase the top line number (i.e. Premiums).
Our government has capped APRs for credit cards. Then, banks added fees, reduced services. Then govt capped fees.

Whackamole.

Is this a legit analogy? If so, is either side wrong? How would you handle it, if you were King?

My only notion for healthcare is to change the incentives. Reward wellness vs treat disease.

> My only notion for healthcare is to change the incentives.

That's only a recipe for lobbying over what bureaucrats designate "wellness" and "disease".

The best way to fix healthcare is to cut out more middlemen. The person whose health is being cared for (or their loved ones) needs to decide when another $10k scan or $65k pill regimen is worth it compared to using the cash to pay off their (grand)kids' student loans.

If all of this is filtered through insurers, lawyers, and government officials, there is incentives to meet budgets and increase revenue, but there's no win for the actual people and families all this is supposed to be taking care of.

Wellness is easily measured. Life expectancy, sick days, patient surveys. The fight happens during rationing (bureaucratic triage).

So you'd eliminate pharmacy benefit management companies?

https://en.wikipedia.org/wiki/Pharmacy_benefit_management#Li...

Or are you saying eliminate insurers (insurance) altogether?

The government can't be rational about what a balanced meal looks like or how tomatoes (fruit? veg?) fit in the picture.

There's a reason food lobbying groups exist.

> Or are you saying eliminate insurers (insurance) altogether?

I'm not saying there's an easy answer, but as long as all health transactions are paying for one party's health with another party's money, we'll have bad outcomes.

If the problem is that it's expensive to live, then let's talk about that problem (negative income tax credit, minimum income, optimize for cost of living, etc.) instead of trying to bureaucracy (private or public) our way into utopia.

According to the government a slice of pizza counts as a serving of vegetables because of the tomato sauce.
This may not be what you mean, but this comment triggered a thought. If it were possible to really measure wellness and reward or punish providers accordingly, the new "health insurance" would be a sort of reinsurance that such providers purchased to control risks related to their patients' health. No, it wouldn't be cheap, and yes, the costs would be passed on to someone.
Who has 65k cash to toss around like that? :)

If the doctor can't effectively predict what benefit said scan or pill regimen will provide (50% chance of 10 year life extension,90% chance of 2 years...? not even getting into quality of life), how does the consumer make informed choices? This approach devolves to if you have money you get care, if not, you don't, regardless of need or efficacy. Similar to any other market.

There or some easy decisions to make in medical care (set the broken bone, stop the bleeding), but less so when it comes to areas driving price increases.

Adding layers is potentially problematic, as is removing them. Single payer and no-payer are attractive solutions due to ideological simplicity.

I wonder if a tax funded provision of care rationed similarly to the organ transplant system would be workable...

> Who has 65k cash to toss around like that? :)

Most people who are paying that much for pills are retired middle class people trying to stay healthy into old age. Or alive for a few more years. But people should be able to pay for typical levels of healthcare. The fact that this isn't true is a question of economics and employment opportunities, not entirely problem with the "healthcare system" itself.

> There or some easy decisions to make in medical care (set the broken bone, stop the bleeding), but less so when it comes to areas driving price increases.

> Single payer and no-payer are attractive solutions due to ideological simplicity.

I agree with this first sentence more than the second. These decisions are never all that simple. And they are made even more complicated or perverse when you ignore that the best alternative for a bureaucrat (private or public) approving treatments is saying "no" and going home for a Scotch. The best alternative for the patient is saying "yes" and spending $300k and live another year in diapers.

If people were spending their own money, they would also think about:

- grandkids that want to go to college

- underfunded soup kitchens in their neighborhoods

- not reverse mortgaging the family home they wanted to pass down to their loved ones

- going on a "farewell" cruise

And that's just one category of health care decisions. There are more mundane ones like "Should I try the generic treatment for a week to see if it helps? Or should I put $3k into the boutique one and save the hassle?" I wouldn't know how a (even highly qualified, well meaning) bureaucrat in Atlanta would make a better decision than the person getting treated.

> Most people who are paying that much for pills are retired middle class people trying to stay healthy into old age

What of all the people whose lifespan is cut early by disease? Getting cancer or some other serious disease at a young age just due to sheer dumb luck is less common, but still quite prevalent. Or let's just think about traffic accidents? It's just preposterous to me that people should ever have to worry about sudden illness or injury. If you're looking for bloat and overspending, why not slash the defense budget instead?

> The best alternative for the patient is saying "yes" and spending $300k and live another year in diapers.

Wait, wasn't this exactly what republicans criticized obamacare for? The accusation was that there were "death panels" that would decide whether it's "worth it" to put all that money into that one dying patient, or whether it's better spent elsewhere. So we don't like it when obamacare appears to be doing it, but now that obamacare is gone we want it?

> What of all the people whose lifespan is cut early by disease?

Actual insurance and government welfare programs are for things you can't otherwise save for. Let's not burn straw men, here.

> If you're looking for bloat and overspending, why not slash the defense budget instead?

That's fine, too. But paying for the healthcare of an entire country is far more expensive than even U.S. defense spending. And cutting defense spending doesn't address the structural problems of having party A pay party B to pay party C to produce good health in party D.

> The accusation was that there were "death panels" that would decide whether it's "worth it" to put all that money into that one dying patient, or whether it's better spent elsewhere.

I want the patient to be able to decide (for instance) that another year on the lam from the reaper isn't worth the high price tag. That only really works if the savings goes to what they want: family, charity, community, etc. and not back into the general fund of a government or corporation.

One issue here is that health care can get really complicated really fast. Which makes it quite a bit more difficult for many consumers to make an informed decision.

Because health care is complicated, in an idea world, there would be a layer to distill all of the complicated scientific information into simpler, scientifically sound recommendations. In theory, this is what your doctor should do. In practice, this system is currently broken in the United States. It is easy to demonstrate cases of over-treatment in the current medical system. (http://www.newyorker.com/magazine/2015/05/11/overkill-atul-g...).

If the consumer was paying for everything with full price visibility, would this change? I'm not sure. The problem here is a large amount of people would have no idea whether the prescription / test the doctor is requesting is worthwhile or bunk.

In fact, I think some of the over-treatment today is by "by patient request". One reason given for why doctors write antibiotic prescriptions for viral infections (which as we should know is scientifically useless and may in fact be harmful due to the potential of allergic reactions) is to make patients feel better by making it seem like something is being done. (http://ideas.time.com/2012/04/16/why-doctors-uselessly-presc...). The same probably goes for even more expensive cases (MRIs for most back pain cases, for instance).

Think of what we see in the "alternative medicine" market. There may be a few hidden undiscovered gems in that field, but in the majority of cases people pay sometimes a ton of money for what ends up being a rather expensive placebo effect at best (or harmful snake oil at worst). All of this both because medical science is difficult to understand (where glossy marketing is easy to understand), and also because in many cases medical science has unsatisfactory answers for people desperate to be healed.

I'm not saying that knowledgeable bureaucrats are completely correct (ideally the doctor and patient are making the informed decisions together, not the bureaucrats) but I can't see not having some layer that aims to eliminate the current perverse incentives in the system.

Who has 65k cash to toss around like that? :)

If the doctor can't effectively predict what benefit said scan or pill regimen will provide (50% chance of 10 year life extension,90% chance of 2 years...? not even getting into quality of life), how does the consumer make informed choices? This approach devolves to if you have money you get care, if not, you don't, regardless of need or efficacy. Similar to any other market.

There or some easy decisions to make in medical care (set the broken bone, stop the bleeding), but less so when it comes to areas driving price increases.

Adding layers is potentially problematic, as is removing them. Single payer and no-payer are attractive solutions due to ideological simplicity.

I wonder if a tax funded provision of care rationed similarly to the organ transplant system would be workable...

HHS doesn't actually pay medical costs with that 95%. They take a 5% cut for their own overhead and then turn around and pay a group of private insurance companies to provide coverage with the rest.
I think the 5% includes the administrative cost of the third-party administrators as well as HHS itself. The 95% is payments to hospitals, doctors, pharmacies, etc.
I believe the ACA itself was a poison pill that would make the system so inefficient to drive it to collapse and bring about single payer.

The failure of Republicans to implement an alternative has pretty much guaranteed it.

I think this is right. If not a deliberate poison pill, implementation of the ACA was a strong "foot in the door" technique for those who want single payer. Though, even before the ACA, the government is deeply involved in the system (employer based tax incentives, Medicare, etc)

I don't find that surprising though. I think Obama himself said that ACA was just a start and would have to be revisited. (Citation needed, I know.) What's amazing is just how bad conservatives have been at articulating the free market case for health care and insurance.

I'm hoping Donald Trump is the poison pill that will bring about the Progressive movement.
He's a poison pill alright, but I'm not sure who's going to feel the poison.

He's either the end of the Democrat establishment (the election and post-election hysterics are unheard of in my lifetime), or the end of the Republican establishment (if he crashes and burns as bad as certain circles think he will).

Healthcare costs grew under the ACA, but the rate of growth slowed. I don't think it was more inefficient than continuing with the old system would have been. Right now though they're working to defund as many parts of the ACA as they can, so even if it doesn't get repealed, it's going to be in bad shape from now on.
It's important to recognize that capitation provides a powerful economic incentive to provide as little care as possible. Every pill, X-ray, MRI, or surgery comes straight out of the insurer's bottom line. You'd like to see an allergist/rheumatologist/orthopedic surgeon? How about the thirty-third of Nevember? Instead you can talk to a high-schooler in another country with a wellness script.
I don't see how reporting a politican's comments were "carrying water". At the time Obama said that you could keep your doctor I don't think there was any way to disprove his statement, because the ACA hadn't been enacted yet.
I'm not carrying water for the ACA, which I think is not-very-good law, and I remember back when even congressional Democrats thought it was "the stupid compromise we can get Max Baucus and Joe Lieberman to allow to pass the Senate" and not a hill worth dying on.
What's the alternative?

I remember similar things said of their prior ombudsmans (ombudsmen? ombudespersons? ha). I stopped reading NYT when they helped sell Bush's Folly in Iraq, so I don't have a dog in this fight.

I ask because I've been eagerly awaiting the next iteration of journalism. Though I can't guess what that'd look like.

Frederic Filloux (mondaynote.com) has some notions. Mostly how to make it viable.

And I really wanted the Nerd TV 2.0 format experiment to happen (publish all source material as-is, much like we're asking for academics).

And lastly I really wanted the wikipedia model, as envisioned by Clay Shirky in Cognitive Surplus, to become more mainstream, but I don't know why that format isn't replicated more widely.

> false equivalence and the need to sell newspapers to both sides of the aisle

What does that mean? Are you saying you would prefer the paper take a more hardline stance in favor of one side because you perceive that side to be objectively superior in some manner?

Case in point: in the Washington Post story regarding Jared Kushners attempts to bypass government oversight of diplomatic negotiations with Russia, both WaPo and NYT both received comments from Kushner confidants alleging these contacts were intended to facilitate cooperation of military actions in Syria. WaPo refused to publish these comments unless they could specify that they were Kushner confidants speaking in his defense. NYT had no such qualms, instead allowing these partisans to be presented as anonymous neutral parties, in exchange for their continued access.

Allowing Kushner to present his spin as neutral is a de facto false equivalence and shifting the 'objective' reality, which other organizations (WaPo) who practice investigative journalism, as opposed to access journalism, refused to allow themselves to be used for.

Everybody is a political hack. There is no such thing as objective reporting. You are expressing a sentiment which reflects a suspicious and conspiratorial perspective on the Russia dealings. Rush Limbaugh would express a sentiment reflecting a dismissive attitude towards that topic. Both you and Rush believe that you are reporting the objective truth, but you would point out the false equivalence of comparing your enlightened and objective opinion with Rush's obvious partisan hackery. And so it goes.
TIL that reporting on the potential conflicts of interest in a sources comments are signs of political hackery.
You do yourself a disservice by misrepresenting my point. A lot of folks throw around the phrase "false equivalence" to mean that the current president is abnormally and objectively bad, and that those opposed to the administration posses some sort of moral and intellectual superiority. This is the broader hypocrisy to which I am referring.
I said it's a false equivalence to present trumps detractors as detractors, and trumps supporters as a neutral party.
(comment deleted)
Unfortunately, the thing that will always stick in my memory about Liz Spayd is her criticism of the New York Times for refusing to give more credence to the claim that junk emails about Trump hotels were a secret communications channel with Russia: https://www.nytimes.com/2017/01/20/public-editor/trump-russi... (Seriously. Every single piece of evidence pointed to the communications being mass marketing emails, from the companies actually operating the server to The Intercept obtaining copies of some of the emails.)
Wouldn't be surprised if in a few years there will be studies about the "Russia" propaganda campaign. How first it was very effective, everyone repeating the same things, the infamous "dossier" with sexual allegations, communication via mass marketing emails and so on. How it was beautiful and effective, and then everyone went too far and it ended up as a very expensive, counterproductive and embarrassing affair.

Also I wonder to what degree the journalists and anchors involved actually believe those things, or they simply know what to sell and do it regardless of their personal convictions in the matter.

I'm not sure you understand. It's not about what journalists genuinely believe, it's about what narrative creates the most polarizing, emotional and long-standing controversy, because those traits result in eyeballs and therefore revenue.

That's why Trayvon Martin was such a massive story... two polarized, completely entrenched sides battling at it endlessly. Truth didn't matter, the hyped up emotionalism of it all was a windfall for the media.

I'm not sure you understand: the Times didn't publish those stories, precisely because they didn't feel they had proof, and in doing so they were willing to miss out on the eyeballs and revenue that would have come with publishing.
Did you read what you're replying to? The New York Times didn't buy into some sort of "propaganda campaign", nor were they part of it.

If anything it proves that journalism is doing quite well at the NYT, and the narrative of "bias" and "propaganda campaigns" is undermining civil society, and unfairly accusing people doing the best they can and mostly succeeding.

Supposedly a good five or so publications passed over the Trump hotel communications with Russia story for the same reason, but ultimately it didn't matter that much because the moment one publication (Slate) was willing to publish their article went massively viral on social media, the Clinton campaign helped spread it, and huge numbers of people saw and believed it. We've effectively reduced the accuracy of the media as a whole to whatever the sloppiest, most view-hungry outlet is willing to put out that fits everyone's preferred narrative.
> Did you read what you're replying to? The New York Times didn't buy into some sort of "propaganda campaign", nor were they part of it.

I was replying to gp's comment about Liz Spayd criticizing Times

You mean that same dossier of which broad swaths were verified by the intelligence community to have been correct?
What was verified? Once they stuck the golden showers bit and the blackmail, other stuff to be credible would need some solid evidence/ Have you seen any? I haven't.

It's a bit like having a conversation with someone and as soon as they start talking about chemtrails it's an uphill battle to take anything from them seriously after that.

The mayflower meeting happened pretty much as described in the Steele dossier. Seth Abramson made a nice tweet storm summary. There's even pictures of the individuals there in question: https://theintellectualist.co/reporter-makes-convincing-case...

Also, its odd to me that people intentionally try to misrepresent raw intelligence. Raw intelligence is so named because it's an amalgamation of different sources and even possible rumors that were deemed pertinent to include. Intelligence officials regularly deal with these, so there are established procedures to verify its contents independently. So to hear you say that since one thing may be wrong, then everything is wrong, therefore it's fake, seems deceitful.

Lastly, we have the curious fact that most every Russian implicated in cooperating with the Steele dossier had been murdered, found dead, or arrested by the FSB shortly after its existence became known to Russian security services.

Id also suggest it's odd for a sitting president to repeatedly obstruct an investigation that he steadfastly maintains will find nothing.

I wouldn't mind if the N.Y. Times crawled up itself and disappeared. The main loss would be the crossword puzzle.
Before I start, Disclosure: I'm neither a Trump or a Hillary supporter and I don't mean the following to be political. I'm just very dissatisfied with the sad reality of how I cannot trust anything online anymore.

For me, the moment all my trust in NYT died was the election day. The front page had this interactive realtime story of how Hillary is supposedly winning by overwhelming margin, when it was very apparent that that's opposite of the truth. People kept questioning "where the hell is this data coming from?".

I believe Newspapers should try to play as much neutral role as possible, but throughout the entire election cycle the narrative was NYT vs. Trump. News organizations and journalists should report stories but should not become the story.

As someone who just wanted to learn what REALLY is going on, I had (and still have) hard time believing anything they say anymore. Other conservative publications at least don't claim to be neutral. A publication shouldn't claim to "tell the truth" and do a whole TV ad about it if they are clearly biased and they're also aware of this.

I know they're being provocative because this sells, and they need to make money, and it's getting harder to make money with newspaper. But they've crossed the line.

People keep talking about "fake news", but what's really dangerous is not some viral website that write some funny photoshopped story. What's really dangerous is influential news organizations like NYT doing deceptively objective-sounding pieces with narratives that mislead the public.

By now I think a lot of people know it's very possible to take a same piece of reality and write completely different versions of it, just by framing them differently. Because of that I have hard time believing anything online anymore, especially the influential ones. This is sad because it used to be the job of journalism to fight against injustice, but nowadays it's more about generating more page views and getting more subscribers, which led to people losing faith in journalism.

It is my understanding of history that most newspapers used to have open political biases; they were even on the take from the party bosses, and it was written into their mission statements. You knew through what lens to filter the content, and they were happy with smaller, more limited readerships. This aim for objectivity in printed reporting was a recent 20th century development.
One of my friends sold all his stocks buying into nytimes predictions of imminent stock market crash worldwide post trump election.
> The front page had this interactive realtime story of how Hillary is supposedly winning by overwhelming margin, when it was very apparent that that's opposite of the truth.

I don't remember that at all. I remember the chances of winning, but that is not in any way a claim that she is winning by a large margin. In fact, reading the articles during the campaign and during election day, there was a lot of talk about even though polling data read one way, Trump wasn't out of it.

But yes, if you just read the headlines and didn't actually read the content, I could see why you'd say this.

Here's one leading up to the election: https://www.nytimes.com/interactive/2016/upshot/presidential...

And here's the page I was talking about: https://www.nytimes.com/elections/forecast/president

Now it says >95% Trump, but on that day it was the other way around--something like >95% Hillary.

The early predictions were based on the early polls, and the later predictions were based on data after election results started to come in. A lot of the States that ended up swinging were on the eastern half, so the rapidly changing outlook makes sense if you look at the bigger picture (rather than just the number at the end).

That being said, predicting the outcome of an election with some 250 million potential voters based on a few hundred thousand polled individuals is a tricky game to play.

> The front page had this interactive realtime story of how Hillary is supposedly winning by overwhelming margin, when it was very apparent that that's opposite of the truth.

I remember that interactive story. It was not what you describe. It was showing Hillary's chances of winning throughout the evening - she started at 90% or so, based on the polling averages they'd been running on a regular basis throughout the entire election season. Then, as the evening wore on, and results were not as the polling would indicate, her chances of winning dropped, dropped, and then obviously went to zero.

Now, I won't argue that in hindsight that was a pretty bad feature to run. I think it vastly overestimated reader ability to understand statistical probabilities, as well as the accuracy of polling data (the Brexit vote ought to have been an indicator to be cautious). But it absolutely wasn't showing false data that indicated Hillary was winning when she was not.

How is it not what I describe? You just described exactly what I described.

Everyone was wondering (both pro-trump and anti-trump) where the "90%" came from. Because it was pretty clear the race was not that obvious. People keep saying nobody saw it coming, but that's their emotion speaking. People just didn't want Donald Trump to win because they thought he's a joke, but objectively speaking people knew clearly that there were tons of his supporters around the U.S., and it was definitely NOT 90% chance that Hillary would win.

To me, THAT's the ultimate "fake news". Slapping a percentage number on it and putting it on the front page is just not right, especially when it's based on very subjective (and eventually false) speculative data.

You said:

"The front page had this interactive realtime story of how Hillary is supposedly winning"

I said:

"it absolutely wasn't showing false data that indicated Hillary was winning when she was not"

They're pretty clearly contradictory. That data, at no point, showed Hillary "supposedly" anything. You're making my point about readers not understanding statistical probabilities. 90% doesn't mean she's winning. It means she's got a 90% chance of winning, and there is a 10% chance she will lose. She lost.

> People keep saying nobody saw it coming, but that's their emotion speaking.

Quite the opposite. The data (that same data you're saying came from nowhere) showed that Hillary had a 90% chance of winning. There was variance in the polls, but pretty much all of them showed Hillary with an edge. Yes, there were a great many reports of people saying that there was a lot of Trump support across the country, but those reports are the very opposite of "objective" - they were thinly spread and anecdotal. Pretty much every newspaper ran stories interviewing those Trump supporters, but there was no hard data to quantify it.

Polling data isn't "subjective" and "speculative". There is a clearly defined methodology behind it. That methodology was incorrect in the 2016 election, and didn't accurate gauge support for two unprecedentedly unpopular candidates. That makes it "incorrect news", not "fake news".

You could argue that the news industry has relied too heavily on this sort of statistical prediction stuff, and I'd agree with you. But in 2012 FiveThirtyEight correctly predicted all but one state, and everyone absolutely lauded Nate Silver for it ("Is Nate Silver a witch?"). But really, what that meant is that each of Nate's predictions were the right side of 50%. And in 2016 they most definitely weren't.

Polling data wasn't even wrong. It was almost spot on in measuring popular vote totals. The mistakes were solely with some state level polls (and really not even mistakes, just high uncertainties) and poll aggregators.

Really, I think most of the incorrect predictions came from one error: assuming a higher level of independence between state results than is actually the case. Nate Silver/538 talk about this some, and they gave Trump the highest (~30%) chance of winning on election day.

Polling can be incredibly hard. If you ask the wrong type of questions you introduce a bias. If you ask the wrong types of people you have sampling error.

All of these things could have occurred, but something definitely was wrong with the polling data. Nate Silver just like NYT gave Hillary a 90% chance at the start of the day, and this was all based on using the polling data as inputs to their models.

>Nate Silver just like NYT gave Hillary a 90% chance at the start of the day

He didn't though, and that's my point. What you're saying is factually untrue, they haven't updated their prediction since election morning, and it sits at 71% Clinton [0] (Updated Nov. 8, 2016). If you don't believe that, then you can take a look at archive.is, which provides timestampped copies. This [4] is an archived copy from well before polls closed (3:30 PM), there are others from earlier in the day.

I don't know what led you to believe that it was a sure clinton win, but that just wasn't the case.

Clinton won the popular vote by somewhere between 2 and 3%. National polls from just before the election (also per 538, [1]):

    +4, +2, +3, +4, +4, +4, +4, +4, +2, +3
An arithmetic average of those gives C + 3.4.

Take Florida, a state trump won, here's polls taken in Novemeber [2]:

    +1, -3, +0, +2, +1, -4
An arithmetic mean gives -.5, Trump won by about a percent.

Take Pennsylvania [3], another close state. November polls again

    +1, +2, +4, +6, +3, -2, -2, +0, +0, +2, +2

An arithmetic mean gives Clinton + 1.18, Trump won by about .7, so this was perhaps the greatest polling blunder, and its still a margin of under 2%, which isn't unheard of.

[0]: https://projects.fivethirtyeight.com/2016-election-forecast/ [1]: https://projects.fivethirtyeight.com/2016-election-forecast/... [2]: https://projects.fivethirtyeight.com/2016-election-forecast/... [3]: https://www.nytimes.com/elections/results/pennsylvania [4]: https://archive.fo/4EQGB

I stand corrected, it was 71.4% The 90% from NYT stood out in my mind.
I see what you're saying now.

But I don't think you got my point either. The detail doesn't matter. Whatever that number was about, it was pretty clear that 90% was not the right number, I'm not even pro-trump and I was confused, imagine what others thought.

Also, it is not ok to be irresponsible just because you're using a "scientific method". This is what cigarette companies did to fund research that "prove" that smoking is healthy. And this happens increasingly more, because writing polarizing articles generate more shares and page views and money.

Lastly, I am not trying to talk politics here, and it's not even just about NYT. Nowadays for a lot of writers it's all about generating more page views. Again, I'm fine with it (even the "fake news" viral sites), but I was pointing out that it is hypocritical to claim that you're "the truth", when you do things that are biased.

> Whatever that number was about, it was pretty clear that 90% was not the right number

Was it? Maybe 90% didn't match up with your intuition, but NYT wasn't making that number up out of whole cloth. That's what the polls were saying, as analyzed by the NYT polling staff.

Maybe they were wrong (maybe they weren't-- things that have a 10% chance of happening happen all the time), but you certainly haven't demonstrated, or even really attempted to demonstrate, bias. All you've really said is:

* Trump won

* Your intuition told you he had a greater than 10% chance of winning before election day

> throughout the entire election cycle the narrative was NYT vs. Trump. News organizations and journalists should report stories but should not become the story.

It's important to remember that this narrative largely came (and continues to come) from Trump, and he benefited greatly from it.

You seem very confident that "90% was not the right number", and I'm curious how you reached that conclusion. Do you have research that indicates the odds were actually way different, or does it just that it feel incorrect?

If I win the lottery, is it misleading for people to state that my odds of winning were one in a million?

Dude, that guy who was supposed to lose by 90% chance actually DID win. And you're telling me to prove this by showing you some research result?

It's not misleading to say your odds of winning the lottery is one in a million because it IS one in a million, mathematically. NYT saying Hillary would win by 90% chance is NOT mathematics. Do you really think your analogy was adequate?

My point was that people believe statistics, just like you believe that because NYT did it, it would be scientific and you should believe it. But that's where the danger lies. And an influential publication such as NYT should be ashamed of themselves for misleading people by trivializing things down to just a percentage and feeding it to the world, in realtime.

I'm sure a lot of Trump fans who saw that poll on NYT that morning who didn't care for voting decided to go vote.

> Dude, that guy who was supposed to lose by 90% chance actually DID win.

Again, that's how probabilities work. Unless the prediction was 100% Hillary win then a Trump win is possible. This isn't a new concept, it happens in sports all the time. Odds are offered for betting, and every now and then there are upsets. That doesn't mean the odds are nonsense.

What's the alternative? News organisations never even attempt to gauge whose campaign is more popular? I don't think that's realistic.

> Dude, that guy who was supposed to lose by 90% chance actually DID win. And you're telling me to prove this by showing you some research result?

Actually yes, that is exactly what I'd like to find out more about. I'm asking why you are so sure that the NYT and other media publication odds were so wrong? Is it just that he won, therefore his odds must have been better than 10%? Most betting markets (which the HN set seems to put a lot of faith in) had Clinton as a 80ish % favorite (http://www.cnbc.com/2016/11/07/betting-sites-see-record-wage...), do you think they were a product of a misled public?

I don't recall the New York Times reporting a 90% chance of victory on election day. I recall her chances according to the Times decreasing several percentage points (down to the high 70s IIRC) during the last few days leading up to the election. If the interactive graphic you are talking about was the 'paths to victory' graphic that I'm thinking of, it probably reflected the most current polling data - which was probably already obsolete - that they had on hand at the moment.

My own observation during the days leading up to the election was that her chances of winning were decreasing at a faster rate than could be reported with reliable polling data, though I still believed that she'd eke out a win.

> Because it was pretty clear the race was not that obvious.

I think that's where the confusion lies. The majority of published polls and pundit calculations pointed that way, and it was earnestly interpreted by well-intentioned journalists that 90% of polls meant 90% probability. Perhaps I'm cutting a little too deep with Hanlon's razor here, but I think systemic biases are a more likely culprit than some kind of media conspiracy.

> Everyone was wondering (both pro-trump and anti-trump) where the "90%" came from

No, they weren't, because everyone paying even a little bit of attention to the issues around aggregating polling numbers new that 538 had already dissected the problem with that long before the election—most prediction models were falsely assuming that variations in outcome from.the poll averages between different states were independent events when history shows that such deviations are strongly correlated. Which is why 538, working with essentially the same polling data as everyone else, had Clinton's chance of losing much higher than NYT and some other predictors.

> To me, THAT's the ultimate "fake news".

Predictions of the future are (and were presented as) analysis, not news. To be "fake news", it has to firsr pretend to be news, that is, reports of things that have occurred.

You are completely correct, the people quibbling with you about statistics have not been paying attention to what is going on and are hopelessly naive about human nature and psychology. They are demonstrating exactly the bias thinking that caused the election results to be projected the way they were.

It wasn't just the New York Times, it was every single news outlet and every single poll. We have been running and studying elections for over 100 years, you would think, by now, that if we had any idea what we were doing, we would have long ago learned how to perform actual predictions using the data available to us.

When people break their legs, we can cure them 100% of the time because medicine has a strong answer for this problem based in thousands of years of medical science. Election polling and prediction is simply not in the category of "solved" problems.

If the statistics were really well understood...as some of the people arguing with you are claiming...WOULDN'T WE HAVE FIGURED THIS OUT BY NOW?

I have had these discussions about polls and statistics with my Liberal friends. Many of them are software engineers. Some of them even work in ML at large scale. They are utterly convinced that the data and statistics just need to be tweaked a little bit more and THEN they will be correct. To quote Donald Trump: WRONG!

There is a simple explanation for what happened that does not get into nit-picking statistical nuances. There is a simple explanation that does not allow people on the other side of the argument to bludgeon less educated opponents into silence by spewing technical jargon about polling data (which is really just a manipulation technique used to silence people by berating their intelligence).

The explanation is that the polling data was simply made up.

Or that many prediction models were created and only the ones favorable to the desired narrative were selected as being "Accurate" by people who are blinded by deep personal political bias.

This outcome scares people who prefer to see the world in a particular way.

People want to believe they aren't biased or that everything they read and see everyday is somehow "clean" or being presented by rational experts. That is a scary thought process...software engineers, in particular, cannot bear to believe that because it violates their underlying belief that the world operates in a logical way.

So instead of thinking like this, they crack open statistics textbooks and start reading chapter 4 at you. Its a defense mechanism to avoid reality.

There are direct financial rewards for telling people exactly what they want to hear. If Nate Silver and 538 consistently told people that Donald Trump would win, it is likely that their page-views would plummet into oblivion. Instead, the vast online hoards who like to be told whatever they want to hear and read news only to have their existing beliefs confirmed swarmed over Nate's websites, clicking on articles and gleefully watching the ads.

What is Nate Silver's punishment for being wrong over and over again? Nothing! There is no identifiable drop-off in his web traffic despite him and his team being wrong about everything this last election.

If what Nate Silver was doing was really "data journalism," shouldn't his following online have dropped off a cliff? It hasn't and it won't because what 538 is doing is not really backed by data and his readers and viewers are not discerning enough to care or believe that what they are reading is just opinion.

I am fairly disgusted by humanity at this point as a result.

Keep strong my brother, you are right. The people quibbling with you are living in an alternate reality where the media isn't incentivized by page views to gravitate towards narratives that result in more clicks. The media is dead.

> The explanation is that the polling data was simply made up.

You don't feel like you are oversimplifying a bit, and applying your own bias? Any pollster knows (and I realize you may ignore this point due to that opening bit) that there are margins for error as long as your sample size is any smaller than 100% of the population. Further, any pollster knows that your data will be skewed by how a question is worded.

The data wasn't made up, and each outcome was within the predicted margin for error. Many people will accuse the statistician of using large margins for error to cover any issues in the data, but that is straight from the opposite end of the spectrum of error.

> People want to believe they aren't biased or that everything they read and see everyday is somehow "clean" or being presented by rational experts. That is a scary thought process...

No one believes that everything they read is clean; anyone who does has (again) oversimplified the world. It isn't black and white. Every day, the margin for error is being examined, polls are being refined, questions are being tweaked. If the pollsters thought they had it all nailed down, why would they even be re-evaluating the method?

The only way to get 100% accurate data would be to poll 100% of the people who are going to vote -- and 0% of the people who aren't. Oddly, that is what happens on voting day -- at great expense, through great manpower, and with great effort. Pollsters simply don't have those kinds of resources.

> There are direct financial rewards for telling people exactly what they want to hear.

People do love to be reinforced, but again you are drastically oversimplifying a more complex issue. No matter what happens, no matter the outcome, people want to read the polls.

I mean, why do we even have them otherwise? So regardless of the error bars or margins, polls are going to keep happening; some will be correct, some won't. Most will fall within fairly narrow margins for error, but often the <50%> marker is in that margin -- so the opposite outcome could happen, even within the poll's own admitted rules.

> So instead of thinking like this, they crack open statistics textbooks and start reading chapter 4 at you. Its a defense mechanism to avoid reality.

Unnecessarily inflammatory, I think. The worst part about this rant is that the polls were so close to accurate -- especially if you count the popular vote, instead of the electoral college outcome. It all seems to be a symptom of a country that is, perhaps, too large and diverse for a federal government. I would argue for States' rights here, or say States should be more largely self governing -- but you do need a mechanism to get money from California (taxed $405B, received $343B in federal expenditures) to Kansas (taxed $9.9B, received $24.2B).

That was a bit of a tangent; the point is that the setup of the United States government, both from the State side and Federal side could use some work.

> What is Nate Silver's punishment for being wrong over and over again?

And what is his reward for how often he is right? He works in an industry with a large margin for error. Are you arguing that statistics should basically be abolished? Fivethirtyeight does far more than political polling, mind.

The other thing is relative; Nate Silver was far closer to the mark than any other publication, and his data were more robust. If you are arguing that Nate should lose traffic, every other organization (including those that argued for a Trump win based on similarly faulty statistics) should receive even more loss?

> If what Nate Silver was doing was really "data journalism," shouldn't his following online have dropped off a cliff?

But he was working with the data he had. Fivethirtyeight doesn't even do its own polls; it aggregates data from other polling organizations. Should Nate have made up the data? That is the exact opposite of what you are arguing, and I think your anger is m...

The polls were off, and/or the race moved in last few days. If you can't see the difference between a statistical model's prediction and a journalist's factual claims, you're unlikely to find the sort of "objectivity" you're looking for.

And, fwiw, your sort of political "objectivity" just doesn't exist. What you want is the sort of fake equivalence as in "Yesterday, 10 people died in an earthquake in Northern Italy, according to local accounts. But Jeff from Kansas writes on Twitter that there was no earthquake and the cause of death was the LHC's creation of a black hole. What's the truth? You decide! Back after this...".

Just because something doesn't exist, doesn't mean you shouldn't strive for it, especially if you're in journalism.

It's like saying "Nobody is perfectly ethical. So trying to be ethical is just being fake."

Also you are using a funny example to make a point that's not relevant to this instance. That has nothing to do with news organizations siding with certain political party and pretending to be unbiased.

"For the past three years, my assignment has been to try to help this newspaper live up to its own high journalistic standards as it covered a historic presidential election, two wars, the Great Recession, violence in the Middle East and more. I have deplored the overuse of anonymous sources, warned against the creep of opinion into news analysis and worried about the preservation of Times quality on the Internet. But, in truth, I have sometimes felt less like a keeper of the flame and more like an internal affairs cop."

- Departing NYT Public Editor Clark Hoyt

SOURCE: https://mobile.nytimes.com/2010/06/13/opinion/13pubed.html?r...

>“There is nothing more important to our mission, or our business, than strengthening our connection with our readers,” Sulzberger added. “A relationship that fundamental cannot be outsourced to a single intermediary.”

This is a disturbing statement. Shouldn't a serious newspaper's mission be to investigate and report facts without regard to the "connection" it creates or dissolves with the reader? I'm sure Infowars has a deep connection with its readers.

This is a profoundly ridiculous statement but I think it represents what the NYT has always been quite accurately.