57 comments

[ 4.6 ms ] story [ 141 ms ] thread
Whatever "news" you get is going to suffer from both conscious and unconscious bias on the part of the reporter(s), editors, et al.

You can only accept information from others as "someone wanted to communicate this to me" and speculate on their real reasons for doing so.

We just have to do the best we can with the limited, imperfect information available to us. Our very senses lie to us.

If your notion of rationality requires there to be Perfect Truth then you're SOL right at the start.

While I agree, we really do THINK in terms of absolute truth, making this type of communication more powerful than it ought be. My observation: We will unconciously take known bs as fact when we see it, unless we reject it soon upon perception and react negatively to it. Our bs detector sucks, and is often stymed by our ocassional mental laziness/exhaustion, and becomes less critical with every repetition until called out and reframed. I observe with myself and others, that first learned bias sticks until challenged. Add the in-group/tribal/social/political/identity bias to that and it becomes even stronger.
> we really do THINK in terms of absolute truth

Who does? Not everybody, everywhere, certainly.

Holding fixed abstract ideas in mind for long periods of time, with conviction and a drive to keep them all consistent is a very peculiar, intellectual way of seeing the world, mostly only useful for debate team members, monks, academics, and political pundits. Especially for the countless political and metaphysical ideas that have no relevance to daily actions.

Most people don’t need to do that, and many who do it now have learned to perform it because it’s come to seem normal or even celebrated. But even a lot of them are terrible at it, hanging onto all kinds of sloppy beliefs and inconsistencies.

By default, most people just do shit and say shit and carry on with their day. Their natural BS detector looks like “God, the news program is awful. Turn it off.” and their natural political orientation is whatever makes for the most satisfying conversation with their neighbors.

What you might want to do, instead of all the complexity you’re trying to navigate and systematize, is to just unplug a bit and busy your hands instead of your brain.

Oh, it gets worse.

One of our biases is towards conservation of mental energy. If we can, we choose the easiest path. Which means that we actively prefer experts who are smart, well-informed, and certain of their beliefs. Because we can then be comfortable handing our thinking over to them. Why bother questioning the expert when we'd surely agree if we just knew enough? And then confirmation bias keeps us from ever questioning their results.

The result, as https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... documents, is that our most popular and highest paid pundits have those characteristics. But those experts are ALSO the ones who are worst at making predictions about the world. As I pointed out in https://www.lesswrong.com/posts/TMgfapzbt5qp4Hszf/doubt-cert..., we should really be concluding "likely has cognitive bias" instead of "the expert says it, so case closed".

The documented result is that we wind up trusting the experts that we should trust least!

"Likely cognitive bias" is not a conclusion, it's an invitation to do more research. The catch is that the amount of research you can do is extremely limited compared to what would be needed to verify the facts you hear, so you can't actually escape the need to trust and manage your trust relationships. An incredibly common mistake is to reject a decent but imperfect source in favor of a bad source, and for this reason I consider "trust less" actively harmful.

Trust A over B when C (subject to real time limits on computing C) = ok

Trust Less = harmful

It is both an invitation and a conclusion.

As you note, we need to decide who to trust, and we can't do sufficient research. So we need heuristics.

The heuristics that I recommend are these:

1. Certainty, especially if there is debate among experts, indicates likely cognitive bias.

2. The ability to discuss how multiple perspectives would lead different people to different conclusions, suggests good reasoning.

3. A documented history of quantitative reasoning, along with data showing historical accuracy, is the best evidence of a trustworthy source.

I base those heuristics on the research in https://www.amazon.com/Expert-Political-Judgment-Good-Know/d.... Applying this heuristic to "sounds certain" results in a conclusion, that is an invitation to further research that may or may not happen.

Did you (or to what degree) take epistemic soundness and logic into consideration during your analysis, or are you only coming at it from Bayesian perspective?

To what degree did you take the degree of rationality you've achieved into consideration (Rationalists aspire to rationality, though they often forget that) during your analysis?

There's a lot more going on than bias. There's laziness on the part of the reporter, and incompetence in the field they are reporting on. For example, whenever I read a journalist's article about something technical, like airplanes, they screw it all up, simply because they have no idea how airplanes work and how they are designed.
"News" companies fired all the journalists and replaced them with a quarter as many "writers" instead. Even when these new people have the training and qualifications to actually be a journalist, that's not their job anymore. For the most part, news corporations literally cannot afford to do journalism like they used to, because internet advertising crashed the price of ads in journalism sources, and everyone decided that seeing one inaccurate story meant the entire system should be thrown away and replaced with pundits slinging ass-pulled takes and lines to vaguely apply to whatever happened today, to an audience happy to never self reflect about all the things the pundit gets wrong.

One group cannot speak on all things, they just cannot have all the required knowledge, experience, and practice to accurately judge more than a few corners of reality. The modern jack of all trades can only really have a surface level understanding of most things.

I'm not sure modern journalists are any better than the journalists of yore. It was easier for the yore journalists to get away with bias and incompetence, because there wasn't an alternative source of news.
>Critically, if we are persuaded by either camp, we will find most of the sources in that camp believable. And most in the other camp to be not believable.

You don't believe one camp or the other. You look at how each camp reports on the same event. You use them against each other to get closer to the truth, like a trial where both sides argue their case and you are the judge.

This still falls prey to the most insidious type of lying: lying by omission.

So many times I've seen primary sources as reported by a user posting a video on Twitter, and later I'll read news coverage of that same type (from either side), and they'll omit essential pieces of information that don't fit their narrative, or include irrelevant information that pushes their narrative. The latter category is often an attempt to frame your thinking or make you think a certain way about a target.

In your example you looked at sources from two different sides of the debate and correctly identify the omissions and irrelevant information.

To me that says that looking at the two different sides and judging their arguments to arrive at the truth works.

A final test would be to see if your arrived truth is close to reality, without relying on human observers. This is done by using your truth to make predictions and see if those predictions come tru. To see if your model of reality has predictive power.

I agree, though "works" and "truth" sound a bit optimistic to me.

Let's say you'll probably improve on the accuracy of your picture, if you take more different sides of the story into account.

You're right. I'm abusing the word "truth" and "works". But you get a clearer picture and avoid the most obvious attempts to mislead.
This only works in the case where both camps are acting/reporting in good faith AND you're capable of identifying bad faith actions/reports. Bad faith actors want you to think there's two genuine sides to issues so they can pull the Overton window further to their side. They don't care about truth or objective reality but "winning".
I really don't think this has helped me try to understand anything to do with the Palestine-Israel conflict.

For example: a hospital in gaza is bombed by israel forces. Hamas claims X number of civilian deaths and says nothing about terrorists existing in the building. IDF doesn't claim any number of civilian deaths but does say that, generally, terrorists hide in these hospitals. No one confirms basic facts like who are the people being targeted, were they actually killed, and how many innocent civilians were calculated as an acceptable loss to destroy the targets.

Overall I feel I'm left with even less information than when I started, since all either side can confirm is that Israel bombed a hospital. And this happens for a bunch of things: Israel bombs a neighborhood the UN has labeled a refugee camp. Hamas commits an act of terror with some number of hostages taken.

I cannot even confirm if the food, water, and fuel being sent humanitarian-ly to Gaza is enough for the population there. One side claims there is a huge crisis of resources, the other side claims there's no crisis whatsoever.

When there is a war there is no credible information.

Lying is not only seen as morally justified but as your duty because your life and your compatriot's lives are on the line every second of every day of the conflict.

The issue of rationality in analysis is it is forced to take on the base assumptions of the thing it's analyzing. If you start from false premises you will reach bad conclusions. This is the difference between rationality and reason.

Media literacy, in particular a rather (dare I say) postmodern approach to understanding the process of narrative construction, helps a whole lot to see through the ideological underpinnings. It actually helps quite a bit to triangulate an approximation of the truth by understanding not only the partisan bias in interpreting facts but also what different outlets might choose to include or not in their stories in order to further ideological arguments as well as how they might try to think for the reader to lead them from bare facts to interpretations that feel like facts.

It should be noted that if you ask most people whether foreign news contains propaganda, they'll say it almost certainly does, but if you ask them whether our news contains propaganda, they'll say it's pretty unlikely, minor, or they can easily spot and dismiss it.
what makes it even more murky is that misleading/wrong information can be either from propaganda or bias. I wonder if the polarized major news channels are simply doing so because of market share, and pushing the line towards misinformation/misleading simply as rage-bait to keep their views' allegiance. Whereas when I constantly hear FUD about $BAD_COUNTRY I have to assume thats likely propaganda coming from the top.
Doesn't match my observations. I often hear complaints about "mainstream media", and some even consider the fact that a news source is "mainstream" sufficient evidence to dismiss all its claims.
I would be willing to bet that a large majority of Americans would say either that CNN/MSNBC are propaganda or Fox News is propaganda and that a sizeable minority would say both are.
A disadvantage of attaining hegemony is that lacking unifying external enemies, humans tend to squabble into internal camps. See the Diadochi after Alexander, Blues & Greens for the Byzantines, Guelphs & Ghibellines during the HRE, etc.
As an outsider, I would not characterize the current divisions in the US as a mere "squabble into internal camps".
The swiss Sonderbund war (an 1847 declared protestant/catholic civil war), with ~100 deaths, was resolved much less[0] bloodily than the 1861 US civil war (hundreds of thousands dead). I was rating "squabble" because unless I am miscounting, the total number of partisan deaths directly[1] attributable to current US divisions are still only ~10.

[0] come to think of it, less even than the "Alamo": like the US civil war, also in defense of slavery.

[1] NB possible "stochastic terrorism" not included; excess deaths due to fear of vaccination definitely not included.

> NB possible "stochastic terrorism" not included; excess deaths due to fear of vaccination definitely not included.

Yeah, even counting official domestic terrorism deaths we're into much larger numbers.

But I get your perspective ... I guess I'm worried about the direction.

I enjoyed this blog post but, broadly, conversation on the topic of news epistemology has always irked me because i detect an implication that there exists an "unbiased" recounting of events. This is naive.
Just the Facts as reported by Joe Friday (is a show I wish existed and I would watch)
The PBS Newshour does a pretty good job of that, but it’s not sensationalistic, so most people won’t watch it. It’s dry and kind of boring.
dry and boring can really be a good thing, though.
> conversation on the topic of news epistemology has always irked me because i detect an implication that there exists an "unbiased" recounting of events. This is naive.

I would say, rather, that there is an ideal unbiased recounting of events that is approached with various degrees of fidelity and we all sort of agree to call reporting that is reasonably close to this ideal as "unbiased".

This is not naïveté, since pretty much everyone readily admits that truly unbiased reporting is impossible. But some people are doing their best and some ... are not.

While I believe that there is an unbiased truth, my conclusion is that it is not necessarily something that I can discover.

I do not believe that those in the news are in a significantly better position to discover it either.

This strongly suggests that we should assume that all recountings are biased. It is just sometimes easier to find the bias than it is at other times.

It is less so that it is impossible than that our culture utterly rejects such approaches.
The problem with this analysis is that it assumes that all inputs from partisan news sources are providing a consistent set of observations with differing levels of accuracy. This is demonstrably not the case. Partisan news[1] typically acts more like an extremely lossy compression algorithm that throws away information that doesn't already conform to a predetermined pattern. That information is not retained as "low accuracy", it simply doesn't exist in their reporting. Many people still consider these types of sources as fairly reliable today. On occasion (or perhaps disturbingly often) extremely partisan news sources insert (i.e. repeat, amplify) observations that could be thought to have even a negative accuracy (falsehoods) [2].

The difference in conclusions is thus: suppose we had a computing device that could make the NP complete step (#3 in the post) P instead. It would mean that in such a scenario we could rationally analyze partisan news.

What I'm saying is that even given such a device, we could not. Given [1] above we would also need a device that could generate all possible observations and accuracies to attempt to fill in the missing information - which requires infinite computation. But [2] makes it even more intractable, where we have to not only compute all possible observations, but also consider all scenarios where there is no observation to report (with it's own set of accuracies) -- an even larger infinity.

The article's supposition can be resolved simply by observing enough partisan news (reported from every partisan angle) that you can put error bars on the reported information. Reality is such that you could observe all partisan news and still not know what's going on because there would simply be information gaps that would be irretrievable without infinite computation.

I explicitly make no such assumptions. And the fact that I didn't, really matters here.

While I assume that you get true (though filtered) statements from some on each side, I'm also assuming that there are unreliable (read dishonest) sources as well. And each side is making the case that the other side can't be trusted. Which therefore means that we should expect (as actually happens) to be getting inconsistent information from different sources.

I didn't walk through it in detail, but that is the source of the problem. I start out convinced by one side. Then I observe someone I trust confirming someone I don't trust. Now I wind up trusting someone on the other side more. But as I iterate through my cascade of belief updates, this creates a feedback loop. Strengthening my belief that that person is trustworthy, strengthens my belief in other things that they claimed. Which causes me to trust others on that side more, and trust "my" side less. Which causes me to reevaluate more statements, and so on.

The critical question is, "Does this feedback loop die down, or does it continue until I switch sides?" That's the numerically intractable problem which computer scientists have proven to be NP-hard. And being forced to choose between two networks of conflicting beliefs is exactly the situation which causes the biggest problems.

Moving on, I agree that being able to evaluate the Bayesian update would not make this computable. I offered a straightforward example - unproven statements of mathematics, encoded as statements about economics. But your current line of argument is exactly the argument of the post that I was responding to.

I don't really disagree with the concept in the post, other than to say that it's always assuming that there's some information (observations) coming in on any given topic. The nuance I'm arguing for is that truly partisan news is often as much a practice of complete omission than inaccurately reported observations -- something that's....umm....omitted here, thus providing a salient example. This results in an over focus on belief networks and switching beliefs, when the result is more complex.

Many people hold no particular beliefs whatsoever on most topics. When pressed they'll generate a "belief proxy" at random until the partisan sources their mental models are fed by gives them an observation to hold to. It's perhaps not mathematical, but it's highly observable -- particularly on topics that are likely to be highly partisan.

I'm focused on what we do with information we receive.

I have no idea what one is supposed to do about unknown information which was not received.

As Donald Rumsfeld famously said when working his way through the permutations in a confusion matrix:

"Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones."

Probably more actionable, https://www.amazon.com/Psychology-Intelligence-Analysis-Rich...

It's like survivorship bias. You only see the things that aren't targeted for omission.

Sometimes you can tell a lot by what a given source leaves out about a given story. You find what's "allowed into the narrative" and how that evolves over time. Can be more interesting than the story itself.

It seems like you should be able to extract statements of fact from competing sources, say Fox and CNN. The overlap of facts would likely be true enough. The distinct sets then become color as to why those facts were included/excluded and the bias is then it's own set of news.
It is possible to apply some filtration to one's choices of news sources. Do they publish and enforce a code of journalistic ethics? Have they ever publicly disciplined a reporter for making things up or otherwise violating that code? Do they prominently and frequently publish corrections, either "in-band" or on their Web page? Do they refuse to provide a platform of respectability to people spreading [md]isformation?

None of these are guarantees of trustworthiness, but a lack of them is in my experience a warning signal.

I feel that reporting which is done in a neutral tone tends to correlate well with organizations and individuals who are interested in factually true reporting. Also, people who are concerned about the truth will tend to use more weasel words, such as “appears to” or “alleges”, because those words leave open the possibility of being wrong.
Alas, it seems that in practice humans are often motivated by something "short, quippy, and wrong" [0] over something with more details like demarcating areas of ambiguity.

So the "better" sources have an uphill fight to be influential.

[0] https://www.youtube.com/watch?v=wmVkJvieaOA&list=PLJA_jUddXv...

(comment deleted)
I am fond of the "STASM" analytical framework given in: https://www.gutenberg.org/files/48612/48612-h/48612-h.htm#Pa...

> The point will invariably arise: "This tells me how to listen to a foreign radio. Okay, I'll get the news, the lectures, the plays—all the rest of it. But so what? How am I going to know what's the truth and what's propaganda? How can I tell 'em apart? Tell me that!"

> The answer is simple: "If you agree with it, it's truth. If you don't agree, it's propaganda. Pretend that it is all propaganda. See what happens on your analysis reports."

(If I understand TFA, Linebarger's proposed 1948 sol'n seems to lie in not updating beliefs, or at least not so aggressively?)

I've never seen that one.

I would say that his solution is to analyze it twice, once from your current view, one from a cynical one. Look at where you get changes. Those are places to look for possible errors in your current thinking.

I'll have to try that out and see what I think of it.

It's a well-known adage that a lie by omission is still a lie. Complete coverage of a topic or event is not just a positive analysis of factual claims, but also a negative analysis of facts omitted. This is an incredibly fuzzy realm where it is actually impossible to know if a story is truly complete. Thus, the entire pursuit of "objective journalism" to me seems a folly.

I much prefer that partisan actors pursue their incentives to include competing details, from which I can synthesize a Bayesian analysis of what is most likely to have happened. Often, the attempt to slander partisan coverage of events is done to cover up for partisan reporting masquerading as "objective" while omitting or massaging crucial details that only the slandered opposition will point out.

Linebarger's Psychological Warfare (1954) suggests making substantial use —in 'white' propaganda, at least— of omission:

> The Japanese who obediently hated the Americans when it was their duty to do so nevertheless could not help looking at maps that showed where the Americans actually were. Nazis who despised us and everything we stood for nevertheless studied the photographs of our new light bombers. The appeal of credible fact is universal; propaganda does not consist of doctoring the fact with moralistic blather, but of selecting that fact which is correct, interesting, and bad for the enemy to know.

https://www.gutenberg.org/files/48612/48612-h/48612-h.htm#Pa...

(unfortunately he seems to hold the wartime propagandist to higher standards than we currently see in peacetime "journalism"?)

Edit:

> Almost all good propaganda—no matter what kind—is true. It uses truth selectively.

We all want to be lied to. The alternative, watching CSPAN live-stream congress or the senate is so boring that you can even see the senators or congress people sleeping in the chambers!

We have the tools to find the truth, at least for many parts of our government. If you go look at CSPAN programming which allows the public to comment, or watch "debates", you'll notice a lot of empty chambers and very old people with nothing better to do being the only participants.

This is why we turn to partisan news. Shame on us.

> We have the tools to find the truth, at least for many parts of our government. If you go look at CSPAN programming which allows the public to comment, or watch "debates", you'll notice a lot of empty chambers and very old people with nothing better to do being the only participants.

> This is why we turn to partisan news. Shame on us.

You're omitting an important point to be a scold: "finding the truth" by combing through raw CSPAN feeds and primary documents takes such an extreme amount of time and effort, that literally no one has the ability to do it. Think you can? Watch a TV like this https://www.youtube.com/watch?v=cFwPFQXQHNk, except with 1000 channels instead of six, all playing programs you have to closely attend to. The "very old people with nothing better to do" are some of the only ones with the time to even dabble, and even they fail.

The best you can do is figure out a tiny, tiny little part, and most people don't even have the time for even that. So people divide up the work and rely on proxies, and others realize that exploiting those proxies can bring power.

It's an impossible problem. Shame on no none.

I don’t think I quite understand what a belief is or what is being updated in this context.

If I have lots of beliefs, like every fact in a news story is a belief, then even an O(n) updating process seems overwhelming.

If my beliefs are just big things, like “people in states governed by political party X tend to have better outcomes by the metrics I care about,” then I think I only have like a handful of beliefs and I don’t really care about the big-O cost of updating them.

The most valuable thing I ever read about trust was this gem by a bank risk policy consultant:

One Minute MBA – Avoiding Projects Pursued By Morons 101 https://dsquareddigest.wordpress.com/2004/05/27/108573518762...

Fibbers’ forecasts are worthless. Case after miserable case after bloody case we went through, I tell you, all of which had this moral. Not only that people who want a project will tend to make innacurate projections about the possible outcomes of that project, but about the futility of attempts to “shade” downward a fundamentally dishonest set of predictions. If you have doubts about the integrity of a forecaster, you can’t use their forecasts at all.