78 comments

[ 2.8 ms ] story [ 85.9 ms ] thread
This is a good blog post. Two thoughts about it:

- Contradictory facts often shouldn't change beliefs because it is extremely rare for a single fact in isolation to undermine a belief. If you believe in climate change and encounter a situation where a group of scientists were proven to have falsified data in a paper on climate change, it really isn't enough information to change your belief in climate change, because the evidence of climate change is much larger than any single paper. It's only really after reviewing a lot of facts on both sides of an issue that you can really know enough to change your belief about something.

- The facts we're exposed to today are often extremely unrepresentative of the larger body of relevant facts. Say what you want about the previous era of corporate controlled news media, at least the journalists in that era tried to present the relevant facts to the viewer. The facts you are exposed to today are usually decided by an algorithm that is trying to optimize for engagement. And the people creating the content ("facts") that you see are usually extremely motivated/biased participants. There is zero effort by the algorithms or the content creators to present a reasonably representative set of facts on both sides of an issue

The idea that people believe in climate change (or evolution) is odd considering people don't say they believe in General Relativity or atomic theory of chemistry. They just accept those as the best explanations for the evidence we have. But because climate change and evolution run counter to some people's values (often religious but also financially motivated), they get called beliefs.
The best way to lie is not presenting false facts, it's curating facts to suit your narrative. It's also often that you accidentally lie to yourself or others in this way. See a great many news stories.
> If you believe in climate change and encounter a situation where a group of scientists were proven to have falsified data in a paper on climate change, it really isn't enough information to change your belief in climate change, because the evidence of climate change is much larger than any single paper.

Although your wider point is sound that specific example should undermine your belief quite significantly if you're a rational person.

1. It's a group of scientists and their work was reviewed, so they are probably all dishonest.

2. They did it because they expected it to work.

3. If they expected it to work it's likely that they did it before and got away with it, or saw others getting away with it, or both.

4. If there's a culture of people falsifying data and getting away with it, that means there's very likely to be more than one paper with falsified data. Possibly many such papers. After all, the authors have probably authored papers previously and those are all now in doubt too, even if fraud can't be trivially proven in every case.

5. Scientists often take data found in papers at face value. That's why so many claims are only found to not replicate years or decades after they were published. Scientists also build on each other's data. Therefore, there are likely to not only be undetected fraudulent papers, but also many papers that aren't directly fraudulent but build on them without the problem being detected.

6. Therefore, it's likely the evidence base is not as robust as previously believed.

7. Therefore, your belief in the likelihood of their claims being true should be lowered.

In reality how much you should update your belief will depend on things like how the fraud was discovered, whether there were any penalties, and whether the scientists showed contrition. If the fraud was discovered by people outside of the field, nothing happened to the miscreants and the scientists didn't care that they got caught, the amount you should update your belief should be much larger than if they were swiftly detected by robust systems, punished severely and showed genuine regret afterwards.

> Say what you want about the previous era of corporate controlled news media, at least the journalists in that era tried to present the relevant facts to the viewer.

If you think this reduced bias, you couldn't be more wrong - it only made the bias harder to debunk. Deciding which facts are "relevant" is one easy way to bias reporting, but the much easier, much more effective way is deciding which stories are "relevant". Journalists have their own convictions and causes, motivating which incidents they cast as isolated and random, and get buried in the news, and which are part of a wider trend, a "conversation that we as a nation must have", etc., getting front-page treatment.

A typical example: And third, the failure of its findings to attract much notice, at least so far, suggests that scholars, medical institutions and members of the media are applying double standards to such studies. - https://www.economist.com/united-states/2024/10/27/the-data-... (unpaywalled: https://archive.md/Mwjb4)

> the previous era of corporate controlled news media... The facts you are exposed to today are usually decided by an algorithm

... But that algorithm is still corporate controlled.

See also: the Chinese robber fallacy.

Even if only 0.1% of Chinese people engaged in theft, and that would be a much lower rate than in any developed country, you'd still get a million Chinese thieves. You could show a new one every day, bombarding people with images and news reports of how untrustworthy Chinese people are. The news reports themselves wouldn't even be misinformation, as all the people shown would actually be guilty of the crimes they were accused of. Nevertheless, people would draw the wrong conclusion.

> If we want to counter manipulation and polarization, we need to focus on strengthening the structural integrity and resilience of our own belief systems. This means fostering internal coherence, building bridges between different templates, and cultivating narratives that are not just factually accurate, but also emotionally compelling and structurally robust.

I fear we are already increasingly too late on much of these things because there also exists communities that maintain structural integrity by resisting bridge building between different cultural templates. I.E. You must refuse bridge building in order to maintain your own community, such as wizard, blackpill, or MGTOW actively discouraging bonding with women as equal people to men.

For some reason the article seems to really like the — symbol, even as far as replacing most of it's commas with it
[flagged]
"Don't be snarky."

"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."

"When disagreeing, please reply to the argument instead of calling names."

https://news.ycombinator.com/newsguidelines.html

"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."

I don't think there's anything more curmudgeonly or generic than just pasting rules.

Crackpots don't deserve thoughtful replies and practically they cannot be given thoughtful replies. Thoughtfully replying to every crackpot would absorb all the non crackpot resources.

And before it's suggested that "calling someone a crackpot is an ad-hominem" or the more advanced "who is the arbiter of what is a crackpot? Wouldn't that just let you dismiss any argument you don't like by saying it's made by a crackpot?"

This theory has sufficient characteristics of a crackpot theory:

  Immune to falsification
    There's no actual theory here
  Exhibiting specific rhetorical markers 
    Central position of Galileo or Einstein 
    Dense jargon that mimics academic language but lacks coherent meaning
      "When belief systems interact, the real contest isn’t just about exchanging arguments—it’s about trying to reshape each other’s underlying structures. Each side looks for ways to weaken, sever, or absorb the connections that make the other system stable."
        What does it mean for two belief systems to interact?
        What does reshaping the other sides structure mean?
        How do you absorb connections and how is that an advantage?  What does it even mean?
        The notion of stability is vague and undefined, but does all the heavy lifting in this argument.  This shows a complete lack of even familiarity with how actual philosophical and academic writing is conducted or how arguments such as this must be constructed.
  Characterized by specific behavioral patterns
    Claiming fundamental insights in a field where they lack sufficient training to have mastered the basics
    Bypassing peer review to appeal directly to media or public 
    Misunderstanding or misrepresenting fundamental facts    
      example: "The Church’s liturgical calendar—including the calculation of Easter—relied on the apparent movements of the Sun and Moon around a stationary Earth."
I certainly agree that it's generic (and therefore boring). Such mod comments are an out-of-band side channel—not at all the intended use of the site, necessary for it to function (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...). (I can't see what's curmudgeonly about it though)

Even if you're right about the OP's theory, your post was obviously against HN's rules. It would be better to take responsibility for that instead of pointing the finger at someone else. Even assuming you're right about their ideas, you damage the community here by posting snarky putdowns. Most readers won't notice the subtle difference in this snarky putdown; they'll just take it as another indication that it's ok to be snarky and aggressive here, which it's not.

p.s. I was going to add something about your account history, but looking at more recent comments, they seem to be much more in keeping with HN's intended spirit than I remember from the past. It might be that my memory is tricking me, but if not, thanks for that—it is greatly appreciated!

I'm glad to hear that you are becoming mature enough to appreciate my comments. Kudos to you as well!

I don't know if a link to you saying the thing you are saying again is as convincing as you would hope. Now, obviously I'm just being obnoxious, but to respond to your question: It's curmudgeonly because you are pasting rules verbatim without engaging. Your second message here is a model of a positive interaction, treating participants as people first and rule violators to be managed second.

I appreciate what you do as a moderator, and thank you, genuinely, for doing it. As someone who has been here a long time and posted hundreds of long, very thoughtful and what I try to make be well supported or well reasoned posts, it's slightly irritating to just get rules pasted at me.

I have come to believe that there is no such thing as 'true rationality' in the universe. There are true events and true facts, but rationality is a shared framework for communication. Rationality exists between people.

People always have a framing story or perspective or viewpoint or system prompt for how they understand facts and events.

If you want to influence beliefs you have to understand the framing story that a person is using - even when that framing story is invalid or untrue.

Also, if you want to influence beliefs, you have to provide some emotional validation. You can't remove a load bearing core belief from someone's story, you can only replace it.

---

Another partial explanation is trauma - you can think about 'conspiracy theories' in a number of ways, but these low information, high satisfaction theories often arise after traumatic experiences. You can't properly address the facts of the situation while a person is hurting.

We should expect to see more conspiracy theories after natural and unnatural disasters. Think wildfires caused space lasers, floods caused by cloud seeding, storms caused by radar installations, melting of steel beams by various means. The people who believe these things are generally not having a good time in life.

---

BONUS Link: Tim Minchin - Confirmation Bias

https://www.youtube.com/watch?v=G1juPBoxBdc

Yes. There are true facts, but the concept of "rationality" presupposes that there is one correct way to interpret these facts and translate them into behavior.

Two people observe someone beating another person. One person moves forward to intervene and stop the violence. The other moves away to protect themselves. Which person has acted rationally? They may have both acted in complete alignment with their personal philosophies, and they may each view the other as irrational.

"Rationality" is completely subjective to your own values and belief systems. Human behavior is infinitely more complex than formal logic allows.

Aside: For the Mermaid graph, what library or how is it being shown like this?
my understanding (which is definitely not exhaustive!) is that the case between Galileo and the church was way more nuanced than is popularly retold, and had nothing whatsoever to do with Biblical literalism like the passage in Joshua about making the sun stand still.

Paul Feyerabend has a book called Against Method in which he essentially argues that it was the Catholic Church who was following the classical "scientific method" of weighing evidence between theories, and Galileo's hypothesis was rationally judged to be inferior to the existing models. Very fun read.

> the case between Galileo and the church was way more nuanced than is popularly retold

Ex historian here. This is true. It’s a complicated episode and its interpretation is made more murky by generations of people trying to use it to make a particular rhetorical point. Paul Feyerabend is guilty of this too, although he’s at least being very original in the contrarian philosophy of science he’s using it for.

If anyone is interested in the episode for its own sake (which is rare actually, unless you’re a renaissance history buff first and foremost), I’d probably recommend John Heilbron’s biography which has a pretty balanced take on the whole thing.

> and had nothing whatsoever to do with Biblical literalism like the passage in Joshua about making the sun stand still.

The church is and was a large, often heterogenous institution. For some the issue was about conflict with literal interpretations of the bible, not merely the predominate allegorical interpretations (a more widely held concern, at least as a pedagogic matter). AFAIU, while the pope wasn't of this mind, some of the clerics tapped to investigate were. See, e.g., the 1616 Consultant's Report,

> All said that this proposition is foolish and absurd in philosophy, and formally heretical since it explicitly contradicts in many places the sense of Holy Scripture, according to the literal meaning of the words and according to the common interpretation and understanding of the Holy Fathers and the doctors of theology.

https://www.vaticanobservatory.org/sacred-space-astronomy/in...

Persevered through the article and comments in hopes someone would point this out.
As I’ve grown older and witnessed history in action I’ve begun to understand that reality is much, much more complicated than the simple narratives of history we lean on as a society.

Just think of how many different competing narratives are currently in existence surrounding this tumultuous point in history and realize that at some point some of these narratives will become dominant. Over time as the events leave social memory the key conclusions will likely be remembered but a lot of the reasoning behind them will not. As it exits living memory most of the nuance and context is lost. Over time we may change the narrative by reconsidering aspects that were forgotten, recontextualizing events based on modern concepts and concerns, misunderstanding what happened, or even surreptitiously “modifying” what happened for political ends. Or to put it more plainly, history is written by the victors and can be rewritten as time goes on and the victors change.

The important thing is not why they thought they were right but the fact they could not tolerate being wrong, or even tolerate dissidence on that one little inconsequential thing.

That's why you have people today pushing for flat earth and creationism.

Because their whole shtick is we are always right about absolutely everything.

I just recently watched a lecture about this and was fascinated.

Specifically, the (incorrect) model of the universe that was used in Europe at the time had been refined to the point that it was absurdly accurate. Even had they adopted a heliocentric model, there would have been no direct benefit for for a long, long time. If anything, Galileo's work was rife with errors and mathematical problems that would have taken a lot of work to figure out.

So the argument was to take on a bunch of technical debt and switching costs for almost no benefits.

Feelings aren't facts but they are important for persuasion. The methods most able to create radical change are the gentlest

https://en.wikipedia.org/wiki/Rogerian_argument

I disagree with Rapoport's taxonomy, not least "Chinese brainwashing" in the Korean war was not Pavolivan and was rather closer to the T-group method developed in Bethel, ME.

https://en.wikipedia.org/wiki/T-groups

Interesting. I often hear/read the term "steelmanning" instead, as in the opposite of constructing a straw man argument.
[flagged]
I'm in the same boat. In my case it's especially disturbing because prior to this whole gestures broadly thing, we had nuanced debates on many topics. Now, he just dismisses everything as 'narratives' if he disagrees with it.

Sadly, it seems to be based on the propaganda style I've seen out of authoritarians in general. It doesn't hurt 'their side' to break everything down to narratives, because at the end of the day, you just have to swallow the current one and you don't have to think about it again.

In another way it seems almost like a form of burnout. Like 'fuck this, I'm settling on cognitive ease from now on'.

I saw an interesting interview on the subject: https://youtu.be/6Ibk5vJ-4-o?t=1678

It argues the way to engage with these people is to first understand the psychological manipulation tactics they have been subjected to. That what you should focus on is not their false beliefs, but the underlying reasons why they were vulnerable to that manipulation in the first place; and to realize that everybody can become a victim of such cults helps to empathize with them. Don't give up on your friend.

If he is close and trusts you then this might work as well for you (has for me):

Don't start discussions with a concrete (recent) topic, but focus on fundamental things: Values, hopes, fears... Strengthen their argument. Tell them how you feel and why instead of making judgements.

The topical stuff is always under attack through misinformation, social media spam and propaganda. It's far more effective to honestly nurture human connection and focusing on core values. Without the fog of BS, people can see what's happening more easily themselves.

The only way to help these kind of people is by reactivating their critical thinking. This is only possible if you tell them something that is so absurd, that even they have to reject it. For example: „Trump is Putins father.“ „Kennedy was killed by aliens, because he wanted to land on the moon.“ „We live on the inside of a hollow world with the sun in the middle.“ „The mothership will come soon and send all humans with a chip inside their brain to Beteigeuze.“ At some point they start rejecting your outlandish ideas, because their brain simply can‘t stand that nonsense. And this is the first step in the right direction for these kind of people.
I'm wary of making an "arguments are soldiers" assumption where facts are mostly useful for making arguments, in an attempt to change people's minds.

We should be curious about what's going on in the world regardless of what ideologies we might find appealing. Knowing what's going on in the world is an end in itself. An article with some interesting evidence in it is useful even if you disagree with the main argument.

Facts may not change minds, but we should still support people who do the reporting that brings us the facts.

CS Peirce has a famous essay "The Fixation of Belief" where he describes various processes by which we form beliefs and what it takes to surprise/upset/unsettle them.

The essay: https://www.peirce.org/writings/p107.html

This blog post gestures at that idea while being an example of what Peirce calls the "a priori method". A certain framework is first settled upon for (largely) aesthetic reasons and then experience is analyzed in light of that framework. This yields comfortable conclusions (for those who buy the framework, anyhow).

For Peirce, all inquiry begins with surprise, sometimes because we've gone looking for it but usually not. About the a priori method, he says:

“[The a priori] method is far more intellectual and respectable from the point of view of reason than either of the others which we have noticed. But its failure has been the most manifest. It makes of inquiry something similar to the development of taste; but taste, unfortunately, is always more or less a matter of fashion, and accordingly metaphysicians have never come to any fixed agreement, but the pendulum has swung backward and forward between a more material and a more spiritual philosophy, from the earliest times to the latest. And so from this, which has been called the a priori method, we are driven, in Lord Bacon's phrase, to a true induction.”

Wow. I'm reminded of a great essay/blgo I read years ago that I'll never find again that said a good, engaging talk/presentation has to have an element of surprise. More specifically, you start with an exposition of what your audience already knows/believes, then you introduce your thesis which is SURPRISING in terms of what they already know. Not too out of the realm of belief, but just enough.

The bigger/more thought-diverse the audience, the harder this is to do.

Some of the core ideas here seem good, but the node/edge distinction feels too fuzzy. The node "Climate Change Threat" is a claim. Is the node "Efficiency" a claim? Can one challenge the existence of Efficiency? If one instead challenges the benefit of Efficiency, isn't that an edge attack?

I could give a bunch of other examples where the nodes in the article don't feel like apples-to-apples things. I feel less motivated to try to internalize the article due to this.

The edges are labeled by transitive verbs, where the arrow points from the subject of that verb to the direct object. (I'm counting particle verbs, like "leads to", as verbs.) The nodes are labeled by nouns. If you can change a noun to a verb, I guess you would be changing what is an edge and what is a node.

Example: In the article's first diagram, there is a node labeled "Innovation". This could be replaced by a node labeled "Capitalist" and a node labeled "Improvement", with an arrow from the first to the second labeled "innovates."

So yes, if you can replace a node by an edge (and vice versa, although I don't give an example), this node vs. edge thing is fuzzy.

I think the structure inherently enables each node to be a claim (like "this thing exists"), but that there's value in making a node even if that node's claim is not particularly disagreeable, because the edges to that node might be disagreeable, or to provide more detail about how one node relates to another (e.g. through some intermediate node). In this case, maybe the main value in modeling "Efficiency" is to convey how innovation might lead to profit.

To me, it feels less fuzzy when you assume that all nodes and edges imply their own claims, and that it's just a matter of whether or not those claims are worth arguing. The fuzziness imo is based on the fact that the curator picks which nodes and edges exist, which therefore determines which claims exist and can be agreed or disagreed with, not to mention the overall legibility of the graph itself. But I would argue that a causal graph like this is better at representing reality than something like an argument tree, and that, while it might be fuzzy to determine which nodes should exist, at least there's less opinion involved about where nodes should be placed in relation to each other. Which imo makes the structure easier to refine given time and feedback.

I'm struggling to understand this article. I think it's for a couple of reasons:

1. The capitalism graph seems OK but the climate change graph doesn't look right. I've never heard anyone argue that "resilient communities" automatically lead to "policy changes". What does that mean? If you have a resilient community already, why would you need to change anything? It seems to suggest that people with this belief system would end up in an infinite loop of wanting to change policies even when the original motivating problem is solved, which sounds like a very uncharitable view of climate activists.

2. After setting up this very abstract argument, the author ends by claiming, "The evidence, and the argument of this post, suggest [truth doesn't determine what people believe]: structure, coherence, and emotional resonance are far more important for the persistence and spread of beliefs". But he hasn't supplied any arguments. He outlined an abstract theoretical model, but it makes no testable predictions and he doesn't try to prove it's correct. Then he claims there are no real debates in the west about climate change, vaccines, or race, it's all driven by the evil Ruskies "creating social chaos". This claim isn't linked in any way to the first part with the graphs.

I've written about this belief twice in the past.

https://blog.plan99.net/fake-science-part-ii-bots-that-are-n...

https://blog.plan99.net/did-russian-bots-impact-brexit-ad66f...

It's all based on a bunch of academic papers that don't replicate and which use pseudo-scientific methodologies. They misuse ML in ways that generate noise, identify random people as "Russian bots", conclude that "Russian bots" support every possible opinion simultaneously and from there assume there must be some nefarious psychological strategy behind it. In reality they're just doing bad social science and casting the results through the prism of their ideological biases. It works because social science is full of people who are easily impressed by maths they don't understand, and who are surrounded by people with identical ideologies to themselves (often extreme ones). So there's nobody to give them a reality check. Eventually people who understand computer science come along and write a rebuttal, but academia is a closed system so they just ignore it and keep pumping journalists/politicians full of conspiracy theories and misinformation.

Given that, it's kind of ironic that the author is writing about the difficulty of changing people's minds with truth.

> graph seems OK but

The point of the argument is agnostic to the contents and structure of the graphs. They are only there to illustrate that a) there exists a conflict; b) both sides of this conflict have a graph; c) even though these graphs inform positions on the same policy, they are composed of completely unrelated ideas.

> But he hasn't supplied any arguments. He outlined an abstract theoretical model, but it makes no testable predictions and he doesn't try to prove it's correct.

You're meant, I think, to find the argument intuitively persuasive. It's easy to map the model's concept onto one's own beliefs, at least if you consider yourself to be rational (and most people do, even if they end up believing absurdities).

I think there is a testable prediction: if you just go in guns blazing to a "culture war" argument and try to convince people of your viewpoint, you are not going to make any progress. Further, in order to even challenge individual beliefs, you will have to understand how they relate to the rest of the other side's memeplex.

> Then he claims there are no real debates in the west about climate change, vaccines, or race, it's all driven by the evil Ruskies "creating social chaos". This claim isn't linked in any way to the first part with the graphs.

Blaming Russia for this is indeed very much out of pocket, and an example of the kind of culture warring that the article seems to want to discourage. However, there is ample evidence of the existence of the groups cited (granted there are others from other countries as well), even if they can't really explain more than a small part of the problem — at least directly. I think it's fair to say that a small number of agitators can produce large amounts of social tension, if they hit just the right talking points (qv. https://slatestarcodex.com/2018/10/30/sort-by-controversial/).

More importantly, I'd say the dearth of "real debate" is abundantly clear from looking at pretty much any social media. Even on sites that allow users to take either side of an issue, even on the subset of those where one side isn't clearly being continually persecuted and driven off, you find very heavy siloing of each side into its own echo chamber.

Structure for facts and information is just communication.

You can have the facts but not be persuasive due to poor communication skills.

The core of the problem lies not in facts failing to persuade, but in our obsession with trying to change minds.

We've developed systems to facilitate this. Parliamentary debate, for instance, was meant to force parties to justify their positions through public reasons, not private convictions. Religious institutions, too, have long shaped minds with varying degrees of success.

But attempts to reshape humanity, especially on a grand scale, have consistently produced devastating and unintended consequences.

We now live in an age where political expedience trumps truth; what matters is not whether something is right, but whether it plays well. The public is expected to absorb politicized half-truths while being shielded from the real issues....because complexity isn’t expedient. The current obsession with labeling ideas as “misinformation” or “disinformation” is a desperate, often incoherent attempt to control discourse, and it breeds more cynicism than clarity.

In the end, good ideas tend to survive, but not on any schedule we can manage. Trying to micromanage thought or the flow of information is not only futile, it’s unworthy of the very rationality we claim to protect.

You say the misinformation label is counterproductive. But what if we cannot even agree on what the facts are? There's no productive discussion to be had. We cannot solve problems collaboratively when facts are ignored or denied.
> Religious institutions, too, have long shaped minds with varying degrees of success.

Given that a huge portion of the world's population is religious (a quick google search say 84%), I'd say with a very high success.

In the "Climate Change Threat" example, one vector of attack is when the policy changes do not lead to renewable energy adoption or to reduced emissions.

That justifies the questioning of whether the climate change was really motivating the policy change or just being used as pretext.

That idea suggests there might be another way to get to the desired goal, if that goal is renewable energy adoption (quite aside from whether that goal results in reduced emissions).

We have solar panels on our house, and recently installed a heat pump with gas furnace backup in place of an AC (for summer) + gas furnace (for winter). We also replaced our gas water heater (near the end of its life) with a heat pump water heater, and I drive an EV (not Tesla :)). All these were partly paid for by various tax rebates. The result is that our electric bill is zero, and our gas bill has plummeted (I think it will be nearly zero), and I spend zero at the gas station and the oil change place. One should be able to sell that idea to anyone who wants to reduce their expenses, and expects to live in a house for a few years to a decade to break even, and to drive their car for five years or so.

Of course the current administration is doing its best to eliminate those tax benefits...

Minds can’t be changed when a decision was based on emotion.

One day I told a friend how I could make no sense of my girlfriend’s behavior in some situation.

He said to me “you still think people make decisions on logic. Many people make their decisions on how they feel emotionally. Logic and facts have nothing to do with it.”

Suddenly a light switched on and i realized that I’m a typical computer person who thinks that everything is based on logic and if you can just explain clearly enough, explain the facts, then the other person will change their mind when they see the facts. It doesn’t work that way.

Computer people have real trouble getting their head around this concept.

In 'Thus Spoke Zarathustra' the argument is made that the most important cultural changes happen outside the debate, where new structures of thought are being built without being noticed. As without a competing thought structure we are unable to even perceive the new structure. It is the dissonances and the debates that lets us introspect our own ideas. Without the dissonance we do not notice new ideas taking hold of us and changing ourselves, and it is only unnoticed that truly radical changes can take place.
In the seminaries of the world they don't teach how to make up clever stories to entrap people. For that you go to Marketing, PR or Sales school. And ofcourse the people who come out of these schools think they are very clever because they sold some widgets or politicians to the masses by some deadline.

But have you heard of a sales org or a marketing dept that has been running for thousand years? They barely ever survive few decades as a coherent unit if ever.

For the curious go check what the neighborhood seminary teaches.

The Church (and all other religious systems) haven't stood for thousands of year through the fall of empires, nations, civil wars, revolutions, plagues, famines, collapse of economic systems, internal schisms, enlightenment, progress in science/tech etc because of the stories they tell.

In fact the stories have been rewritten, branched, mutated, merged with other stories thousands of times to the point we have thousands of different versions of these stories. There is no "narrative domination".

The Church has survived because when people Suffer due to the fall of empire/nations/banks/economies, war, plagues, famine, disasters etc where else do people go?

Do they all head to house of the local system analyst/graph theorist?

This seems way too logical. Humans are not, for the most part, rational and logical creatures.
Very engaging look at a very difficult topic to approach analytically.

I'm reminded of something I learned about the founder of Stormfront, the internet's first white supremacist forum. His child went on to attend college away from home, her first time away from her family, and over a period of roughly two years, she attended dinners with a group of Jewish students who challenged each of her beliefs one at a time. Each time, as she accepted the evidence her friends presented to her about a particular belief, she nonetheless would integrate the new information with her racist worldview. This continued piece by piece until there was nothing left of her racist worldview at all.

It's both heartening and disheartening at the same time, because if this person can change her mind after almost two decades of constant indoctrination during her formative years, then surely anyone can change their mind. That's the heartening part: the disheartening part is, of course, that the effort it took is far from scalable at present and much more difficult to apply to someone who remains plugged into whatever information sources they are getting their current fix of nonsense from.

This accurately describes how my brain works. My thought process is like a bunch of graph nodes, and when new information doesn't "fit", it puts tensions on the links, and I want to resolve that tension. I can...feel it happening inside my mind when I think, more or less? -- It's hard to describe

Resolving that tension may occur in several ways, in order of increasing significance:

- Rejecting the new information

- Refining the graph (splitting a node representing a concept into multiple sub-nodes representing sub-concepts with their own relationships)

- Making local modifications to the graph

- Making sweeping architectural changes to the graph as a whole

The author seems to imply that cognitive biases are an inherent qualitative problem that is fundamentally forced to arise from this graph structure. I personally respectfully disagree. In my view, cognitive biases are a quantitative problem, incorrectly setting the threshold at which a large reorganization should occur. "Extraordinary claims require extraordinary proof" is qualitatively a sound epistemological principle -- but to correctly apply it, you must quantitatively set a reasonable threshold for "extraordinary."

I feel like we need to get better at understanding the graph structure of people we disagree with. The best example I can think of is the abortion debate [1]: If you accept the premise "Life begins at conception" [2], the pro-life camp has an enormously strong case; the rest of the graph between that premise and "Abortion should be illegal" is very strong (it's mostly tremendously well-reinforced nodes in near-universal moral foundations, like "Do unto others" or "Murder should be illegal").

Arguments against abortion are frequently just bad when looked at from the graph point of view: They often don't directly confront the premise "Life begins at conception," nor do they attack the graph between the premise and conclusion. [3]

[1] I'm personally in the pro-choice camp; I do not accept the premise that a human fetus has the same moral status as a fully grown human.

[2] "Life" here is not in the technical biological sense, but something more akin to "The ethical standing of human-equivalent sentience." (Bacteria and protozoa and so on are biologically alive, but nobody moralizes about killing them en masse by, e.g., cooking your food.)

[3] If you're curious about my own views on this specific subject, I've talked about them here before: https://news.ycombinator.com/item?id=36255493#36270990

I've had this mental model for a while now, but this post lays it out better than I could've. I think the most important part of the post is this part of the conclusion:

> For years, our main defense against misinformation and manipulation has been to double down on “truth”—to fact-check, debunk, and moderate. These efforts are important, but they rest on the assumption that truth is the main determinant of what people believe. The evidence, and the argument of this post, suggest otherwise: structure, coherence, and emotional resonance are far more important for the persistence and spread of beliefs.

I'm still friends with one or two people who are hat-wearing MAGA supporters. WE stopped talking politics after 2018 or so, but between 2016 and 2018, and still occasionally since, I get a glimpse into their belief graph. Sometimes their facts are incorrect, but that's less common than simply them interpreting the same facts in a different light. Occasionally they'll have an interpretation of a fact pattern that I find more compelling than the interpretation I find in more liberal spaces. (The Democrat party is, after all, not the best at hypocrisy.) These patterns are the place where the point of the blog post comes out most clearly: most people aren't motivated by facts and logic; they're motivated by a vast network of feelings and emotions where each point reinforces all the other points and an individual fact is less important for its truth than its reinforcement of the overall belief graph.

The most interesting thing about the MAGA belief graph though is its overall structure and maintenance. There is approximately a third of the US that simply has an entirely different basis of belief in the world than the other two thirds. How is it maintained? How does normal everyday contact between the two groups not reconcile the foundations of the two belief systems? It's not a difference in facts, although that does come up occasionally. For example, the sudden change in the truth of the Epstein client list and the effort of the MAGA belief system maintainers (news orgs, influencers, etc) to excise it from the belief graph has had some interesting effects.

But the interesting part is the methods used, the way the belief system reacts to influencers and others that shape the belief system, and how particular facts and opinions are used to reinforce the effects of both new and existing parts of the belief graph. Looking at my MAGA acquaintances and seeing their belief system from the outside has made those methods and reactions more legible, and has allowed me to notice some of the times those same methods and reactions pop up in other communities. For example, I dislike the focus on fact-checking, because too often the facts are the same on both sides, and it's only a difference in interpretation. Then people who agree with the fact checkers prove to themselves that the other side is unable to see truth, while people who disagree with the fact checkers prove to themselves that the other side twists truth into lies. Yet people still push for fact checking despite the fact it only reinforces both sides opinion of themselves rather than having any chance of changing the mind of anyone on the other side.

Unfortunately I am lazy or else I would've taken notes of examples of the methods and reactions used to reinforce a belief system, rather than just vague half-recollected memories that form my own belief graph. Regardless, I think it's important for people to look at their own belief system and, when presented with new facts or arguments, examining them and how they fit into their belief system, and see if maybe the argument is relying less on pure facts and more on emotional ties to the rest of their belief system.

(comment deleted)
> Sun, stand thou still upon Gibeon; and thou, Moon, in the valley of Ajalon. And the sun stood still, and the moon stayed

Who am I to doubt the Church's interpretation but this seems like it literally reads as if the scripture is saying that the sun stood still and the Earth revolves around it?