Acclaimed science author Carl Sagan illustrated this challenge with his “dragon in the garage” analogy. If someone claims to have a dragon that is invisible, silent, intangible, and undetectable by any means, there is no practical difference between the dragon’s existence and non-existence. Similarly, without verifiable evidence, the existence of an immortal soul remains unproven.
> If someone claims to have a dragon that is invisible, silent, intangible, and undetectable by any means, there is no practical difference between the dragon’s existence and non-existence.
Unsurprising that the longest subthread here is one criticizing the premise.
While skeptical, he did not have much skepticism against mainstream theories.
I think it needs another item in the list:
For any theory/ hypothesis: how well does it stand against the null-hypothesis?
For example: How much physical evidence is there really for the string-theory?
And I would upgrade this one:
If there’s a chain of physical evidence (was argument), every link in the chain must work (including the premise) — not just most of them
And when breaking these items do not mean that something is false. It means that the arguments and evidence is incomplete. Don't jump to conclusions when you think that the arguments or evidence is invalid (that is how some people even think that the moonlanding was a hoax).
> And I would upgrade this one: If there’s a chain of physical evidence (was argument), every link in the chain must work (including the premise) — not just most of them
We still use Newtonian physics plenty, despite bits of it not working due to relativity.
> And I would upgrade this one: If there’s a chain of physical evidence (was argument), every link in the chain must work (including the premise) — not just most of them
From The Demon Haunted World:
"In the middle 1970s an astronomer I admire put together a modest manifesto called “Objections to Astrology” and asked me to endorse it. I struggled with his wording, and in the end found myself unable to sign—not because I thought astrology has any validity whatever, but because I felt (and still feel) that the tone of the statement was authoritarian. It criticized astrology for having origins shrouded in superstition. But this is true as well for religion, chemistry, medicine, and astronomy, to mention only four. The issue is not what faltering and rudimentary knowledge astrology came from, but what is its present validity.
...
The statement stressed that we can think of no mechanism by which astrology could work. This is certainly a relevant point but by itself it’s unconvincing. No mechanism was known for continental drift (now subsumed in plate tectonics) when it was proposed by Alfred Wegener in the first quarter of the twentieth century to explain a range of puzzling data in geology and paleontology. (Ore-bearing veins of rocks and fossils seemed to run continuously from Eastern South America to West Africa; were the two continents once touching and the Atlantic Ocean new to our planet?) The notion was roundly dismissed by all the great geophysicists, who were certain that continents were fixed, not floating on anything, and therefore unable to “drift.” Instead, the key twentieth-century idea in geophysics turns out to be plate tectonics; we now understand that continental plates do indeed float and “drift” (or better, are carried by a kind of conveyor belt driven by the great heat engine of the Earth’s interior), and all those great geophysicists were simply wrong. Objections to pseudoscience on the grounds of unavailable mechanism can be mistaken—although if the contentions violate well-established laws of physics, such objections of course carry great weight."
> For any theory/ hypothesis: how well does it stand against the null-hypothesis? For example: How much physical evidence is there really for the string-theory?
That's an unfortunate choice of example - the problem with string theory is that there is no null hypothesis. We know that our other theories are not self-consistent when unified, but we don't have a theory that is self-consistent, that could serve as the null hypothesis.
The people for whom this stuff isn't glaringly obvious, relatively early in life, will never get it. Except, maybe, specific instances that directly affect them in a bad way. Switch "brands" and they'll be fooled again. They'll probably even double-down on it.
The article links to an article about Sagans' prediction of the decline of america. Strangely fitting nowadays.
> I have a foreboding of an America in my children’s or grandchildren’s time — when the United States is a service and information economy; when nearly all the manufacturing industries have slipped away to other countries; when awesome technological powers are in the hands of a very few, and no one representing the public interest can even grasp the issues; when the people have lost the ability to set their own agendas or knowledgeably question those in authority; when, clutching our crystals and nervously consulting our horoscopes, our critical faculties in decline, unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness…
Asimov and Feynman also spoke about similar things (along with many others)
In 1980, Asimov famously wrote The Cult of Ignorance[0], criticizing the rise of anti-intellectualism. Where there was a strong political push of "don't trust the experts". He criticizes claims that sound familiar today "America has a right to know" on the basis of this being meaningless without literacy. He clarifies that literacy is far more than being able to actually read words on a page, but to interpret and process them. Asimov isn't being pretentious, his definition is consistent with how we determine reading levels[2] and his critique would be that most people do not have that of a Freshman in High School. Hell, it is even in his fiction! It is even in The Foundation and is literally the premise of Profession[3].
Feynman is a bit more scattered, but I think his discussion about the education system in Brazil (in the 50's) says a lot[4]. He talks a lot about how the students could recite the equations, ace all the tests, and achieve everything that looks to be, at least on paper, perfectly academic; but how the students did not really have the deeper understanding of the equations. It is a discussion about literacy. Were he around today I'm sure he'd use the phrase "metric hacking". Anyone that knows Feynman may also be thinking about his Cargo Cult Science[5](a commencement speech at Cal Tech (1974)). This is where his famous quote
The first principle is that you must not fool yourself--and you are the easiest person to fool. So you have to be very careful about that.
comes from. But there is a lot of important context surrounding this and it is worth knowing about.
[3] Profession has been in discussion lately, directly relating to this topic. If you haven't read it I'll say it is one of my favorite's of his. Not as good as Foundation but up there with Nightfall https://en.wikipedia.org/wiki/Profession_(novella)
I wanted to add Asimov's The Relativity of Wrong. Sometimes I feel it should be required reading before arguing on the internet. I find myself coming back to read it at least once a year
- Try not to get overly attached to a hypothesis just because it’s yours. It’s only a way station in the pursuit of knowledge. Ask yourself why you like the idea. Compare it fairly with the alternatives.
- See if you can find reasons for rejecting it. If you don’t, others will.
This is good advice IME. Get well acquainted (like REALLY well acquainted) with opposing viewpoints, such that you could argue them better than their proponents. See also "Argue Well by Losing" by Phil Haack [1].
Somewhat relatedly, the ancients viewed Rhetoric as the purest expression of intelligence. It required you to have deep knowledge of a topic, including all arguments in favour and against (implying deep empathy with the audience), and the ability to form coherent and meaningful argument. Modern political "debate" is ludicrous in comparison.
> Somewhat relatedly, the ancients viewed Rhetoric as the purest expression of intelligence. It required you to have deep knowledge of a topic, including all arguments in favour and against (implying deep empathy with the audience), and the ability to form coherent and meaningful argument. Modern political "debate" is ludicrous in comparison.
"Rhetoric" is an unfortunately overloaded term, as modern political "debate" is often nothing more than (the other definition of) rhetoric.
I wonder how well Sagan's own "baloney" holds up against his kit. Historians despise the guy for all the stuff he made up about the library of Alexandria, Hypatia, Eratosthenes, etc... People still repeat a lot of that to this day.
> I wonder how well Sagan's own "baloney" holds up against his kit. Historians despise the guy for all the stuff he made up about the library of Alexandria, Hypatia, Eratosthenes, etc... People still repeat a lot of that to this day.
Yeah, but he's a saint of science-fandom, so don't question him. Instead, admire and follow him, and encourage others to do likewise.
I loved watching Cosmos when I was a kid but as I got older I developed a dislike for Sagan. He strikes me as supremely arrogant and probably insufferable to be around. I don't know that of course, I never met him. Just a feeling.
Sagan made solid contributions to Planetary Science in the 60's and 70's.
His role as PBS educator, SF author, etc. needs to be considered as a separate thing.
I also loved James Burke and his Connections series, but as it got into the later seasons the so-called "connections" got tenuous and sometimes quite strained.
You can go through all the classic PBS science shows and find problems, Stephen Hawking's Universe was basically unwatchable because they refused to engage with the math.
"As I write, the number-one videocassette rental in America is the movie Dumb and Dumber. “Beavis and Butthead” remain popular (and influential) with young TV viewers. The plain lesson is that study and learning—not just of science, but of anything—are avoidable, even undesirable.”
Mike Judge's satire was lost on Sagan. Carl took his knee-jerk reaction and ran with it.
I think it was lost on a lot of the audience at the time as well, who saw Beavis and Butthead as cool rather than a critique of anti-intellectualism. That's the problem with satire in general -- no position is so absurd that somebody won't take it at face value rather than satire.
I think the notion of considering all points of view depends on the assumption that people are arguing in good faith. When this breaks down, I don’t think we can just throw up our hands and give up, but the baloney detection kit needs to be updated. I don’t have a blog-worthy list of answers, but it’s something I at least think about.
One thing we can do is a kind of meta-analysis, where we check on the condition of our own baloney detection kit. For instance, if I reject an idea and it later turns out to be true, did my BDK fail? Does it need to be updated? Or are a few scattered failures OK? You can treat the BDK as a testable hypothesis like anything else.
Sagan's kit is one of several similar resources I'd turned up ... over a decade ago now ... when I'd begun considering the matter of epistemics in media, particularly online discourse. The situation's not improved.
It includes in addition to Sagan: Rory Coker's precis on pseudoscience, the Venn Diagram of Irrational Nonsense, the concept of falsifiability, an informative (if excruciatingly painful) BBC docu on stupidity, Frankfurt's "On Bullshit", Ferguson on why youth culture made everything suck, Brandolino's Law, Silver's Bullshitter's Inequality, a relationship between the Kübler-Ross model and the Dunning-Kruger Effect, The Periodic Table of Irrational Nonsense, and Adams's 'B'-Ark.
While each of these are good to keep in mind while reading, I don't find them very exhaustive or derivable. I prefer the Theory of Constraints (Eli Goldratt)'s "Thinking Tools", specifically the "categories of legitimate reservation". Depending on source, there are between six and eight.
1. Is it clear?
2. Does it actually exist? Is it true?
3. Does the cause actually cause the effect?
4. For the proposed cause, do its other implied effects exist?
A key meta-requirement is to want to think critically about issues.
If there is no desire to discover the truth of a matter and evaluate it against supporting evidence and opposing claims, then all efforts at inculcating critical thinking are dead in the water. On the other hand, if there is a genuine desire to assess arguments and claims critically, there are plenty of resources today that can teach you how.
This is a never-ending process. But the desire to think critically has to be in place before it can even begin. Critical thinking cannot occur without a strong commitment to epistemic hygiene.
In India, the problem is that many people do not even want to think critically. We tend to gravitate toward beliefs that buttress our tribal affiliations. Our tribes are defined by our worldviews, and our tribes must prevail. Hence our worldviews must be proven true, regardless of whether they are in fact true.
There is a striking indifference toward truth as a value - ironic for a country whose national motto is "Truth alone triumphs." Many people have yet to realize that truth - satya - is not something you place on a pedestal and worship, but something you actively pursue, overturning long-held beliefs where necessary.
32 comments
[ 2.8 ms ] story [ 59.5 ms ] threadCarl Sagan’s Baloney Detection Kit: Tools for Thinking Critically & Knowing Pseudoscience When You See It
https://www.rxjourney.net/the-possibility-of-life-after-deat...
Unsurprising that the longest subthread here is one criticizing the premise.
I think it needs another item in the list: For any theory/ hypothesis: how well does it stand against the null-hypothesis? For example: How much physical evidence is there really for the string-theory?
And I would upgrade this one: If there’s a chain of physical evidence (was argument), every link in the chain must work (including the premise) — not just most of them
And when breaking these items do not mean that something is false. It means that the arguments and evidence is incomplete. Don't jump to conclusions when you think that the arguments or evidence is invalid (that is how some people even think that the moonlanding was a hoax).
We still use Newtonian physics plenty, despite bits of it not working due to relativity.
From The Demon Haunted World:
"In the middle 1970s an astronomer I admire put together a modest manifesto called “Objections to Astrology” and asked me to endorse it. I struggled with his wording, and in the end found myself unable to sign—not because I thought astrology has any validity whatever, but because I felt (and still feel) that the tone of the statement was authoritarian. It criticized astrology for having origins shrouded in superstition. But this is true as well for religion, chemistry, medicine, and astronomy, to mention only four. The issue is not what faltering and rudimentary knowledge astrology came from, but what is its present validity.
...
The statement stressed that we can think of no mechanism by which astrology could work. This is certainly a relevant point but by itself it’s unconvincing. No mechanism was known for continental drift (now subsumed in plate tectonics) when it was proposed by Alfred Wegener in the first quarter of the twentieth century to explain a range of puzzling data in geology and paleontology. (Ore-bearing veins of rocks and fossils seemed to run continuously from Eastern South America to West Africa; were the two continents once touching and the Atlantic Ocean new to our planet?) The notion was roundly dismissed by all the great geophysicists, who were certain that continents were fixed, not floating on anything, and therefore unable to “drift.” Instead, the key twentieth-century idea in geophysics turns out to be plate tectonics; we now understand that continental plates do indeed float and “drift” (or better, are carried by a kind of conveyor belt driven by the great heat engine of the Earth’s interior), and all those great geophysicists were simply wrong. Objections to pseudoscience on the grounds of unavailable mechanism can be mistaken—although if the contentions violate well-established laws of physics, such objections of course carry great weight."
That's an unfortunate choice of example - the problem with string theory is that there is no null hypothesis. We know that our other theories are not self-consistent when unified, but we don't have a theory that is self-consistent, that could serve as the null hypothesis.
> I have a foreboding of an America in my children’s or grandchildren’s time — when the United States is a service and information economy; when nearly all the manufacturing industries have slipped away to other countries; when awesome technological powers are in the hands of a very few, and no one representing the public interest can even grasp the issues; when the people have lost the ability to set their own agendas or knowledgeably question those in authority; when, clutching our crystals and nervously consulting our horoscopes, our critical faculties in decline, unable to distinguish between what feels good and what’s true, we slide, almost without noticing, back into superstition and darkness…
https://www.openculture.com/2025/02/carl-sagan-predicts-the-...
In 1980, Asimov famously wrote The Cult of Ignorance[0], criticizing the rise of anti-intellectualism. Where there was a strong political push of "don't trust the experts". He criticizes claims that sound familiar today "America has a right to know" on the basis of this being meaningless without literacy. He clarifies that literacy is far more than being able to actually read words on a page, but to interpret and process them. Asimov isn't being pretentious, his definition is consistent with how we determine reading levels[2] and his critique would be that most people do not have that of a Freshman in High School. Hell, it is even in his fiction! It is even in The Foundation and is literally the premise of Profession[3].
Feynman is a bit more scattered, but I think his discussion about the education system in Brazil (in the 50's) says a lot[4]. He talks a lot about how the students could recite the equations, ace all the tests, and achieve everything that looks to be, at least on paper, perfectly academic; but how the students did not really have the deeper understanding of the equations. It is a discussion about literacy. Were he around today I'm sure he'd use the phrase "metric hacking". Anyone that knows Feynman may also be thinking about his Cargo Cult Science[5](a commencement speech at Cal Tech (1974)). This is where his famous quote
comes from. But there is a lot of important context surrounding this and it is worth knowing about.[0] Note: 1980 was an election year, and one with a sweeping victory...[1] https://people.bath.ac.uk/mnsbr/papers/Asimov-Newsweek-Janua...
[1] https://en.wikipedia.org/wiki/1980_United_States_presidentia...
[2] https://nces.ed.gov/nationsreportcard/reading/achieve.aspx
[3] Profession has been in discussion lately, directly relating to this topic. If you haven't read it I'll say it is one of my favorite's of his. Not as good as Foundation but up there with Nightfall https://en.wikipedia.org/wiki/Profession_(novella)
[4] https://enlightenedidiot.net/random/feynman-on-brazilian-edu...
[5] https://sites.cs.ucsb.edu/~ravenben/cargocult.html
[Edit]
I wanted to add Asimov's The Relativity of Wrong. Sometimes I feel it should be required reading before arguing on the internet. I find myself coming back to read it at least once a year
https://hermiene.net/essays-trans/relativity_of_wrong.html
As a committed centrist, I am very good at fairly scrutinizing everything. /s
- See if you can find reasons for rejecting it. If you don’t, others will.
This is good advice IME. Get well acquainted (like REALLY well acquainted) with opposing viewpoints, such that you could argue them better than their proponents. See also "Argue Well by Losing" by Phil Haack [1].
Somewhat relatedly, the ancients viewed Rhetoric as the purest expression of intelligence. It required you to have deep knowledge of a topic, including all arguments in favour and against (implying deep empathy with the audience), and the ability to form coherent and meaningful argument. Modern political "debate" is ludicrous in comparison.
[1] https://haacked.com/archive/2013/10/21/argue-well-by-losing....
"Rhetoric" is an unfortunately overloaded term, as modern political "debate" is often nothing more than (the other definition of) rhetoric.
[0] https://news.ycombinator.com/item?id=46997906
Human directed selection is a thing.
Have you ever seen a pug or do I need 58 articles with a bibliography 20 miles long to tell you they exist?
Yeah, but he's a saint of science-fandom, so don't question him. Instead, admire and follow him, and encourage others to do likewise.
His role as PBS educator, SF author, etc. needs to be considered as a separate thing.
I also loved James Burke and his Connections series, but as it got into the later seasons the so-called "connections" got tenuous and sometimes quite strained.
You can go through all the classic PBS science shows and find problems, Stephen Hawking's Universe was basically unwatchable because they refused to engage with the math.
Mike Judge's satire was lost on Sagan. Carl took his knee-jerk reaction and ran with it.
One thing we can do is a kind of meta-analysis, where we check on the condition of our own baloney detection kit. For instance, if I reject an idea and it later turns out to be true, did my BDK fail? Does it need to be updated? Or are a few scattered failures OK? You can treat the BDK as a testable hypothesis like anything else.
My catalogue is here: <https://web.archive.org/web/20200121211018/https://old.reddi...> (archive).
It includes in addition to Sagan: Rory Coker's precis on pseudoscience, the Venn Diagram of Irrational Nonsense, the concept of falsifiability, an informative (if excruciatingly painful) BBC docu on stupidity, Frankfurt's "On Bullshit", Ferguson on why youth culture made everything suck, Brandolino's Law, Silver's Bullshitter's Inequality, a relationship between the Kübler-Ross model and the Dunning-Kruger Effect, The Periodic Table of Irrational Nonsense, and Adams's 'B'-Ark.
1. Is it clear?
2. Does it actually exist? Is it true?
3. Does the cause actually cause the effect?
4. For the proposed cause, do its other implied effects exist?
5. Is the cause sufficient for the effect?
6. Are cause and effect reversed?
If there is no desire to discover the truth of a matter and evaluate it against supporting evidence and opposing claims, then all efforts at inculcating critical thinking are dead in the water. On the other hand, if there is a genuine desire to assess arguments and claims critically, there are plenty of resources today that can teach you how.
This is a never-ending process. But the desire to think critically has to be in place before it can even begin. Critical thinking cannot occur without a strong commitment to epistemic hygiene.
In India, the problem is that many people do not even want to think critically. We tend to gravitate toward beliefs that buttress our tribal affiliations. Our tribes are defined by our worldviews, and our tribes must prevail. Hence our worldviews must be proven true, regardless of whether they are in fact true.
There is a striking indifference toward truth as a value - ironic for a country whose national motto is "Truth alone triumphs." Many people have yet to realize that truth - satya - is not something you place on a pedestal and worship, but something you actively pursue, overturning long-held beliefs where necessary.