How to detect baloney the Carl Sagan way | Michael Shermer | Big Think

How to detect baloney the Carl Sagan way
----------------------------------------------------------------------------------
In 1995, just a few months before his death, astrophysicist Carl Sagan published The Demon-Haunted World: Science as a Candle in the Dark. In that book, he wrote a chapter called 'The Fine Art of Baloney Detection', and from it sprang what skeptics call the 'baloney detection kit'. This is a set of tools for critical thinking that has continued to develop since Sagan's death, 22 years ago. Here, skeptic and science writer Michael Shermer explains key lessons from Sagan, and from his own college freshman course 'Skepticism 101', where teaches students ten basic questions that will help them debunk untruths, and call out baloney when they see it.

1. How reliable is the source of the claim?

2. Does the source make similar claims?

3. Have the claims been verified by somebody else?

4. Does this fit with the way the world works?

5. Has anyone tried to disprove the claim?

6. Where does the preponderance of evidence point?

7. Is the claimant playing by the rules of science?

8. Is the claimant providing positive evidence?

9. Does the new theory account for as many phenomena as the old theory?

10. Are personal beliefs driving the claim?
----------------------------------------------------------------------------------
MICHAEL SHERMER:

Dr. Michael Shermer is the Founding Publisher of Skeptic magazine, a monthly columnist for Scientific American, and Presidential Fellow at Chapman University.
----------------------------------------------------------------------------------
TRANSCRIPT:

Michael Shermer: Back in the late '90s we introduced the Baloney Detection Kit, inspired by Carl Sagan’s 'Demon-Haunted World' where he had a chapter on the Baloney Detection Kit. He had his set of questions; I kind of developed my own because I started encountering other people that disagreed with me, you know, “we never went to the moon” people, conspiracy people, whatever, and I thought okay so: How do we know—if I don’t know what’s coming down the pike in ten years from now, if I am going to teach my students how to think critically, what are the key points, like just basic questions they can ask?

So it begins with one: how reliable is the source of the claim? Here’s the claim, how reliable is it? What’s the evidence for it? What’s the quality of the evidence? Where does it come from? Who said that? Is this some fake news alternative site thing or is it the Wall Street Journal or The New York Times? I mean, the source really matters.

Has anyone tried to disprove the claim? This is super important because everybody thinks they’re right and every website has testimonials about this product or that idea; the question is not what do your supporters think but what do the people who don’t agree with you think? Because that’s what I want to know.

Has anyone run experiments to try to disprove your theory? In science, this is as basic as it gets. Karl Popper called this the principle of falsification, that is we can’t ever prove a theory correct, but we can disprove it by having an experiment that shows it’s wrong.

If you can’t falsify it, what are you really doing? And my favorite story on this, by the way—let me just have a little sidetrack here from Carl Sagan, he’s got this great little section in his book 'Demon-Haunted World': “There is a dragon in my garage. I have a dragon in my garage. Do you want to see it? Let me show you.” So I pull up the garage door I go, “Look. Can you see the dragon?” And you look in there and you go, “I don’t see anything.”

“Oh, sorry, this is an invisible dragon.”

“An invisible dragon?”

“Yeah, yeah he’s invisible.”

“Well, what if we put some flour on the ground and then we’ll get the footprints of the dragon.”

“Well, no, see, this is a special dragon that hovers above the ground, it floats. It’s an invisible floating dragon.”

“An invisible floating dragon. Okay. Wait, I have some infrared cameras here we can detect the heat of the dragon.”

”No, see this is a cold-blooded dragon. It doesn’t give off any heat.”

“What about the fire? We can detect the fire that the dragon spits out.”

“No, it spits out cold fire.”

You see the problem? If there’s no way for me to falsify that there’s a dragon there, what’s the difference between an invisible floating heatless dragon and no dragon at all? None.

And of course we can apply this to god or any other supernatural/paranormal-type phenomenon. If I can’t debunk it, if I can’t falsify it, if there’s no way to test it, then how will we ever know...

[davedmk]

I've been saying for over twenty years that critical thinking needs to be somewhere in the public school K-12 curriculum. Teaching kids HOW to think is, in my opinion, far more valuable than teaching them WHAT to think.

[chesterwilberforce]

My favorite Sagan quote (paraphrased) is that when he was asked to square religion with science his answer was essentially "I find that the older I get, the more tolerant of ambiguity I become. I simply don't have to know all the answers." If we all didn't insist on having to be right all the time about our beliefs, it would indeed be a kinder gentler world.

[lyraserpentine]

1. Reliability of source, Evidence, & Quality of evidence.
2. Has anyone tried to disprove the claim/theory; has it been tested or replicated?
3. Do personal beliefs influence the evidence? Is there an agenda?
4. Does the new idea being proposed account for the ideas of the old idea and the new anomalies?
5. Does the claimant play by the rules of science?

[michael.forkert]

Carl Sagan

One of the saddest lessons of history is this: "If we have been bamboozled long enough, we tend to reject any evidence of the bamboozle. We are no longer interested in finding out the truth. The bamboozle has captured us. It's simply too painful to acknowledge, even to ourselves, that we have been taken. Once you give a charlatan power over you, you almost never get it back”.

That’s the bamboozler’s confession.

[Showmetheevidence-]

I was lucky enough to have a lecturer in Uni that often said; “Prove me wrong” & “Question everything”… basically teaching us these lessons. Those 2 comments have been more important in my life than all the theories and textbooks we had to study!

[amritangshubaruah]

It's all in Carl Sagan's book 'The Demon-Haunted World'. One of the best and informative books I've read.

[musicauthority]

Carl Sagan was so brilliant for coming up with his baloney detection kit. among his many brilliant achievements.

[jonahtwhale]

Things were just as bad in Sagan's day. His own writings reveal this. He recalls a childhood experience of going to the library to find a book about the stars. The librarians gave him a volume on Hollywood Celebrities.

[heethn]

I guess I've been doing this my whole life without realizing it. They didn't like all my questions in church when I was little and I could never figure out why I was the only one asking them.

[marcochimio]

I teach at a local college, and it still blows me away how few of my students understand WHAT constitutes EVIDENCE (even after giving them the rules WITH correct & incorrect examples).

[pky]

Wow, so many people got hooked on the Wall Street Journal comment that they didn't bother listening to the rest of the video or even understand what the comment meant :S He didn't say "Believe the Wall Street Journal", he was saying "The Source Matters" & then goes on to ask questions like, "Would this source have an agenda?", "Is there a way to verify the claims independently?". He teaches a skepticism course for gods sake! And to those crying "fake news", there's a world of difference between "fake news" (false claims) & bad journalism (most journalism). Most of us have been aware of the inadequacies of journalism for donkey's years. If your old enough not to be a millennial & it's a new concept to you, then "fake news" isn't your biggest problem.

[CausticLemons]

Everyone complaining about the examples he used are missing the point. He wasn't saying the WSJ or NYT are the most reliable sources he was comparing them to no name blogs, conspiracy sites, and others that don't have the same level of scrutiny and accountability. No, I do not trust the WSJ 100% of the time but given the choice between them and a random YouTuber the choice is pretty simple.

[JohnnoDorber]

Carl Sagan was just one of those legendary scientists. I doubt if our grandchildren's grandchildren won't hear about him. Baloney test indeed! It sort of sounds like what I know of as 'Empirical Method'. Every theory is valid until it is disproved. Many unsuccessful attempts to disprove a theory may give a consensus of accuracy but an innovative experiment may still topple a beloved and long-standing theory.

[midnightrambler]

"Demon Haunted World" changed my life it put the final nail in the coffin of my remaining religious beliefs.

[Chiccone]

Sagan, Randi, even Carlin were all great bullshit detectors and I miss all of them even though Randi is still alive but ill.

[furdfelmer]

"Broca's Brain", by Carl, was one of the most inspirational books I ever read, back in the day, , , , the insights and concepts within this book have never faded all these decades later.
Carl, and others, taught me the difference between "smart" and "wise"...he was both, and very rare among those as, "celebrity experts". Most in that category are smart, but only a tiny fragment are wise.
Richard Feynman, ranks up there with Carl....both of these men, not only were expert in their fields, but pure genius in teaching others how to achieve "critical thinking" skills.

[imapseudonym]

We need another Carl Sagan, and NOW.
The ill-educated, knuckle-dragging mouth breathers are taking over.
Both Sagan and Asimov warned us about the upcoming "culture of ignorance" but neither of them could ever imagine it would ever get so bad.

[saganworshipper]

"We're all in this greenhouse together."~CS

[patbrennan]

what realy made carl a great man was simple, he told the truth..

[ekenny]

Appreciate the quick critical thinking revision course. I always try to apply a healthy dose of skepticism (especially when someone wants my money or my vote). One important exception though, human experience usually cannot be proved, tested or measured. A family member of mine was an amputee who experienced 'phantom limb pain' and asked what could be done to address it. The doctor could have said 'I can prove to you that I amputated that limb one month ago. What proof can you offer that it hurts?' Nowadays, maybe an MRI or brain scan could 'prove' the experience of pain, but for centuries before, no proof could be offered. We were grateful to the doctors who used the scientific method to devise 'mirror therapy' for phantom limb pain. This therapy can drastically reduce the need for pain medication. Just an example of an exception.