Is Science Reliable?

It seems like every few months, there’s some kind of news about problems with the scientific publishing industry. Why does this keep happening? And what can be done to fix the system?

Hosted by: Hank Green
----------
----------
Dooblydoo thanks go to the following Patreon supporters -- we couldn't make SciShow without them! Shout out to Justin Ove, Andreas Heydeck, Justin Lentz, Will and Sonja Marple, Benny, Chris Peters, Tim Curwick, Philippe von Bergen, Patrick, Fatima Iqbal, Lucy McGlasson, Mark Terrio-Cameron, Accalia Elementia, Kathy & Tim Philip, charles george, Kevin Bealer, Thomas J., and Patrick D. Ashmore.
----------
----------
Looking for SciShow elsewhere on the internet?
----------
Sources:

[srpskihayk]

A favorite quote of mine "Science does not lie, but scientists do."

[Imagine-Baggins]

Sounds to me like money is at the root of most of these problems...

[Skywalker-zuod]

This is a sentinel episode of SciShow. THANK YOU FOR THIS. IT IS INCREDIBLY IMORTANT. BEST EPISODE YET.

[EpicB]

I wonder how many people will interpret this video wrong.

[BlueYoshi]

A phrase I heard thrown around in my Biomedical Research class is "publish or perish", and I think that's a pretty accurate summary of what you were describing from 5:24 to 6:34. It's definitely a real issue

[ohno]

More reliable than a 5000 year old story book

[MattRieckman]

I am actually absolutely shocked you are being so open about this. Skeptics have been pointing out the exact problems described in this video for years and only got mocked and ridiculed in response. Fanatical, dogmatic belief in the infallibility of the current publishing and peer review system has been so pervasive that I was not expecting open honesty from any science educators within my lifetime.

[HappyHusbandnWife]

Finally SciShow address the underline problem of "the scientific publishing industry". I do agree with everything were mention in this vid, such as:
+ The inconsistenc of those study
+ Human factor/errors
+ Lack of knowledge in math( which is being introduced/integrated to others field now),
+ Pressure to public "positive" result.
+ And the culprit MONEY, that's right MONEY
Well done on none bias report SciShow !!!

[alexwillenbrink]

I work in a biological sciences lab and holy crap THIS IS TOTALLY A THING AND ANNOYS THE BAGIZASS OUT OF ME. Not being able to publish results because lack of significance becomes extremely frustrating because nobody wants to admit they've wasted a ungodly amount of time on something they thought would pan out. Changing the way you interpret the results can easily make data that shows little or no significance look very nice and very significant. Unfortunately, that's not proper science...but it happens a lot.

[notlun]

You can't expect serious comments in less then 15 minutes after the initial upload time, comment section is literally only shitposters

[jeremyj.]

Thank you very much for highlighting this! I feel like the worst factor is that replication studies are often considered "not worth the time" because, well, that thing has already been done and it wouldn´t be as exciting for the publishing journals or even simply the scientists themselves.

[NarekGaming]

I'm in college and I had to do educational research. The one major problem I came up with is that most research journals, magazines and more all require some sort of fee to access articles. The reason why I am bringing this up was if a free online journal were to come out which allows research articles to be published, it could get out to more people and allow for more researchers to test other team's work and would also allow for negative results to be published as well.

[DustinRodriguez_]

The beautiful thing about science is that when we are wrong, it's not because there was a flaw in science, it's because there was a flaw in the PEOPLE doing it. We _CAN_ examine our beliefs and identify exactly where flaws might lie. It makes things more complicated, sure, but reality is complicated. It's a personal choice if someone, for instance, chooses to trust something radically flawed such as psychology. But we know it's flawed, and that performing legitimate scientific studies in that field is impossible (in the few cases where it IS possible, it turns out to be neuroscience, not psychology). If a subject is very important, and people care very strongly about it, we can go to the absolute limits of our willingness to tolerate error and ability to investigate the matter.

But in every single case where a thing previously held up as scientifically valid is found to be untrue, we can show exactly where prior scientists failed to meet the standards set by science itself. We believed leaded gasoline was safe because the standard of proof was stupendously bad, those doing the research did only a couple horribly flawed studies that no one in their right mind should have trusted, but a few people cut corners to make their careers look better. And they're personally responsible for the deaths of millions of people, disease in many more, and an unknowable amount of human suffering as a result. They absolutely should have known better, and their names should really be more widely known so that people can spit when they think of them. And we should definitely remember them any time someone studies a handful of WEIRD (western, educated, industrialized, rich, democractic) undergrads and claims to have made a finding about the entire human race. And if those studies are used to back some side (regardless of whether it's "your" side or not) of an argument influencing public policy, they should be called out and the people suggesting we affect peoples lives based on them should be seen as the dangerous, shortsighted, biased, malicious people they are.

A real scientific study makes findings which are parsimonious - they are restricted to only the group from which their subjects were drawn (randomly). Any wider applicability of their findings may be reasoned about, and even conjectured, but only with ample warning to the reader that they are exactly that - mere conjecture. If a psychology study were to do this, and do it with integrity, the paper would end up being laughable. There would be 60 pages of irrelevant-sounding criteria that they didn't control for (doing a study on the effects of violent media on the youth? Did you take care to examine the diet of your subjects to ensure there was not a statistically abnormal amount of vegetarians or junk-food-eaters or high-carb or high-fat or low-fiber people present? No? Then you have to explain that. I've never even seen a study on media effects on a population that even controlled for consumption of the EXACT media they are claiming to have drawn conclusions about! Not once! 90% of their study participants could have just gotten off a weekend-long Halo LAN party binge and they would not have a clue because they don't even care enough to ASK. The vast majority of studies also totally conflate witnessing actual violence in person and watching violence on a screen, making the completely baseless assumption that all of the brain processing involved in taking a flat image at the wrong proportions while every single other sense is feeding completely contradictory sensory information and figuring out that it's supposed to represent a scene with human beings has zero effect upon the brains ability to process it differently from in-person experience of actual violence which puts them in real danger. That, not violent media, makes me want to slap somebody.)

The terrible truth that most people don't want to realize is that there is no free lunch, and no simple answers. Even things which appear simple are complex. The desire for simplicity is understandable, an outgrowth of the base structure of the human brain, universal, and utterly wrong. Like most of our traits that came from long evolution, it is perfectly honed to make humans into survival machines - on an African savannah with no language skills and in a small tribe surrounded by abundant food and predatory animals where 'survival' meant making it a little past puberty and producing at least 1 child as young as possible. For a situation where we have language, complex social structures, no danger from predatory animals, and advanced stores of knowledge, everything that comes from that evolutionary development is misleading at best, and most often dangerously destructive. It's what enables a few researchers to trick themselves into thinking that cutting a few corners and not "being a hardass" about scientific rigor will be harmless, so that before they know it millions of people are profoundly and negatively affected by those little errors.

[Remlap]

Oh God, P-Values are my trigger words. I'm getting horrible flashbacks to my advanced statistics class. If this gets any worse, I'm going to have to create a Patreon.

[ReidarWasenius]

Very important topic!! Thanks for highlighting these issues.

[treverbostelaar]

So glad you guys made this video. I'm an undergraduate researcher and my PI and postdocs have been complaining about this for a while now.

[rebelbeammasterx]

The science of science should be called, wait for it, "Scientology" wait, that's not right.

[Aziraphale]

I find it interesting that many religious believers see the built-in error correction of the scientific method as a flaw, rather than a feature.

[nathantron]

I wish I could work for SciShow... I would love to do research for their shows, it's amazing, and I can't learn enough!

[eryntey]

Such an important topic! Thanks for covering it!