Is there a reproducibility crisis in science?

Reproducibility is a hot topic in science at the moment, but is there a crisis? Nature asked 1,576 scientists this question as part of an online survey. Most agree that there is a crisis and over 70% said they'd tried and failed to reproduce another group's experiments.

[Mandragara]

Publish or perish will be a nail in the coffin of western science.

[ContraFidem]

I really can't understand how a huge portion of the respondents have been unable to reproduce others' or their own results, but still believe that most published research is trustworthy.

[anadus]

So if I were to repeat this survey, I wonder if I can reproduce the same insights

[tnichlsn]

maybe start by making tenure decisions less on length of publication lists and more on teaching skills.

[shuchko]

Excellent!!! I am looking forward to see these suggestions finally implemented across the science community. By the way, industrial research would benefit most from data that can be reproduced. In my experience 90 % of published data in chemistry is difficult to reproduce. Industry has long ago endorsed programs like six sigma and experimental design. These are not geared towards process optimization alone, but also for proving reproducbility, identifying confounded variables etc. one more thing, if you shake write it down in the experimental

[Zajcooo]

now reproduce your survey results!
jk

[Martin-posz]

We need to put serious money into reproducing results and hunting down fraud in the scientific community.

[SamuelHurstClayton]

Yes...But is this survey REPRODUCIBLE?

[davidsweeney]

The more you publish the faster you become a professor. Remove this requirement and you remove the reproducibility problem. Simple!

[TonmoyKabiraj]

If the same protocols and/or procedures are followed but the result or finding is not reproducible, then it's not a valid research.

[iamgoddard]

Didn't you ask how often they _do_ reproduce findings?

Without knowing the relative frequency of mismatching _vs_ matching results, what you're reporting isn't meaningful. Reading a lot of peer-reviewed studies that investigate similar phenomena, I do not see 70% of studies coming to contrary conclusions. If this 'crisis' was real or widespread, it would manifest in a sea of confusion in science. In contrast, by and large science converges toward consensus results. There may be exceptions in subfields with little research.

[theragingmoderate]

Their is a "future prediction model crisis" too

[agranero]

00:58 That is why James Bond always asked his martini shaken, not stirred. Seriously now: negative results are as important as positive ones. Imagine the waste of time and resources trying something already done without success because failures are not published. Besides not always raw data is available for analysis, the attempts to publish data from research specially in medicine encountered heavy resistance of researchers alleging privacy issues. You could publish raw data without compromising privacy of test subjects or they could waive the right for data be published, or data could be published on a need to know basis between researchers. If you give no access to raw data, you don't know what was discarded and why. The same is valid for physics where most of the data collected from Higgs boson experiments is discarded on the fly by computers, because of the huge quantity of data generated.
Now reason with me: a research that can't have raw data analyzed to preserve privacy of test subjects, a discovery that can't be done again on another accelerator (because there is none), data collection that can't be analyzed using other assumptions deserve to be published?
Maybe: on the Journal of Irreproducible Results, because researchers are asking: please trust us, we know what we are doing. Well scientific method is based on heavy skepticism not without reason: Fermi discovering a non existent element and winning the Nobel for that (not only that), Deryagin and poly-water, Kammerer, N-rays, water memory, phlogisthon, caloric, Lorentz electron theory, steady state cosmology, Piltdown man and more recently cold fusion. Even Mendel data, despite of its conclusions being correct seems to have been "sanitized" of wrong results. Milikan gave "stars" to data collected from oil drop experiment, considering some and discarding others, despite of getting the correct electron charge . We never would know that without access to their raw data.

[nash]

Depends on the methodology, questions asked, cred of the person, but mostly expectations. Do we need to study whether a gun fired lead bullet into your heart will cause your death, or do we have to subject animals to increasing levels of a toxin to find out just at what point death overcomes them due to the toxicity of the substance. So many animals are killed/sacrificed for extending human life for a bit longer with no regard for the animals who died to secure that finding.

[michaelrothschild]

Oh big surprise your group does not find this to be a problem. It's like someone being shot in front of you and you say you don't your own eye witness testimony.

[Arizzaable]

how statistic can help reproducibility crisis?

Grouping all the sciences together seems fairly unhelpful

[gs]

The peer review process cannot effectively determine whether the work is reproducible. In addition, these journals care mostly about high impact results, and proving reproducibility does not benefit that purpose at all, and it requires a huge amount of data, slowing down the publication. Therefore almost nobody cares. People after you got tripped? Who cares?

[jscotthatcher]

damn your non solid white background in the video making me think my screen was dirty. ; P

[violet-trash]

We all know the problem isn't with physicists and chemists. 🤷‍♀️