IASSIST GVC 2021: Research Reproducibility (2021-05-17)

0:06 ReprohackNL 2019: Enhancing research reproducibility at Dutch Universities
16:06 Computational reproducibility: A simplified framework for data curators
31:00 Computational reproducibility: Examining verification errors and frictions

ReprohackNL 2019: Enhancing research reproducibility at Dutch Universities
University Libraries around the world play a crucial role in Open Science, contributing to more transparent, reproducible and reusable research. The Center for Digital Scholarship (CDS) at Leiden University (LU) is a scholarly lab located in the LU Library. The CDS employes two complementary strategies to improve open data literacy among Leiden’s scholars: existing top-down structures are used to provide Open Science training and services, while bottom-up initiatives are actively supported by offering CDS’s expertise and facilities. A prime example of how bottom-up initiatives can blossom with the help of the CDS is the ReproHack initiative. ReproHack – a reproducibility hackathon – is a grass-root initiative by young scholars with the goal of improving research reproducibility in three ways: First, hackathon attendees learn about reproducibility tools and challenges by reproducing published results and providing feedback to authors on their attempt. Second, authors can nominate their own work and receive feedback on their reproducibility efforts. Third, the collaborative atmosphere at the event helps building an interdisciplinary community among researchers, who often lack such support in their own departments.
A first ReproHack in the Netherlands took place on November 30th, 2019, co-organised by the CSD at the LU Library with 44 participants from the fields of psychology, engineering, biomedicine, and computer science, 3 had submitted their own work to the hackathon. From 19 papers with code and data, 24 feedback forms were filled, 5 papers were successfully reproduced and 6 where almost reproduced. Two speakers framed the event, the first one introducing participants to current developments on tools for reproducible research and the second one putting reproducibility into a broader context.
Kristina Hettne, Leiden University Libraries
Linda Nab, Leiden University Medical Center
Paloma Rojas Saunero, Erasmus Medical Center
Daniela Gawehns, Leiden University

Computational reproducibility: A simplified framework for data curators
Phrases like the ‘data deluge’ and the ‘reproducibility crisis’ may serve to further the impression that data curation is hard and that research data management is “basically fighting against chaos” (Briney, 2019). If trying to manage research data is chaotic, then the management of computationally-derived data presents an even bigger challenge due to the multiplicity of operating systems, coding languages, dependencies, and file types. This is exacerbated by the reality that most researchers and librarians are not formally trained as programmers. This purpose of this presentation is to propose an approach of ‘just enough’ data curation by arguing that partial reproducibility is better than nothing at all (Broman, n.d.). By focusing on incremental progress rather than prescriptive rules, researchers and curators can build their knowledge and skills as the need arises. A computational reproducibility framework, developed for the Canadian Data Curation Forum, will serve as the model for this approach, which combines learning about reproducibility with improving reproducibility.
Sandra Sawchuk, Mount Saint Vincent University
Shahira Khair, University of Victoria

Computational reproducibility: Examining verification errors and frictions
Data archives, libraries, and publishers are extending their services to support computational reproducibility of results reported in manuscripts. Computational reproducibility is having enough information about the data, code, and compute environment to re-run and reproduce analyses. While archives and publishers are adopting policies and audit workflows to verify the results in a manuscript, many opponents express concerns about the additional effort, time, and specialized expertise being placed on authors. What are the challenges that researchers face in complying with computational reproducibility and transparency policies?
Cheryl Thompson, UNC Odum Institute
Thu-Mai Christian, UNC Odum Institute