Wrap-ups and recordings of the Leiden University Libraries & Elsevier seminars on Reproducible Research
Leiden University Libraries (UBL) in partnership with Elsevier hosted a series of online seminars on the challenges involved in achieving reproducibility in research. The seminars aimed to identify best practices that can help to overcome central challenges around reproducibility, and to convey several concrete guidelines that can help researchers during their attempts to make their own research transparent and verifiable. While discussions of crucial theoretical concepts got ample attention, the seminars also showcased concrete experiences gained during various case studies. The seminars were held on Thursday 22 April, 29 April and Wednesday 12 May. A wrap-up for each session is available in the programme below, as well as recordings of individual lectures.
Reproducible Research
The aim to achieve reproducibility forms one of the central drivers of the open science movement. It has often been assumed that, when researchers offer access to their data, and when they diligently document their methods, their work can in turn enable peers to replicate their analyses, and to verify their conclusions. It has been stressed, moreover, that reproducibility can produce value for a wide range of stakeholders in the field of academia. For researchers, reproducibility ensures transparency, and it enables peers to assess the reliability of academic findings. By conducting research in a reproducible manner, researchers may enhance their accountability, and, additionally, their reputation.
Reproducibility may also benefit students and scholars in training. By studying and replicating research methodology designed by experienced researchers, they can familiarise themselves with the conventions and the minutiae of specific research methods. This may, in turn, inspire students to emulate these examples. Achieving greater verifiability of academic findings can be of importance to academic publishers too, since such efforts could be viewed as a form of peer review. By replicating particular results, publishers could ensure the reliability of the findings they disseminate. Reproducible research can be valuable to funding agencies, finally, as it ultimately fosters the reuse of research materials. When data can be reused beyond the context in which they were created, financial resources could be utilised in a much more effective and judicious manner.
Challenges
Over the last few years, numerous authors have decried the emergence of a ‘replication crisis’ in fields such as Medicine, Psychology and Biology. Despite recent progress, the transition to reproducible research still faces multiple challenges. Publishing data in agreement with the FAIR data management principles can be very labour-intensive, and the need to document workflows and methodologies invariably demands a string sense of discipline. For researchers aiming to make their research fully reproducible, the concrete steps to be followed are often unclear. Researchers attempting to replicate specific studies have often found that data sets referenced in publications are often inaccessible or incomplete, and papers frequently lack the details needed to repeat certain analyses. In the case of older research, experiments may have been conducted using outdated or obsolete software. And even when the data and the methods are fully transparent, replicated research might still yield results that diverge from the findings of the original studies.
Programme
Click the title of the seminar to read the wrap-up for the session. Recordings of each lecture are available on youtube. Follow the link under the title of a lecture to watch a specific recording.
Seminar 1: Rationale and benefits (Thursday 22 April)
This seminar addressed the overall rationale of reproducible research. The speakers in the session explored the scholarly and the societal benefits of reproducibility in research, along with the actions that may need to be taken to expedite the cultural change that is needed to make the transition to reproducible research practices.
- Welcome (15 min): Introduction to the lecture series.
- Lecture (30 min): Dorothy Bishop, Professor of Developmental Neuropsychology, University of Oxford. Reproducibility: A crisis for scientists or society?
- Lecture (30 min): Brian Nosek, co-Founder and Executive Director of the Center for Open Science and Professor in the Department of Psychology at the University of Virginia. Improving reproducibility of research.
- Panel discussion (40 min): Actions need to effectuate a cultural change. Panel discussion focusing on stakeholder analysis: Researchers, Publishers, Funders.
Seminar 2: Theoretical session (Thursday 29 April)
The second session aimed to discuss the phenomenon of reproducibility mainly on a conceptual level. As one of its central questions, this seminar examined the scope and limits of the reproducibility. Should the call for replication be targeted mainly towards those disciplines following a quantitative or empirical approach?
- Welcome (20 min): Recapitulation of session 1 and introduction of the topic.
- Lecture (30 min): Christof Schöch, Professor of Digital Humanities, Trier Center for Digital Humanities. A Typology of Reproducible Research: Concepts, Terms, Examples.
- Lecture (30 min): Bart Penders, Associate Professor Metamedica, School for Public Health and Prim Care, Fac. Health, Medicine and Life Sciences, Maastricht University. The right tool for the job? Replication as research norm.
- Lecture (30 min): Peter Verhaar, Lecturer at Book & Digital Media Studies, Leiden University. Illustration: accountability and reproducibility in the digital humanities.
Seminar 3: Practical session (Wednesday 12 May)
The speakers in the third session explained the concrete steps researchers can take to make their research reproducible. It offered insights on the aspects of the research project that need to be documented, and on the type of resources that need to be made available.
- Welcome (5 min): Recapitulation of session 1 & 2 and introduction of the topic.
- Lecture (20 min): Catriona Fennell, Director Journal Services, Elsevier. The role of the publisher.
- Lecture (40 min): Kristina Hettne, Digital Scholarship Librarian, Leiden University Libraries, How to ‘ReproHack’: Learn how to reproduce the results of a study.
- Lecture (25 min): Gabriele Hayden, Research Data Management and Reproducibility Librarian at the University of Oregon, Improving reproducibility: practical steps for researchers.
- Panel (15 min): Kristina Hettne and Gabriele Hayden: What does a replication study entail on a concrete level? Discussion of main challenges.
Collaboration with Elsevier
In 2019, Leiden University Libraries and Elsevier jointly hosted a symposium on Tools Criticism and a public lecture by visiting Scaliger Professor Ted Underwood, Professor of Information Sciences and of English, University of Illinois at Urbana-Champaign: "The Role of the Humanities in an Information Age". Elsevier also sponsors the UBL Fellowship Program for Digital Scholarship. This fellowship offers researchers the unique opportunity to conduct computational research in close cooperation with the Centre for Digital Scholarship (CDS) at Leiden University Libraries. The fellowship program supports researchers during a two-month stay in Leiden. In the fellowship, special attention will be paid to the ways in which research findings can be made reproducible and FAIR. Paolo Rossini, for instance, spent two months at the CDS thanks to an Elsevier Fellowship for Digital Scholarship. Using digital tools, he reconstructed the networks of René Descartes and his followers from epistolary sources.