Click the star to add/remove an item to/from your individual schedule.
You need to be logged in to avail of this functionality.
Log in
- Convenor:
-
Lukas Röseler
(University of Münster)
- Format:
- Pre-conference virtual symposium
Short Abstract:
We discuss why there are still only relatively few replication studies in the social sciences and present multiple projects from the FORRT Replication Hub that enable future replications, facilitate the execution of replications, and reward publication of replication research.
Description:
In this sequence of presentations, we discuss why replications - despite their high relevance - are rarely conducted and published and provide solutions that are structured around the COS's pyramid of culture change:
- Replications are often not possible due to deficiencies in reporting standards and data availability.
- There is no consensus on standards of replicability, leaving the door open for successful and failed replications to be dismissed as irrelevant.
- There is still a lack of interest in robust research by commercial journals and funders due to an overreliance on bibliometrics in research assessment.
- In sum, replication research is not rewarding unless it is based on simple studies (e.g., online surveys), reported in batches of hundreds of findings, and yields far reaching results about the replicability of research from entire fields or journals.
Many metascientific studies rely on replication attempts with questionable generalizability. To support researchers in conducting replication research in a sustainable way and to provide a solid basis for meta-science on replicability in the long run, we present multiple projects from the FORRT Replication Hub that solve these problems:
- Open educational resources enable future replications.
- An interdisciplinary and collaborative guide to replications and reproductions will facilitate the execution of replications.
- Replication rankings and journal databases allow researchers to choose journals based on robustness and openness instead of misleading bibliometrics.
- A diamond open access journal will reward the publication of replication research and allow for discussions about replication methods spanning multiple disciplines.
| Register to attend | https://cos-io.zoom.us/webinar/register/WN_mo4IRqnoSQWerakDAHQsZw#/registration |
Accepted papers:
Short abstract:
Robust science needs venues that value rigor, transparency, and inclusion. I introduce Replication Research (R2), a diamond open access journal for replications, reproductions, and methods across sciences. R2’s open, community-built model sets new standards for responsible publishing.
Long abstract:
Robust, reproducible science requires a dedicated venue that values rigor, responsible communication, and academic inclusivity. In this talk, I present the creation of Replication Research (R2, https://replicationresearch.org), a diamond open access journal designed to publish replications, reproductions, and methodological advances across the social, cognitive, behavioral, and medical sciences. R2’s constitution and workflows were developed through an open, community-driven discussion series that foregrounded core values—scientific accuracy, societal responsibility, high standards for open science, and supportive mentorship. I outline our approach, including responsible peer review practices (with openness balanced by protections for early career researchers) and thorough reproducibility checks. By centering transparency, rigorous standards, and community engagement, R2 aims to legitimize replication as a vital scholarly contribution and set a new benchmark for responsible research publishing.
Short abstract:
Discovering replication studies is challenging due to scattered resources. This talk presents FORRT’s solutions such as the FORRT Replication Database and an Annotator, which centralize and tag replications for easy access—boosting meta-research, and integration into scientific practice.
Long abstract:
Lukas Wallrich: Enhancing the Discoverability of Replications
The lack of centralized, accessible resources for replication studies hinders their integration into research synthesis and practice. To address this, FORRT has been developing innovative tools designed to enhance the discoverability and visibility of replication research. In this talk, I will introduce the FORRT Replication Database (FReD)—a curated, searchable repository of replication findings— the FReD Annotator App, which streamlines the process of tagging and documenting replication studies, and future tools that are currently in development. I will showcase how these tools can be integrated into researchers’ workflows, supporting meta-research, transparency, and cumulative science.
Short abstract:
Replication is vital yet undervalued in science. FORRT drives culture change by promoting replication through education, advocacy, and resources. This talk outlines our initiatives to normalize, reward, and embed replication in research, advancing a more trustworthy scientific ecosystem.
Long abstract:
Despite being vital for cumulative science, replication research has historically been undervalued in academia. The Framework for Open and Reproducible Research Training (FORRT) aims to address this by fostering research culture change through education, advocacy, and community engagement. In this talk, I will outline FORRT’s initiatives to normalize, recognize, and incentivize replication efforts. I will discuss our educational resources, mentorship programs, and efforts to promote open science as standard practice, as well as highlight strategies for integrating replication work into hiring, promotion, and publication criteria. By facilitating a shift toward appreciating replication as essential scholarship, FORRT strives to build a more robust and trustworthy scientific ecosystem.
Short abstract:
Student-led replication studies boost research training and strengthen science by verifying findings. This talk highlights their impact on transparency, and reproducibility, and advocates for integrating replications into curricula with strong mentorship and collaborative models.
Long abstract:
Student-led replication studies offer a powerful solution to the lack of replication studies: they not only provide invaluable training in rigorous research methods, but also strengthen the scientific record through collective efforts to verify existing findings. In this talk, I will discuss the impact of student replications on improving statistical power, research transparency, and the broader culture of reproducibility. Drawing on empirical evidence and practical examples, I will make the case for integrating replications into student curricula and outline effective models for mentorship, collaboration, and publication—ensuring that the next generation of scientists is equipped to build robust and trustworthy science.